On 2008-05-17 16:59, Alexandre Vassalotti wrote:
> On Sat, May 17, 2008 at 5:05 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>> I'd like to bring a potential problem to attention that is caused
>> by the recent module renaming approach:
>>
>> Object serialization protocols like e.g. pickle usually store the
>> complete module path to the object class together with the object.
>
> Thanks for bringing this up. I was aware of the problem myself, but I
> hadn't yet worked out a good solution to it.
>
>
>> It can also happen in storage setups where Python
>> objects are stored using e.g. pickle, ZODB being a prominent
>> example. As soon as a Python 2.6 application starts writing to
>> such storages, Python 2.5 and lower versions will no longer be
>> able to read back all the data.
>>
>
> The opposite problem exists for Python 3.0, too. Pickle streams
> written by Python 2.x applications will not be readable by Python 3.0.
> And, one solution to this is to use Python 2.6 to regenerate pickle
> stream.
>
> Another solution would be to write a 2to3 pickle converter using the
> pickletools module. It is surely not the most elegant or robust
> solution, but I could work.
I'm not really worried much about going from 2.x to 3.x.
Breakage is allowed for that transition.
However, the case is different for going from 2.5 to 2.6. Breakage
should be avoided if at all possible.
>> Now, I think there's a way to solve this puzzle:
>>
>> Instead of renaming the modules (e.g. Queue -> queue), we leave
>> the code in the existing modules and packages and instead add
>> the new module names and package structure with pointers and
>> redirects to the existing 2.5 modules.
>
> This would certainly work for simple modules, but what about packages?
> For packages, you can't use the ``sys.modules[__name__] = Queue`` to
> preserve module identity. Therefore, pickle will use the new package
> name when writing its streams. So, we are back to the same problem
> again.
>
> A possible solution could be writing a compatibility layer for the
> Pickler class, which would map new module names to their old at
> runtime. Again, this is neither an elegant, nor robust, solution, but
> it should work in most cases.
While it's possible to fix pickle (at least the Python version),
this would not help with other serialization formats that rely
on the .__module__ attribute mapping to an existing module.
It's better to address the problem at the module level.
Perhaps I have a misunderstanding of the reasoning behind
doing the renaming in the 2.x branch, but it appears that
the only reason is to get used to the new names. That's a
rather low priority argument in comparison to the breakage
the renaming will cause in the 2.x branch.
I think it's much better to have 2to3.py do the renaming
and only add warnings to the renamed modules in 2.x
(without actually applying any renaming).
It would also be possible to seed sys.modules with module
proxy objects (see e.g. mx.Misc.LazyModule from egenix-mx-base)
which only turn into real module object if the module is
referenced.
This would allow adding a "from __future__ import new_module_names"
which then results in loading proxies for all renamed modules
(without actually loading the modules until they are used under
their new names).
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, May 18 2008)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________
:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4