[I tried to post this on SourceForge, but as usual, it hates my guts]
I have been hacking on ways to make lighter-weight Python objects using the
__slots__ mechanism that came with Python 2.2 new-style class. Everything
has gone swimmingly until I noticed that slots do not get pickled/cPickled
at all!
Here is a simple test case:
import pickle,cPickle
class Test(object):
__slots__ = ['x']
def __init__(self):
self.x = 66666
test = Test()
pickle_str = pickle.dumps( test )
cpickle_str = cPickle.dumps( test )
untest = pickle.loads( pickle_str )
untestc = cPickle.loads( cpickle_str )
print untest.x # raises AttributeError
print untextc.x # raises AttributeError
Clearly, this is incorrect behavior. The problem is due a change in object
reflection semantics. Previously (before type-class unification), a
standard Python object instance always contained a __dict__ that listed all
of its attributes. Now, with __slots__, some classes that do store
attributes have no __dict__ or one that only contains what did not fit into
slots.
Unfortunately, there is no trivial way to know what slots a particular class
instance really has. This is because the __slots__ list in classes and
instances can be mutable! Changing these lists _does not_ change the object
layout at all, so I am unsure why they are not stored as tuples and the
'__slots__' attribute is not made read-only. To be pedantic, the C
implementation does have an immutable and canonical list(s) of slots, though
they are well buried within the C extended type implementation.
So, IMHO this bug needs to be fixed in two steps:
First, I propose that class and instance __slots__ read-only and the lists
made immutable. Otherwise, pickle, cPickle, and any others that want to use
reflection will be SOL. There is certainly good precedent in several places
for this change (e.g., __bases__, __mro__, etc.) I can submit a fairly
trivial patch to do so. This change requires Guido's input, since I am
guessing that I am simply not privy to the method, only the madness.
Second, after the first issue is resolved, pickle and cPickle must then be
modified to iterate over an instance's __slots__ (or possibly its class's)
and store any attributes that exist. i.e., some __slots__ can be empty and
thus should not be pickled. I can also whip up patches for this, though I'll
wait to see how the first issue shakes out.
Regards,
-Kevin
PS: There may be another problem when when one class inherits from another
and both have a slot with the same name.
e.g.:
class Test(object):
__slots__ = ['a']
class Test2(Test):
__slots__ = ['a']
test=Test()
test2=Test2()
test2.__class__ = Test
This code results in this error:
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: __class__ assignment: 'Test' object layout differs from 'Test2'
However, Test2's slot 'a' entirely masks Test's slot 'a'. So, either
there should be some complex attribute access scheme to make both slots
available OR (in my mind, the preferable solution) slots with the same
name can simply be re-used or coalesced. Now that I think about it,
this has implications for pickling objects as well. I'll likely leave
this patch for Guido -- it tickles some fairly hairy bits of typeobject.
Cool stuff, but the rabbit hole just keeps getting deeper and deeper....
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com
Fax: (216) 986-0714 WWW: http://www.theopalgroup.com
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4