On 04:58 pm, jaedan31 at gmail.com wrote: >Jesse Noller <jnoller <at> gmail.com> writes: >>We already have an implementation that spawns a >>subprocess and then pushes the required state to the child. The >>fundamental need for things to be pickleable *all the time* kinda >>makes it annoying to work with. > >This requirement puts a fairly large additional strain on working with >unwieldy, wrapped C++ libraries in a multiprocessing environment. >I'm not very knowledgeable on the internals of the system, but would >it be possible to have some kind of fallback system whereby if an >object >fails to pickle we instead send information about how to import it? >This >has all kinds of limitations - it only works for importable things >(i.e. not >instances), it can potentially lose information dynamically added to >the >object, etc., but I thought I would throw the idea out there so someone >knowledgeable can decide if it has any merit. It's already possible to define pickling for arbitrary objects. You should be able to do this for the kinds of importable objects you're talking about, and perhaps even for some of the actual instances (though that depends on how introspectable they are from Python, and whether the results of this introspection can be used to re-instantiate the object somewhere else). Take a look at the copy_reg module. Jean-Paul
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4