On 5 October 2017 at 18:45, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> After we move to not sharing the GIL between interpreters:
>
> Channel.send(obj): # in interp A
> incref(obj)
> if type(obj).tp_share == NULL:
> raise ValueError("not a shareable type")
> set_owner(obj) # obj.owner or add an obj -> interp entry to global
> table
> ch.objects.append(obj)
>
> Channel.recv(): # in interp B
> orig = ch.objects.pop(0)
> obj = orig.tp_share()
> set_shared(obj, orig) # add to a global table
> return obj
>
This would be hard to get to work reliably, because "orig.tp_share()" would
be running in the receiving interpreter, but all the attributes of "orig"
would have been allocated by the sending interpreter. It gets more reliable
if it's *Channel.send* that calls tp_share() though, but moving the call to
the sending side makes it clear that a tp_share protocol would still need
to rely on a more primitive set of "shareable objects" that were the
permitted return values from the tp_share call.
And that's the real pay-off that comes from defining this in terms of the
memoryview protocol: Py_buffer structs *aren't* Python objects, so it's
only a regular C struct that gets passed across the interpreter boundary
(the reference to the original objects gets carried along passively as part
of the CIV - it never gets *used* in the receiving interpreter).
> bytes.tp_share():
> obj = blank_bytes(len(self))
> obj.ob_sval = self.ob_sval # hand-wavy memory sharing
> return obj
>
This is effectively reinventing memoryview, while trying to pretend it's an
ordinary bytes object. Don't reinvent memoryview :)
> bytes.tp_free(): # under no-shared-GIL:
> # most of this could be pulled into a macro for re-use
> orig = lookup_shared(self)
> if orig != NULL:
> current = release_LIL()
> interp = lookup_owner(orig)
> acquire_LIL(interp)
> decref(orig)
> release_LIL(interp)
> acquire_LIL(current)
> # clear shared/owner tables
> # clear/release self.ob_sval
> free(self)
>
I don't think we should be touching the behaviour of core builtins solely
to enable message passing to subinterpreters without a shared GIL.
The simplest possible variant of CIVs that I can think of would be able to
avoid that outcome by being a memoryview subclass, since they just need to
hold the extra reference to the original interpreter, and include some
logic to swtich interpreters at the appropriate time.
That said, I think there's definitely a useful design question to ask in
this area, not about bytes (which can be readily represented by a
memoryview variant in the receiving interpreter), but about *strings*: they
have a more complex internal layout than bytes objects, but as long as the
receiving interpreter can make sure that the original string continues to
exist, then you could usefully implement a "strview" type to avoid having
to go through an encode/decode cycle just to pass a string to another
subinterpreter.
That would provide a reasonable compelling argument that CIVs *shouldn't*
be implemented as memoryview subclasses, but instead defined as
*containing* a managed view of an object owned by a different interpreter.
That way, even if the initial implementation only supported CIVs that
contained a memoryview instance, we'd have the freedom to define other
kinds of views later (such as strview), while being able to reuse the same
CIV machinery.
Cheers,
Nick.
--
Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20171005/2c041f5c/attachment.html>
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4