Aahz <aahz@pythoncraft.com> writes:
> On Sun, Dec 29, 2002, David Abrahams wrote:
>>
>> Python extension module B calls shared library Q and uses Q's callback
>> interface. Because some of the callbacks need to use the Python API,
>> and *might* be invoked by threads, they must all acquire the GIL.
>> Because they also might be invoked by direct calls into Q, B must
>> always release the GIL before calling anything in Q.
>
> So you're saying that the callback functions in B acquire the GIL?
Yes.
>> Problem: using B while A is loaded breaks A: because B has installed
>> callbacks in Q that acquire the GIL, A must also release the GIL
>> before calling into Q.
>
> Why? The callbacks in B will simply hang until they acquire the GIL.
If A doesn't release the GIL, one of its direct calls into Q from
Python may invoke a callback in B, which tries to acquire the lock
when it is already held. This is a no-no.
I realize that the docs for PyEval_AcquireLock() say: "If this thread
already has the lock, a deadlock ensues", but the behavior we're
seeing is consistent with a scenario where trying to acquire an
already-held is a no-op and releasing it is unconditional. Eventually
the GIL release in B's callback takes effect and when A returns to
Python there is no thread state.
> I think this thread might be better handled on c.l.py, at least
> until it's understood well enough to be clear whether something does
> need to change in Python.
I'm pretty certain I understand what's going on. If it would be
better for you to take this to c.l.py, I'm happy to do so, but AFAICT
there _is_ a Python core issue here: there's no way to find out
whether you've already got the GIL**, so if you _might_ have to acquire
it, you must always acquire it.
**If I'm wrong about that, please let me know. It isn't obvious from
the documentation.
--
David Abrahams
dave@boost-consulting.com * http://www.boost-consulting.com
Boost support, enhancements, training, and commercial distribution
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4