On Thu, Mar 29, 2012 at 17:26, Victor Stinner <victor.stinner at gmail.com> wrote: > The problem is not the frame, but the Python thread state referenced by the > frame. It's a private attribute. My patch just updates this reference before > running the generator (and it clears the reference when the generator > excution is stopped or finished). Right--my thought was the the situation we saw might be similarly triggered because we were storing an exception with traceback (and associated frames) generated by thread A, and then re-throwing it from thread B some time after thread A has exited. The frame attached to the exception's traceback would still, then, be referencing a nonexistent thread state, correct? My concern was that this might one instance of a broader problem for folks who embed Python and see the attractive PyGILState_Ensure() API. It already prevents us from using subinterpreters (which for us might have been a better solution than repeated initialize/finalize, with its known issues). We recently made a change to dispose of the traceback before storing the exception, and that appears to eliminate the corruption we were seeing, so that's making me more suspicious. > You may leak memory if your threads have a short lifetime and you create > many threads. For example if one thread is only used to process one request > and then is destroyed. Absolutely--this particular hack was only used for a thread created outside Python that had to call into the VM. Their behavior is well-defined in our case--two particular OS threads, with lifetimes longer than those of the interpreters we create and finalize. -- Tim Lesher <tlesher at gmail.com>
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4