On 29/03/2012 21:35, Tim Lesher wrote:
> From a theoretical standpoint, I can't quite decide what the real error is:
>
> 1) the fact that PyGILState_Release() destroys a temporary thread
> state that may still be referenced by some objects, or
> 2) the fact that some code is trying to keep frame objects after the
> creating thread state no longer exists.
>
> This week I've been leaning toward 2), but then I realized that
> keeping frames post-thread-death is not that uncommon (for example,
> debuggers and other diagnostic techniques like
> http://nedbatchelder.com/blog/200711/rethrowing_exceptions_in_python.html).
The problem is not the frame, but the Python thread state referenced by
the frame. It's a private attribute. My patch just updates this
reference before running the generator (and it clears the reference when
the generator excution is stopped or finished).
> Locally we added some (unfortunate) code to our 3.1.2 port to wrap
> PyGILState_Ensure(), which I thought had sidestepped the issue for us:
>
> void takeGIL()
> {
> PyGILState_Ensure();
> // This has the side effect of keeping such thread states alive until
> // the interpreter is finalized; however, all thread state objects get
> // unconditionally deleted during Py_Finalize, so they won't leak.
> PyThreadState* pThreadState = PyGILState_GetThisThreadState();
> if (pThreadState->gilstate_counter == 1)
> {
> ++pThreadState->gilstate_counter;
> }
> }
>
> But clearly that can't be a correct answer (and it may not even be a
> functioning one, given that I'm seeing a similar issue again).
You may leak memory if your threads have a short lifetime and you create
many threads. For example if one thread is only used to process one
request and then is destroyed.
Victor
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4