Last month there was a flurry of discussion, around
http://www.python.org/pipermail/python-dev/2000-July/014208.html
about problems arising when combining threading and forking. I've
been reading through the python-dev archives and as far as I can tell
this problem has not yet been resolved.
Well, I think I understand what's going on and I have a patch that
fixes the problem.
Contrary to some folklore, you *can* use fork() in threaded code; you
just have to be a bit careful about locks...
Rather than write up a long-winded explanation myself, allow me to
quote:
-----------------------------------------------------------------
from "man pthread_atfork":
... recall that fork(2) duplicates the whole memory space,
including mutexes in their current locking state, but only the
calling thread: other threads are not running in the child
process. Thus, if a mutex is locked by a thread other than
the thread calling fork, that mutex will remain locked
forever in the child process, possibly blocking the execu-
tion of the child process.
and from http://www.lambdacs.com/newsgroup/FAQ.html#Q120
Q120: Calling fork() from a thread
> Can I fork from within a thread ?
Absolutely.
> If that is not explicitly forbidden, then what happens to the
> other threads in the child process ?
There ARE no other threads in the child process. Just the one that
forked. If your application/library has background threads that need
to exist in a forked child, then you should set up an "atfork" child
handler (by calling pthread_atfork) to recreate them. And if you use
mutexes, and want your application/library to be "fork safe" at all,
you also need to supply an atfork handler set to pre-lock all your
mutexes in the parent, then release them in the parent and child
handlers. Otherwise, ANOTHER thread might have a mutex locked when
one thread forks -- and because the owning thread doesn't exist in
the child, the mutex could never be released. (And, worse, whatever
data is protected by the mutex is in an unknown and inconsistent
state.)
-------------------------------------------------------------------
Below is a patch (I will also post this to SourceForge)
Notes on the patch:
1) I didn't make use of pthread_atfork, because I don't know how
portable it is. So, if somebody uses "fork" in a C extension there
will still be trouble.
2) I'm deliberately not cleaning up the old lock before creating
the new one, because the lock destructors also do error-checking.
It might be better to add a PyThread_reset_lock function to all the
thread_*.h files, but I'm hesitant to do this because of the amount
of testing required.
Patch:
Index: Modules/signalmodule.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Modules/signalmodule.c,v
retrieving revision 2.53
diff -c -r2.53 signalmodule.c
*** Modules/signalmodule.c 2000/08/03 02:34:44 2.53
--- Modules/signalmodule.c 2000/08/19 03:37:52
***************
*** 667,672 ****
--- 667,673 ----
PyOS_AfterFork(void)
{
#ifdef WITH_THREAD
+ PyEval_ReInitThreads();
main_thread = PyThread_get_thread_ident();
main_pid = getpid();
#endif
Index: Parser/intrcheck.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Parser/intrcheck.c,v
retrieving revision 2.39
diff -c -r2.39 intrcheck.c
*** Parser/intrcheck.c 2000/07/31 15:28:04 2.39
--- Parser/intrcheck.c 2000/08/19 03:37:54
***************
*** 206,209 ****
--- 206,212 ----
void
PyOS_AfterFork(void)
{
+ #ifdef WITH_THREAD
+ PyEval_ReInitThreads();
+ #endif
}
Index: Python/ceval.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Python/ceval.c,v
retrieving revision 2.191
diff -c -r2.191 ceval.c
*** Python/ceval.c 2000/08/18 19:53:25 2.191
--- Python/ceval.c 2000/08/19 03:38:06
***************
*** 142,147 ****
--- 142,165 ----
Py_FatalError("PyEval_ReleaseThread: wrong thread state");
PyThread_release_lock(interpreter_lock);
}
+
+ /* This function is called from PyOS_AfterFork to ensure that newly
+ created child processes don't hold locks referring to threads which
+ are not running in the child process. (This could also be done using
+ pthread_atfork mechanism, at least for the pthreads implementation) */
+ void
+ PyEval_ReInitThreads(void)
+ {
+ if (!interpreter_lock)
+ return;
+ /*XXX Can't use PyThread_free_lock here because it does too
+ much error-checking. Doing this cleanly would require
+ adding a new function to each thread_*.h. Instead, just
+ create a new lock and waste a little bit of memory */
+ interpreter_lock = PyThread_allocate_lock();
+ PyThread_acquire_lock(interpreter_lock, 1);
+ main_thread = PyThread_get_thread_ident();
+ }
#endif
/* Functions save_thread and restore_thread are always defined so
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4