[Guido van Rossum] > Christian Tismer just did an exhaustive search for thread unsafe use > of Python operations, and found two weaknesses. One is > posix.listdir(), which I had already found; the other is > file.writelines(). Here's a program that demonstrates the bug; > basically, while writelines is walking down the list, another thread > could truncate the list, causing PyList_GetItem() to fail or a string > object to be deallocated while writelines is using it. On my SOlaris > 7 system it typically crashes in the first or second iteration. > > It's easy to fix: just don't use release the interpreter lock (get rid > of Py_BEGIN_ALLOW_THREADS c.s.). This would however prevent other > threads from doing any work while this thread may be blocked for I/O. > > An alternative solution is to put Py_BEGIN_ALLOW_THREADS and > Py_END_ALLOW_THREADS just around the fwrite() call. This is safe, but > would require a lot of lock operations and would probably slow things > down too much. > > Ideas? 2.5: 1: Before releasing the lock, make a shallow copy of the list. 1.5: As in #1, but iteratively peeling off "the next N" values, for some N balancing the number of lock operations against the memory burden (I don't care about the speed of a shallow copy here ...). 2. Pull the same trick list.sort() uses: make the list object immutable for the duration (I know you think that's a hack, and it is <wink>, but it costs virtually nothing and would raise an approriate error when they attempted the insane mutation). I actually like #2 best now, but won't in the future, because file_writelines() should really accept an argument of any sequence type. This makes 1.5 a better long-term hack. although-adding-1.5-to-1.6-is-confusing<wink>-ly y'rs - tim
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4