On Tue, Oct 28, 2003 at 11:12:21AM +0100, Alex Martelli wrote: > On Monday 27 October 2003 10:56 pm, Gregory P. Smith wrote: > > What about the behaviour of multiple iterators for the same dict being > > used at once (either interleaved or by multiple threads; it shouldn't > > matter)? I expect that works fine in python. > > If the dict is not being modified, or if the only modifications on it are > assigning different values for already-existing keys, multiple iterators > on the same unchanging dict do work fine in one or more threads. > But note that iterators only "read" the dict, don't change it. If any > change to the set of keys in the dict happens, all bets are off. ... > > This is something the _DBWithCursor iteration interface does not currently > > support due to its use of a single DBCursor internally. > > > > _DBWithCursor is currently written such that the cursor is never closed > > once created. This leaves tons of potential for deadlock even in single > > threaded apps. Reworking _DBWithCursor into a _DBThatUsesCursorsSafely > > such that each iterator creates its own cursor in an internal pool > > and other non cursor methods that would write to the db destroy all > > cursors after saving their current() position so that the iterators can > > reopen+reposition them is a solution. > > Woof. I think I understand what you're saying. However, writing to a > dict (in the sense of changing the sets of keys) while one is iterating > on the dict is NOT supported in Python -- basically "undefined behavior" > (which does NOT include possibilities of crashes and deadlocks, though). > So, maybe, we could get away with something a bit less rich here? I just implemented and committed something about that rich. I believe I could simplify it: have __iter__() and iteritems() return if their cursor was closed out from underneath them instead of the current attempt to reopen a cursor, reposition themselves, and keep going [which could still have unpredictable results since a db modification could rearrange the keys in some types of databases]. > So, maybe I _should_ just fix popitem that way and see if all tests pass? > I dunno -- it feels a bit like fixing the symptoms and leaving some deep > underlying problems intact... My commit fixed the deadlock problem for the single threaded case and wrote a test case to prove it. I opened a SF bug to track fixing the deadlock possibilities in the multithreaded case (and a memory leak i believe i added). -g
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4