Guido & I talked out A Plan, and he's going to implement it while I take a nap. Outline: 1. It sucks that PyDict_Next() can resize a dict. And while staring at all this in the debugger, it was plain appalling that the mere act of doing gc ran around turning empty dicts into non-empty ones because of it (not a bug, just irksome waste). 2. It sucks that PyDict_SetItem() can resize a dict even when the number of active slots doesn't change. The plan for addressing those: A. Rip out the PyDict_Next() resizing hack. B. In PyDict_SetItem(), first look at the number of free slots, and resize the dict if it would be impossible to add a new active slot (I *suspect* this can be reduced to making a minimal dict when the incoming dict is empty). Remember the number of used slots. Do the insert. Look at the number of used slots now: do "the usual" resize logic if and only the number of used slots changed across the insert call. That much should suffice to stop Jeremy's old bugs, and the bug I bumped into here. It's not enough, though. Allocating a tuple (or any other gc'ed type) can still cause gc to run, then gc can delete __del__-free cycles, then deleting those can cause objects with __del__ methods to become unreachable too, and then any Python code whatsoever can run, incl. but not limited to code that dicts, and also incl. allowing other threads to run. So code inside dict methods can't assume much of anything across container allocations, even after fixing all the bugs we've bumped into so far. So at least dict_items() and dict_popitem() remain unsafe after these changes, although we don't have a concrete test case to prove that and it would be mondo difficult to create one. Nevertheless, Python users are effective proof of the million monkeys hypothesis <wink>. These remaining problems require case-by-case analysis and rewriting. could-be-the-biggest-one-line-fix-in-history-ly y'rs - tim
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4