[Aahz] > My take is that programs with a million live objects and no cycles are > common enough that gc should be designed to handle that smoothly. Well, millions of live objects is common but isn't a problem. The glitch we're looking at it is surprising slowdown with millions of live *container* objects. The latter isn't so common. > I don't think that a programmer casually writing such applications > (say, processing information from a database) should be expected to > understand gc well enough to tune it. People casually writing applications pushing the limits of their boxes are in for more surprises than just this <wink>. > Having read the entire discussion so far, and *NOT* being any kind of > gc expert, I would say that Tim's adaptive solution makes the most > sense to me. For years, we told people with cyclic data to figure out > how to fix the problem themselves; now that we have gc available, I > don't think we should punish everyone else. We're not trying to punish anyone, but innocent users with lots of containers can lose big despite our wishes: if we don't check them for cycles, they can run out of memory; if we do check them for cycles, it necessarily consumes time. As a datapoint, here are the times (in seconds) for justzip() on my box after my checkin to precompute the result size (list.append behavior is irrelevant now): gc disabled: 0.64 gc enabled: 7.32 magic=2(*): 2.63 magic=3(*): 2.02 (*) This is gcmodule.c fiddled to add this block after "collections1 = 0;" in the first branch of collect_generations(): if (n == 0) threshold2 *= magic; else if (threshold2 > 5) threshold2 /= magic; magic=1 is equivalent to the current code. That's all an "adaptive scheme" need amount to, provided the "*=" part were fiddled to prevent threshold2 from becoming insanely large. Boosting magic above 3 didn't do any more good in this test. At magic=3 it still takes 3+ times longer than with gc disabled, but that's a whale of a lot better than the current 11+ times longer. Note that with gc disabled, any cycle in any of the 1,000,001 containers this test creates would leak forever -- casual users definitely get something back for the time spent.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4