"Tim Peters" <tim.one at comcast.net> writes: > [Michael Hudson] >> ... >> And then I find out running test_descr in a loop leaks 166(!) >> references *anyway* (run the attached blah.py in a debug build, and >> some print test or other that I don't understand fails when run like >> this, so I fiddled it) >> >> Should I have been expecting this, or is it time for some ref-leak >> hunting? Actually, the process seems to grow -- slowly -- when you >> loop, so it's probably a real problem. I gather it's best to ask >> about this sort of thing when Tim is off sick from work :-) > > Only if you want help <wink>. Actually, I found the worst culprit this morning. Some dunderhead had got the decrefs wrong on error return from type_set_bases... I'll check my fix in momentarily. > Since Zope has worse leak problems than > Python, Zope testing has more tools to dig into it: > > http://cvs.zope.org/Zope3/test.py > > That's the Zope3 unittest test driver. The TrackRefs class is a handy > little beast, which uses a debug build's sys.getobjects() to keep track of > how many objects of each *type* exist across test runs, and how many > refcounts total they account for. After the first few iterations, TrackRefs > doesn't itself distort the results it prints. It will tell you the types of > the objects that are leaking, and also whether objects are actually leaking, > or that merely refcounts to existing objects are leaking (e.g., a common > kind of "leak" is to forget to decref Py_None, but memory doesn't actually > grow then). Cool! I'd blundered my way to something like an ad hoc version of this, next time I'll use something that someone's actually thought about :-) > Do use 2.3 if you try this. Well, yeah. Turned out the worst leaks weren't in 2.2 code anyway. > The "list of all objects" before 2.3 turned out to be missing many > objects that tend to leak (None, booleans, any number of type > objects, essentially all statically allocated objects), and far > fewer are missing in 2.3. > > It's typically easy to find out which types of objects are leaking this way. > Then it's typical to add more code to TraceRefs.update() to get more > details. In Zope use, it's surprising how often that still left us > scratching our heads! But Zope has lots of extension types coded in C, > massive interconnections among them, and none of them originally > participated in cyclic gc. Python leaks are usually easier to plug (for > example, in Python, they're almost always in new code). Bingo! I did all these tests incorporating my safety fixes to PyObject_GenericGetAttr so I think they're probably solid, too. Cheers, mwh -- The PROPER way to handle HTML postings is to cancel the article, then hire a hitman to kill the poster, his wife and kids, and fuck his dog and smash his computer into little bits. Anything more is just extremism. -- Paul Tomblin, asr
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4