On Mon, Oct 11, 2004 at 09:21:57PM -0400, Tim Peters wrote: > [Jason Evans] > > I think you're in for a rude awakening if you expect any language's gc > support to help you manage arbitrary blobs of memory obtained from the > system malloc(). I don't expect that. What I do expect is for there to be clean and efficient mechanisms for integrating custom Python object types into Python. Python more or less meets this expectation. > If all the nodes in your object graph derived from PyObject, Python's gc > would do it all for you. If you want to plant pointers in and among > arbitrary C structs, then you'll have to implement your own gc, or hope > that you can bash a general gc thingie (like BDW -- and I'm not sure > there is anything else of that sort usefully available, so "like" is > probably optimistic) into guessing the layout of your structs. No, this is not necessary. What is necessary is for my code to accurately maintain and report all references to Python objects during traversal. The Crux code does so, and works as designed. > > Crux was designed this way in order to be able to implement various low level > > algorithms such that they never have to call interpreter-related code. > > I don't know why that was thought to be desirable, or even exactly > what it means. One plausible meaning is that you don't want to tie > the "various low level algorithms" to Python, because you hope to > reuse them in other, non-Python contexts. And, in addition to that, > you don't want, or don't know how, to abstract the necessary > operations into macros or functions that are pluggable for the > different contexts you have in mind. First, "hoping" to be able use my code in other contexts is what allowed me to transition from using a different embeddable interpreter to using Python inside of two days. Had I melded all of the scripting glue code with the low level tree code, it would not have been nearly as easy to transition to using Python. Second, I have the scripting language glue and low level tree code separated for two reasons: 1) performance, and 2) modularization. There's no question of whether I want to use cpp magic to be able to munge two large piles of code into one huge pile of code, nor whether I'm capable of doing so. To do so would simply take the code in a bad direction. > [Martin] > >> Hmmm. Even if a ring member holds only a single reference to another > >> ring member, would not calling visit() on this member in turn invoke > >> visit() on the other one, which would, in turn, invoke visit() on > >> the next one, and so on? > > [Jason] > > Yes, this is true. > > It's false, but I covered that one yesterday. It is true, given my interpretation of the original context of the dialogue. We were discussing whether it would work for a node object to hold a reference to a single ring object, and rely on the ring objects to report their neighbors. Indeed this would work. > > The problem I ran into had to do with the node logically holding references to all > > the ring objects, but only reporting one of them. > > If a ring object doesn't derive from PyObject, then visit() must not > be passed a pointer to a ring object. Doing so would account for many > catastrophic meanings of "doesn't work" <wink>. If you want to see the Crux source code, I'll be happy to provide you with a source archive. In the meanwhile, please rest assured that the code doesn't suck nearly as badly as you are insinuating here. > We can help you use Python's gc, but not if you're unwilling or unable > to change anything about what you do. Prior to switching to using Python, Crux integrated with a language that implemented atomic mark and sweep garbage collection. I have had to change a tremendous amount about Crux in order to use a reference counted interpreter. In fact, I made some changes to Crux this morning that were a direct result of your original response to my email (ring objects now report references to themselves, which makes reference counting of ring objects much simpler). Many of the issues I ran into had to do with the impedence mismatch between my previous experience and the way Python works. I appreciate you taking the time to respond to me, especially since you've helped me gain a deeper understanding of how Python's GC works. However, please get it out of your mind that I am incompetent and/or set in my ways. > > I should have said tp_clear here. In Modules/gcmodule.c:delete_garbage(), > > the clear() call does not pay attention to the return value. > > Heh. So true! See > > http://mail.python.org/pipermail/python-dev/2003-April/034435.html > > > It would be possible to check the return value, and defer deallocation if the > > return value were non-zero, and ob_refcnt were 1. (However, it would probably > > have to be a separate list of objects, contrary to what I claimed before.) > > I'm not sure, but I think your desire for this was still based on > misunderstanding of what Python's gc does. Of the original issues I brought up, I think this one potentially still has merit, though I'm not sure that it's of sufficiently general utility. The idea is to allow extension modules some control over the order in which objects are deallocated, so that it is possible to tear down interconnections in a particular order. However, it occurs to me that since tp_clear can leave references in place, it might be possible to control the object destruction order by leaving a DAG of references in place, and letting Python deallocate objects in dependency order. Does this work? Thanks, Jason
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4