This is a re-post in plain text of a message I sent yesterday in HTML. Anyone not "consumed with interest" in the buffer object should probably skip it. Scott Gilbert wrote: >--- Todd Miller <jmiller@stsci.edu> wrote: > >>>I don't understand what you say, but I believe you. >>> >>I meant we call PyBuffer_FromReadWriteObject and the resulting buffer >>lives longer than the extension function call that created it. I have >>heard that it is possible for the original object to "move" leaving the >>buffer object pointer to it dangling. >> > >Yes. The PyBufferObject grabs the pointer from the PyBufferProcs >supporting object when the PyBufferObject is created. If the PyBufferProcs >supporting object reallocates the memory (possibly from a resize) the > Thanks for the example. > >PyBufferObject can be left with a bad pointer. This is easily possible if >you try to use the array module arrays as a buffer. > This is good to know. > > >I've submitted a patch to fix this particular problem (among others), but >there are still enough things that the buffer object can't do that >something new is needed. > I understand. I saw your patches and they sounded good to me. > >>> >>>>>Maybe instead of the buffer() function/type, there should be a way to >>>>>allocate raw memory? >>>>> >>>>Yes. It would also be nice to be able to: >>>> >>>>1. Know (at the python level) that a type supports the buffer C-API. >>>> >>>Good idea. (I guess right now you can see if calling buffer() with an >>>instance as argument works. :-) >>> >>>>2. Copy bytes from one buffer to another (writeable buffer). >>>> > >And the copy operations shouldn't create any large temporaries: > I agree with this completely. I could summarize my opinion by saying that while I regard the current buffering system as pretty complete, the buffer object places emphasis on the wrong behavior. In terms of modelling memory regions, strings are the wrong way to go. > > > buf1 = memory(50000) > buf2 = memory(50000) > # no 10K temporary should be created in the next line > buf1[10000:20000] = buf2[30000:40000] > >The current buffer object could be used like this, but it would create a >temporary string. > Looking at buffering most of this week, the fact that mmap slicing also returns strings is one justification I've found for having a buffer object, i.e., mmap slicing is not a substitute for the buffer object. The buffer object makes it possible to partition a mmap or any bufferable object into pseudo-independent, possibly writable, pieces. One justification to have a new buffer object is pickling (one of Scott's posts alerted me to this). I think the behavior we want for numarray is to be able to pickle a view of a bufferable object more or less like a string containing the buffer image, and to unpickle it as a memory object. The prospect of adding pickling support makes me wonder if seperating the allocator and view aspects of the buffer object is a good idea; I thought it was, but now I wonder. > >So getting an efficient copy operation seems to require that slices just >create new "views" to the same memory. > Other justifications for a new buffer object might be: 1. The ability to partition any bufferable object into regions which can be passed around. These regions would themselves be buffers. 2. The ability to efficiently pickle a view of any bufferable object. > >>>Maybe you would like to work on a requirements gathering for a memory >>>object >>> >>Sure. I'd be willing to poll comp.lang.python (python-list?) and >>collate the results of any discussion that ensues. Is that what you had >>in mind? >> > > >In the PEP that I'm drafting, I've been calling the new object "bytes" >(since it is just a simple array of bytes). Now that you guys are >referring to it as the "memory object", should I change the name? Doesn't >really matter, but it might avoid confusion to know we're all talking about >the same thing. > Calling this a memory type sounds the best to me. The question I have not resolved for myself is whether there should be one type which "does it all" or two types, a memory allocator and a bufferable object manipulator. > > > >__________________________________________________ >Do You Yahoo!? >Yahoo! Autos - Get free new car price quotes >http://autos.yahoo.com >
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4