----- Original Message ----- From: "Tony Rentschler" <tonyr at walterr.bellatlantic.net> Subject: Re: string concatentation > Thanks, the list idiom is just what I was looking for. I'm new > to Python and am still learning how best to accomplish certain tasks. In > this case, I can't read() the whole file in one fell swoop because the > upstream program sends it line by line. If the list method is more efficient, > does that then mean that Python lists allocate more space than required as > items are added so as to grow efficiently? I just reviewed the source for 2.0, and it appears that a list is represented as a dynamic-allocated C array of generic Python objects. This means that creating a string object (via the .readline() method) counts as one malloc(), and appending it to the list requires a realloc() (actually PyMem_RESIZE()). The other way requires a malloc() for the object creation, then a copy of the string data (twice, once per each source string). Note that in this case, the number of bytes of data malloc()ed per pass gets progressively larger, but in the other case the total number malloc()ed per pass remains *relatively* constant. I don't know but I think the real reason for the advantage of the list paradigm is all those excessive string copies. Also, there will be a free() call approximately every pass (more often I think as the string gets larger). Also the heap will get messier and messier because each new malloc() gets a larger block of memory, which probably won't match previously free()ed blocks. Anyway, the list method has always worked better for me. Anyone with more of a clue than I about this want to comment?
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4