On Feb 26, 2004, at 10:03 PM, Raymond Hettinger wrote:
>>> I had been led astray because I was experimenting with using
>>> cStringIO.writelines() as a basis for implementing str.join() for
>>> general iterables without creating an intermediate tuple. Right
> now,
>>> ''.join(it) will unexpectedly consume much more memory than really
>>> needed.
>
> [Jeremy Fincher]
>> This sounds like a cool idea, are you still going to implement it?
>
>
> One way or another, I'll make str.join() smarter and faster than it is
> now. Which way proves to be better is still open.
In trying to time this in pure python, I discovered that
cStringIO.writelines doesn't take generators.. apparently only objects
that support len(...) are allowed
Currently, it seems that ''.join is faster than cStringIO.writelines
anyway, even if you just pass it a big list.
from cStringIO import StringIO
def join2(sep, seq):
sio = StringIO()
if not sep:
sio.writelines(seq)
else:
# does not work
def join2():
yield seq.next()
for s in seq:
yield sep
yield s
sio.writelines(join2())
return sio.getvalue()
import time
from itertools import repeat
import operator
def timeit(fn, *args, **kwargs):
lst = []
for ig in xrange(100):
t0 = time.time()
fn(*args, **kwargs)
lst.append(time.time() - t0)
print fn.__name__
print '', 'max:', max(lst)
print '', 'min:', min(lst)
print '', 'avg:', reduce(operator.add, lst)/len(lst)
if __name__ == '__main__':
strings = [' ' * 1000] * 1000
print 'no separator'
print '------------'
timeit(join2, '', strings)
timeit(''.join, strings)
[crack:~] bob% python strio.py
no separator
------------
join2
max: 0.0453701019287
min: 0.0157110691071
avg: 0.0181557154655
join
max: 0.0425598621368
min: 0.00337600708008
avg: 0.00432039260864
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4