[Guido, on reworking select()]
> Yes, I think we do need this. PyObject_AsFileDescriptor() calls the
> fileno() method which could be a Python call.
Ya, but for me to believe that *matters*, I have to believe people worried
about high-performance select are passing objects with Python-implemented
fileno methods to begin with. That's quite a stretch, isn't it? They have
to pay for fileno() calls regardless, the question is only whether they pay
it once or twice per object.
> Maybe we could use a more sophisticated construct where we use a short
> array allocated on the stack for lists of fewer than 100 (say) files
> but use malloc for larger ones?
Another idea: screw it <wink>. We have to allocate three lists anyway to
hold the results. Allocate them at the start instead of the end, and then
they can do double-duty, holding the cached fileno() results across the
select() call.
Fleshing it out a little,
def list2set(inputlist, fdset):
"Return (list of file descriptors, biggest fd seen); set fdset slots."
n = len(inputlist)
result = [None] * n
if _MSC_VER:
biggest = 0
else:
biggest = -1
for i in range(n):
obj = inputlist[i]
fd = PyObject_AsFileDescriptor(obj)
if fd == -1:
return NULL, -1
result[i] = obj, fd
FD_SET(fd, fdset)
if not _MSC_VER:
biggest = max(biggest, fd)
return result, biggest
# iolist is the result of an earlier list2set() call. It gets shrunk
# to hold just the objects corresponding to the fds in fdset.
def set2list(iolist, fdset):
lastindex = 0
for obj, fd in iolist:
if FD_ISSET(fd, fdset):
iolist[lastindex] = obj
lastindex += 1
if lastindex < len(iolist):
del iolist[lastindex:]
So it still does the perversely <wink> optimized caching, but no stack space
required, no free() calls, only the 3 malloc() calls that are needed anyway,
and up to 3 realloc() calls to shrink the output lists. It does create a
bunch of 2-tuples, but they're on a free list and come cheap.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4