> [Tim] > >> Well, it would be easy to fiddle the HAVE_GETC_UNLOCKED method > >> to keep the file locked until the line was complete, and I > >> wouldn't be opposed to making life saner on platforms that allow it. > > [Guido] > > Hm... That would be possible, except for one unfortunate detail: > > _PyString_Resize() may call PyErr_BadInternalCall() which touches > > thread state. [Tim] > FLOCKFILE/FUNLOCKFILE are independent of Python's notion of thread state. > IOW, do FLOCKFILE once before the for(;;), and FUNLOCKFILE once on every > *exit* path thereafter. We can block/unblock Python threads as often as > desired between those *file*-locking brackets. The only thing the repeated > FLOCKFILE/FUNLOCKFILE calls do to my eyes now is to create the *possibility* > for multiple readers to get partial lines of the file. I don't want to call FLOCKFILE while holding the Python lock, as this means that *if* we're blocked in FLOCKFILE (e.g. we're reading from a pipe or socket), no other Python thread can run! > > ... > > NO, NO NO! Mixing reads and writes on the same stream wasn't what we > > are locking against at all. (As you've found out, it doesn't even > > work.) > > On Windows, yes, but that still seems to me to be a bug in MS's code. If > anyone had reported a core dump on any other platform, I'd be more tractable > <wink> on this point. Yes, it's a Windows bug. > > We're only trying to protect against concurrent *reads*. > > As above, I believe that we could do a better job of that, then, on > platforms that HAVE_GETC_UNLOCKED, by protecting not only against core dumps > but also against .readline() not delivering an intact line from the file. See above for a reason why I think that's not safe. I think that applications that want to do this can do their own locking. (They'll find out soon enough that readline() isn't atomic. :-) > >> But since FLOCKFILE is in effect, other threads *trying* to write > >> to the stream we're reading will get blocked anyway. Seems to give us > >> potential for deadlocks. > > > Only if tyeh are holding other locks at the same time. > > I'm not being clear, then. Thread X does f.readline(), on a > HAVE_GETC_UNLOCKED platform. get_line allows other threads to run and > invokes FLOCKFILE on f->f_fp. get_line's GETC in thread X eventually hits > the end of the stdio buffer, and does its platform's version of _filbuf. > _filbuf may wait (depending on the nature of the stream) for more input to > show up. Simultaneously, thread Y attempts to write some data to f. But > the *FLOCKFILE* lock prevents it from doing anything with f. So X is > waiting for Y to write data inside platform _filbuf, but Y is waiting for X > to release the platform stream lock inside some platform stream-output > routine (if I'm being clear now, Python locks have nothing to do with this > scenario: it's the platform stream lock). I don't think that _filbuf can possibly wait for another thread to write data to the same stream object. A single stream object doesn't act like a pipe, even if it is open for simultaneous reading and writing. So if there's no more data in the file, _fulbuf will simply return with an EOF status, not wait for the data that the other thread would write. > I think this is purely the user's fault if it happens. Just pointing it out > as another insecurity we're probably not able to protect users from. I don't think this can happen. > > ... > > Yeah. But this is insane use -- see my comments on SF. It's only > > worth fixing because it could be used to intentionally crash Python -- > > but there are easier ways... > > If it's unique to MS (as I suspect), I see no reason to even consider trying > to fix it in Python. Unless the Perl Mongers use it to crash Zope <wink>. OK. It's unique to MS. So close the bug report with a "won't fix" resolution. There's no point in having bug reports remain open that we know we can't fix. --Guido van Rossum (home page: http://www.python.org/~guido/)
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4