Showing content from https://mail.python.org/pipermail/python-dev/2000-August.txt below:
From greg@cosc.canterbury.ac.nz Tue Aug 1 00:45:02 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 01 Aug 2000 11:45:02 +1200 (NZST) Subject: [Python-Dev] Negative slice steps considered unhealthy (extended slicing for lists) In-Reply-To: Message-ID: <200007312345.LAA10291@s454.cosc.canterbury.ac.nz> I think there are some big conceptual problems with allowing negative steps in a slice. With ordinary slices, everything is very clear if you think of the indices as labelling the points between the list elements. With a step, this doesn't work any more, and you have to think in terms of including the lower index but excluding the upper index. But what do "upper" and "lower" mean when the step is negative? There are several things that a[i:j:-1] could plausibly mean: [a[i], a[i-1], ..., a[j+1]] [a[i-1], a[i-2], ..., a[j]] [a[j], a[j-1], ..., a[i+1]] [a[j-1], a[j-2], ..., a[i]] And when you consider negative starting and stopping values, it just gets worse. These have no special meaning to range(), but in list indexing they do. So what do they mean in a slice with a step? Whatever is chosen, it can't be consistent with both. In the face of such confusion, the only Pythonic thing would seem to be to disallow these things. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Tue Aug 1 01:01:45 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 01 Aug 2000 12:01:45 +1200 (NZST) Subject: [Python-Dev] PEP 203 Augmented Assignment In-Reply-To: <200007281147.GAA04007@cj20424-a.reston1.va.home.com> Message-ID: <200008010001.MAA10295@s454.cosc.canterbury.ac.nz> > The way I understand this, mixing indices and slices is used all > the time to reduce the dimensionality of an array. I wasn't really suggesting that they should be disallowed. I was trying to point out that their existence makes it hard to draw a clear distinction between indexing and slicing. If it were the case that a[i,j,...,k] was always equivalent to a[i][j]...[k] then there would be no problem -- you could consider each subscript individually as either an index or a slice. But that's not the way it is. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Tue Aug 1 01:07:08 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 01 Aug 2000 12:07:08 +1200 (NZST) Subject: [Python-Dev] Should repr() of string should observe locale? In-Reply-To: Message-ID: <200008010007.MAA10298@s454.cosc.canterbury.ac.nz> Tim Peters: > The problem isn't that repr sticks in backslash escapes, the problem is that > repr gets called when repr is inappropriate. Seems like we need another function that does something in between str() and repr(). It would be just like repr() except that it wouldn't put escape sequences in strings unless absolutely necessary, and it would apply this recursively to sub-objects. Not sure what to call it -- goofy() perhaps :-) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From bwarsaw@beopen.com Tue Aug 1 01:25:43 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Mon, 31 Jul 2000 20:25:43 -0400 (EDT) Subject: [Python-Dev] Should repr() of string should observe locale? References: <200008010007.MAA10298@s454.cosc.canterbury.ac.nz> Message-ID: <14726.6407.729299.113509@anthem.concentric.net> >>>>> "GE" == Greg Ewing writes: GE> Seems like we need another function that does something in GE> between str() and repr(). I'd bet most people don't even understand why there has to be two functions that do almost the same thing. -Barry From guido@beopen.com Tue Aug 1 04:32:18 2000 From: guido@beopen.com (Guido van Rossum) Date: Mon, 31 Jul 2000 22:32:18 -0500 Subject: [Python-Dev] test_re fails with re==pre In-Reply-To: Your message of "Mon, 31 Jul 2000 23:59:34 +0200." <20000731215940.28A11E266F@oratrix.oratrix.nl> References: <20000731215940.28A11E266F@oratrix.oratrix.nl> Message-ID: <200008010332.WAA25069@cj20424-a.reston1.va.home.com> > Test_re now works fine if re is sre, but it still fails if re is pre. > > Is this an artifact of the test harness or is there still some sort of > incompatibility lurking in there? It's because the tests are actually broken for sre: it prints a bunch of "=== Failed incorrectly ..." messages. We added these as "expected output" to the test/output/test_re file. The framework just notices there's a difference and blames pre. Effbot has promised a new SRE "real soon now" ... --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From guido@beopen.com Tue Aug 1 05:01:34 2000 From: guido@beopen.com (Guido van Rossum) Date: Mon, 31 Jul 2000 23:01:34 -0500 Subject: [Python-Dev] Negative slice steps considered unhealthy (extended slicing for lists) In-Reply-To: Your message of "Tue, 01 Aug 2000 11:45:02 +1200." <200007312345.LAA10291@s454.cosc.canterbury.ac.nz> References: <200007312345.LAA10291@s454.cosc.canterbury.ac.nz> Message-ID: <200008010401.XAA25180@cj20424-a.reston1.va.home.com> > I think there are some big conceptual problems with allowing > negative steps in a slice. > > With ordinary slices, everything is very clear if you think > of the indices as labelling the points between the list > elements. > > With a step, this doesn't work any more, and you have to > think in terms of including the lower index but excluding the > upper index. > > But what do "upper" and "lower" mean when the step is negative? > There are several things that a[i:j:-1] could plausibly mean: > > [a[i], a[i-1], ..., a[j+1]] > > [a[i-1], a[i-2], ..., a[j]] > > [a[j], a[j-1], ..., a[i+1]] > > [a[j-1], a[j-2], ..., a[i]] > > And when you consider negative starting and stopping values, > it just gets worse. These have no special meaning to range(), > but in list indexing they do. So what do they mean in a slice > with a step? Whatever is chosen, it can't be consistent with > both. > > In the face of such confusion, the only Pythonic thing would > seem to be to disallow these things. You have a point! I just realized today that my example L[9:-1:-1] does *not* access L[0:10] backwards, because of the way the first -1 is interpreted as one before the end of the list L. :( But I'm not sure we can forbid this behavior (in general) because the NumPy folks are already using this. Since these semantics are up to the object, and no built-in objects support extended slices (yet), I'm not sure that this behavior has been documented anywhere except in NumPy. However, for built-in lists I think it's okay to forbid a negative step until we've resolved this... This is something to consider for patch 100998 which currently implements (experimental) extended slices for lists... --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From ping@lfw.org Tue Aug 1 01:02:40 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Mon, 31 Jul 2000 17:02:40 -0700 (PDT) Subject: [Python-Dev] Reordering opcodes (PEP 203 Augmented Assignment) In-Reply-To: Message-ID: On Mon, 31 Jul 2000, Mark Hammond wrote: > IDLE and Pythonwin are able to debug arbitary programs once they have > started - and they are both written in Python. But only if you start them *in* IDLE or Pythonwin, right? > * You do not want to debug the IDE itself, just a tiny bit of code running > under the IDE. Making the IDE take the full hit simply because it wants to > run a debugger occasionally isnt fair. Well, running with trace hooks in place is no different from the way things run now. > The end result is that all IDEs will run with debugging enabled. Right -- that's what currently happens. I don't see anything wrong with that. > * Python often is embedded, for example, in a Web Server, or used for CGI. > It should be possible to debug these programs directly. But we don't even have a way to do this now. Attaching to an external running process is highly system-dependent trickery. If printing out tracebacks and other information isn't enough and you absolutely have to step the program under a debugger, the customary way of doing this now is to run a non-forking server under the debugger. In that case, you just start a non-forking server under IDLE which sets -g, and you're fine. Anyway, i suppose this is all rather moot now that Vladimir has a clever scheme for tracing even without SET_LINENO. Go Vladimir! Your last proposal sounded great. -- ?!ng From Fredrik Lundh" Message-ID: <001a01bffb80$87514860$f2a6b5d4@hagrid> greg wrote: > I think there are some big conceptual problems with allowing > negative steps in a slice. wasn't "slices" supposed to work the same way as "ranges"? from PEP-204: "Extended slices do show, however, that there is already a perfectly valid and applicable syntax to denote ranges in a way that solve all of the earlier stated disadvantages of the use of the range() function" > In the face of such confusion, the only Pythonic thing would > seem to be to disallow these things. ...and kill PEP-204 at the same time. From tim_one@email.msn.com Tue Aug 1 07:16:41 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 1 Aug 2000 02:16:41 -0400 Subject: [Python-Dev] Should repr() of string should observe locale? In-Reply-To: <14726.6407.729299.113509@anthem.concentric.net> Message-ID: [Barry A. Warsaw] > I'd bet most people don't even understand why there has to be two > functions that do almost the same thing. Indeed they do not. The docs are too vague about the intended differences between str and repr; in 1.5.2 and earlier, string was just about the only builtin type that actually had distinct str() and repr() implementations, so it was easy to believe that strings were somehow a special case with unique behavior; 1.6 extends that (just) to floats, where repr(float) now displays enough digits so that the output can be faithfully converted back to the float you started with. This is starting to bother people in the way that distinct __str__ and __repr__ functions have long frustrated me in my own classes: the default (repr) at the prompt leads to bloated output that's almost always not what I want to see. Picture repr() applied to a matrix object! If it meets the goal of producing a string sufficient to reproduce the object when eval'ed, it may spray megabytes of string at the prompt. Many classes implement __repr__ to do what __str__ was intended to do as a result, just to get bearable at-the-prompt behavior. So "learn by example" too often teaches the wrong lesson too. I'm not surprised that users are confused! Python is *unusual* in trying to cater to more than one form of to-string conversion across the board. It's a mondo cool idea that hasn't got the praise it deserves, but perhaps that's just because the current implementation doesn't really work outside the combo of the builtin types + plain-ASCII strings. Unescaping locale printables in repr() is the wrong solution to a small corner of the right problem. From Fredrik Lundh" Message-ID: <006401bffb81$89a7ed20$f2a6b5d4@hagrid> ping wrote: > > * Python often is embedded, for example, in a Web Server, or used = for CGI. > > It should be possible to debug these programs directly. >=20 > But we don't even have a way to do this now. Attaching to an > external running process is highly system-dependent trickery. not under Python: just add an import statement to the script, tell the server to reload it, and off you go... works on all platforms. From paul@prescod.net Tue Aug 1 07:34:53 2000 From: paul@prescod.net (Paul Prescod) Date: Tue, 01 Aug 2000 02:34:53 -0400 Subject: [Python-Dev] New winreg module really an improvement? References: Message-ID: <39866F8D.FCFA85CB@prescod.net> Mark Hammond wrote: > > > Interesting; I'd understood from Paul that you'd given approval to > > this module. > > Actually, it was more more along the lines of me promising to spend some > time "over the next few days", and not getting to it. However, I believe > it was less than a week before it was just checked in. It was checked in the day before the alpha was supposed to go out. I thought that was what you wanted! On what schedule would you have preferred us to do it? > I fear this may be a general symptom of the new flurry of activity; no-one > with a real job can keep up with this list, meaning valuable feedback on > many proposals is getting lost. For example, DigiCool have some obviously > smart people, but they are clearly too busy to offer feedback on anything > lately. That is a real shame, and a good resource we are missing out on. >From my point of view, it was the appearance of _winreg that prompted the "flurry of activity" that led to winreg. I would never have bothered with winreg if I were not responding to the upcoming "event" of the defacto standardization of _winreg. It was clearly designed (and I use the word loosely) by various people at Microsoft over several years -- with sundry backwards and forwards compatibility hacks embedded. I'm all for slow and steady, deliberate design. I'm sorry _winreg was rushed but I could only work with the time I had and the interest level of the people around. Nobody else wanted to discuss it. Nobody wanted to review the module. Hardly anyone here even knew what was in the OLD module. > I am quite interested to hear from people like Gordon and Bill > about their thoughts. I am too. I would *also* be interested in hearing from people who have not spent the last five years with the Microsoft API because _winreg was a very thin wrapper over it and so will be obvious to those who already know it. I have the feeling that an abstraction over the APIs would never be as "comfortable" as the Microsoft API you've been using for all of these years. -- Paul Prescod - Not encumbered by corporate consensus "I don't want you to describe to me -- not ever -- what you were doing to that poor boy to make him sound like that; but if you ever do it again, please cover his mouth with your hand," Grandmother said. -- John Irving, "A Prayer for Owen Meany" From paul@prescod.net Tue Aug 1 08:16:30 2000 From: paul@prescod.net (Paul Prescod) Date: Tue, 01 Aug 2000 03:16:30 -0400 Subject: [Python-Dev] New winreg module really an improvement? References: Message-ID: <3986794E.ADBB938C@prescod.net> (reorganizing the important stuff to the top) Mark Hammond wrote: > Still-can't-see-the-added-value-ly, I had no personal interest in an API for the windows registry but I could not, in good conscience, let the original one become the standard Python registry API. Here are some examples: (num_subkeys, num_values, last_modified ) = winreg.QueryInfoKey( key ) for i in range( num_values ): (name,value)=winreg.EnumValue( key, i ) if name==valuename: print "found" Why am I enumerating but not using the Python enumeration protocol? Why do I have to get a bogus 3-tuple before I begin enumerating? Where else are the words "Query" and "Enum" used in Python APIs? and winreg.SetValueEx( key, "ProgramFilesDir", None, winreg.REG_SZ, r"c:\programs" ) Note that the first argument is the key object (so why isn't this a method?) and the third argument is documented as bogus. In fact, in the OpenKey documentation you are requested to "always pass 0 please". All of that was appropriate when winreg was documented "by reference" to the Microsoft documentation but if it is going to be a real, documented module in the Python library then the bogus MS junk should go. The truth is I would prefer NOT to work on winreg and leave both versions out of the library. But Unless someone else is willing to design and implement a decent API, I took that burden upon myself rather than see more weird stuff in the Python API. So the value add is: * uses Python iteration protocol * uses Python mapping protocol * uses Python method invocation syntax * uses only features that will be documented * does not expose integers as object handles (even for HKLM etc.) * uses inspectable, queryable objects even as docstrings * has a much more organized module dictionary (do a dir( _winreg)) If you disagree with those principles then we are in trouble. If you have quibbles about specifics then let's talk. > Ive just updated the test suite so that test_winreg2.py actually works. > > It appears that the new winreg.py module is still in a state of flux, but > all work has ceased. The test for this module has lots of placeholders > that are not filled in. Worse, the test code was checked in an obviously > broken state (presumably "to be done", but guess who the bunny who had to > do it was :-( The tests ran fine on my machine. Fred had to make minor changes before he checked it in for me because of module name changes. It's possible that he mistyped a search and replace or something...or that I had a system dependency. Since I changed jobs I no longer have access to Visual C++ and have not had luck getting GCC to compile _winreg. This makes further testing difficult until someone cuts a Windows binary build of Python (which is perpetually imminent). The test cases are not all filled in. The old winreg test tested each method on average one time. The new winreg tries harder to test each in a variety of situations. Rather than try to keep all cases in my head I created empty function bodies. Now we have clear documentation of what is done and tested and what is to be tested still. Once an alpha is cut, (or I fix my compiler situation) I can finish that process. > Browsing the source made it clear that the module docstrings are still > incomplete (eg "For information on the key API, open a key and look at its > docstring."). The docstrings are not complete, but they are getting better and the old winreg documentation was certainly not complete either! I admit I got into a little bit of recursive projects wherein I didn't want to write the winreg, minidom, SAX, etc. documentation twice so I started working on stuff that would extract the docstrings and generate LaTeX. That's almost done and I'll finish up the documentation. That's what the beta period is for, right? > Eg, the specific example I had a problem with was: > > key[value] > > Returns a result that includes the key index! This would be similar to a > dictionary index _always_ returning the tuple, and the first element of the > tuple is _always_ the key you just indexed. There is a very clearly labelled (and docstring-umented) getValueData method: key.getValueData("FOO") That gets only the value. Surely that's no worse than the original: winreg.QueryValue( key, "FOO" ) If this is your only design complaint then I don't see cause for alarm yet. Here's why I did it that way: You can fetch data values by their names or by their indexes. If you've just indexed by the name then of course you know it. If you've just fetched by the numeric index then you don't. I thought it was more consistent to have the same value no matter how you indexed. Also, when you get a value, you should also get a type, because the types can be important. In that case it still has to be a tuple, so it's just a question of a two-tuple or a three-tuple. Again, I thought that the three-tuple was more consistent. Also, this is the same return value returned by the existing EnumValue function. > Has anyone else actually looked at or played with this, and still believe > it is an improvement over _winreg? I personally find it unintuitive, and > will personally continue to use _winreg. If we can't find anyone to > complete it, document it, and stand up and say they really like it, I > suggest we pull it. I agree that it needs more review. I could not get anyone interested in a discussion of how the API should look, other than pointing at old threads. You are, of course, welcome to use whatever you want but I think it would be productive to give the new API a workout in real code and then report specific design complaints. If others concur, we can change it. -- Paul Prescod - Not encumbered by corporate consensus "I don't want you to describe to me -- not ever -- what you were doing to that poor boy to make him sound like that; but if you ever do it again, please cover his mouth with your hand," Grandmother said. -- John Irving, "A Prayer for Owen Meany" From mwh21@cam.ac.uk Tue Aug 1 07:59:11 2000 From: mwh21@cam.ac.uk (Michael Hudson) Date: 01 Aug 2000 07:59:11 +0100 Subject: [Python-Dev] Negative slice steps considered unhealthy (extended slicing for lists) In-Reply-To: "Fredrik Lundh"'s message of "Tue, 1 Aug 2000 08:20:01 +0200" References: <200007312345.LAA10291@s454.cosc.canterbury.ac.nz> <001a01bffb80$87514860$f2a6b5d4@hagrid> Message-ID: "Fredrik Lundh" writes: > greg wrote: > > > I think there are some big conceptual problems with allowing > > negative steps in a slice. > > wasn't "slices" supposed to work the same way as "ranges"? The problem is that for slices (& indexes in general) that negative indices have a special interpretation: range(10,-1,-1) range(10)[:-1] Personally I don't think it's that bad (you just have to remember to write :: instead of :-1: when you want to step all the way back to the beginning). More serious is what you do with out of range indices - and NumPy is a bit odd with this one, it seems: >>> l = Numeric.arrayrange(10) >>> l[30::-2] array([0, 8, 6, 4, 2, 0]) What's that initial "0" doing there? Can someone who actually understands NumPy explain this? Cheers, M. (PS: PySlice_GetIndices is in practice a bit useless because when it fails it offers no explanation of why! Does any one use this function, or should I submit a patch to make it a bit more helpful (& support longs)?) -- -Dr. Olin Shivers, Ph.D., Cranberry-Melon School of Cucumber Science -- seen in comp.lang.scheme From tim_one@email.msn.com Tue Aug 1 08:57:06 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 1 Aug 2000 03:57:06 -0400 Subject: [Python-Dev] Should repr() of string should observe locale? In-Reply-To: <200008010007.MAA10298@s454.cosc.canterbury.ac.nz> Message-ID: [Greg Ewing] > Seems like we need another function that does something in > between str() and repr(). It would be just like repr() except > that it wouldn't put escape sequences in strings unless > absolutely necessary, and it would apply this recursively > to sub-objects. > > Not sure what to call it -- goofy() perhaps :-) In the previous incarnation of this debate, a related (more str-like than repr-like) intermediate was named ssctsoos(). Meaning, of course , "str() special casing the snot out of strings". It was purely a hack, and I was too busy working at Dragon at the time to give it the thought it needed. Now I'm too busy working at PythonLabs <0.5 wink>. not-a-priority-ly y'rs - tim From MarkH@ActiveState.com Tue Aug 1 08:59:22 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Tue, 1 Aug 2000 17:59:22 +1000 Subject: [Python-Dev] New winreg module really an improvement? In-Reply-To: <39866F8D.FCFA85CB@prescod.net> Message-ID: I am going to try very hard to avoid antagonistic statements - it doesnt help anyone or anything when we are so obviously at each others throats. Let me start by being conciliatory: I do appreciate the fact that you made the effort on the winreg module, and accept it was done for all the right reasons. The fact that I dont happen to like it doesnt imply any personal critisism - I believe we simply have a philosophical disagreement. But then again, they are the worst kinds of disagreements to have! > > Actually, it was more more along the lines of me promising to > spend some > > time "over the next few days", and not getting to it. > However, I believe > > it was less than a week before it was just checked in. > > It was checked in the day before the alpha was supposed to go out. I > thought that was what you wanted! On what schedule would you have > preferred us to do it? Im not sure, but one that allowed everyone with relevant input to give it. Guido also stated he was not happy with the process. I would have preferred to have it miss the alpha than to go out with a design we are not happy with. > >From my point of view, it was the appearance of _winreg that prompted > the "flurry of activity" that led to winreg. I would never have bothered > with winreg if I were not responding to the upcoming "event" of the > defacto standardization of _winreg. It was clearly designed (and I use > the word loosely) by various people at Microsoft over several years -- > with sundry backwards and forwards compatibility hacks embedded. Agreed. However, the main problem was that people were assuming win32api was around to get at the registry. The win32api module's registry functions have been around for _ages_. None of its users have ever proposed a more Pythonic API. Thus I find it a little strange that someone without much experience in the API should find it such an abomination, while experienced users of the API were clearly happy (ok - maybe "happy" isnt the right word - but no unhappy enough to complain :-) If nothing else, it allows the proliferation of documentation on the Win32 API to apply to Python. This is clearly not true with the new module. This is also a good case for using the .NET API. However, it still would not provide Python indexing, iteration etc. However, as I state below, Im not convinced this is a problem. > I'm all for slow and steady, deliberate design. I'm sorry _winreg was > rushed but I could only work with the time I had and the interest level > of the people around. Nobody else wanted to discuss it. Nobody wanted to > review the module. Hardly anyone here even knew what was in the OLD > module. I dont belive that is fair. As I said, plenty of people has used win32api, and were sick of insisting their users install my extensions. distutils was the straw that broke the serpents back, IIRC. It is simply the sheer volume of people who _did_ use the win32api registry functions that forced the new winreg module. The fact that no one else wanted to discuss it, or review it, or generally seemed to care should have been indication that the new winreg was not really required, rather than taken as proof that a half-baked module that has not had any review should be checked in. > I am too. I would *also* be interested in hearing from people who have > not spent the last five years with the Microsoft API because _winreg was > a very thin wrapper over it and so will be obvious to those who already > know it. Agreed - but it isnt going to happen. There are not enough people on this list who are not experienced with Windows, but also intend getting that experience during the beta cycle. I hope you would agree that adding an experimental module to Python simply as a social experiment is not the right thing to do. Once winreg is released, it will be too late to remove, even if the consesus is that it should never have been released in the first place. > I have the feeling that an abstraction over the APIs would never be as > "comfortable" as the Microsoft API you've been using for all of these > years. Again agreed - although we should replace the "you've" with "you and every other Windows programmer" - which tends to make the case for _winreg stronger, IMO. Moving to the second mail: > All of that was appropriate when winreg was documented "by reference" to > the Microsoft documentation but if it is going to be a real, documented > module in the Python library then the bogus MS junk should go. I agree in principal, but IMO it is obvious this will not happen. It hasnt happened yet, and you yourself have moved into more interesting PEPs. How do you propose this better documentation will happen? > The truth is I would prefer NOT to work on winreg and leave both > versions out of the library. Me too - it has just cost me work so far, and offers me _zero_ benefit (if anyone in the world can assume that the win32api module is around, it surely must be me ;-). However, this is a debate you need to take up with the distutils people, and everyone else who has asked for registry access in the core. Guido also appears to have heard these calls, hence we had his complete support for some sort of registry module for the core. > So the value add is: ... > If you disagree with those principles then we are in trouble. If you > have quibbles about specifics then let's talk. I'm afraid we are in a little trouble ;-) These appear dubious to me. If I weigh in the number of calls over the years for a more Pythonic API over the win32api functions, I become more convinced. The registry is a tree structure similar to a file system. There havent been any calls I have heard to move the os.listdir() function or the glob module to a more "oo" style? I dont see a "directory" object that supports Python style indexing or iteration. I dont see any of your other benefits being applied to Python's view of the file system - so why is the registry so different? To try and get more productive: Bill, Gordon et al appear to have the sense to stay out of this debate. Unless other people do chime in, Paul and I will remain at an impasse, and no one will be happy. I would much prefer to move this forward than to vent at each other regarding mails neither of us can remember in detail ;-) So what to do? Anyone? If even _one_ experienced Windows developer on this list can say they believe "winreg" is appropriate and intuitive, I am happy to shut up (and then simply push for better winreg documentation ;-) Mark. From Moshe Zadka Tue Aug 1 09:36:29 2000 From: Moshe Zadka (Moshe Zadka) Date: Tue, 1 Aug 2000 11:36:29 +0300 (IDT) Subject: [Python-Dev] Access to the Bug Database Message-ID: Hi! I think I need access to the bug database -- but in the meantime, anyone who wants to mark 110612 as closed is welcome to. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From tim_one@email.msn.com Tue Aug 1 09:40:53 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 1 Aug 2000 04:40:53 -0400 Subject: [Python-Dev] New winreg module really an improvement? In-Reply-To: Message-ID: FWIW, I ignored all the winreg modules, and all the debate about them. Why? Just because Mark's had been in use for years already, so was already battle-tested. There's no chance that any other platform will ever make use of this module, and given that its appeal is thus solely to Windows users, it was fine by me if it didn't abstract *anything* away from MS's Win32 API. MS's APIs are hard enough to understand without somebody else putting their own layers of confusion <0.9 wink> on top of them. May as well complain that the SGI-specific cd.open() function warns that if you pass anything at all to its optional "mode" argument, it had better be the string "r" (maybe that makes some kind of perverse sense to SGI weenies? fine by me if so). So, sorry, but I haven't even looked at Paul's code. I probably should, but-- jeez! --there are so many other things that *need* to get done. I did look at Mark's (many months ago) as part of helping him reformat it to Guido's tastes, and all I remember thinking about it then is "yup, looks a whole lot like the Windows registry API -- when I need it I'll be able to browse the MS docs lightly and use it straight off -- good!". So unless Mark went and did something like clean it up , I still think it's good. From tim_one@email.msn.com Tue Aug 1 10:27:59 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 1 Aug 2000 05:27:59 -0400 Subject: [Python-Dev] Access to the Bug Database In-Reply-To: Message-ID: [Moshe Zadka] > I think I need access to the bug database Indeed, you had no access to the SF bug database at all. Neither did a bunch of others. I have a theory about that: I mentioned several weeks ago that IE5 simply could not display the Member Permissions admin page correctly, after it reached a certain size. I downloaded a stinking Netscape then, and that worked fine until it reached *another*, larger size, at which point the display of some number of the bottom-most entries (like where moshez lives) gets totally screwed up *sometimes*. My *suspicion* is that if an admin changes a permission while either IE5 or NS is in this screwed-up state, it wreaks havoc with the permissions of the members whose display lines are screwed up. It's a weak suspicion , but a real one: I've only seen NS screw up some contiguous number of the bottom-most lines, I expect all the admins are using NS, and it was all and only a contiguous block of developers at the bottom of the page who had their Bug Manager permissions set to None (along with other damaged values) when I brought up the page. So, admins, look out for that! Anyway, I just went thru and gave every developer admin privileges on the SF Bug Manager. Recall that it will probably take about 6 hours to take effect, though. > -- but in the meantime, anyone who wants to mark 110612 as > closed is welcome to. No, they're not: nobody who knows *why* the bug is being closed should even think about closing it. It's still open. you're-welcome -ly y'rs - tim From Vladimir.Marangozov@inrialpes.fr Tue Aug 1 10:53:36 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Tue, 1 Aug 2000 11:53:36 +0200 (CEST) Subject: [Python-Dev] SET_LINENO and python options In-Reply-To: <14725.63622.190585.197392@beluga.mojam.com> from "Skip Montanaro" at Jul 31, 2000 05:07:02 PM Message-ID: <200008010953.LAA02082@python.inrialpes.fr> Skip Montanaro wrote: > > Isn't that what the code object's co_lnotab is for? I thought the idea was > to dispense with SET_LINENO altogether and just compute line numbers using > co_lnotab on those rare occasions (debugging, tracebacks, etc) when you > needed them. Don't worry about it anymore. It's all in Postponed patch #101022 at SF. It makes the current "-O" the default (and uses co_lnotab), and reverts back to the current default with "-d". I give myself a break on this. You guys need to test it now and report some feedback and impressions. If only to help Guido making up his mind and give him a chance to pronounce on it . [?!ng] > Anyway, i suppose this is all rather moot now that Vladimir has a > clever scheme for tracing even without SET_LINENO. Go Vladimir! > Your last proposal sounded great. Which one? They are all the latest . See also the log msg of the latest tiny patch update at SF. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From nhodgson@bigpond.net.au Tue Aug 1 11:47:12 2000 From: nhodgson@bigpond.net.au (Neil Hodgson) Date: Tue, 1 Aug 2000 20:47:12 +1000 Subject: [Python-Dev] New winreg module really an improvement? References: Message-ID: <010501bffba5$db4ebf90$8119fea9@neil> > So what to do? Anyone? If even _one_ experienced Windows developer on > this list can say they believe "winreg" is appropriate and intuitive, I am > happy to shut up (and then simply push for better winreg documentation ;-) Sorry but my contribution isn't going to help much with breaking the impasse. Registry code tends to be little lumps of complexity you don't touch once it is working. The Win32 Reg* API is quite ugly - RegCreateKeyEx takes/returns 10 parameters but you only normally want 3 and the return status and everyone asks for KEY_ALL_ACCESS until the installation testers tell you it fails for non-Administrators. So it would be good if the API was simpler and defaulted everything you don't need to set. But I only hack the registry about once a year with Python. So if its closer to the Win32 API then that helps me to use existing knowledge and documentation. When writing an urllib patch recently, winreg seemed OK. Is it complete enough? Are the things you can't do with it important for its role? IMO, if winreg can handle the vast majority of cases (say, 98%) then its a useful tool and people who need RegSetKeySecurity and similar can go to win32api. Do the distutils developers know how much registry access they need? Enough fence sitting for now, Neil From MarkH@ActiveState.com Tue Aug 1 12:08:58 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Tue, 1 Aug 2000 21:08:58 +1000 Subject: [Python-Dev] New winreg module really an improvement? In-Reply-To: <010501bffba5$db4ebf90$8119fea9@neil> Message-ID: Just to clarify (or confuse) the issue: > When writing an urllib patch recently, winreg seemed OK. Is > it complete > enough? Are the things you can't do with it important for its > role? IMO, if > winreg can handle the vast majority of cases (say, 98%) then its a useful > tool and people who need RegSetKeySecurity and similar can go to > win32api. Note that Neil was actually using _winreg - the exposure of the raw Win32 API. Part of my applying the patch was to rename the usage of "winreg" to "_winreg". Between the time of you writing the original patch and it being applied, the old "winreg" module was renamed to "_winreg", and Paul's new "winreg.py" was added. The bone of contention is the new "winreg.py" module, which urllib does _not_ use. Mark. From jim@interet.com Tue Aug 1 14:28:40 2000 From: jim@interet.com (James C. Ahlstrom) Date: Tue, 01 Aug 2000 09:28:40 -0400 Subject: [Python-Dev] InfoWorld July 17 looks at Zope and Python References: <397DB146.C68F9CD0@interet.com> <398654A8.37EB17BA@prescod.net> Message-ID: <3986D088.E82E2162@interet.com> Paul Prescod wrote: > > Would you mind giving me the jist of the review? 20-word summary, if you > don't mind. Please note that I don't necessarily agree with the reviews. Also, there is no such thing as bad publicity. Page 50: "Zope is a powerful application server. Version 2.2 beta scales well, but enterprise capability, Python language raise costs beyond the competition's." Author claims he must purchase ZEO for $25-50K which is too expensive. Zope is dedicated to OOP, but shops not doing OOP will have problems understanding it. Python expertise is necessary, but shops already know VB, C++ and JavaScript. Page 58: "After many tutorials, I'm still waiting to become a Zope addict." Zope is based on Python, but that is no problem because you do most programming in DTML which is like HTML. It is hard to get started in Zope because of lack of documentation, it is hard to write code in browser text box, OOP-to-the-max philosophy is unlike a familiar relational data base. Zope has an unnecessarily high nerd factor. It fails to automate simple tasks. My point in all this is that we design features to appeal to computer scientists instead of "normal users". JimA From billtut@microsoft.com Tue Aug 1 14:57:37 2000 From: billtut@microsoft.com (Bill Tutt) Date: Tue, 1 Aug 2000 06:57:37 -0700 Subject: [Python-Dev] New winreg module really an improvement? Message-ID: <58C671173DB6174A93E9ED88DCB0883D0A610A@red-msg-07.redmond.corp.microsoft.com> Mark wrote: > To try and get more productive: Bill, Gordon et al appear to have the > sense to stay out of this debate. Unless other people do chime in, Paul > and I will remain at an impasse, and no one will be happy. I would much > prefer to move this forward than to vent at each other regarding mails > neither of us can remember in detail ;-) I'm actually in the process of checking it out, and am hoping to compose some comments on it later today. I do know this about abstracting the registry APIs. If it doesn't allow you to do everything you can do with the normal APIs, then you've failed in your abstraction. (Which is probably why I've never yet seen a successful abstraction of the API. :) ) The registry is indeed a bizarre critter. Key's have values, and values have values. Ugh.... It's enough to drive a sane man bonkers, and here I was being annoyed by the person who originally designed the NT console APIs, silly me.... Bill From gmcm@hypernet.com Tue Aug 1 17:16:54 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Tue, 1 Aug 2000 12:16:54 -0400 Subject: [Python-Dev] New winreg module really an improvement? In-Reply-To: References: <39866F8D.FCFA85CB@prescod.net> Message-ID: <1246975873-72274187@hypernet.com> [Mark] > To try and get more productive: Bill, Gordon et al appear to > have the sense to stay out of this debate. Wish I had that much sense... I'm only +0 on the philosophy of a friendly Pythonic wrapper: the registry is only rarely the "right" solution. You need it when you have small amounts of persistent data that needs to be available to multiple apps and / or Windows. I actively discourage use of the registry for any other purposes. So making it easy to use is of very low priority for me. In addition, I doubt that a sane wrapper can be put over it. At first blush, it looks like a nested dict. But the keys are ordered. And a leaf is more like a list of tuples [(value, data), ]. But if you pull up regedit and look at how it's used, the (user- speak) "value" may be a (MS-speak) "key", "value" or "data". Just note the number of entries where a keyname has one (value, data) pair that consists of ("(Default)", "(value not set)"). Or the number where keyname must be opened, but the (value, data) pair is ("(Default)", something). (It doesn't help that "key" may mean "keyname" or "keyhandle", and "name" may mean "keyname" or "valuename" and "value" may mean "valuename" or "datavalue".) IOW, this isn't like passing lists (instead of FD_SETs) to select. No known abstract container matches the registry. My suspicion is that any attempt to map it just means the user will have to understand both the underlying API and the mapping. As a practical matter, it looks to me like winreg (under any but the most well-informed usage) may well leak handles. If so, that would be a disaster. But I don't have time to check it out. In sum: - I doubt the registry can be made to look elegant - I use it so little I don't really care - Gordon From paul@prescod.net Tue Aug 1 17:52:45 2000 From: paul@prescod.net (Paul Prescod) Date: Tue, 01 Aug 2000 12:52:45 -0400 Subject: [Python-Dev] Winreg recap Message-ID: <3987005C.9C45D7B6@prescod.net> I specifically asked everyone here if an abstraction was a good idea. I got three + votes and no - votes. One of the + votes requested that we still ship the underlying module. Fine. I was actually pointed (on python-dev) to specs for an abstraction layer that AFAIK had been designed *on Python-dev*. Back then, I said: > > I've just had a chance to look at the winreg module. I think that it is > > too low-level. Mark Hammond said: > I agree. There was a proposal (from Thomas Heller, IIRC) to do just this. > I successfully argued there should be _2_ modules for Python - the raw > low-level API, which guarantees you can do (almost) anything. A > higher-level API could cover the 80% of cases. > ... > I have no real problem with your proposed design, as long as it it written > in Python, _using_ the low-level API. It could be called "registry" or I > would even be happy for "winreg.pyd" -> "_winreg.pyd" and your new module > to be called "winreg.py" Gordon pointed me to the spec. I took it and expanded on it to cover a wider range of cases. So now I go off and code it up and in addition to complaining about one detail, I'm also told that there is no real point to having a high level API. Windows users are accustomed to hacks and pain so crank it up! > FWIW, I ignored all the winreg modules, and all the debate about them. Why? > Just because Mark's had been in use for years already, so was already > battle-tested. There's no chance that any other platform will ever make use > of this module, and given that its appeal is thus solely to Windows users, > it was fine by me if it didn't abstract *anything* away from MS's Win32 API. It is precisely because it is for Windows users -- often coming from VB, JavaScript or now C# -- that it needs to be abstracted. I have the philosophy that I come to Python (both the language and the library) because I want things to be easy and powerful at the same time. Whereever feasible, our libraries *should* be cleaner and better than the hacks that they cover up. Shouldn't they? I mean even *Microsoft* abstracted over the registry API for VB, JavaScript, C# (and perhaps Java). Are we really going to do less for our users? To me, Python (language and library) is a refuge from the hackiness of the rest of the world. -- Paul Prescod - Not encumbered by corporate consensus "I don't want you to describe to me -- not ever -- what you were doing to that poor boy to make him sound like that; but if you ever do it again, please cover his mouth with your hand," Grandmother said. -- John Irving, "A Prayer for Owen Meany" From paul@prescod.net Tue Aug 1 17:53:31 2000 From: paul@prescod.net (Paul Prescod) Date: Tue, 01 Aug 2000 12:53:31 -0400 Subject: [Python-Dev] New winreg module really an improvement? References: <200007281206.HAA04102@cj20424-a.reston1.va.home.com> Message-ID: <3987008B.35D5C2A2@prescod.net> Guido van Rossum wrote: > > I vaguely remember that I wasn't happy with the way this was handled > either, but was too busy at the time to look into it. (I can't say > whether I'm happy with the module or not, since I've never tried to > use it. But I do feel unhappy about the process.) I was also unhappy with the process but from a differEnt perspective. A new module appeared in the Python library: _winreg It was based on tried and true code: _winreg, but it's API had many placeholder arguments (where Microsoft had placeholder arguments) used function call syntax for things that were clearly methods (as Microsoft did for C), had an enumeration mechanism that seems, to me, be very unPythonic, had many undocumented features and constants, and the documented methods and properties often have weird Microsoft conventions (e.g. SetValueEx). The LaTeX documentation for the old winreg says of one method: "This is Lame Lame Lame, DO NOT USE THIS!!!" Now I am still working on new winreg. I got involved in a recursive project to avoid writing the docs twice in two different formats. We are still in the beta period so there is no need to panic about documentation yet. I would love nothing more than to hear that Windows registry handling is hereby delayed until Python 7 or until someone more interested wants to work on it for the love of programming. But if that is not going to happen then I will strongly advise against falling back to _winreg which is severely non-Pythonic. > I vaguely remember that Paul Prescod's main gripe with the _winreg API > was that it's not object-oriented enough -- but that seems his main > gripe about most code these days. :-) In this case it wasn't a mild preference, it was a strong allergic reaction! > Paul, how much experience with using the Windows registry did you have > when you designed the new API? I use it off and on. There are still corners of _winreg that I don't understand. That's part of why I thought it needed to be covered up with something that could be fully documented. To get even the level of understanding I have, of the *original* _winreg, I had to scour the Web. The perl docs were the most helpful. :) Anyhow, Mark isn't complaining about me misunderstanding it, he's complaining about my mapping into the Python object model. That's fair. That's what we python-dev is for. As far as Greg using _winreg, my impression was that that code predates new winreg. I think that anyone who reads even just the docstrings for the new one and the documentation for the other is going to feel that the new one is at the very least more organized and thought out. Whether it is properly designed is up to users to decide. -- Paul Prescod - Not encumbered by corporate consensus "I don't want you to describe to me -- not ever -- what you were doing to that poor boy to make him sound like that; but if you ever do it again, please cover his mouth with your hand," Grandmother said. -- John Irving, "A Prayer for Owen Meany" From guido@beopen.com Tue Aug 1 19:20:23 2000 From: guido@beopen.com (Guido van Rossum) Date: Tue, 01 Aug 2000 13:20:23 -0500 Subject: [Python-Dev] New winreg module really an improvement? In-Reply-To: Your message of "Tue, 01 Aug 2000 03:16:30 -0400." <3986794E.ADBB938C@prescod.net> References: <3986794E.ADBB938C@prescod.net> Message-ID: <200008011820.NAA30284@cj20424-a.reston1.va.home.com> Paul wrote: > I had no personal interest in an API for the windows registry but I > could not, in good conscience, let the original one become the > standard Python registry API. and later: > I use it off and on. There are still corners of _winreg that I don't > understand. That's part of why I thought it needed to be covered up with > something that could be fully documented. To get even the level of > understanding I have, of the *original* _winreg, I had to scour the Web. > The perl docs were the most helpful. :) I believe this is the crux of the problem. Your only mistake was that you criticized and then tried to redesign a (poorly designed) API that you weren't intimately familiar with. My boss tries to do this occasionally; he has a tendency to complain that my code doesn't contain enough classes. I tell him to go away -- he only just started learning Python from a book that I've never seen, so he wouldn't understand... Paul, I think that the best thing to do now is to withdraw winreg.py, and to keep (and document!) the _winreg extension with the understanding that it's a wrapper around poorly designed API but at least it's very close to the C API. The leading underscore should be a hint that this is not a module for every day use. Hopefully someday someone will eventually create a set of higher level bindings modeled after the Java, VB or C# version of the API. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From fdrake@beopen.com Tue Aug 1 18:43:16 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Tue, 1 Aug 2000 13:43:16 -0400 (EDT) Subject: [Python-Dev] New winreg module really an improvement? In-Reply-To: <200008011820.NAA30284@cj20424-a.reston1.va.home.com> References: <3986794E.ADBB938C@prescod.net> <200008011820.NAA30284@cj20424-a.reston1.va.home.com> Message-ID: <14727.3124.622333.980689@cj42289-a.reston1.va.home.com> Guido van Rossum writes: > and to keep (and document!) the _winreg extension with the > understanding that it's a wrapper around poorly designed API but at > least it's very close to the C API. The leading underscore should be > a hint that this is not a module for every day use. It is documented (as _winreg), but I've not reviewed the section in great detail (yet!). It will need to be revised to not refer to the winreg module as the preferred API. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From Moshe Zadka Tue Aug 1 19:30:48 2000 From: Moshe Zadka (Moshe Zadka) Date: Tue, 1 Aug 2000 21:30:48 +0300 (IDT) Subject: [Python-Dev] Bug Database Message-ID: I've just had a quick view over the database, and saw what we can prune at no cost: 110647 -- Segmentation fault in "%.1200d" % 1. Fixed for me... 110649 -- Core dumps on compiling big expressions ('['+'1,'*100000+'1]'). Fixed for me -- now throws a SyntaxError 110653 -- Complain about how class foo: def __init__(self): self.bar1 = bar def bar(self): pass Creates cycles. A notabug if I ever saw one. 110654 -- 1+0j tested false. The bug was fixed. 110679 -- math.log(0) dumps core. Gives OverflowError for me...(I'm using a different OS, but the same CPU family (intel)) 110710 -- range(10**n) gave segfault. Works for me -- either works, or throws MemoryError 110711 -- apply(foo, bar, {}) throws MemoryError. Works for me. (But might be an SGI problem) 110712 -- seems to be a duplicate of 110711 110715 -- urllib.urlretrieve() segfaults under kernel 2.2.12. Works for me with 2.2.15. 110740, 110741, 110743, 110745, 110746, 110747, 110749, 110750 -- dups of 11715 I've got to go to sleep now.... -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From jeremy@beopen.com Tue Aug 1 19:47:47 2000 From: jeremy@beopen.com (Jeremy Hylton) Date: Tue, 1 Aug 2000 14:47:47 -0400 (EDT) Subject: [Python-Dev] Bug Database In-Reply-To: References: Message-ID: <14727.6995.164586.983795@bitdiddle.concentric.net> Thanks for doing some triage, Moshe! I am in the process of moving the bug database from jitterbug to SourceForge. There are still a few kinks in the process, which I am trying to finish today. There are two problems you will see with the current database: * Many bugs do not contain all the followup messages that Jitterbug has. I am working to add them. * There are many duplicates of some bugs. The duplicates are the result of the debugging process for my Jitterbug -> SF conversion script. I will remove these before I am done. Any bug numbered higher than 110740 is probably a duplicate at this point. The conversion process has placed most of the Jitterbug entries in the SF bug tracker. The PR number is in the SF summary and most of the relevant Jitterbug headers (submittor, data, os, version) are part of the body of the SF bug. Any followups to the Jitterbug report are stored as followup comments in SF. The SF bug tracker has several fields that we can use to manage bug reports. * Category: Describes what part of Python the bug applies to. Current values are parser/compiler, core, modules, library, build, windows, documentation. We can add more categories, e.g. library/xml, if that is helpful. * Priority: We can assign a value from 1 to 9, where 9 is the highest priority. We will have to develop some guidelines for what those priorities mean. Right now everthing is priority 5 (medium). I would hazard a guess that bugs causing core dumps should have much higher priority. * Group: These reproduce some of the Jitterbug groups, like trash, platform-specific, and irreproducible. These are rough categories that we can use, but I'm not sure how valuable they are. * Resolution: What we plan to do about the bug. * Assigned To: We can now assign bugs to specific people for resolution. * Status: Open or Closed. When a bug has been fixed in the CVS repository and a test case added to cover the bug, change its status to Closed. New bug reports should use the sourceforge interface. Jeremy From guido@beopen.com Tue Aug 1 21:14:39 2000 From: guido@beopen.com (Guido van Rossum) Date: Tue, 01 Aug 2000 15:14:39 -0500 Subject: [Python-Dev] Bug Database In-Reply-To: Your message of "Tue, 01 Aug 2000 14:47:47 -0400." <14727.6995.164586.983795@bitdiddle.concentric.net> References: <14727.6995.164586.983795@bitdiddle.concentric.net> Message-ID: <200008012014.PAA31076@cj20424-a.reston1.va.home.com> > * Category: Describes what part of Python the bug applies to. Current > values are parser/compiler, core, modules, library, build, windows, > documentation. We can add more categories, e.g. library/xml, if that > is helpful. Before it's too late, would it make sense to try and get the categories to be the same in the Bug and Patch managers? --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From m.favas@per.dem.csiro.au Tue Aug 1 21:30:42 2000 From: m.favas@per.dem.csiro.au (Mark Favas) Date: Wed, 02 Aug 2000 04:30:42 +0800 Subject: [Python-Dev] regression test failure in test_tokenize? Message-ID: <39873372.1C6F8CE1@per.dem.csiro.au> Current CVS (Wed Aug 2 04:22:16 WST 2000) fails on Tru64 Unix: ./python Lib/test/regrtest.py test_tokenize.py test_tokenize test test_tokenize failed -- Writing: "57,4-57,5:\011NUMBER\011'3'", expected: "57,4-57,8:\011NUMBER\011'3." 1 test failed: test_tokenize Test produces (snipped): 57,4-57,5: NUMBER '3' Test should produce (if supplied output correct): 57,4-57,8: NUMBER '3.14' Is this just me, or an un-checked checkin? (I noticed some new sre bits in my current CVS version.) Mark -- Email - m.favas@per.dem.csiro.au Mark C Favas Phone - +61 8 9333 6268, 0418 926 074 CSIRO Exploration & Mining Fax - +61 8 9383 9891 Private Bag No 5, Wembley WGS84 - 31.95 S, 115.80 E Western Australia 6913 From akuchlin@mems-exchange.org Tue Aug 1 21:47:57 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Tue, 1 Aug 2000 16:47:57 -0400 Subject: [Python-Dev] regression test failure in test_tokenize? In-Reply-To: <39873372.1C6F8CE1@per.dem.csiro.au>; from m.favas@per.dem.csiro.au on Wed, Aug 02, 2000 at 04:30:42AM +0800 References: <39873372.1C6F8CE1@per.dem.csiro.au> Message-ID: <20000801164757.B27333@kronos.cnri.reston.va.us> On Wed, Aug 02, 2000 at 04:30:42AM +0800, Mark Favas wrote: >Current CVS (Wed Aug 2 04:22:16 WST 2000) fails on Tru64 Unix: >Is this just me, or an un-checked checkin? (I noticed some new sre bits >in my current CVS version.) test_tokenize works fine using the current CVS on Linux; perhaps this is a 64-bit problem in sre manifesting itself? --amk From Fredrik Lundh" <20000801164757.B27333@kronos.cnri.reston.va.us> Message-ID: <02ac01bffbfd$bc6b27a0$f2a6b5d4@hagrid> andrew wrote: > On Wed, Aug 02, 2000 at 04:30:42AM +0800, Mark Favas wrote: > >Current CVS (Wed Aug 2 04:22:16 WST 2000) fails on Tru64 Unix: > >Is this just me, or an un-checked checkin? (I noticed some new sre = bits > >in my current CVS version.) >=20 > test_tokenize works fine using the current CVS on Linux; perhaps this > is a 64-bit problem in sre manifesting itself? I've confirmed (and fixed) the bug reported by Mark. It was a nasty little off-by-one error in the "branch predictor" code... But I think I know why you didn't see anything: Guido just checked in the following change to re.py: *** 21,26 **** # =20 ! engine =3D "sre" ! # engine =3D "pre" =20 if engine =3D=3D "sre": --- 21,26 ---- # =20 ! # engine =3D "sre" ! engine =3D "pre" =20 if engine =3D=3D "sre": From guido@beopen.com Tue Aug 1 23:21:51 2000 From: guido@beopen.com (Guido van Rossum) Date: Tue, 01 Aug 2000 17:21:51 -0500 Subject: [Python-Dev] regression test failure in test_tokenize? In-Reply-To: Your message of "Tue, 01 Aug 2000 23:16:15 +0200." <02ac01bffbfd$bc6b27a0$f2a6b5d4@hagrid> References: <39873372.1C6F8CE1@per.dem.csiro.au> <20000801164757.B27333@kronos.cnri.reston.va.us> <02ac01bffbfd$bc6b27a0$f2a6b5d4@hagrid> Message-ID: <200008012221.RAA05722@cj20424-a.reston1.va.home.com> > But I think I know why you didn't see anything: Guido just checked > in the following change to re.py: > > *** 21,26 **** > # > > ! engine = "sre" > ! # engine = "pre" > > if engine == "sre": > --- 21,26 ---- > # > > ! # engine = "sre" > ! engine = "pre" > > if engine == "sre": Ouch. did I really? I didn't intend to! I'll back out right away... --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From barry@scottb.demon.co.uk Wed Aug 2 00:01:29 2000 From: barry@scottb.demon.co.uk (Barry Scott) Date: Wed, 2 Aug 2000 00:01:29 +0100 Subject: [Python-Dev] Preventing 1.5 extensions crashing under 1.6/2.0 Python In-Reply-To: <000701bff108$950ec9f0$060210ac@private> Message-ID: <000801bffc0c$6d985490$060210ac@private> If someone in the core of Python thinks a patch implementing what I've outlined is useful please let me know and I will generate the patch. Barry From MarkH@ActiveState.com Wed Aug 2 00:13:31 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Wed, 2 Aug 2000 09:13:31 +1000 Subject: [Python-Dev] Preventing 1.5 extensions crashing under 1.6/2.0 Python In-Reply-To: <000801bffc0c$6d985490$060210ac@private> Message-ID: > If someone in the core of Python thinks a patch implementing > what I've outlined is useful please let me know and I will > generate the patch. Umm - I'm afraid that I dont keep my python-dev emils for that long, and right now I'm too lazy/busy to dig around the archives. Exactly what did you outline? I know it went around a few times, and I can't remember who said what. For my money, I liked Fredrik's solution best (check Py_IsInitialized() in Py_InitModule4()), but as mentioned that only solves for the next version of Python; it doesnt solve the fact 1.5 modules will crash under 1.6/2.0 It would definately be excellent to get _something_ in the CNRI 1.6 release, so the BeOpen 2.0 release can see the results. But-I-doubt-anyone-will-release-extension-modules-for-1.6-anyway ly, Mark. From jeremy@beopen.com Wed Aug 2 00:56:27 2000 From: jeremy@beopen.com (Jeremy Hylton) Date: Tue, 1 Aug 2000 19:56:27 -0400 (EDT) Subject: [Python-Dev] Bug Database In-Reply-To: <200008012014.PAA31076@cj20424-a.reston1.va.home.com> References: <14727.6995.164586.983795@bitdiddle.concentric.net> <200008012014.PAA31076@cj20424-a.reston1.va.home.com> Message-ID: <14727.25515.570860.775496@bitdiddle.concentric.net> >>>>> "GvR" == Guido van Rossum writes: >> * Category: Describes what part of Python the bug applies to. >> Current values are parser/compiler, core, modules, library, >> build, windows, documentation. We can add more categories, >> e.g. library/xml, if that is helpful. GvR> Before it's too late, would it make sense to try and get the GvR> categories to be the same in the Bug and Patch managers? Yes, as best we can do. We've got all the same names, though the capitalization varies sometimes. Jeremy From gstein@lyra.org Wed Aug 2 02:26:51 2000 From: gstein@lyra.org (Greg Stein) Date: Tue, 1 Aug 2000 18:26:51 -0700 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/PC _winreg.c,1.7,1.8 In-Reply-To: <200007280344.UAA12335@slayer.i.sourceforge.net>; from mhammond@users.sourceforge.net on Thu, Jul 27, 2000 at 08:44:43PM -0700 References: <200007280344.UAA12335@slayer.i.sourceforge.net> Message-ID: <20000801182651.S19525@lyra.org> This could be simplified quite a bit by using PyObject_AsReadBuffer() from abstract.h ... Cheers, -g On Thu, Jul 27, 2000 at 08:44:43PM -0700, Mark Hammond wrote: > Update of /cvsroot/python/python/dist/src/PC > In directory slayer.i.sourceforge.net:/tmp/cvs-serv12325 > > Modified Files: > _winreg.c > Log Message: > Allow any object supporting the buffer protocol to be written as a binary object. > > Index: _winreg.c > =================================================================== > RCS file: /cvsroot/python/python/dist/src/PC/_winreg.c,v > retrieving revision 1.7 > retrieving revision 1.8 > diff -C2 -r1.7 -r1.8 > *** _winreg.c 2000/07/16 12:04:32 1.7 > --- _winreg.c 2000/07/28 03:44:41 1.8 > *************** > *** 831,837 **** > *retDataSize = 0; > else { > ! if (!PyString_Check(value)) > ! return 0; > ! *retDataSize = PyString_Size(value); > *retDataBuf = (BYTE *)PyMem_NEW(char, > *retDataSize); > --- 831,844 ---- > *retDataSize = 0; > else { > ! void *src_buf; > ! PyBufferProcs *pb = value->ob_type->tp_as_buffer; > ! if (pb==NULL) { > ! PyErr_Format(PyExc_TypeError, > ! "Objects of type '%s' can not " > ! "be used as binary registry values", > ! value->ob_type->tp_name); > ! return FALSE; > ! } > ! *retDataSize = (*pb->bf_getreadbuffer)(value, 0, &src_buf); > *retDataBuf = (BYTE *)PyMem_NEW(char, > *retDataSize); > *************** > *** 840,847 **** > return FALSE; > } > ! memcpy(*retDataBuf, > ! PyString_AS_STRING( > ! (PyStringObject *)value), > ! *retDataSize); > } > break; > --- 847,851 ---- > return FALSE; > } > ! memcpy(*retDataBuf, src_buf, *retDataSize); > } > break; > > > _______________________________________________ > Python-checkins mailing list > Python-checkins@python.org > http://www.python.org/mailman/listinfo/python-checkins -- Greg Stein, http://www.lyra.org/ From guido@beopen.com Wed Aug 2 05:09:38 2000 From: guido@beopen.com (Guido van Rossum) Date: Tue, 01 Aug 2000 23:09:38 -0500 Subject: [Python-Dev] Still no new license -- but draft text available Message-ID: <200008020409.XAA01355@cj20424-a.reston1.va.home.com> We still don't have a new license for Python 1.6; Bob Kahn and Richard Stallman need to talk before a decision can be made about how to deal with the one remaining GPL incompatibility. While we're all waiting, we're preparing the CNRI 1.6 release at SourceForge (part of the deal is that the PythonLabs group finishes the 1.6 release for CNRI). The last thing I committed today was the text (dictated by Bob Kahn) for the new LICENSE file that will be part of the 1.6 beta 1 release. (Modulo any changes that will be made to the license text to ensure GPL compatibility.) Since anyone with an anonymous CVS setup can now read the license anyway, I might as well post a copy here so that you can all get used to it... --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) ======== LICENSE ======================================================= A. HISTORY OF THE SOFTWARE Python originated in 1991 at Stichting Mathematisch Centrum (CWI) in the Netherlands as an outgrowth of a language called ABC. Its principal author was Guido van Rossum, although it included smaller contributions from others at CWI and elsewhere. The last version of Python issued by CWI was Python 1.2. In 1995, Mr. van Rossum continued his work on Python at the Corporation for National Research Initiatives (CNRI) in Reston, Virginia where several versions of the software were generated. Python 1.6 is the last of the versions developed at CNRI. B. TERMS AND CONDITIONS FOR ACCESSING OR OTHERWISE USING Python 1.6, beta 1 1. CNRI LICENSE AGREEMENT PYTHON 1.6, beta 1 CNRI OPEN SOURCE LICENSE AGREEMENT IMPORTANT: PLEASE READ THE FOLLOWING AGREEMENT CAREFULLY. BY CLICKING ON "ACCEPT" WHERE INDICATED BELOW, OR BY COPYING, INSTALLING OR OTHERWISE USING PYTHON 1.6, beta 1 SOFTWARE, YOU ARE DEEMED TO HAVE AGREED TO THE TERMS AND CONDITIONS OF THIS LICENSE AGREEMENT. 1. This LICENSE AGREEMENT is between the Corporation for National Research Initiatives, having an office at 1895 Preston White Drive, Reston, VA 20191 ("CNRI"), and the Individual or Organization ("Licensee") accessing and otherwise using Python 1.6, beta 1 software in source or binary form and its associated documentation, as released at the www.python.org Internet site on August 5, 2000 ("Python 1.6b1"). 2. Subject to the terms and conditions of this License Agreement, CNRI hereby grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce, analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python 1.6b1 alone or in any derivative version, provided, however, that CNRI's License Agreement is retained in Python 1.6b1, alone or in any derivative version prepared by Licensee. Alternately, in lieu of CNRI's License Agreement, Licensee may substitute the following text (omitting the quotes): "Python 1.6, beta 1, is made available subject to the terms and conditions in CNRI's License Agreement. This Agreement may be located on the Internet using the following unique, persistent identifier (known as a handle): 1895.22/1011. This Agreement may also be obtained from a proxy server on the Internet using the URL:http://hdl.handle.net/1895.22/1011". 3. In the event Licensee prepares a derivative work that is based on or incorporates Python 1.6b1or any part thereof, and wants to make the derivative work available to the public as provided herein, then Licensee hereby agrees to indicate in any such work the nature of the modifications made to Python 1.6b1. 4. CNRI is making Python 1.6b1 available to Licensee on an "AS IS" basis. CNRI MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, CNRI MAKES NO AND DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 1.6b1 WILL NOT INFRINGE ANY THIRD PARTY RIGHTS. 5. CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE SOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS A RESULT OF USING, MODIFYING OR DISTRIBUTING PYTHON 1.6b1, OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. SOME STATES DO NOT ALLOW THE LIMITATION OR EXCLUSION OF LIABILITY SO THE ABOVE DISCLAIMER MAY NOT APPLY TO LICENSEE. 6. This License Agreement will automatically terminate upon a material breach of its terms and conditions. 7. This License Agreement shall be governed by and interpreted in all respects by the law of the State of Virginia, excluding conflict of law provisions. Nothing in this License Agreement shall be deemed to create any relationship of agency, partnership, or joint venture between CNRI and Licensee. This License Agreement does not grant permission to use CNRI trademarks or trade name in a trademark sense to endorse or promote products or services of Licensee, or any third party. 8. By clicking on the "ACCEPT" button where indicated, or by copying installing or otherwise using Python 1.6b1, Licensee agrees to be bound by the terms and conditions of this License Agreement. ACCEPT 2. CWI PERMISSIONS STATEMENT AND DISCLAIMER Copyright (c) 1991 - 1995, Stichting Mathematisch Centrum Amsterdam, The Netherlands. All rights reserved. Permission to use, copy, modify, and distribute this software and its documentation for any purpose and without fee is hereby granted, provided that the above copyright notice appear in all copies and that both that copyright notice and this permission notice appear in supporting documentation, and that the name of Stichting Mathematisch Centrum or CWI not be used in advertising or publicity pertaining to distribution of the software without specific, written prior permission. STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. ======================================================================== From guido@beopen.com Wed Aug 2 05:42:30 2000 From: guido@beopen.com (Guido van Rossum) Date: Tue, 01 Aug 2000 23:42:30 -0500 Subject: [Python-Dev] BeOpen statement about Python license Message-ID: <200008020442.XAA01587@cj20424-a.reston1.va.home.com> Bob Weiner, BeOpen's CTO, has this to say about the Python license: Here's the official word from BeOpen.com regarding any potential license change on Python 1.6 (the last CNRI Python release) and subsequent versions: The Python license is fully open source compliant, as certified by the Open Source Initiative. That means that if you look at www.opensource.org/osd.html, then this license complies with those 9 precepts, allowing broad freedom of use, distribution and modification. The Python license will continue to allow fully proprietary software development. The license issues are down to one point which we are working to resolve together with CNRI and involving potential GPL-compatibility. It is a small point regarding a requirement that the license be interpreted under the terms of Virginia law. One lawyer has said that this doesn't affect GPL-compatibility but Richard Stallman of the FSF has felt differently; he views it as a potential additional restriction of rights beyond those listed in the GPL. So work continues to resolve on this point before the license is published or attached to any code. We are presently waiting for follow-up from Stallman on this point. In summary, BeOpen.com is actively working to keep Python the extremely open platform it has traditionally been and to resolve legal issues such as this in ways that benefit Python users worldwide. CNRI is working along the same lines as well. Please assure yourselves and your management that Python continues to allow for both open and closed software development. Regards, Bob Weiner I (Guido) hope that this, together with the draft license text that I just posted, clarifies matters for now! I'll post more news as it happens, --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From thomas@xs4all.net Wed Aug 2 07:12:54 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 2 Aug 2000 08:12:54 +0200 Subject: [Python-Dev] CVS feature wish ? :) In-Reply-To: <200008012122.OAA22327@slayer.i.sourceforge.net>; from gvanrossum@users.sourceforge.net on Tue, Aug 01, 2000 at 02:22:20PM -0700 References: <200008012122.OAA22327@slayer.i.sourceforge.net> Message-ID: <20000802081254.V266@xs4all.nl> On Tue, Aug 01, 2000 at 02:22:20PM -0700, Guido van Rossum wrote: > Update of /cvsroot/python/python/dist/src/Lib > In directory slayer.i.sourceforge.net:/tmp/cvs-serv22316 > Modified Files: > re.py > Log Message: > My fix to the URL accidentally also switched back to the "pre" module. > Undo that! This kind of thing is one of the reasons I wish 'cvs commit' would give you the entire patch you're about to commit in the log-message-edit screen, as CVS: comments, rather than just the modified files. It would also help with remembering what the patch was supposed to do ;) Is this possible with CVS, other than an 'EDITOR' that does this for you ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From paul@prescod.net Wed Aug 2 08:30:30 2000 From: paul@prescod.net (Paul Prescod) Date: Wed, 02 Aug 2000 03:30:30 -0400 Subject: [Python-Dev] New winreg module really an improvement? References: <3986794E.ADBB938C@prescod.net> <200008011820.NAA30284@cj20424-a.reston1.va.home.com> Message-ID: <3987CE16.DB3E72B8@prescod.net> Guido van Rossum wrote: > > ... > > I believe this is the crux of the problem. Your only mistake was that > you criticized and then tried to redesign a (poorly designed) API that > you weren't intimately familiar with. I don't think that this has been demonstrated. We have one complaint about one method from Mark and silence from everyone else (and about everything else). The Windows registry is weird in its terminology, but it isn't brain surgery. Yes, I had to do some research on what various things do but I expect that almost anyone would have had to do that. Some of the constants in the module are meant to be used with functions that are not even exposed in the module. This indicates to me that nobody has clearly thought out all of the details (and also that _winreg is not a complete binding to the API). I probably understand the original API as well as anyone and more than most, by now. Anyhow, the list at the bottom should demonstrate that I understand the API at least as well as the Microsoftie that invented the .NET API for Java, VB and everything else. > Hopefully someday someone will eventually create a set of higher level > bindings modeled after the Java, VB or C# version of the API. Mark sent me those specs and I believe that the module I sent out *is* very similar to that higher level API. Specifically (>>> is Python version) Equals (inherited from Object) >>> __cmp__ key.Name >>> key.name key.SubKeyCount >>> len( key.getSubkeys() ) key.ValueCount >>> len( key.getValues() ) Close >>> key.close() CreateSubKey >>> key.createSubkey() DeleteSubKey >>> key.deleteSubkey() DeleteSubKeyTree >>> (didn't get around to implementing/testing something like this) DeleteValue >>> key.deleteValue() GetSubKeyNames >>> key.getSubkeyNames() GetValue >>> key.getValueData() GetValueNames >>> key.getValueNames() OpenRemoteBaseKey >>> key=RemoteKey( ... ) OpenSubKey >>> key.openSubkey SetValue >>> key.setValue() ToString >>> str( key ) My API also has some features for enumerating that this does not have. Mark has a problem with one of those. I don't see how that makes the entire API "unintuitive", considering it is more or less a renaming of the .NET API. -- Paul Prescod - Not encumbered by corporate consensus "I don't want you to describe to me -- not ever -- what you were doing to that poor boy to make him sound like that; but if you ever do it again, please cover his mouth with your hand," Grandmother said. -- John Irving, "A Prayer for Owen Meany" From Fredrik Lundh" <3986794E.ADBB938C@prescod.net> <200008011820.NAA30284@cj20424-a.reston1.va.home.com> Message-ID: <004d01bffc50$522fa2a0$f2a6b5d4@hagrid> guido wrote: > Paul, I think that the best thing to do now is to withdraw winreg.py, > and to keep (and document!) the _winreg extension with the > understanding that it's a wrapper around poorly designed API but at > least it's very close to the C API. The leading underscore should be > a hint that this is not a module for every day use. how about letting _winreg export all functions with their win32 names, and adding a winreg.py which looks some- thing like this: from _winreg import * class Key: .... HKEY_CLASSES_ROOT =3D Key(...) ... where the Key class addresses the 80% level: open keys and read NONE/SZ/EXPAND_SZ/DWORD values (through a slightly extended dictionary API). in 2.0, add support to create keys and write values of the same types, and you end up supporting the needs of 99% of all applications. > Hopefully someday someone will eventually create a set of higher level > bindings modeled after the Java, VB or C# version of the API. how about Tcl? I'm pretty sure their API (which is very simple, iirc) addresses the 99% level... From Moshe Zadka Wed Aug 2 08:00:40 2000 From: Moshe Zadka (Moshe Zadka) Date: Wed, 2 Aug 2000 10:00:40 +0300 (IDT) Subject: [Python-Dev] Cookies.py in the core (was Tangent to Re: [Tutor] CGI and Python (fwd)) Message-ID: Do we have a procedure for putting more batteries in the core? I'm not talking about stuff like PEP-206, I'm talking about small, useful modules like Cookies.py. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez ---------- Forwarded message ---------- Date: Tue, 01 Aug 2000 12:03:12 PDT From: Brian Wisti To: tutor@python.org Subject: Tangent to Re: [Tutor] CGI and Python >In contrast, i've been motivated with questions like yours which pop up >every now and then to create a separate chapter entrily devoted to CGI pro- >gramming and in it, to provide an example that starts out simple and builds >to something a little more complex. there will be lots of screen captures >too so that you can see what's going on. finally, there will be a more >"advanced" section towards the end which does the complicated stuff that >everyone wants to do, like cookies, multivalued fields, and file uploads >with multipart data. sorry that the book isn't out yet... trying to get >the weeds out of it right NOW! ;-) > I'm looking forward to seeing the book! Got a question, that is almost relevant to the thread. Does anybody know why cookie support isn't built in to the cgi module? I had to dig around to find Cookie.py, which (excellent module that it is) should be in the cgi package somewhere. Just a random thought from the middle of my workday... Later, Brian Wisti ________________________________________________________________________ Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com _______________________________________________ Tutor maillist - Tutor@python.org http://www.python.org/mailman/listinfo/tutor From mal@lemburg.com Wed Aug 2 10:12:01 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 02 Aug 2000 11:12:01 +0200 Subject: [Python-Dev] Still no new license -- but draft text available References: <200008020409.XAA01355@cj20424-a.reston1.va.home.com> Message-ID: <3987E5E1.A2B20241@lemburg.com> Guido van Rossum wrote: > > We still don't have a new license for Python 1.6; Bob Kahn and Richard > Stallman need to talk before a decision can be made about how to deal > with the one remaining GPL incompatibility. While we're all waiting, > we're preparing the CNRI 1.6 release at SourceForge (part of the deal > is that the PythonLabs group finishes the 1.6 release for CNRI). The > last thing I committed today was the text (dictated by Bob Kahn) for > the new LICENSE file that will be part of the 1.6 beta 1 release. > (Modulo any changes that will be made to the license text to ensure > GPL compatibility.) > > Since anyone with an anonymous CVS setup can now read the license > anyway, I might as well post a copy here so that you can all get used > to it... Is the license on 2.0 going to look the same ? I mean we now already have two seperate licenses and if BeOpen adds another two or three paragraphs will end up with a license two pages long. Oh, how I loved the old CWI license... Some comments on the new version: > A. HISTORY OF THE SOFTWARE > > Python originated in 1991 at Stichting Mathematisch Centrum (CWI) in > the Netherlands as an outgrowth of a language called ABC. Its > principal author was Guido van Rossum, although it included smaller > contributions from others at CWI and elsewhere. The last version of > Python issued by CWI was Python 1.2. In 1995, Mr. van Rossum > continued his work on Python at the Corporation for National Research > Initiatives (CNRI) in Reston, Virginia where several versions of the > software were generated. Python 1.6 is the last of the versions > developed at CNRI. > > B. TERMS AND CONDITIONS FOR ACCESSING OR OTHERWISE USING Python 1.6, beta 1 > > 1. CNRI LICENSE AGREEMENT > > PYTHON 1.6, beta 1 > > CNRI OPEN SOURCE LICENSE AGREEMENT > > IMPORTANT: PLEASE READ THE FOLLOWING AGREEMENT CAREFULLY. > > BY CLICKING ON "ACCEPT" WHERE INDICATED BELOW, OR BY COPYING, > INSTALLING OR OTHERWISE USING PYTHON 1.6, beta 1 SOFTWARE, YOU ARE > DEEMED TO HAVE AGREED TO THE TERMS AND CONDITIONS OF THIS LICENSE > AGREEMENT. > > 1. This LICENSE AGREEMENT is between the Corporation for National > Research Initiatives, having an office at 1895 Preston White Drive, > Reston, VA 20191 ("CNRI"), and the Individual or Organization > ("Licensee") accessing and otherwise using Python 1.6, beta 1 software > in source or binary form and its associated documentation, as released > at the www.python.org Internet site on August 5, 2000 ("Python > 1.6b1"). > > 2. Subject to the terms and conditions of this License Agreement, CNRI > hereby grants Licensee a nonexclusive, royalty-free, world-wide > license to reproduce, analyze, test, perform and/or display publicly, > prepare derivative works, distribute, and otherwise use Python 1.6b1 > alone or in any derivative version, provided, however, that CNRI's > License Agreement is retained in Python 1.6b1, alone or in any > derivative version prepared by Licensee. I don't the latter (retaining the CNRI license alone) is not possible: you always have to include the CWI license. > Alternately, in lieu of CNRI's License Agreement, Licensee may > substitute the following text (omitting the quotes): "Python 1.6, beta > 1, is made available subject to the terms and conditions in CNRI's > License Agreement. This Agreement may be located on the Internet > using the following unique, persistent identifier (known as a handle): > 1895.22/1011. This Agreement may also be obtained from a proxy server > on the Internet using the URL:http://hdl.handle.net/1895.22/1011". Do we really need this in the license text ? It's nice to have the text available on the Internet, but why add long descriptions about where to get it from to the license text itself ? > 3. In the event Licensee prepares a derivative work that is based on > or incorporates Python 1.6b1or any part thereof, and wants to make the > derivative work available to the public as provided herein, then > Licensee hereby agrees to indicate in any such work the nature of the > modifications made to Python 1.6b1. In what way would those indications have to be made ? A patch or just text describing the new features ? What does "make available to the public" mean ? If I embed Python in an application and make this application available on the Internet for download would this fit the meaning ? What about derived work that only uses the Python language reference as basis for its task, e.g. new interpreters or compilers which can read and execute Python programs ? > 4. CNRI is making Python 1.6b1 available to Licensee on an "AS IS" > basis. CNRI MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR > IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, CNRI MAKES NO AND > DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS > FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 1.6b1 WILL NOT > INFRINGE ANY THIRD PARTY RIGHTS. > > 5. CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE > SOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS > AS A RESULT OF USING, MODIFYING OR DISTRIBUTING PYTHON 1.6b1, OR ANY > DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. SOME > STATES DO NOT ALLOW THE LIMITATION OR EXCLUSION OF LIABILITY SO THE > ABOVE DISCLAIMER MAY NOT APPLY TO LICENSEE. I would make this "...SOME STATES AND COUNTRIES...". E.g. in Germany the above text would only be valid after an initial 6 month period after installation, AFAIK (this period is called "Gewährleistung"). Licenses from other vendors usually add some extra license text to limit the liability in this period to the carrier on which the software was received by the licensee, e.g. the diskettes or CDs. > 6. This License Agreement will automatically terminate upon a material > breach of its terms and conditions. Immediately ? Other licenses usually include a 30-60 day period which allows the licensee to take actions. With the above text, the license will put the Python copy in question into an illegal state *prior* to having even been identified as conflicting with the license. > 7. This License Agreement shall be governed by and interpreted in all > respects by the law of the State of Virginia, excluding conflict of > law provisions. Nothing in this License Agreement shall be deemed to > create any relationship of agency, partnership, or joint venture > between CNRI and Licensee. This License Agreement does not grant > permission to use CNRI trademarks or trade name in a trademark sense > to endorse or promote products or services of Licensee, or any third > party. Would the name "Python" be considered a trademark in the above sense ? > 8. By clicking on the "ACCEPT" button where indicated, or by copying > installing or otherwise using Python 1.6b1, Licensee agrees to be > bound by the terms and conditions of this License Agreement. > > ACCEPT > > 2. CWI PERMISSIONS STATEMENT AND DISCLAIMER > > Copyright (c) 1991 - 1995, Stichting Mathematisch Centrum Amsterdam, > The Netherlands. All rights reserved. > > Permission to use, copy, modify, and distribute this software and its > documentation for any purpose and without fee is hereby granted, > provided that the above copyright notice appear in all copies and that > both that copyright notice and this permission notice appear in > supporting documentation, and that the name of Stichting Mathematisch > Centrum or CWI not be used in advertising or publicity pertaining to > distribution of the software without specific, written prior > permission. > > STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO > THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND > FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE > FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES > WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN > ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT > OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. ...oh how I loved this one ;-) -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From jack@oratrix.nl Wed Aug 2 10:43:05 2000 From: jack@oratrix.nl (Jack Jansen) Date: Wed, 02 Aug 2000 11:43:05 +0200 Subject: [Python-Dev] Winreg recap In-Reply-To: Message by Paul Prescod , Tue, 01 Aug 2000 12:52:45 -0400 , <3987005C.9C45D7B6@prescod.net> Message-ID: <20000802094305.C3006303181@snelboot.oratrix.nl> > I specifically asked everyone here if an abstraction was a good idea. I > got three + votes and no - votes. One of the + votes requested that we > still ship the underlying module. Fine. I was actually pointed (on > python-dev) to specs for an abstraction layer that AFAIK had been > designed *on Python-dev*. This point I very much agree to: if we can abstract 90% of the use cases of the registry (while still giving access to the other 10%) in a clean interface we can implement the same interface for Mac preference files, unix dot-files, X resources, etc. A general mechanism whereby a Python program can get at a persistent setting that may have factory defaults, installation overrides and user overrides, and that is implemented in the logical way on each platform would be very powerful. The initial call to open the preference database(s) and give identity information as to which app you are, etc is probably going to be machine dependent, but from that point on there should be a single API. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From Moshe Zadka Wed Aug 2 11:16:40 2000 From: Moshe Zadka (Moshe Zadka) Date: Wed, 2 Aug 2000 13:16:40 +0300 (IDT) Subject: [Python-Dev] More Non-Bugs Message-ID: Bug 110651 -- re.compile('[\\200-\\400]') segfaults -- it doens't for me -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From thomas@xs4all.net Wed Aug 2 11:41:12 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 2 Aug 2000 12:41:12 +0200 Subject: [Python-Dev] More Non-Bugs In-Reply-To: ; from moshez@math.huji.ac.il on Wed, Aug 02, 2000 at 01:16:40PM +0300 References: Message-ID: <20000802124112.W266@xs4all.nl> On Wed, Aug 02, 2000 at 01:16:40PM +0300, Moshe Zadka wrote: > Bug 110651 -- re.compile('[\\200-\\400]') segfaults -- it doens't for me You can close bugs now, right, Moshe ? If not, you should be able to :P Just do what I do: close them, assign them to yourself, set the status to 'Works For Me', explain in the log message what you did to test it, and forward a copy of the mail you get from SF to the original complainee. A lot of the bugs are relatively old, so a fair lot of them are likely to be fixed already. If they aren't fixed for the complainee (or someone else), the bug can be re-opened and possibly updated at the same time. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From Moshe Zadka Wed Aug 2 12:05:06 2000 From: Moshe Zadka (Moshe Zadka) Date: Wed, 2 Aug 2000 14:05:06 +0300 (IDT) Subject: [Python-Dev] More Non-Bugs In-Reply-To: <20000802124112.W266@xs4all.nl> Message-ID: On Wed, 2 Aug 2000, Thomas Wouters wrote: > On Wed, Aug 02, 2000 at 01:16:40PM +0300, Moshe Zadka wrote: > > > Bug 110651 -- re.compile('[\\200-\\400]') segfaults -- it doens't for me > > You can close bugs now, right, Moshe? I can, but to tell the truth, after what Tim posted here about closing bugs, I'd appreciate a few more eyeballs before I close them. > A lot of the bugs are relatively old, so a fair lot of them are likely to be > fixed already. If they aren't fixed for the complainee (or someone else), > the bug can be re-opened and possibly updated at the same time. Hmmmmm.....OK. But I guess I'll still wait for a goahead from the PythonLabs team. BTW: Does anyone know if SF has an e-mail notification of bugs, similar to that of patches? If so, enabling it to send mail to a mailing list similar to patches@python.org would be cool -- it would enable much more peer review. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From thomas@xs4all.net Wed Aug 2 12:21:47 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 2 Aug 2000 13:21:47 +0200 Subject: [Python-Dev] More Non-Bugs In-Reply-To: ; from moshez@math.huji.ac.il on Wed, Aug 02, 2000 at 02:05:06PM +0300 References: <20000802124112.W266@xs4all.nl> Message-ID: <20000802132147.L13365@xs4all.nl> On Wed, Aug 02, 2000 at 02:05:06PM +0300, Moshe Zadka wrote: > On Wed, 2 Aug 2000, Thomas Wouters wrote: > > On Wed, Aug 02, 2000 at 01:16:40PM +0300, Moshe Zadka wrote: > > > Bug 110651 -- re.compile('[\\200-\\400]') segfaults -- it doens't for me > > You can close bugs now, right, Moshe? > I can, but to tell the truth, after what Tim posted here about closing > bugs, I'd appreciate a few more eyeballs before I close them. That's why I forward the message to the original submittor. The list of bugs is now so insanely large that it's pretty unlikely a large number of eyeballs will caress them. Marking them closed (or atleast marking them *something*, like moving them to the right catagory) and forwarding the summary to the submittor is likely to have them re-check the bug. Tim was talking about 'closing it without reason', without knowing why it should be closed. 'Works for me' is a valid reason to close the bug, if you have the same (kind of) platform, can't reproduce the bug and have a strong suspicion it's already been fixed. (Which is pretty likely, if the bugreport is old.) > BTW: Does anyone know if SF has an e-mail notification of bugs, similar > to that of patches? If so, enabling it to send mail to a mailing list > similar to patches@python.org would be cool -- it would enable much more > peer review. I think not, but I'm not sure. It's probably up to the project admins to set that, but I think if they did, they'd have set it before. (Then again, I'm not sure if it's a good idea to set it, yet... I bet the current list is going to be quickly cut down in size, and I'm not sure if I want to see all the notifications! :) But once it's running, it would be swell. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From Vladimir.Marangozov@inrialpes.fr Wed Aug 2 13:13:41 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Wed, 2 Aug 2000 14:13:41 +0200 (CEST) Subject: [Python-Dev] More Non-Bugs In-Reply-To: from "Moshe Zadka" at Aug 02, 2000 01:16:40 PM Message-ID: <200008021213.OAA06073@python.inrialpes.fr> Moshe Zadka wrote: > > Bug 110651 -- re.compile('[\\200-\\400]') segfaults -- it doens't for me You get a compiled SRE object, right? But SRE is the new 're' and the old 're' is 'pre'. Make the example: import pre; pre.compile('[\\200-\\400]') and I suspect you'll get the segfault (I did). -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From Moshe Zadka Wed Aug 2 13:17:31 2000 From: Moshe Zadka (Moshe Zadka) Date: Wed, 2 Aug 2000 15:17:31 +0300 (IDT) Subject: [Python-Dev] More Non-Bugs In-Reply-To: <200008021213.OAA06073@python.inrialpes.fr> Message-ID: On Wed, 2 Aug 2000, Vladimir Marangozov wrote: > Moshe Zadka wrote: > > > > Bug 110651 -- re.compile('[\\200-\\400]') segfaults -- it doens't for me > > You get a compiled SRE object, right? Nope -- I tested it with pre. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From jeremy@beopen.com Wed Aug 2 13:31:55 2000 From: jeremy@beopen.com (Jeremy Hylton) Date: Wed, 2 Aug 2000 08:31:55 -0400 (EDT) Subject: [Python-Dev] More Non-Bugs In-Reply-To: References: <20000802124112.W266@xs4all.nl> Message-ID: <14728.5307.820982.137908@bitdiddle.concentric.net> >>>>> "MZ" == Moshe Zadka writes: MZ> Hmmmmm.....OK. But I guess I'll still wait for a goahead from MZ> the PythonLabs team. BTW: Does anyone know if SF has an e-mail MZ> notification of bugs, similar to that of patches? If so, MZ> enabling it to send mail to a mailing list similar to MZ> patches@python.org would be cool -- it would enable much more MZ> peer review. Go ahead and mark as closed bugs that are currently fixed. If you can figure out when they were fixed (e.g. what checkin), that would be best. If not, just be sure that it really is fixed -- and write a test case that would have caught the bug. SF will send out an email, but sending it to patches@python.org would be a bad idea, I think. Isn't that list attached to Jitterbug? Jeremy From Moshe Zadka Wed Aug 2 13:30:16 2000 From: Moshe Zadka (Moshe Zadka) Date: Wed, 2 Aug 2000 15:30:16 +0300 (IDT) Subject: [Python-Dev] More Non-Bugs In-Reply-To: <14728.5307.820982.137908@bitdiddle.concentric.net> Message-ID: On Wed, 2 Aug 2000, Jeremy Hylton wrote: > SF will send out an email, but sending it to patches@python.org would > be a bad idea, I think. I've no problem with having a seperate mailing list I can subscribe to. Perhaps it should be a mailing list along the lines of Python-Checkins.... -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From guido@beopen.com Wed Aug 2 15:02:00 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 02 Aug 2000 09:02:00 -0500 Subject: [Python-Dev] CVS feature wish ? :) In-Reply-To: Your message of "Wed, 02 Aug 2000 08:12:54 +0200." <20000802081254.V266@xs4all.nl> References: <200008012122.OAA22327@slayer.i.sourceforge.net> <20000802081254.V266@xs4all.nl> Message-ID: <200008021402.JAA02711@cj20424-a.reston1.va.home.com> > > My fix to the URL accidentally also switched back to the "pre" module. > > Undo that! > > This kind of thing is one of the reasons I wish 'cvs commit' would give you > the entire patch you're about to commit in the log-message-edit screen, as > CVS: comments, rather than just the modified files. It would also help with > remembering what the patch was supposed to do ;) Is this possible with CVS, > other than an 'EDITOR' that does this for you ? Actually, I have made it a habit to *always* do a cvs diff before I commit, for exactly this reason. That's why this doesn't happen more often. In this case I specifically remember reviewing the diff and thinking that it was alright, but not scrolling towards the second half of the diff. :( --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From guido@beopen.com Wed Aug 2 15:06:00 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 02 Aug 2000 09:06:00 -0500 Subject: [Python-Dev] Cookies.py in the core (was Tangent to Re: [Tutor] CGI and Python (fwd)) In-Reply-To: Your message of "Wed, 02 Aug 2000 10:00:40 +0300." References: Message-ID: <200008021406.JAA02743@cj20424-a.reston1.va.home.com> > Do we have a procedure for putting more batteries in the core? I'm > not talking about stuff like PEP-206, I'm talking about small, useful > modules like Cookies.py. Cookie support in the core would be a good thing. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From fdrake@beopen.com Wed Aug 2 14:20:52 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Wed, 2 Aug 2000 09:20:52 -0400 (EDT) Subject: [Python-Dev] More Non-Bugs In-Reply-To: <14728.5307.820982.137908@bitdiddle.concentric.net> References: <20000802124112.W266@xs4all.nl> <14728.5307.820982.137908@bitdiddle.concentric.net> Message-ID: <14728.8244.745008.301891@cj42289-a.reston1.va.home.com> Jeremy Hylton writes: > SF will send out an email, but sending it to patches@python.org would > be a bad idea, I think. Isn't that list attached to Jitterbug? No, but Barry is working on getting a new list set up for SourceForge to send bug messages to. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From gvwilson@nevex.com Wed Aug 2 14:22:01 2000 From: gvwilson@nevex.com (Greg Wilson) Date: Wed, 2 Aug 2000 09:22:01 -0400 (EDT) Subject: [Python-Dev] CVS headaches / Subversion reminder Message-ID: Those of you who are having troubles with (or have complaints about) CVS on SourceForge might want to check out Subversion, a "better CVS" being developed as part of Tigris: subversion.tigris.org Jason Robbins (project manager, jrobbins@collab.net) told me in Monterey that they are still interested in feature requests, alternatives, etc. There may still be room to add features like showing the full patch during checkin (as per Thomas Wouters' earlier mail). Greg p.s. I'd be interested in hearing from anyone who's ever re-written a medium-sized (40,000 lines) C app in Python --- how did you decide how much of the structure to keep, and how much to re-think, etc. Please mail me directly to conserve bandwidth; I'll post a summary if there's enough interest. From fdrake@beopen.com Wed Aug 2 14:26:28 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Wed, 2 Aug 2000 09:26:28 -0400 (EDT) Subject: [Python-Dev] Cookies.py in the core (was Tangent to Re: [Tutor] CGI and Python (fwd)) In-Reply-To: <200008021406.JAA02743@cj20424-a.reston1.va.home.com> References: <200008021406.JAA02743@cj20424-a.reston1.va.home.com> Message-ID: <14728.8580.460583.760620@cj42289-a.reston1.va.home.com> Guido van Rossum writes: > > Do we have a procedure for putting more batteries in the core? I'm > > not talking about stuff like PEP-206, I'm talking about small, useful > > modules like Cookies.py. > > Cookie support in the core would be a good thing. There's also some cookie support in Grail (limited); that uses a Netscape-style client-side database. Note that the Netscape format is insufficient for the most recent cookie specifications (don't know the RFC #), but I understood from AMK that browser writers are expecting to actually implement that (unlike RFC 2109). If we stick to an in-process database, that wouldn't matter, but I'm not sure if that solves the problem for everyone. Regardless of the format, there's a little bit of work to do here. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From guido@beopen.com Wed Aug 2 15:32:02 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 02 Aug 2000 09:32:02 -0500 Subject: [Python-Dev] Cookies.py in the core (was Tangent to Re: [Tutor] CGI and Python (fwd)) In-Reply-To: Your message of "Wed, 02 Aug 2000 09:26:28 -0400." <14728.8580.460583.760620@cj42289-a.reston1.va.home.com> References: <200008021406.JAA02743@cj20424-a.reston1.va.home.com> <14728.8580.460583.760620@cj42289-a.reston1.va.home.com> Message-ID: <200008021432.JAA02937@cj20424-a.reston1.va.home.com> > Guido van Rossum writes: > > > Do we have a procedure for putting more batteries in the core? I'm > > > not talking about stuff like PEP-206, I'm talking about small, useful > > > modules like Cookies.py. > > > > Cookie support in the core would be a good thing. > > There's also some cookie support in Grail (limited); that uses a > Netscape-style client-side database. > Note that the Netscape format is insufficient for the most recent > cookie specifications (don't know the RFC #), but I understood from > AMK that browser writers are expecting to actually implement that > (unlike RFC 2109). If we stick to an in-process database, that > wouldn't matter, but I'm not sure if that solves the problem for > everyone. > Regardless of the format, there's a little bit of work to do here. I think Cookie.py is for server-side management of cookies, not for client-side. Do we need client-side cookies too???? --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From Moshe Zadka Wed Aug 2 14:34:29 2000 From: Moshe Zadka (Moshe Zadka) Date: Wed, 2 Aug 2000 16:34:29 +0300 (IDT) Subject: [Python-Dev] Cookies.py in the core (was Tangent to Re: [Tutor] CGI and Python (fwd)) In-Reply-To: <200008021432.JAA02937@cj20424-a.reston1.va.home.com> Message-ID: On Wed, 2 Aug 2000, Guido van Rossum wrote: > I think Cookie.py is for server-side management of cookies, not for > client-side. Do we need client-side cookies too???? Not until we write a high-level interface to urllib which is similar to the Perlish UserAgent module -- which is something that should be done if Python wants to be a viable clients-side langugage. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From fdrake@beopen.com Wed Aug 2 14:37:50 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Wed, 2 Aug 2000 09:37:50 -0400 (EDT) Subject: [Python-Dev] Cookies.py in the core (was Tangent to Re: [Tutor] CGI and Python (fwd)) In-Reply-To: <200008021432.JAA02937@cj20424-a.reston1.va.home.com> References: <200008021432.JAA02937@cj20424-a.reston1.va.home.com> <200008021406.JAA02743@cj20424-a.reston1.va.home.com> <14728.8580.460583.760620@cj42289-a.reston1.va.home.com> Message-ID: <14728.9262.635980.220234@cj42289-a.reston1.va.home.com> Guido van Rossum writes: > I think Cookie.py is for server-side management of cookies, not for > client-side. Do we need client-side cookies too???? I think this would be highly desirable; we've seen enough requests for it on c.l.py. Moshe Zadka writes: > Not until we write a high-level interface to urllib which is similar > to the Perlish UserAgent module -- which is something that should > be done if Python wants to be a viable clients-side langugage. Exactly! It has become very difficult to get anything done on the Web without enabling cookies, and simple "screen scraping" tools need to have this support as well. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From fdrake@beopen.com Wed Aug 2 15:05:41 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Wed, 2 Aug 2000 10:05:41 -0400 (EDT) Subject: [Python-Dev] test_parser.py Message-ID: <14728.10933.534904.378463@cj42289-a.reston1.va.home.com> At some point I received a message/bug report referring to test_parser.py, which doesn't exist in the CVS repository (and never has as far as I know). If someone has a regression test for the parser module hidden away, I'd love to add it to the CVS repository! It's time to update the parser module, and a good time to cover it in the regression test! -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From guido@beopen.com Wed Aug 2 16:11:20 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 02 Aug 2000 10:11:20 -0500 Subject: [Python-Dev] Still no new license -- but draft text available In-Reply-To: Your message of "Wed, 02 Aug 2000 11:12:01 +0200." <3987E5E1.A2B20241@lemburg.com> References: <200008020409.XAA01355@cj20424-a.reston1.va.home.com> <3987E5E1.A2B20241@lemburg.com> Message-ID: <200008021511.KAA03049@cj20424-a.reston1.va.home.com> > Is the license on 2.0 going to look the same ? I mean we now > already have two seperate licenses and if BeOpen adds another > two or three paragraphs will end up with a license two pages > long. Good question. We can't really keep the license the same because the old license is very specific to CNRI. I would personally be in favor of using the BSD license for 2.0. > Oh, how I loved the old CWI license... Ditto! > Some comments on the new version: > > 2. Subject to the terms and conditions of this License Agreement, CNRI > > hereby grants Licensee a nonexclusive, royalty-free, world-wide > > license to reproduce, analyze, test, perform and/or display publicly, > > prepare derivative works, distribute, and otherwise use Python 1.6b1 > > alone or in any derivative version, provided, however, that CNRI's > > License Agreement is retained in Python 1.6b1, alone or in any > > derivative version prepared by Licensee. > > I don't the latter (retaining the CNRI license alone) is not > possible: you always have to include the CWI license. Wow. I hadn't even noticed this! It seems you can prepare a derivative version of the license. Well, maybe. > > Alternately, in lieu of CNRI's License Agreement, Licensee may > > substitute the following text (omitting the quotes): "Python 1.6, beta > > 1, is made available subject to the terms and conditions in CNRI's > > License Agreement. This Agreement may be located on the Internet > > using the following unique, persistent identifier (known as a handle): > > 1895.22/1011. This Agreement may also be obtained from a proxy server > > on the Internet using the URL:http://hdl.handle.net/1895.22/1011". > > Do we really need this in the license text ? It's nice to have > the text available on the Internet, but why add long descriptions > about where to get it from to the license text itself ? I'm not happy with this either, but CNRI can put anything they like in their license, and they seem very fond of this particular bit of advertising for their handle system. I've never managed them to convince them that it was unnecessary. > > 3. In the event Licensee prepares a derivative work that is based on > > or incorporates Python 1.6b1or any part thereof, and wants to make the > > derivative work available to the public as provided herein, then > > Licensee hereby agrees to indicate in any such work the nature of the > > modifications made to Python 1.6b1. > > In what way would those indications have to be made ? A patch > or just text describing the new features ? Just text. Bob Kahn told me that the list of "what's new" that I always add to a release would be fine. > What does "make available to the public" mean ? If I embed > Python in an application and make this application available > on the Internet for download would this fit the meaning ? Yes, that's why he doesn't use the word "publish" -- such an action would not be considered publication in the sense of the copyright law (at least not in the US, and probably not according to the Bern convention) but it is clearly making it available to the public. > What about derived work that only uses the Python language > reference as basis for its task, e.g. new interpreters > or compilers which can read and execute Python programs ? The language definition is not covered by the license at all. Only this particular code base. > > 4. CNRI is making Python 1.6b1 available to Licensee on an "AS IS" > > basis. CNRI MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR > > IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, CNRI MAKES NO AND > > DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS > > FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 1.6b1 WILL NOT > > INFRINGE ANY THIRD PARTY RIGHTS. > > > > 5. CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE > > SOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS > > AS A RESULT OF USING, MODIFYING OR DISTRIBUTING PYTHON 1.6b1, OR ANY > > DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. SOME > > STATES DO NOT ALLOW THE LIMITATION OR EXCLUSION OF LIABILITY SO THE > > ABOVE DISCLAIMER MAY NOT APPLY TO LICENSEE. > > I would make this "...SOME STATES AND COUNTRIES...". E.g. in > Germany the above text would only be valid after an initial > 6 month period after installation, AFAIK (this period is > called "Gewährleistung"). Licenses from other vendors usually > add some extra license text to limit the liability in this period > to the carrier on which the software was received by the licensee, > e.g. the diskettes or CDs. I'll mention this to Kahn. > > 6. This License Agreement will automatically terminate upon a material > > breach of its terms and conditions. > > Immediately ? Other licenses usually include a 30-60 day period > which allows the licensee to take actions. With the above text, > the license will put the Python copy in question into an illegal > state *prior* to having even been identified as conflicting with the > license. Believe it or not, this is necessary to ensure GPL compatibility! An earlier draft had 30-60 days. But the GPL doesn't, so this was deemed incompatible. There's an easy workaround though: you fix your compliance and download a new copy, which gives you all the same rights again. > > 7. This License Agreement shall be governed by and interpreted in all > > respects by the law of the State of Virginia, excluding conflict of > > law provisions. Nothing in this License Agreement shall be deemed to > > create any relationship of agency, partnership, or joint venture > > between CNRI and Licensee. This License Agreement does not grant > > permission to use CNRI trademarks or trade name in a trademark sense > > to endorse or promote products or services of Licensee, or any third > > party. > > Would the name "Python" be considered a trademark in the above > sense ? No, Python is not a CNRI trademark. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From trentm@ActiveState.com Wed Aug 2 16:04:17 2000 From: trentm@ActiveState.com (Trent Mick) Date: Wed, 2 Aug 2000 08:04:17 -0700 Subject: [Python-Dev] CVS feature wish ? :) In-Reply-To: <20000802081254.V266@xs4all.nl>; from thomas@xs4all.net on Wed, Aug 02, 2000 at 08:12:54AM +0200 References: <200008012122.OAA22327@slayer.i.sourceforge.net> <20000802081254.V266@xs4all.nl> Message-ID: <20000802080417.A16446@ActiveState.com> On Wed, Aug 02, 2000 at 08:12:54AM +0200, Thomas Wouters wrote: > On Tue, Aug 01, 2000 at 02:22:20PM -0700, Guido van Rossum wrote: > > Update of /cvsroot/python/python/dist/src/Lib > > In directory slayer.i.sourceforge.net:/tmp/cvs-serv22316 > > > Modified Files: > > re.py > > Log Message: > > My fix to the URL accidentally also switched back to the "pre" module. > > Undo that! > > This kind of thing is one of the reasons I wish 'cvs commit' would give you > the entire patch you're about to commit in the log-message-edit screen, as > CVS: comments, rather than just the modified files. It would also help with > remembering what the patch was supposed to do ;) Is this possible with CVS, > other than an 'EDITOR' that does this for you ? > As Guido said, it is probably prefered that one does a cvs diff prior to checking in. But to answer your question *unauthoritatively*, I know that CVS allows you the change the checkin template and I *think* that it offers a script hook to be able to generate it (not sure). If so then one could use that script hook to put in the (commented) patch. Trent -- Trent Mick TrentM@ActiveState.com From trentm@ActiveState.com Wed Aug 2 16:14:16 2000 From: trentm@ActiveState.com (Trent Mick) Date: Wed, 2 Aug 2000 08:14:16 -0700 Subject: [Python-Dev] More Non-Bugs In-Reply-To: <14728.5307.820982.137908@bitdiddle.concentric.net>; from jeremy@beopen.com on Wed, Aug 02, 2000 at 08:31:55AM -0400 References: <20000802124112.W266@xs4all.nl> <14728.5307.820982.137908@bitdiddle.concentric.net> Message-ID: <20000802081416.B16446@ActiveState.com> On Wed, Aug 02, 2000 at 08:31:55AM -0400, Jeremy Hylton wrote: > >>>>> "MZ" == Moshe Zadka writes: > > MZ> Hmmmmm.....OK. But I guess I'll still wait for a goahead from > MZ> the PythonLabs team. BTW: Does anyone know if SF has an e-mail > MZ> notification of bugs, similar to that of patches? If so, > MZ> enabling it to send mail to a mailing list similar to > MZ> patches@python.org would be cool -- it would enable much more > MZ> peer review. > > Go ahead and mark as closed bugs that are currently fixed. If you can > figure out when they were fixed (e.g. what checkin), that would be > best. If not, just be sure that it really is fixed -- and write a > test case that would have caught the bug. I think that unless (1) you submitted the bug or can be sure that "works for me" is with the exact same configuration as the person who did; or (2) you can identify where in the code the bug was and what checkin (or where in the code) fixed it then you cannot close the bug. This is the ideal case, with incomplete bug reports and extremely stale one then these strict requirements are probably not always practical. Trent -- Trent Mick TrentM@ActiveState.com From jack@oratrix.nl Wed Aug 2 16:16:06 2000 From: jack@oratrix.nl (Jack Jansen) Date: Wed, 02 Aug 2000 17:16:06 +0200 Subject: [Python-Dev] Still no new license -- but draft text available In-Reply-To: Message by Guido van Rossum , Wed, 02 Aug 2000 10:11:20 -0500 , <200008021511.KAA03049@cj20424-a.reston1.va.home.com> Message-ID: <20000802151606.753EF303181@snelboot.oratrix.nl> I'm not sure I'm entirely happy with point 3. Depending on how you define "derivative work" and "make available" it could cause serious problems. I assume that this clause is meant so that it is clear that MacPython and PythonWin and other such versions may be based on CNRI Python but are not the same. However, if you're building a commercial application that uses Python as its implementation language this "indication of modifications" becomes rather a long list. Just imagine that a C library came with such a license ("Anyone incorporating this C library or part thereof in their application should indicate the differences between their application and this C library":-). Point 2 has the same problem to a lesser extent, the sentence starting with "Python ... is made available subject to the terms and conditions..." is fine for a product that is still clearly recognizable as Python, but would look silly if Python is just used as the implementation language. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From thomas@xs4all.net Wed Aug 2 16:39:40 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 2 Aug 2000 17:39:40 +0200 Subject: [Python-Dev] CVS feature wish ? :) In-Reply-To: <200008021402.JAA02711@cj20424-a.reston1.va.home.com>; from guido@beopen.com on Wed, Aug 02, 2000 at 09:02:00AM -0500 References: <200008012122.OAA22327@slayer.i.sourceforge.net> <20000802081254.V266@xs4all.nl> <200008021402.JAA02711@cj20424-a.reston1.va.home.com> Message-ID: <20000802173940.X266@xs4all.nl> On Wed, Aug 02, 2000 at 09:02:00AM -0500, Guido van Rossum wrote: > > > My fix to the URL accidentally also switched back to the "pre" module. > > > Undo that! > > This kind of thing is one of the reasons I wish 'cvs commit' would give you > > the entire patch you're about to commit in the log-message-edit screen, as > > CVS: comments, rather than just the modified files. It would also help with > > remembering what the patch was supposed to do ;) Is this possible with CVS, > > other than an 'EDITOR' that does this for you ? > Actually, I have made it a habit to *always* do a cvs diff before I > commit, for exactly this reason. Well, so do I, but none the less I'd like it if the patch was included in the comment :-) I occasionally forget what I was doing (17 xterms, two of which are running 20-session screens (6 of which are dedicated to Python, and 3 to Mailman :), two irc channels with people asking for work-related help or assistance, one telephone with a 'group' number of same, and enough room around me for 5 or 6 people to stand around and ask questions... :) Also, I sometimes wonder about the patch while I'm writing the comment. (Did I do that right ? Didn't I forget about this ? etc.) Having it included as a comment would be perfect, for me. I guess I'll look at the hook thing Trent mailed about, and Subversion, if I find the time for it :P -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From mal@lemburg.com Wed Aug 2 18:22:06 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 02 Aug 2000 19:22:06 +0200 Subject: [Python-Dev] Still no new license -- but draft text available References: <200008020409.XAA01355@cj20424-a.reston1.va.home.com> <3987E5E1.A2B20241@lemburg.com> <200008021511.KAA03049@cj20424-a.reston1.va.home.com> Message-ID: <398858BE.15928F47@lemburg.com> Guido van Rossum wrote: > > > Is the license on 2.0 going to look the same ? I mean we now > > already have two seperate licenses and if BeOpen adds another > > two or three paragraphs will end up with a license two pages > > long. > > Good question. We can't really keep the license the same because the > old license is very specific to CNRI. I would personally be in favor > of using the BSD license for 2.0. If that's possible, I don't think we have to argue about the 1.6 license text at all ;-) ... but then: I seriously doubt that CNRI is going to let you put 2.0 under a different license text :-( ... > > Some comments on the new version: > > > > 2. Subject to the terms and conditions of this License Agreement, CNRI > > > hereby grants Licensee a nonexclusive, royalty-free, world-wide > > > license to reproduce, analyze, test, perform and/or display publicly, > > > prepare derivative works, distribute, and otherwise use Python 1.6b1 > > > alone or in any derivative version, provided, however, that CNRI's > > > License Agreement is retained in Python 1.6b1, alone or in any > > > derivative version prepared by Licensee. > > > > I don't think the latter (retaining the CNRI license alone) is > > possible: you always have to include the CWI license. > > Wow. I hadn't even noticed this! It seems you can prepare a > derivative version of the license. Well, maybe. I think they mean "derivative version of Python 1.6b1", but in court, the above wording could cause serious trouble for CNRI ... it seems 2.0 can reuse the CWI license after all ;-) > > > Alternately, in lieu of CNRI's License Agreement, Licensee may > > > substitute the following text (omitting the quotes): "Python 1.6, beta > > > 1, is made available subject to the terms and conditions in CNRI's > > > License Agreement. This Agreement may be located on the Internet > > > using the following unique, persistent identifier (known as a handle): > > > 1895.22/1011. This Agreement may also be obtained from a proxy server > > > on the Internet using the URL:http://hdl.handle.net/1895.22/1011". > > > > Do we really need this in the license text ? It's nice to have > > the text available on the Internet, but why add long descriptions > > about where to get it from to the license text itself ? > > I'm not happy with this either, but CNRI can put anything they like in > their license, and they seem very fond of this particular bit of > advertising for their handle system. I've never managed them to > convince them that it was unnecessary. Oh well... the above paragraph sure looks scary to a casual license reader. Also I'm not sure about the usefulness of this paragraph since the mapping of a URL to a content cannot be considered a legal binding. They would at least have to add some crypto signature of the license text to make verification of the origin possible. > > > 3. In the event Licensee prepares a derivative work that is based on > > > or incorporates Python 1.6b1or any part thereof, and wants to make the > > > derivative work available to the public as provided herein, then > > > Licensee hereby agrees to indicate in any such work the nature of the > > > modifications made to Python 1.6b1. > > > > In what way would those indications have to be made ? A patch > > or just text describing the new features ? > > Just text. Bob Kahn told me that the list of "what's new" that I > always add to a release would be fine. Ok, should be made explicit in the license though... > > What does "make available to the public" mean ? If I embed > > Python in an application and make this application available > > on the Internet for download would this fit the meaning ? > > Yes, that's why he doesn't use the word "publish" -- such an action > would not be considered publication in the sense of the copyright law > (at least not in the US, and probably not according to the Bern > convention) but it is clearly making it available to the public. Ouch. That would mean I'd have to describe all additions, i.e. the embedding application, in most details in order not to breach the terms of the CNRI license. > > What about derived work that only uses the Python language > > reference as basis for its task, e.g. new interpreters > > or compilers which can read and execute Python programs ? > > The language definition is not covered by the license at all. Only > this particular code base. Ok. > > > 4. CNRI is making Python 1.6b1 available to Licensee on an "AS IS" > > > basis. CNRI MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR > > > IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, CNRI MAKES NO AND > > > DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS > > > FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 1.6b1 WILL NOT > > > INFRINGE ANY THIRD PARTY RIGHTS. > > > > > > 5. CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE > > > SOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS > > > AS A RESULT OF USING, MODIFYING OR DISTRIBUTING PYTHON 1.6b1, OR ANY > > > DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. SOME > > > STATES DO NOT ALLOW THE LIMITATION OR EXCLUSION OF LIABILITY SO THE > > > ABOVE DISCLAIMER MAY NOT APPLY TO LICENSEE. > > > > I would make this "...SOME STATES AND COUNTRIES...". E.g. in > > Germany the above text would only be valid after an initial > > 6 month period after installation, AFAIK (this period is > > called "Gewährleistung"). Licenses from other vendors usually > > add some extra license text to limit the liability in this period > > to the carrier on which the software was received by the licensee, > > e.g. the diskettes or CDs. > > I'll mention this to Kahn. > > > > 6. This License Agreement will automatically terminate upon a material > > > breach of its terms and conditions. > > > > Immediately ? Other licenses usually include a 30-60 day period > > which allows the licensee to take actions. With the above text, > > the license will put the Python copy in question into an illegal > > state *prior* to having even been identified as conflicting with the > > license. > > Believe it or not, this is necessary to ensure GPL compatibility! An > earlier draft had 30-60 days. But the GPL doesn't, so this was deemed > incompatible. There's an easy workaround though: you fix your > compliance and download a new copy, which gives you all the same > rights again. Hmm, but what about the 100.000 copies of the embedding application that have already been downloaded -- I would have to force them to redownload the application (or even just a demo of it) in order to reestablish the lawfulness of the copy action. Not that I want to violate the license in any way, but there seem to be quite a few pitfalls in the present text, some of which are not clear at all (e.g. the paragraph 3). > > > 7. This License Agreement shall be governed by and interpreted in all > > > respects by the law of the State of Virginia, excluding conflict of > > > law provisions. Nothing in this License Agreement shall be deemed to > > > create any relationship of agency, partnership, or joint venture > > > between CNRI and Licensee. This License Agreement does not grant > > > permission to use CNRI trademarks or trade name in a trademark sense > > > to endorse or promote products or services of Licensee, or any third > > > party. > > > > Would the name "Python" be considered a trademark in the above > > sense ? > > No, Python is not a CNRI trademark. I think you or BeOpen on behalf of you should consider registering the mark before someone else does it. There are quite a few "PYTHON" marks registered, yet all refer to non- computer business. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From akuchlin@mems-exchange.org Wed Aug 2 20:57:09 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Wed, 2 Aug 2000 15:57:09 -0400 Subject: [Python-Dev] Python HOWTO project created Message-ID: <20000802155709.D28691@kronos.cnri.reston.va.us> [CC'ed to python-dev and doc-sig -- followups set to doc-sig] I've created a py-howto project on SourceForge to hold the Python HOWTO documents. http://sourceforge.net/projects/py-howto/ Currently me, Fred, Moshe, and ESR are listed as developers and have write access to CVS; if you want write access, drop me a note. Web pages and a py-howto-checkins mailing list will be coming soon, after a bit more administrative fiddling around on my part. Should I also create a py-howto-discuss list for discussing revisions, or is the doc-sig OK? Fred, what's your ruling about this? --amk From guido@beopen.com Wed Aug 2 22:54:47 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 02 Aug 2000 16:54:47 -0500 Subject: [Python-Dev] New winreg module really an improvement? In-Reply-To: Your message of "Wed, 02 Aug 2000 03:30:30 -0400." <3987CE16.DB3E72B8@prescod.net> References: <3986794E.ADBB938C@prescod.net> <200008011820.NAA30284@cj20424-a.reston1.va.home.com> <3987CE16.DB3E72B8@prescod.net> Message-ID: <200008022154.QAA04109@cj20424-a.reston1.va.home.com> OK. Fine. You say your module is great. The Windows weenies here don't want to touch it with a ten-foot pole. I'm not going to be able to dig all the way to the truth here -- I don't understand the Registry API at all. I propose that you and Mark Hammond go off-line and deal with Mark's criticism one-on-one, and come back with a compromise that you are both happy with. I don't care what the compromise is, but both of you must accept it. If you *can't* agree, or if I haven't heard from you by the time I'm ready to release 2.0b1 (say, end of August), winreg.py bites the dust. I realize that this gives Mark Hammond veto power over the module, but he's a pretty reasonable guy, *and* he knows the Registry API better than anyone. It should be possible for one of you to convince the other. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From fdrake@beopen.com Wed Aug 2 22:05:20 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Wed, 2 Aug 2000 17:05:20 -0400 (EDT) Subject: [Python-Dev] Re: [Doc-SIG] Python HOWTO project created In-Reply-To: <20000802155709.D28691@kronos.cnri.reston.va.us> References: <20000802155709.D28691@kronos.cnri.reston.va.us> Message-ID: <14728.36112.584563.516268@cj42289-a.reston1.va.home.com> Andrew Kuchling writes: > Should I also create a py-howto-discuss list for discussing revisions, > or is the doc-sig OK? Fred, what's your ruling about this? It's your project, your choice. ;) I've no problem with using the Doc-SIG for this if you like, but a separate list may be a good thing since it would have fewer distractions! -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From guido@beopen.com Wed Aug 2 23:18:26 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 02 Aug 2000 17:18:26 -0500 Subject: [Python-Dev] Still no new license -- but draft text available In-Reply-To: Your message of "Wed, 02 Aug 2000 19:22:06 +0200." <398858BE.15928F47@lemburg.com> References: <200008020409.XAA01355@cj20424-a.reston1.va.home.com> <3987E5E1.A2B20241@lemburg.com> <200008021511.KAA03049@cj20424-a.reston1.va.home.com> <398858BE.15928F47@lemburg.com> Message-ID: <200008022218.RAA04178@cj20424-a.reston1.va.home.com> [MAL] > > > Is the license on 2.0 going to look the same ? I mean we now > > > already have two seperate licenses and if BeOpen adds another > > > two or three paragraphs will end up with a license two pages > > > long. [GvR] > > Good question. We can't really keep the license the same because the > > old license is very specific to CNRI. I would personally be in favor > > of using the BSD license for 2.0. [MAL} > If that's possible, I don't think we have to argue about the > 1.6 license text at all ;-) ... but then: I seriously doubt that > CNRI is going to let you put 2.0 under a different license text :-( ... What will happen is that the licenses in effect all get concatenated in the LICENSE file. It's a drag. > > > Some comments on the new version: > > > > > > 2. Subject to the terms and conditions of this License Agreement, CNRI > > > > hereby grants Licensee a nonexclusive, royalty-free, world-wide > > > > license to reproduce, analyze, test, perform and/or display publicly, > > > > prepare derivative works, distribute, and otherwise use Python 1.6b1 > > > > alone or in any derivative version, provided, however, that CNRI's > > > > License Agreement is retained in Python 1.6b1, alone or in any > > > > derivative version prepared by Licensee. > > > > > > I don't think the latter (retaining the CNRI license alone) is > > > possible: you always have to include the CWI license. > > > > Wow. I hadn't even noticed this! It seems you can prepare a > > derivative version of the license. Well, maybe. > > I think they mean "derivative version of Python 1.6b1", but in > court, the above wording could cause serious trouble for CNRI You're right of course, I misunderstood you *and* the license. Kahn explains it this way: [Kahn] | Ok. I take the point being made. The way english works with ellipsis or | anaphoric references is to link back to the last anchor point. In the above | case, the last referent is Python 1.6b1. | | Thus, the last phrase refers to a derivative version of Python1.6b1 | prepared by Licensee. There is no permission given to make a derivative | version of the License. > ... it seems 2.0 can reuse the CWI license after all ;-) I'm not sure why you think that: 2.0 is a derivative version and is thus bound by the CNRI license as well as by the license that BeOpen adds. > > > > Alternately, in lieu of CNRI's License Agreement, Licensee may > > > > substitute the following text (omitting the quotes): "Python 1.6, beta > > > > 1, is made available subject to the terms and conditions in CNRI's > > > > License Agreement. This Agreement may be located on the Internet > > > > using the following unique, persistent identifier (known as a handle): > > > > 1895.22/1011. This Agreement may also be obtained from a proxy server > > > > on the Internet using the URL:http://hdl.handle.net/1895.22/1011". > > > > > > Do we really need this in the license text ? It's nice to have > > > the text available on the Internet, but why add long descriptions > > > about where to get it from to the license text itself ? > > > > I'm not happy with this either, but CNRI can put anything they like in > > their license, and they seem very fond of this particular bit of > > advertising for their handle system. I've never managed them to > > convince them that it was unnecessary. > > Oh well... the above paragraph sure looks scary to a casual > license reader. But it's really harmless. > Also I'm not sure about the usefulness of this paragraph since > the mapping of a URL to a content cannot be considered a > legal binding. They would at least have to add some crypto > signature of the license text to make verification of the > origin possible. Sure. Just don't worry about it. Kahn again: | They always have the option of using the full text in that case. So clearly he isn't interested in taking it out. I'd let it go. > > > > 3. In the event Licensee prepares a derivative work that is based on > > > > or incorporates Python 1.6b1or any part thereof, and wants to make the > > > > derivative work available to the public as provided herein, then > > > > Licensee hereby agrees to indicate in any such work the nature of the > > > > modifications made to Python 1.6b1. > > > > > > In what way would those indications have to be made ? A patch > > > or just text describing the new features ? > > > > Just text. Bob Kahn told me that the list of "what's new" that I > > always add to a release would be fine. > > Ok, should be made explicit in the license though... It's hard to specify this precisely -- in fact, the more precise you specify it the more scary it looks and the more likely they are to be able to find fault with the details of how you do it. In this case, I believe (and so do lawyers) that vague is good! If you write "ported to the Macintosh" and that's what you did, they can hardly argue with you, can they? > > > What does "make available to the public" mean ? If I embed > > > Python in an application and make this application available > > > on the Internet for download would this fit the meaning ? > > > > Yes, that's why he doesn't use the word "publish" -- such an action > > would not be considered publication in the sense of the copyright law > > (at least not in the US, and probably not according to the Bern > > convention) but it is clearly making it available to the public. > > Ouch. That would mean I'd have to describe all additions, > i.e. the embedding application, in most details in order not to > breach the terms of the CNRI license. No, additional modules aren't modifications to CNRI's work. A change to the syntax to support curly braces is. > > > > 4. CNRI is making Python 1.6b1 available to Licensee on an "AS IS" > > > > basis. CNRI MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR > > > > IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, CNRI MAKES NO AND > > > > DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS > > > > FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 1.6b1 WILL NOT > > > > INFRINGE ANY THIRD PARTY RIGHTS. > > > > > > > > 5. CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE > > > > SOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS > > > > AS A RESULT OF USING, MODIFYING OR DISTRIBUTING PYTHON 1.6b1, OR ANY > > > > DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. SOME > > > > STATES DO NOT ALLOW THE LIMITATION OR EXCLUSION OF LIABILITY SO THE > > > > ABOVE DISCLAIMER MAY NOT APPLY TO LICENSEE. > > > > > > I would make this "...SOME STATES AND COUNTRIES...". E.g. in > > > Germany the above text would only be valid after an initial > > > 6 month period after installation, AFAIK (this period is > > > called "Gewährleistung"). Licenses from other vendors usually > > > add some extra license text to limit the liability in this period > > > to the carrier on which the software was received by the licensee, > > > e.g. the diskettes or CDs. > > > > I'll mention this to Kahn. His response: | Guido, Im not willing to do a study of international law here. If you | can have the person identify one country other than the US that does | not allow the above limitation or exclusion of liability and provide a | copy of the section of their law, ill be happy to change this to read | ".... SOME STATES OR COUNTRIES MAY NOT ALLOW ...." Otherwise, id just | leave it alone (i.e. as is) for now. Please mail this info directly to Kahn@CNRI.Reston.Va.US if you believe you have the right information. (You may CC me.) Personally, I wouldn't worry. If the German law says that part of a license is illegal, it doesn't make it any more or less illegal whether the license warns you about this fact. I believe that in the US, as a form of consumer protection, some states not only disallow general disclaimers, but also require that licenses containing such disclaimers notify the reader that the disclaimer is not valid in their state, so that's where the language comes from. I don't know about German law. > > > > 6. This License Agreement will automatically terminate upon a material > > > > breach of its terms and conditions. > > > > > > Immediately ? Other licenses usually include a 30-60 day period > > > which allows the licensee to take actions. With the above text, > > > the license will put the Python copy in question into an illegal > > > state *prior* to having even been identified as conflicting with the > > > license. > > > > Believe it or not, this is necessary to ensure GPL compatibility! An > > earlier draft had 30-60 days. But the GPL doesn't, so this was deemed > > incompatible. There's an easy workaround though: you fix your > > compliance and download a new copy, which gives you all the same > > rights again. > > Hmm, but what about the 100.000 copies of the embedding application > that have already been downloaded -- I would have to force them > to redownload the application (or even just a demo of it) in > order to reestablish the lawfulness of the copy action. It's better not to violate the license. But do you really think that they would go after you immediately if you show good intentions to rectify? > Not that I want to violate the license in any way, but there > seem to be quite a few pitfalls in the present text, some of > which are not clear at all (e.g. the paragraph 3). I've warned Kahn about this effect of making the license bigger, but he simply disagrees (and we agree to disagree). I don't know what else I could do about it, apart from putting a FAQ about the license on python.org -- which I intend to do. > > > > 7. This License Agreement shall be governed by and interpreted in all > > > > respects by the law of the State of Virginia, excluding conflict of > > > > law provisions. Nothing in this License Agreement shall be deemed to > > > > create any relationship of agency, partnership, or joint venture > > > > between CNRI and Licensee. This License Agreement does not grant > > > > permission to use CNRI trademarks or trade name in a trademark sense > > > > to endorse or promote products or services of Licensee, or any third > > > > party. > > > > > > Would the name "Python" be considered a trademark in the above > > > sense ? > > > > No, Python is not a CNRI trademark. > > I think you or BeOpen on behalf of you should consider > registering the mark before someone else does it. There are > quite a few "PYTHON" marks registered, yet all refer to non- > computer business. Yes, I do intend to do this. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From Fredrik Lundh" Guido asked me to update my old SRE benchmarks, and post them to python-dev. Summary: -- SRE is usually faster than the old RE module (PRE). -- SRE is faster than REGEX on anything but very trivial patterns and short target strings. And in some cases, it's even faster than a corresponding string.find... -- on real-life benchmarks like XML parsing and Python tokenizing, SRE is 2-3 times faster than PRE. -- using Unicode strings instead of 8-bit strings doesn't hurt performance (for some tests, the Unicode version is 30-40% faster on my machine. Go figure...) -- PRE is still faster for some patterns, especially when using long target strings. I know why, and I plan to fix that before 2.0 final. enjoy /F -------------------------------------------------------------------- These tests were made on a P3/233 MHz running Windows 95, using a local build of the 0.9.8 release (this will go into 1.6b1, I suppose). -------------------------------------------------------------------- parsing xml: running xmllib.py on hamlet.xml (280k): sre8 7.14 seconds sre16 7.82 seconds pre 17.17 seconds (for the sre16 test, the xml file was converted to unicode before it was fed to the unmodified parser). for comparision, here's the results for a couple of fast pure-Python parsers: rex/pre 2.44 seconds rex/sre 0.59 seconds srex/sre 0.16 seconds (rex is a shallow XML parser, based on code by Robert Cameron. srex is an even simpler shallow parser, using sre's template mode). -------------------------------------------------------------------- parsing python: running tokenize.py on Tkinter.py (156k): sre8 3.23 seconds pre 7.57 seconds -------------------------------------------------------------------- searching for literal text: searching for "spam" in a string padded with "spaz" (1000 bytes on each side of the target): string.find 0.112 ms sre8.search 0.059 pre.search 0.122 unicode.find 0.130 sre16.search 0.065 (yes, regular expressions can run faster than optimized C code -- as long as we don't take compilation time into account ;-) same test, without any false matches: string.find 0.035 ms sre8.search 0.050 pre.search 0.116 unicode.find 0.031 sre16.search 0.055 -------------------------------------------------------------------- compiling regular expressions compiling the 480 tests in the standard test suite: sre 1.22 seconds pre 0.05 seconds or in other words, pre (using a compiler written in C) can compile just under 10,000 patterns per second. sre can only compile about 400 pattern per second. do we care? ;-) (footnote: sre's pattern cache stores 100 patterns. pre's cache hold 20 patterns, iirc). -------------------------------------------------------------------- benchmark suite to round off this report, here's a couple of "micro benchmarks". all times are in milliseconds. n=3D 0 5 50 250 1000 5000 ----- ----- ----- ----- ----- ----- ----- pattern 'Python|Perl', string '-'*n+'Perl'+'-'*n sre8 0.014 0.013 0.013 0.016 0.027 0.079 sre16 0.014 0.014 0.015 0.018 0.025 0.076 pre 0.107 0.109 0.114 0.116 0.135 0.259 regex 0.011 0.011 0.012 0.016 0.033 0.122 pattern 'Python|Perl', string 'P'*n+'Perl'+'P'*n sre8 0.013 0.016 0.030 0.100 0.358 1.716 sre16 0.014 0.015 0.030 0.094 0.347 1.649 pre 0.115 0.112 0.158 0.351 1.085 5.002 regex 0.010 0.016 0.060 0.271 1.022 5.162 (false matches causes problems for pre and regex) pattern '(Python|Perl)', string '-'*n+'Perl'+'-'*n sre8 0.014 0.016 0.030 0.099 0.362 1.684 sre16 0.015 0.016 0.030 0.094 0.340 1.623 pre 0.110 0.111 0.112 0.119 0.143 0.267 regex 0.012 0.012 0.013 0.017 0.034 0.124 (in 0.9.8, sre's optimizer doesn't grok named groups, and it doesn't realize that this pattern has to start with a "P") pattern '(?:Python|Perl)', string '-'*n+'Perl'+'-'*n sre8 0.013 0.013 0.014 0.016 0.027 0.079 sre16 0.015 0.014 0.016 0.018 0.026 0.075 pre 0.108 0.135 0.113 0.137 0.140 0.275 regex skip (anonymous groups work better) pattern 'Python', string '-'*n+'Python'+'-'*n sre8 0.013 0.013 0.014 0.019 0.039 0.148 sre16 0.013 0.013 0.014 0.020 0.043 0.187 pre 0.129 0.105 0.109 0.117 0.191 0.277 regex 0.011 0.025 0.018 0.016 0.037 0.127 pattern 'Python', string 'P'*n+'Python'+'P'*n sre8 0.040 0.012 0.021 0.026 0.080 0.248 sre16 0.012 0.013 0.015 0.025 0.061 0.283 pre 0.110 0.148 0.153 0.338 0.925 4.355 regex 0.013 0.013 0.041 0.155 0.535 2.628 (as we saw in the string.find test, sre is very fast when there are lots of false matches) pattern '.*Python', string '-'*n+'Python'+'-'*n sre8 0.016 0.017 0.026 0.067 0.217 1.039 sre16 0.016 0.017 0.026 0.067 0.218 1.076 pre 0.111 0.112 0.124 0.180 0.386 1.494 regex 0.015 0.022 0.073 0.408 1.669 8.489 pattern '.*Python.*', string '-'*n+'Python'+'-'*n sre8 0.016 0.017 0.030 0.089 0.315 1.499 sre16 0.016 0.018 0.032 0.090 0.314 1.537 pre 0.112 0.113 0.129 0.186 0.413 1.605 regex 0.016 0.023 0.076 0.387 1.674 8.519 pattern '.*(Python)', string '-'*n+'Python'+'-'*n sre8 0.020 0.021 0.044 0.147 0.542 2.630 sre16 0.019 0.021 0.044 0.154 0.541 2.681 pre 0.115 0.117 0.141 0.245 0.636 2.690 regex 0.019 0.026 0.097 0.467 2.007 10.264 pattern '.*(?:Python)', string '-'*n+'Python'+'-'*n sre8 0.016 0.017 0.027 0.065 0.220 1.037 sre16 0.016 0.017 0.026 0.070 0.221 1.066 pre 0.112 0.119 0.136 0.223 0.566 2.377 regex skip pattern 'Python|Perl|Tcl', string '-'*n+'Perl'+'-'*n sre8 0.013 0.015 0.034 0.114 0.407 1.985 sre16 0.014 0.016 0.034 0.109 0.392 1.915 pre 0.107 0.108 0.117 0.124 0.167 0.393 regex 0.012 0.012 0.013 0.017 0.033 0.123 (here's another sre compiler problem: it fails to realize that this pattern starts with characters from a given set [PT]. pre and regex both use bitmaps...) pattern 'Python|Perl|Tcl', string 'P'*n+'Perl'+'P'*n sre8 0.013 0.018 0.055 0.228 0.847 4.165 sre16 0.015 0.027 0.055 0.218 0.821 4.061 pre 0.111 0.116 0.172 0.415 1.354 6.302 regex 0.011 0.019 0.085 0.374 1.467 7.261 (but when there are lots of false matches, sre is faster anyway. interesting...) pattern '(Python|Perl|Tcl)', string '-'*n+'Perl'+'-'*n sre8 0.014 0.018 0.042 0.152 0.575 2.798 sre16 0.015 0.019 0.042 0.148 0.556 2.715 pre 0.112 0.111 0.116 0.129 0.172 0.408 regex 0.012 0.013 0.014 0.018 0.035 0.124 pattern '(?:Python|Perl|Tcl)', string '-'*n+'Perl'+'-'*n sre8 0.014 0.016 0.034 0.113 0.405 1.987 sre16 0.016 0.016 0.033 0.112 0.393 1.918 pre 0.109 0.109 0.112 0.128 0.177 0.397 regex skip pattern '(Python)\\1', string '-'*n+'PythonPython'+'-'*n sre8 0.014 0.018 0.030 0.096 0.342 1.673 sre16 0.015 0.016 0.031 0.094 0.330 1.625 pre 0.112 0.111 0.112 0.119 0.141 0.268 regex 0.011 0.012 0.013 0.017 0.033 0.123 pattern '(Python)\\1', string 'P'*n+'PythonPython'+'P'*n sre8 0.013 0.016 0.035 0.111 0.411 1.976 sre16 0.015 0.016 0.034 0.112 0.416 1.992 pre 0.110 0.116 0.160 0.355 1.051 4.797 regex 0.011 0.017 0.047 0.200 0.737 3.680 pattern '([0a-z][a-z0-9]*,)+', string '-'*n+'a5,b7,c9,'+'-'*n sre8 0.084 0.091 0.143 0.371 1.160 6.165 sre16 0.086 0.090 0.142 0.470 1.258 7.827 pre 0.155 0.140 0.185 0.200 0.280 0.523 regex 0.018 0.018 0.020 0.024 0.137 0.240 (again, sre's lack of "fastmap" is rather costly) pattern '(?:[0a-z][a-z0-9]*,)+', string '-'*n+'a5,b7,c9,'+'-'*n sre8 0.028 0.033 0.077 0.303 1.433 7.140 sre16 0.021 0.027 0.073 0.277 1.031 5.053 pre 0.131 0.131 0.174 0.183 0.227 0.461 regex skip pattern '([a-z][a-z0-9]*,)+', string '-'*n+'a5,b7,c9,'+'-'*n sre8 0.032 0.038 0.083 0.288 1.109 5.404 sre16 0.033 0.038 0.083 0.292 1.035 5.802 pre 0.195 0.135 0.176 0.187 0.233 0.468 regex 0.018 0.018 0.019 0.023 0.041 0.131 pattern '(?:[a-z][a-z0-9]*,)+', string '-'*n+'a5,b7,c9,'+'-'*n sre8 0.022 0.025 0.067 0.302 1.011 8.245 sre16 0.021 0.026 0.066 0.302 1.103 5.372 pre 0.262 0.397 0.178 0.193 0.250 0.817 regex skip pattern '.*P.*y.*t.*h.*o.*n.*', string '-'*n+'Python'+'-'*n sre8 0.021 0.084 0.118 0.251 0.965 5.414 sre16 0.021 0.025 0.063 0.366 1.192 4.639 pre 0.123 0.147 0.225 0.568 1.899 9.336 regex 0.028 0.060 0.258 1.269 5.497 28.334 -------------------------------------------------------------------- From bwarsaw@beopen.com Wed Aug 2 22:40:59 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Wed, 2 Aug 2000 17:40:59 -0400 (EDT) Subject: [Python-Dev] Cookies.py in the core (was Tangent to Re: [Tutor] CGI and Python (fwd)) References: <200008021406.JAA02743@cj20424-a.reston1.va.home.com> Message-ID: <14728.38251.289986.857417@anthem.concentric.net> >>>>> "GvR" == Guido van Rossum writes: >> Do we have a procedure for putting more batteries in the core? >> I'm not talking about stuff like PEP-206, I'm talking about >> small, useful modules like Cookies.py. GvR> Cookie support in the core would be a good thing. I use Tim O'Malley's LGPL'd version (not as contagious as GPL'd) in Mailman with one important patch. I've uploaded it to SF as patch #101055. If you like it, I'm happy to check it in. -Barry From bwarsaw@beopen.com Wed Aug 2 22:42:26 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Wed, 2 Aug 2000 17:42:26 -0400 (EDT) Subject: [Python-Dev] Cookies.py in the core (was Tangent to Re: [Tutor] CGI and Python (fwd)) References: <200008021406.JAA02743@cj20424-a.reston1.va.home.com> <14728.8580.460583.760620@cj42289-a.reston1.va.home.com> <200008021432.JAA02937@cj20424-a.reston1.va.home.com> Message-ID: <14728.38338.92481.102493@anthem.concentric.net> >>>>> "GvR" == Guido van Rossum writes: GvR> I think Cookie.py is for server-side management of cookies, GvR> not for client-side. Do we need client-side cookies too???? Ah. AFAIK, Tim's Cookie.py is server side only. Still very useful -- and already written! -Barry From guido@beopen.com Wed Aug 2 23:44:03 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 02 Aug 2000 17:44:03 -0500 Subject: [Python-Dev] SRE 0.9.8 benchmarks In-Reply-To: Your message of "Wed, 02 Aug 2000 23:37:52 +0200." <015a01bffcc9$ed942bc0$f2a6b5d4@hagrid> References: <015a01bffcc9$ed942bc0$f2a6b5d4@hagrid> Message-ID: <200008022244.RAA04388@cj20424-a.reston1.va.home.com> > Guido asked me to update my old SRE benchmarks, and > post them to python-dev. Thanks, Fredrik! This (plus the fact that SRE now passes all PRE tests) makes me very happy with using SRE as the regular expression engine of choice for 1.6 and 2.0. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From guido@beopen.com Wed Aug 2 23:46:35 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 02 Aug 2000 17:46:35 -0500 Subject: [Python-Dev] Cookies.py in the core (was Tangent to Re: [Tutor] CGI and Python (fwd)) In-Reply-To: Your message of "Wed, 02 Aug 2000 17:40:59 -0400." <14728.38251.289986.857417@anthem.concentric.net> References: <200008021406.JAA02743@cj20424-a.reston1.va.home.com> <14728.38251.289986.857417@anthem.concentric.net> Message-ID: <200008022246.RAA04405@cj20424-a.reston1.va.home.com> > GvR> Cookie support in the core would be a good thing. [Barry] > I use Tim O'Malley's LGPL'd version (not as contagious as GPL'd) in > Mailman with one important patch. I've uploaded it to SF as patch > #101055. If you like it, I'm happy to check it in. I don't have the time to judge this code myself, but hope that others in this group do. Are you sure it's a good thing to add LGPL'ed code to the Python standard library though? AFAIK it is still more restrictive than the old CWI license and probably also more restrictive than the new CNRI license; so it could come under scrutiny and prevent closed, proprietary software development using Python... --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From barry@scottb.demon.co.uk Wed Aug 2 22:50:43 2000 From: barry@scottb.demon.co.uk (Barry Scott) Date: Wed, 2 Aug 2000 22:50:43 +0100 Subject: [Python-Dev] Preventing 1.5 extensions crashing under 1.6/2.0 Python In-Reply-To: Message-ID: <020901bffccb$b4bf4da0$060210ac@private> This is a multi-part message in MIME format. ------=_NextPart_000_020A_01BFFCD4.1683B5A0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit > -----Original Message----- > From: Mark Hammond [mailto:MarkH@activestate.com] > Sent: 02 August 2000 00:14 > To: Barry Scott; python-dev@python.org > Subject: RE: [Python-Dev] Preventing 1.5 extensions crashing under > 1.6/2.0 Python > > > > If someone in the core of Python thinks a patch implementing > > what I've outlined is useful please let me know and I will > > generate the patch. > > Umm - I'm afraid that I dont keep my python-dev emils for that long, and > right now I'm too lazy/busy to dig around the archives. > > Exactly what did you outline? I know it went around a few times, and I > can't remember who said what. For my money, I liked Fredrik's solution > best (check Py_IsInitialized() in Py_InitModule4()), but as mentioned that > only solves for the next version of Python; it doesnt solve the fact 1.5 > modules will crash under 1.6/2.0 This is not a good way to solve the problem as it only works in a limited number of cases. Attached is my proposal which works for all new and old python and all old and new extensions. > > It would definately be excellent to get _something_ in the CNRI 1.6 > release, so the BeOpen 2.0 release can see the results. > But-I-doubt-anyone-will-release-extension-modules-for-1.6-anyway ly, Yes indeed once the story of 1.6 and 2.0 is out I expect folks will skip 1.6. For example, if your win32 stuff is not ported then Python 1.6 is not usable on Windows/NT. > > Mark. Barry ------=_NextPart_000_020A_01BFFCD4.1683B5A0 Content-Type: message/rfc822 Content-Transfer-Encoding: 7bit Content-Disposition: attachment From: "Barry Scott" Sender: To: Subject: RE: [Python-Dev] Preventing 1.5 extensions crashing under 1.6/2.0 Python Date: Tue, 18 Jul 2000 23:36:15 +0100 Message-ID: <000701bff108$950ec9f0$060210ac@private> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0 Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 X-UIDL: 5c981bb22b1a76163266dc06120e339a In-Reply-To: X-BeenThere: python-dev@python.org X-Mailman-Version: 2.0beta4 Mark's comment about what I'd come up with being too complex inspired this simpler solution. Change the init function name to a new name PythonExtensionInit_ say. Pass in the API version for the extension writer to check. If the version is bad for this extension returns without calling any python functions. Add a return code that is true if compatible, false if not. If compatible the extension can use python functions and report and problems it wishes. int PythonExtensionInit_XXX( int invoking_python_api_version ) { if( invoking_python_api_version != PYTHON_API_VERSION ) { /* python will report that the module is incompatible */ return 0; } /* setup module for XXX ... */ /* say this extension is compatible with the invoking python */ return 1; } All 1.5 extensions fail to load on later python 2.0 and later. All 2.0 extensions fail to load on python 1.5. All new extensions work only with python of the same API version. Document that failure to setup a module could mean the extension is incompatible with this version of python. Small code change in python core. But need to tell extension writers what the new interface is and update all extensions within the python CVS tree. Barry _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://www.python.org/mailman/listinfo/python-dev ------=_NextPart_000_020A_01BFFCD4.1683B5A0-- From akuchlin@mems-exchange.org Wed Aug 2 22:55:53 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Wed, 2 Aug 2000 17:55:53 -0400 Subject: [Python-Dev] Cookies.py in the core In-Reply-To: <200008022246.RAA04405@cj20424-a.reston1.va.home.com>; from guido@beopen.com on Wed, Aug 02, 2000 at 05:46:35PM -0500 References: <200008021406.JAA02743@cj20424-a.reston1.va.home.com> <14728.38251.289986.857417@anthem.concentric.net> <200008022246.RAA04405@cj20424-a.reston1.va.home.com> Message-ID: <20000802175553.A30340@kronos.cnri.reston.va.us> On Wed, Aug 02, 2000 at 05:46:35PM -0500, Guido van Rossum wrote: >Are you sure it's a good thing to add LGPL'ed code to the Python >standard library though? AFAIK ... it could come under scrutiny and >prevent closed, proprietary software development using Python... Licence discussions are a conversational black hole... Why not just ask Tim O'Malley to change the licence in return for getting it added to the core? --amk From akuchlin@mems-exchange.org Wed Aug 2 23:00:59 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Wed, 2 Aug 2000 18:00:59 -0400 Subject: [Python-Dev] SRE 0.9.8 benchmarks In-Reply-To: <015a01bffcc9$ed942bc0$f2a6b5d4@hagrid>; from effbot@telia.com on Wed, Aug 02, 2000 at 11:37:52PM +0200 References: <015a01bffcc9$ed942bc0$f2a6b5d4@hagrid> Message-ID: <20000802180059.B30340@kronos.cnri.reston.va.us> On Wed, Aug 02, 2000 at 11:37:52PM +0200, Fredrik Lundh wrote: >-- SRE is usually faster than the old RE module (PRE). Once the compiler is translated to C, it might be worth considering making SRE available as a standalone library for use outside of Python. Most other regex libraries either don't do Perl's extensions, or they don't do Unicode. Bonus points if you can get the Perl6 team interested in it. Hmm... here's an old problem that's returned (recursion on repeated group matches, I expect): >>> p=re.compile('(x)*') >>> p >>> p.match(500000*'x') Segmentation fault (core dumped) --amk From guido@beopen.com Thu Aug 3 00:10:14 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 02 Aug 2000 18:10:14 -0500 Subject: [Python-Dev] Cookies.py in the core In-Reply-To: Your message of "Wed, 02 Aug 2000 17:55:53 -0400." <20000802175553.A30340@kronos.cnri.reston.va.us> References: <200008021406.JAA02743@cj20424-a.reston1.va.home.com> <14728.38251.289986.857417@anthem.concentric.net> <200008022246.RAA04405@cj20424-a.reston1.va.home.com> <20000802175553.A30340@kronos.cnri.reston.va.us> Message-ID: <200008022310.SAA04508@cj20424-a.reston1.va.home.com> From guido@beopen.com Thu Aug 3 00:10:33 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 02 Aug 2000 18:10:33 -0500 Subject: [Python-Dev] Cookies.py in the core In-Reply-To: Your message of "Wed, 02 Aug 2000 17:55:53 -0400." <20000802175553.A30340@kronos.cnri.reston.va.us> References: <200008021406.JAA02743@cj20424-a.reston1.va.home.com> <14728.38251.289986.857417@anthem.concentric.net> <200008022246.RAA04405@cj20424-a.reston1.va.home.com> <20000802175553.A30340@kronos.cnri.reston.va.us> Message-ID: <200008022310.SAA04518@cj20424-a.reston1.va.home.com> > Licence discussions are a conversational black hole... Why not just > ask Tim O'Malley to change the licence in return for getting it added > to the core? Excellent idea. Go for it! --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From guido@beopen.com Thu Aug 3 00:11:39 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 02 Aug 2000 18:11:39 -0500 Subject: [Python-Dev] SRE 0.9.8 benchmarks In-Reply-To: Your message of "Wed, 02 Aug 2000 18:00:59 -0400." <20000802180059.B30340@kronos.cnri.reston.va.us> References: <015a01bffcc9$ed942bc0$f2a6b5d4@hagrid> <20000802180059.B30340@kronos.cnri.reston.va.us> Message-ID: <200008022311.SAA04529@cj20424-a.reston1.va.home.com> > Hmm... here's an old problem that's returned (recursion on repeated > group matches, I expect): > > >>> p=re.compile('(x)*') > >>> p > > >>> p.match(500000*'x') > Segmentation fault (core dumped) Ouch. Andrew, would you mind adding a test case for that to the re test suite? It's important that this doesn't come back! --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From guido@beopen.com Thu Aug 3 00:18:04 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 02 Aug 2000 18:18:04 -0500 Subject: [Python-Dev] Preventing 1.5 extensions crashing under 1.6/2.0 Python In-Reply-To: Your message of "Wed, 02 Aug 2000 22:50:43 +0100." <020901bffccb$b4bf4da0$060210ac@private> References: <020901bffccb$b4bf4da0$060210ac@private> Message-ID: <200008022318.SAA04558@cj20424-a.reston1.va.home.com> > > But-I-doubt-anyone-will-release-extension-modules-for-1.6-anyway ly, > > Yes indeed once the story of 1.6 and 2.0 is out I expect folks > will skip 1.6. For example, if your win32 stuff is not ported > then Python 1.6 is not usable on Windows/NT. I expect to be releasing a 1.6 Windows installer -- but I can't control Mark Hammond. Yet, it shouldn't be hard for him to create a 1.6 version of win32all, should it? > Change the init function name to a new name PythonExtensionInit_ say. > Pass in the API version for the extension writer to check. If the > version is bad for this extension returns without calling any python > functions. Add a return code that is true if compatible, false if not. > If compatible the extension can use python functions and report and > problems it wishes. > > int PythonExtensionInit_XXX( int invoking_python_api_version ) > { > if( invoking_python_api_version != PYTHON_API_VERSION ) > { > /* python will report that the module is incompatible */ > return 0; > } > > /* setup module for XXX ... */ > > /* say this extension is compatible with the invoking python */ > return 1; > } > > All 1.5 extensions fail to load on later python 2.0 and later. > All 2.0 extensions fail to load on python 1.5. > > All new extensions work only with python of the same API version. > > Document that failure to setup a module could mean the extension is > incompatible with this version of python. > > Small code change in python core. But need to tell extension writers > what the new interface is and update all extensions within the python > CVS tree. I sort-of like this idea -- at least at the +0 level. I would choose a shorter name: PyExtInit_XXX(). Could you (or someone else) prepare a patch that changes this? It would be great if the patch were relative to the 1.6 branch of the source tree; unfortunately this is different because of the ANSIfication. Unfortunately we only have two days to get this done for 1.6 -- I plan to release 1.6b1 this Friday! If you don't get to it, prepare a patch for 2.0 would be the next best thing. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From Fredrik Lundh" <200008022318.SAA04558@cj20424-a.reston1.va.home.com> Message-ID: <01d701bffcd7$46a74a00$f2a6b5d4@hagrid> > Yes indeed once the story of 1.6 and 2.0 is out I expect folks > will skip 1.6. For example, if your win32 stuff is not ported then > Python 1.6 is not usable on Windows/NT. "not usable"? guess you haven't done much cross-platform development lately... > Change the init function name to a new name PythonExtensionInit_ say. > Pass in the API version for the extension writer to check. If the > version is bad for this extension returns without calling any python huh? are you seriously proposing to break every single C extension ever written -- on each and every platform -- just to trap an error message caused by extensions linked against 1.5.2 on your favourite platform? > Small code change in python core. But need to tell extension writers > what the new interface is and update all extensions within the python > CVS tree. you mean "update the source code for all extensions ever written." -1 From DavidA@ActiveState.com Thu Aug 3 01:33:02 2000 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 2 Aug 2000 17:33:02 -0700 (Pacific Daylight Time) Subject: [Python-Dev] Fork on Win32 - was (test_fork1 failing...) In-Reply-To: <013901bff821$55dd02e0$8119fea9@neil> Message-ID: > IIRC ActiveState contributed to Perl a version of fork that works on > Win32. Has anyone looked at this? Could it be grabbed for Python? This would > help heal one of the more difficult platform rifts. Emulating fork for Win32 > looks quite difficult to me but if its already done... I've talked to Sarathy about it, and it's messy, as Perl manages PIDs above and beyond what Windows does, among other things. If anyone is interested in doing that work, I can make the introduction. --david From DavidA@ActiveState.com Thu Aug 3 01:35:01 2000 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 2 Aug 2000 17:35:01 -0700 (Pacific Daylight Time) Subject: [Python-Dev] Fork on Win32 - was (test_fork1 failing...) In-Reply-To: <013901bff821$55dd02e0$8119fea9@neil> Message-ID: > IIRC ActiveState contributed to Perl a version of fork that works on > Win32. Has anyone looked at this? Could it be grabbed for Python? This would > help heal one of the more difficult platform rifts. Emulating fork for Win32 > looks quite difficult to me but if its already done... Sigh. Me tired. The message I posted a few minutes ago was actually referring to the system() work, not the fork() work. I agree that the fork() emulation isn't Pythonic. --david From skip@mojam.com (Skip Montanaro) Thu Aug 3 03:32:29 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 2 Aug 2000 21:32:29 -0500 (CDT) Subject: [Python-Dev] METH_VARARGS Message-ID: <14728.55741.477399.196240@beluga.mojam.com> I noticed Andrew Kuchling's METH_VARARGS submission: Use METH_VARARGS instead of numeric constant 1 in method def. tables While METH_VARARGS is obviously a lot better than a hardcoded 1, shouldn't METH_VARARGS be something like Py_METH_VARARGS or PY_METH_VARARGS to avoid potential conflicts with other packages? Skip From akuchlin@mems-exchange.org Thu Aug 3 03:41:02 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Wed, 2 Aug 2000 22:41:02 -0400 Subject: [Python-Dev] Cookies.py in the core In-Reply-To: <200008022310.SAA04518@cj20424-a.reston1.va.home.com>; from guido@beopen.com on Wed, Aug 02, 2000 at 06:10:33PM -0500 References: <200008021406.JAA02743@cj20424-a.reston1.va.home.com> <14728.38251.289986.857417@anthem.concentric.net> <200008022246.RAA04405@cj20424-a.reston1.va.home.com> <20000802175553.A30340@kronos.cnri.reston.va.us> <200008022310.SAA04518@cj20424-a.reston1.va.home.com> Message-ID: <20000802224102.A25837@newcnri.cnri.reston.va.us> On Wed, Aug 02, 2000 at 06:10:33PM -0500, Guido van Rossum wrote: >> Why not just >> ask Tim O'Malley to change the licence in return for getting it added >> to the core? >Excellent idea. Go for it! Mail to timo@bbn.com bounces; does anyone have a more recent e-mail address? What we do if he can't be located? Add the module anyway, abandon the idea, or write a new version? --amk From fdrake@beopen.com Thu Aug 3 03:51:23 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Wed, 2 Aug 2000 22:51:23 -0400 (EDT) Subject: [Python-Dev] METH_VARARGS In-Reply-To: <14728.55741.477399.196240@beluga.mojam.com> References: <14728.55741.477399.196240@beluga.mojam.com> Message-ID: <14728.56875.996310.790872@cj42289-a.reston1.va.home.com> Skip Montanaro writes: > While METH_VARARGS is obviously a lot better than a hardcoded 1, shouldn't > METH_VARARGS be something like Py_METH_VARARGS or PY_METH_VARARGS to avoid > potential conflicts with other packages? I think so, but there are too many third party extension modules that would have to be changed to not also offer the old symbols as well. I see two options: leave things as they are, and provide both versions of the symbols through at least Python 2.1. For the later, all examples in the code and documentation would need to be changed and the non-PY_ versions strongly labelled as deprecated and going away in Python version 2.2 (or whatever version it would be). It would *not* hurt to provide both symbols and change all the examples, at any rate. Aside from deleting all the checkin email, that is! -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From gstein@lyra.org Thu Aug 3 04:06:20 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 2 Aug 2000 20:06:20 -0700 Subject: [Python-Dev] Cookies.py in the core In-Reply-To: <20000802224102.A25837@newcnri.cnri.reston.va.us>; from akuchlin@cnri.reston.va.us on Wed, Aug 02, 2000 at 10:41:02PM -0400 References: <200008021406.JAA02743@cj20424-a.reston1.va.home.com> <14728.38251.289986.857417@anthem.concentric.net> <200008022246.RAA04405@cj20424-a.reston1.va.home.com> <20000802175553.A30340@kronos.cnri.reston.va.us> <200008022310.SAA04518@cj20424-a.reston1.va.home.com> <20000802224102.A25837@newcnri.cnri.reston.va.us> Message-ID: <20000802200620.G19525@lyra.org> On Wed, Aug 02, 2000 at 10:41:02PM -0400, Andrew Kuchling wrote: > On Wed, Aug 02, 2000 at 06:10:33PM -0500, Guido van Rossum wrote: > >> Why not just > >> ask Tim O'Malley to change the licence in return for getting it added > >> to the core? > >Excellent idea. Go for it! > > Mail to timo@bbn.com bounces; does anyone have a more recent e-mail > address? What we do if he can't be located? Add the module anyway, > abandon the idea, or write a new version? If we can't contact him, then I'd be quite happy to assist in designing and writing a new one under a BSD-ish or Public Domain license. I was considering doing exactly that just last week :-) [ I want to start using cookies in ViewCVS; while the LGPL is "fine" for me, it would be nice if the whole ViewCVS package was BSD-ish ] Of course, I'd much rather get a hold of Tim. Cheers, -g -- Greg Stein, http://www.lyra.org/ From bwarsaw@beopen.com Thu Aug 3 05:11:59 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Thu, 3 Aug 2000 00:11:59 -0400 (EDT) Subject: [Python-Dev] Cookies.py in the core (was Tangent to Re: [Tutor] CGI and Python (fwd)) References: <200008021406.JAA02743@cj20424-a.reston1.va.home.com> <14728.38251.289986.857417@anthem.concentric.net> <200008022246.RAA04405@cj20424-a.reston1.va.home.com> Message-ID: <14728.61711.859894.972939@anthem.concentric.net> >>>>> "GvR" == Guido van Rossum writes: GvR> Are you sure it's a good thing to add LGPL'ed code to the GvR> Python standard library though? AFAIK it is still more GvR> restrictive than the old CWI license and probably also more GvR> restrictive than the new CNRI license; so it could come under GvR> scrutiny and prevent closed, proprietary software development GvR> using Python... I don't know, however I have a version of the file with essentially no license on it: # Id: Cookie.py,v 2.4 1998/02/13 16:42:30 timo Exp # by Timothy O'Malley Date: 1998/02/13 16:42:30 # # Cookie.py is an update for the old nscookie.py module. # Under the old module, it was not possible to set attributes, # such as "secure" or "Max-Age" on key,value granularity. This # shortcoming has been addressed in Cookie.py but has come at # the cost of a slightly changed interface. Cookie.py also # requires Python-1.5, for the re and cPickle modules. # # The original idea to treat Cookies as a dictionary came from # Dave Mitchel (davem@magnet.com) in 1995, when he released the # first version of nscookie.py. Is that better or worse? . Back in '98, I actually asked him to send me an LGPL'd copy because that worked better for Mailman. We could start with Tim's pre-LGPL'd version and backport the minor mods I've made. BTW, I've recently tried to contact Tim, but the address in the file bounces. -Barry From guido@beopen.com Thu Aug 3 05:39:47 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 02 Aug 2000 23:39:47 -0500 Subject: [Python-Dev] METH_VARARGS In-Reply-To: Your message of "Wed, 02 Aug 2000 21:32:29 EST." <14728.55741.477399.196240@beluga.mojam.com> References: <14728.55741.477399.196240@beluga.mojam.com> Message-ID: <200008030439.XAA05445@cj20424-a.reston1.va.home.com> > While METH_VARARGS is obviously a lot better than a hardcoded 1, shouldn't > METH_VARARGS be something like Py_METH_VARARGS or PY_METH_VARARGS to avoid > potential conflicts with other packages? Unless someone knows of a *real* conflict, I'd leave this one alone. Yes, it should be Py_*, but no, it's not worth the effort of changing all that. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From gstein@lyra.org Thu Aug 3 05:26:48 2000 From: gstein@lyra.org (Greg Stein) Date: Wed, 2 Aug 2000 21:26:48 -0700 Subject: [Python-Dev] Cookies.py in the core (was Tangent to Re: [Tutor] CGI and Python (fwd)) In-Reply-To: <14728.61711.859894.972939@anthem.concentric.net>; from bwarsaw@beopen.com on Thu, Aug 03, 2000 at 12:11:59AM -0400 References: <200008021406.JAA02743@cj20424-a.reston1.va.home.com> <14728.38251.289986.857417@anthem.concentric.net> <200008022246.RAA04405@cj20424-a.reston1.va.home.com> <14728.61711.859894.972939@anthem.concentric.net> Message-ID: <20000802212648.J19525@lyra.org> On Thu, Aug 03, 2000 at 12:11:59AM -0400, Barry A. Warsaw wrote: >... > I don't know, however I have a version of the file with essentially no > license on it: That implies "no license" which means "no rights to redistribute, use, or whatever." Very incompatible :-) >... > Is that better or worse? . Back in '98, I actually asked him to > send me an LGPL'd copy because that worked better for Mailman. We > could start with Tim's pre-LGPL'd version and backport the minor mods > I've made. Wouldn't help. We need a relicensed version, to use the LGPL'd version, or to rebuild it from scratch. > BTW, I've recently tried to contact Tim, but the address in the file > bounces. I just sent mail to Andrew Smith who has been working with Tim for several years on various projects (RSVP, RAP, etc). Hopefully, he has a current email address for Tim. I'll report back when I hear something from Andrew. Of course, if somebody else can track him down faster... Cheers, -g -- Greg Stein, http://www.lyra.org/ From bwarsaw@beopen.com Thu Aug 3 05:41:14 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Thu, 3 Aug 2000 00:41:14 -0400 (EDT) Subject: [Python-Dev] Re: A small proposed change to dictionaries' "get" method References: Message-ID: <14728.63466.263123.434708@anthem.concentric.net> >>>>> "GM" == Gareth McCaughan writes: GM> Consider the following piece of code, which takes a file GM> and prepares a concordance saying on which lines each word GM> in the file appears. (For real use it would need to be GM> made more sophisticated.) | line_number = 0 | for line in open(filename).readlines(): | line_number = line_number+1 | for word in map(string.lower, string.split(line)): | existing_lines = word2lines.get(word, []) | | existing_lines.append(line_number) | ugh! | word2lines[word] = existing_lines | I've run into this same situation many times myself. I agree it's annoying. Annoying enough to warrant a change? Maybe -- I'm not sure. GM> I suggest a minor change: another optional argument to GM> "get" so that GM> dict.get(item,default,flag) Good idea, not so good solution. Let's make it more explicit by adding a new method instead of a flag. I'll use `put' here since this seems (in a sense) opposite of get() and my sleep addled brain can't think of anything more clever. Let's not argue about the name of this method though -- if Guido likes the extension, he'll pick a good name and I go on record as agreeing with his name choice, just to avoid a protracted war. A trivial patch to UserDict (see below) will let you play with this. >>> d = UserDict() >>> word = 'hello' >>> d.get(word, []) [] >>> d.put(word, []).append('world') >>> d.get(word) ['world'] >>> d.put(word, []).append('gareth') >>> d.get(word) ['world', 'gareth'] Shouldn't be too hard to add equivalent C code to the dictionary object. -Barry -------------------- snip snip -------------------- Index: UserDict.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/UserDict.py,v retrieving revision 1.7 diff -u -r1.7 UserDict.py --- UserDict.py 2000/02/02 15:10:14 1.7 +++ UserDict.py 2000/08/03 04:35:11 @@ -34,3 +34,7 @@ self.data[k] = v def get(self, key, failobj=None): return self.data.get(key, failobj) + def put(self, key, failobj=None): + if not self.data.has_key(key): + self.data[key] = failobj + return self.data[key] From bwarsaw@beopen.com Thu Aug 3 05:45:33 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Thu, 3 Aug 2000 00:45:33 -0400 (EDT) Subject: [Python-Dev] Cookies.py in the core References: <200008021406.JAA02743@cj20424-a.reston1.va.home.com> <14728.38251.289986.857417@anthem.concentric.net> <200008022246.RAA04405@cj20424-a.reston1.va.home.com> <20000802175553.A30340@kronos.cnri.reston.va.us> <200008022310.SAA04518@cj20424-a.reston1.va.home.com> <20000802224102.A25837@newcnri.cnri.reston.va.us> <20000802200620.G19525@lyra.org> Message-ID: <14728.63725.390053.65213@anthem.concentric.net> >>>>> "GS" == Greg Stein writes: GS> If we can't contact him, then I'd be quite happy to assist in GS> designing and writing a new one under a BSD-ish or Public GS> Domain license. I was considering doing exactly that just last GS> week :-) I don't think that's necessary; see my other post. We should still try to contact him if possible though. My request for an LGPL'd copy was necessary because Mailman is GPL'd (and Stallman suggested this as an acceptable solution). It would be just as fine for Mailman if an un-LGPL'd Cookie.py were part of the standard Python distribution. -Barry From bwarsaw@beopen.com Thu Aug 3 05:50:51 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Thu, 3 Aug 2000 00:50:51 -0400 (EDT) Subject: [Python-Dev] Cookies.py in the core (was Tangent to Re: [Tutor] CGI and Python (fwd)) References: <200008021406.JAA02743@cj20424-a.reston1.va.home.com> <14728.38251.289986.857417@anthem.concentric.net> <200008022246.RAA04405@cj20424-a.reston1.va.home.com> <14728.61711.859894.972939@anthem.concentric.net> <20000802212648.J19525@lyra.org> Message-ID: <14728.64043.155392.32408@anthem.concentric.net> >>>>> "GS" == Greg Stein writes: GS> Wouldn't help. We need a relicensed version, to use the LGPL'd GS> version, or to rebuild it from scratch. >> BTW, I've recently tried to contact Tim, but the address in the >> file bounces. GS> I just sent mail to Andrew Smith who has been working with Tim GS> for several years on various projects (RSVP, RAP, GS> etc). Hopefully, he has a current email address for Tim. I'll GS> report back when I hear something from Andrew. Of course, if GS> somebody else can track him down faster... Cool. Tim was exceedingly helpful in giving me a version of the file I could use. I have no doubt that if we can contact him, he'll relicense it in a way that makes sense for the standard distro. That would be the best outcome. -Barry From tim_one@email.msn.com Thu Aug 3 05:52:07 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 3 Aug 2000 00:52:07 -0400 Subject: [Python-Dev] Cookies.py in the core In-Reply-To: <14728.63725.390053.65213@anthem.concentric.net> Message-ID: Guys, these are cookies, not brain surgery! If people like this API, couldn't someone have done a clean-room reimplementation of it in less time than we've spent jockeying over the freaking license? tolerance-for-license-discussions-at-an-all-time-low-ly y'rs - tim From Fredrik Lundh" <200008021406.JAA02743@cj20424-a.reston1.va.home.com> <14728.38251.289986.857417@anthem.concentric.net> <200008022246.RAA04405@cj20424-a.reston1.va.home.com> <20000802175553.A30340@kronos.cnri.reston.va.us> <200008022310.SAA04518@cj20424-a.reston1.va.home.com> <20000802224102.A25837@newcnri.cnri.reston.va.us> Message-ID: <002601bffd21$5f23e800$f2a6b5d4@hagrid> andrew wrote: > Mail to timo@bbn.com bounces; does anyone have a more recent e-mail > address? What we do if he can't be located? Add the module anyway, > abandon the idea, or write a new version? readers of the daily URL might have noticed that he posted a socket timeout wrapper a few days ago: Timothy O'Malley it's the same name and the same signature, so I assume it's the same guy ;-) From wesc@alpha.ece.ucsb.edu Thu Aug 3 08:56:59 2000 From: wesc@alpha.ece.ucsb.edu (Wesley J. Chun) Date: Thu, 3 Aug 2000 00:56:59 -0700 (PDT) Subject: [Python-Dev] Re: Bookstand at LA Python conference Message-ID: <200008030756.AAA23434@alpha.ece.ucsb.edu> > From: Guido van Rossum > Date: Sat, 29 Jul 2000 12:39:01 -0500 > > The next Python conference will be in Long Beach (Los Angeles). We're > looking for a bookstore to set up a bookstand like we had at the last > conference. Does anybody have a suggestion? the most well-known big independent technical bookstore that also does mail order and has been around for about 20 years is OpAmp: OpAmp Technical Books 1033 N. Sycamore Ave Los Angeles, CA 90038 800-468-4322 http://www.opamp.com there really isn't a "2nd place" since OpAmp owns the market, but if there was a #3, it would be Technical Book Company: Technical Book Company 2056 Westwood Blvd Los Angeles, CA 90025 800-233-5150 the above 2 stores are listed in the misc.books.technical FAQ: http://www.faqs.org/faqs/books/technical/ there's a smaller bookstore that's also known to have a good technical book selection: Scholar's Bookstore El Segundo, CA 90245 310-322-3161 (and of course, the standbys are always the university bookstores for UCLA, CalTech, UC Irvine, Cal State Long Beach, etc.) as to be expected, someone has collated a list of bookstores in the LA area: http://www.geocities.com/Athens/4824/na-la.htm hope this helps!! -wesley - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - "Core Python Programming", Prentice Hall PTR, TBP Summer/Fall 2000 http://www.phptr.com/ptrbooks/ptr_0130260363.html Python Books: http://www.softpro.com/languages-python.html wesley.j.chun :: wesc@alpha.ece.ucsb.edu cyberweb.consulting :: silicon.valley, ca http://www.roadkill.com/~wesc/cyberweb/ From tim_one@email.msn.com Thu Aug 3 09:05:31 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 3 Aug 2000 04:05:31 -0400 Subject: [Python-Dev] Go \x yourself Message-ID: Offline, Guido and /F and I had a mighty battle about the meaning of \x escapes in Python. In the end we agreed to change the meaning of \x in a backward-*in*compatible way. Here's the scoop: In 1.5.2 and before, the Reference Manual implies that an \x escape takes two or more hex digits following, and has the value of the last byte. In reality it also accepted just one hex digit, or even none: >>> "\x123465" # same as "\x65" 'e' >>> "\x65" 'e' >>> "\x1" '\001' >>> "\x\x" '\\x\\x' >>> I found no instances of the 0- or 1-digit forms in the CVS tree or in any of the Python packages on my laptop. Do you have any in your code? And, apart from some deliberate abuse in the test suite, I found no instances of more-than-two-hex-digits \x escapes either. Similarly, do you have any? As Guido said and all agreed, it's probably a bug if you do. The new rule is the same as Perl uses for \x escapes in -w mode, except that Python will raise ValueError at compile-time for an invalid \x escape: an \x escape is of the form \xhh where h is a hex digit. That's it. Guido reports that the O'Reilly books (probably due to their Perl editing heritage!) already say Python works this way. It's the same rule for 8-bit and Unicode strings (in Perl too, at least wrt the syntax). In a Unicode string \xij has the same meaning as \u00ij, i.e. it's the obvious Latin-1 character. Playing back the above pretending the new rule is in place: >>> "\x123465" # \x12 -> \022, "3456" left alone '\0223456' >>> "\x65" 'e' >>> "\x1" ValueError >>> "\x\x" ValueError >>> We all support this: the open-ended gobbling \x used to do lost information without warning, and had no benefit whatsoever. While there was some attraction to generalizing \x in Unicode strings, \u1234 is already perfectly adequate for specifying Unicode characters in hex form, and the new rule for \x at least makes consistent Unicode sense now (and in a way JPython should be able to adopt easily too). The new rule gets rid of the unPythonic TMTOWTDI introduced by generalizing Unicode \x to "the last 4 bytes". That generalization also didn't make sense in light of the desire to add \U12345678 escapes too (i.e., so then how many trailing hex digits should a generalized \x suck up? 2? 4? 8?). The only actual use for \x in 8-bit strings (i.e., a way to specify a byte in hex) is still supported with the same meaning as in 1.5.2, and \x in a Unicode string means something as close to that as is possible. Sure feels right to me. Gripe quick if it doesn't to you. as-simple-as-possible-is-a-nice-place-to-rest-ly y'rs - tim From gstein@lyra.org Thu Aug 3 09:16:37 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 3 Aug 2000 01:16:37 -0700 Subject: [Python-Dev] Cookies.py in the core In-Reply-To: ; from tim_one@email.msn.com on Thu, Aug 03, 2000 at 12:52:07AM -0400 References: <14728.63725.390053.65213@anthem.concentric.net> Message-ID: <20000803011637.K19525@lyra.org> On Thu, Aug 03, 2000 at 12:52:07AM -0400, Tim Peters wrote: > Guys, these are cookies, not brain surgery! If people like this API, > couldn't someone have done a clean-room reimplementation of it in less time > than we've spent jockeying over the freaking license? No. -- Greg Stein, http://www.lyra.org/ From gstein@lyra.org Thu Aug 3 09:18:38 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 3 Aug 2000 01:18:38 -0700 Subject: [Python-Dev] Go \x yourself In-Reply-To: ; from tim_one@email.msn.com on Thu, Aug 03, 2000 at 04:05:31AM -0400 References: Message-ID: <20000803011838.L19525@lyra.org> On Thu, Aug 03, 2000 at 04:05:31AM -0400, Tim Peters wrote: >... > Sure feels right to me. Gripe quick if it doesn't to you. +1 -- Greg Stein, http://www.lyra.org/ From Fredrik Lundh" <20000802180059.B30340@kronos.cnri.reston.va.us> Message-ID: <006601bffd24$e25a9360$f2a6b5d4@hagrid> andrew wrote: > > >-- SRE is usually faster than the old RE module (PRE). >=20 > Once the compiler is translated to C, it might be worth considering > making SRE available as a standalone library for use outside of > Python. if it will ever be translated, that is... > Hmm... here's an old problem that's returned (recursion on repeated > group matches, I expect): >=20 > >>> p=3Dre.compile('(x)*') > >>> p > > >>> p.match(500000*'x') > Segmentation fault (core dumped) fwiw, that pattern isn't portable: $ jpython test.py File "test.py", line 3, in ? java.lang.StackOverflowError and neither is: def nest(level): if level: nest(level-1) nest(500000) ...but sure, I will fix that in 0.9.9 (SRE, not Python -- Christian has already taken care of the other one ;-). but 0.9.9 won't be out before the 1.6b1 release... (and to avoid scaring the hell out of the beta testers, it's probably better to leave the test out of the regression suite until the bug is fixed...) From Vladimir.Marangozov@inrialpes.fr Thu Aug 3 09:44:58 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Thu, 3 Aug 2000 10:44:58 +0200 (CEST) Subject: [Python-Dev] Cookies.py in the core In-Reply-To: <20000803011637.K19525@lyra.org> from "Greg Stein" at Aug 03, 2000 01:16:37 AM Message-ID: <200008030844.KAA12666@python.inrialpes.fr> Greg Stein wrote: > > On Thu, Aug 03, 2000 at 12:52:07AM -0400, Tim Peters wrote: > > Guys, these are cookies, not brain surgery! If people like this API, > > couldn't someone have done a clean-room reimplementation of it in less time > > than we've spent jockeying over the freaking license? > > No. Sorry for asking this, but what "cookies in the core" means to you in the first place? A library module.py, C code or both? PS: I can hardly accept the idea that cookies are necessary for normal Web usage. I'm not against them, though. IMO, it is important to keep control on whether they're enabled or disabled. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From Moshe Zadka Thu Aug 3 09:43:04 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 3 Aug 2000 11:43:04 +0300 (IDT) Subject: [Python-Dev] Cookies.py in the core In-Reply-To: <200008030844.KAA12666@python.inrialpes.fr> Message-ID: On Thu, 3 Aug 2000, Vladimir Marangozov wrote: > Sorry for asking this, but what "cookies in the core" means to you in > the first place? A library module.py, C code or both? I think Python is good enough for that. (Python is a great language!) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Moshe preaching to the choir > PS: I can hardly accept the idea that cookies are necessary for normal > Web usage. I'm not against them, though. IMO, it is important to keep > control on whether they're enabled or disabled. Yes, but that all happens client-side -- we were talking server-side cookies. Cookies are a state-management mechanism for a loosely-coupled protocols, and are almost essential in today's web. Not giving support means that Python is not as good a server-side language as it can be. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From Vladimir.Marangozov@inrialpes.fr Thu Aug 3 10:11:36 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Thu, 3 Aug 2000 11:11:36 +0200 (CEST) Subject: [Python-Dev] More Non-Bugs In-Reply-To: from "Moshe Zadka" at Aug 02, 2000 03:17:31 PM Message-ID: <200008030911.LAA12747@python.inrialpes.fr> Moshe Zadka wrote: > > On Wed, 2 Aug 2000, Vladimir Marangozov wrote: > > > Moshe Zadka wrote: > > > > > > Bug 110651 -- re.compile('[\\200-\\400]') segfaults -- it doens't for me > > > > You get a compiled SRE object, right? > > Nope -- I tested it with pre. As of yesterday's CVS (I saw AMK checking in an escape patch since then): ~/python/dev>python Python 2.0b1 (#1, Aug 3 2000, 09:01:35) [GCC egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)] on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam Copyright 1995-2000 Corporation for National Research Initiatives (CNRI) >>> import pre >>> pre.compile('[\\200-\\400]') Segmentation fault (core dumped) -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From Moshe Zadka Thu Aug 3 10:06:23 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 3 Aug 2000 12:06:23 +0300 (IDT) Subject: [Python-Dev] More Non-Bugs In-Reply-To: <200008030911.LAA12747@python.inrialpes.fr> Message-ID: On Thu, 3 Aug 2000, Vladimir Marangozov wrote: > Moshe Zadka wrote: > > > > On Wed, 2 Aug 2000, Vladimir Marangozov wrote: > > > > > Moshe Zadka wrote: > > > > > > > > Bug 110651 -- re.compile('[\\200-\\400]') segfaults -- it doens't for me > > > > > > You get a compiled SRE object, right? > > > > Nope -- I tested it with pre. > > As of yesterday's CVS (I saw AMK checking in an escape patch since then): Hmmmmm....I ought to be more careful then. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From thomas@xs4all.net Thu Aug 3 10:14:24 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 3 Aug 2000 11:14:24 +0200 Subject: [Python-Dev] Go \x yourself In-Reply-To: ; from tim_one@email.msn.com on Thu, Aug 03, 2000 at 04:05:31AM -0400 References: Message-ID: <20000803111424.Z266@xs4all.nl> On Thu, Aug 03, 2000 at 04:05:31AM -0400, Tim Peters wrote: > Sure feels right to me. Gripe quick if it doesn't to you. +1 if it's a compile-time error, +0 if it isn't and won't be made one. The compile-time error makes it a lot easier to track down the issues, if any. (Okay, so everyone should have proper unit testing -- not everyone actually has it ;) I suspect it would be a compile-time error, but I haven't looked at compiling string literals yet ;P -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From mal@lemburg.com Thu Aug 3 10:17:46 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 03 Aug 2000 11:17:46 +0200 Subject: [Python-Dev] SRE 0.9.8 benchmarks References: <015a01bffcc9$ed942bc0$f2a6b5d4@hagrid> Message-ID: <398938BA.CA54A98E@lemburg.com> > > searching for literal text: > > searching for "spam" in a string padded with "spaz" (1000 bytes on > each side of the target): > > string.find 0.112 ms > sre8.search 0.059 > pre.search 0.122 > > unicode.find 0.130 > sre16.search 0.065 > > (yes, regular expressions can run faster than optimized C code -- as > long as we don't take compilation time into account ;-) > > same test, without any false matches: > > string.find 0.035 ms > sre8.search 0.050 > pre.search 0.116 > > unicode.find 0.031 > sre16.search 0.055 Those results are probably due to the fact that string.find does a brute force search. If it would do a last match char first search or even Boyer-Moore (this only pays off for long search targets) then it should be a lot faster than [s|p]re. Just for compares: would you mind running the search routines in mxTextTools on the same machine ? import TextTools TextTools.find(text, what) -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mal@lemburg.com Thu Aug 3 10:55:57 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 03 Aug 2000 11:55:57 +0200 Subject: [Python-Dev] Preventing 1.5 extensions crashing under 1.6/2.0 Python References: <020901bffccb$b4bf4da0$060210ac@private> <200008022318.SAA04558@cj20424-a.reston1.va.home.com> Message-ID: <398941AD.F47CA1C1@lemburg.com> Guido van Rossum wrote: > > > Change the init function name to a new name PythonExtensionInit_ say. > > Pass in the API version for the extension writer to check. If the > > version is bad for this extension returns without calling any python > > functions. Add a return code that is true if compatible, false if not. > > If compatible the extension can use python functions and report and > > problems it wishes. > > > > int PythonExtensionInit_XXX( int invoking_python_api_version ) > > { > > if( invoking_python_api_version != PYTHON_API_VERSION ) > > { > > /* python will report that the module is incompatible */ > > return 0; > > } > > > > /* setup module for XXX ... */ > > > > /* say this extension is compatible with the invoking python */ > > return 1; > > } > > > > All 1.5 extensions fail to load on later python 2.0 and later. > > All 2.0 extensions fail to load on python 1.5. > > > > All new extensions work only with python of the same API version. > > > > Document that failure to setup a module could mean the extension is > > incompatible with this version of python. > > > > Small code change in python core. But need to tell extension writers > > what the new interface is and update all extensions within the python > > CVS tree. > > I sort-of like this idea -- at least at the +0 level. I sort of dislike the idea ;-) It introduces needless work for hundreds of extension writers and effectively prevents binary compatibility for future versions of Python: not all platforms have the problems of the Windows platform and extensions which were compiled against a different API version may very well still work with the new Python version -- e.g. the dynamic loader on Linux is very well capable of linking the new Python version against an extension compiled for the previous Python version. If all this is really necessary, I'd at least suggest adding macros emulating the old Py_InitModule() APIs, so that extension writers don't have to edit their code just to get it recompiled. BTW, the subject line doesn't have anything to do with the proposed solutions in this thread... they all crash Python or the extensions in some way, some nicer, some not so nice ;-) -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From tim_one@email.msn.com Thu Aug 3 10:57:22 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 3 Aug 2000 05:57:22 -0400 Subject: [Python-Dev] Go \x yourself In-Reply-To: <20000803111424.Z266@xs4all.nl> Message-ID: [Thomas Wouters] > +1 if it's a compile-time error, +0 if it isn't and won't be > made one. ... Quoting back from the original msg: > ... will raise ValueError at compile-time for an invalid \x escape ^^^^^^^^^^^^^^^ The pseudo-example was taken from a pseudo interactive prompt, and just as in a real example at a real interactive prompt, each (pseduo)input line was (pseudo)compiled one at a time . From mal@lemburg.com Thu Aug 3 11:04:53 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 03 Aug 2000 12:04:53 +0200 Subject: [Python-Dev] Fork on Win32 - was (test_fork1 failing...) References: Message-ID: <398943C4.AFECEE36@lemburg.com> David Ascher wrote: > > > IIRC ActiveState contributed to Perl a version of fork that works on > > Win32. Has anyone looked at this? Could it be grabbed for Python? This would > > help heal one of the more difficult platform rifts. Emulating fork for Win32 > > looks quite difficult to me but if its already done... > > Sigh. Me tired. > > The message I posted a few minutes ago was actually referring to the > system() work, not the fork() work. I agree that the fork() emulation > isn't Pythonic. What about porting os.kill() to Windows (see my other post with changed subject line in this thread) ? Wouldn't that make sense ? (the os.spawn() APIs do return PIDs of spawned processes, so calling os.kill() to send signals to these seems like a feasable way to control them) -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mal@lemburg.com Thu Aug 3 11:11:24 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 03 Aug 2000 12:11:24 +0200 Subject: [Python-Dev] Re: A small proposed change to dictionaries' "get" method References: <14728.63466.263123.434708@anthem.concentric.net> Message-ID: <3989454C.5C9EF39B@lemburg.com> "Barry A. Warsaw" wrote: > > >>>>> "GM" == Gareth McCaughan writes: > > GM> Consider the following piece of code, which takes a file > GM> and prepares a concordance saying on which lines each word > GM> in the file appears. (For real use it would need to be > GM> made more sophisticated.) > > | line_number = 0 > | for line in open(filename).readlines(): > | line_number = line_number+1 > | for word in map(string.lower, string.split(line)): > | existing_lines = word2lines.get(word, []) | > | existing_lines.append(line_number) | ugh! > | word2lines[word] = existing_lines | > > I've run into this same situation many times myself. I agree it's > annoying. Annoying enough to warrant a change? Maybe -- I'm not > sure. > > GM> I suggest a minor change: another optional argument to > GM> "get" so that > > GM> dict.get(item,default,flag) > > Good idea, not so good solution. Let's make it more explicit by > adding a new method instead of a flag. I'll use `put' here since this > seems (in a sense) opposite of get() and my sleep addled brain can't > think of anything more clever. Let's not argue about the name of this > method though -- if Guido likes the extension, he'll pick a good name > and I go on record as agreeing with his name choice, just to avoid a > protracted war. > > A trivial patch to UserDict (see below) will let you play with this. > > >>> d = UserDict() > >>> word = 'hello' > >>> d.get(word, []) > [] > >>> d.put(word, []).append('world') > >>> d.get(word) > ['world'] > >>> d.put(word, []).append('gareth') > >>> d.get(word) > ['world', 'gareth'] > > Shouldn't be too hard to add equivalent C code to the dictionary > object. The following one-liner already does what you want: d[word] = d.get(word, []).append('world') ... and it's in no way more readable than your proposed .put() line ;-) -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From m.favas@per.dem.csiro.au Thu Aug 3 11:54:05 2000 From: m.favas@per.dem.csiro.au (Mark Favas) Date: Thu, 03 Aug 2000 18:54:05 +0800 Subject: [Python-Dev] (s)re crashing in regrtest (was SRE 0.9.8 benchmarks) Message-ID: <39894F4D.FB11F098@per.dem.csiro.au> [Guido] >> Hmm... here's an old problem that's returned (recursion on repeated >> group matches, I expect): >> >> >>> p=re.compile('(x)*') >> >>> p >> >> >>> p.match(500000*'x') >> Segmentation fault (core dumped) > >Ouch. > >Andrew, would you mind adding a test case for that to the re test >suite? It's important that this doesn't come back! In fact, on my machine with the default stacksize of 2048kb, test_re.py already exercises this bug. (Goes away if I do an "unlimit", of course.) So testing for this deterministically is always going to be dependent on the platform. How large do you want to go (reasonably)? - although I guess core dumps should be avoided... Mark -- Email - m.favas@per.dem.csiro.au Mark C Favas Phone - +61 8 9333 6268, 0418 926 074 CSIRO Exploration & Mining Fax - +61 8 9383 9891 Private Bag No 5, Wembley WGS84 - 31.95 S, 115.80 E Western Australia 6913 From Fredrik Lundh" <398938BA.CA54A98E@lemburg.com> Message-ID: <00eb01bffd3b$8324fb80$f2a6b5d4@hagrid> mal wrote: > Just for compares: would you mind running the search=20 > routines in mxTextTools on the same machine ? > > searching for "spam" in a string padded with "spaz" (1000 bytes on > > each side of the target): > >=20 > > string.find 0.112 ms texttools.find 0.080 ms > > sre8.search 0.059 > > pre.search 0.122 > >=20 > > unicode.find 0.130 > > sre16.search 0.065 > >=20 > > same test, without any false matches (padded with "-"): > >=20 > > string.find 0.035 ms texttools.find 0.083 ms > > sre8.search 0.050 > > pre.search 0.116 > >=20 > > unicode.find 0.031 > > sre16.search 0.055 >=20 > Those results are probably due to the fact that string.find > does a brute force search. If it would do a last match char > first search or even Boyer-Moore (this only pays off for long > search targets) then it should be a lot faster than [s|p]re. does the TextTools algorithm work with arbitrary character set sizes, btw? From Fredrik Lundh" Message-ID: <00fc01bffd3d$91a36460$f2a6b5d4@hagrid> mark favas wrote: > >> >>> p.match(500000*'x') > >> Segmentation fault (core dumped) > > > >Andrew, would you mind adding a test case for that to the re test > >suite? It's important that this doesn't come back! >=20 > In fact, on my machine with the default stacksize of 2048kb, = test_re.py > already exercises this bug. (Goes away if I do an "unlimit", of = course.) > So testing for this deterministically is always going to be dependent = on > the platform. How large do you want to go (reasonably)? - although I > guess core dumps should be avoided... afaik, there was no test in the standard test suite that included run-away recursion... what test is causing this error? (adding a print statement to sre._compile should help you figure that out...) From MarkH@ActiveState.com Thu Aug 3 12:19:50 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Thu, 3 Aug 2000 21:19:50 +1000 Subject: [Python-Dev] (os.kill (was Fork) on Win32 - was (test_fork1 failing...) In-Reply-To: <398943C4.AFECEE36@lemburg.com> Message-ID: > What about porting os.kill() to Windows (see my other post > with changed subject line in this thread) ? Wouldn't that > make sense ? (the os.spawn() APIs do return PIDs of spawned > processes, so calling os.kill() to send signals to these > seems like a feasable way to control them) Signals are a bit of a problem on Windows. We can terminate the thread mid-execution, but a clean way of terminating a thread isn't obvious. I admit I didnt really read the long manpage when you posted it, but is a terminate-without-prejudice option any good? Mark. From MarkH@ActiveState.com Thu Aug 3 12:34:09 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Thu, 3 Aug 2000 21:34:09 +1000 Subject: [Python-Dev] (os.kill (was Fork) on Win32 - was (test_fork1 failing...) In-Reply-To: Message-ID: eek - a bit quick off the mark here ;-] > Signals are a bit of a problem on Windows. We can terminate the thread > mid-execution, but a clean way of terminating a thread isn't obvious. thread = process - you get the idea! > terminate-without-prejudice option any good? really should say > terminate-without-prejudice only version any good? Mark. From m.favas@per.dem.csiro.au Thu Aug 3 12:35:48 2000 From: m.favas@per.dem.csiro.au (Mark Favas) Date: Thu, 03 Aug 2000 19:35:48 +0800 Subject: [Python-Dev] (s)re crashing in regrtest (was SRE 0.9.8 benchmarks) References: <39894F4D.FB11F098@per.dem.csiro.au> <00fc01bffd3d$91a36460$f2a6b5d4@hagrid> Message-ID: <39895914.133D52A4@per.dem.csiro.au> Fredrik Lundh wrote: > > mark favas wrote: > > In fact, on my machine with the default stacksize of 2048kb, test_re.py > > already exercises this bug.> > afaik, there was no test in the standard test suite that > included run-away recursion... > > what test is causing this error? > > (adding a print statement to sre._compile should help you > figure that out...) > > The stack overflow is caused by the test (in test_re.py): # Try nasty case that overflows the straightforward recursive # implementation of repeated groups. assert re.match('(x)*', 50000*'x').span() == (0, 50000) (changing 50000 to 18000 works, 19000 overflows...) -- Email - m.favas@per.dem.csiro.au Mark C Favas Phone - +61 8 9333 6268, 0418 926 074 CSIRO Exploration & Mining Fax - +61 8 9383 9891 Private Bag No 5, Wembley WGS84 - 31.95 S, 115.80 E Western Australia 6913 From guido@beopen.com Thu Aug 3 13:56:38 2000 From: guido@beopen.com (Guido van Rossum) Date: Thu, 03 Aug 2000 07:56:38 -0500 Subject: [Python-Dev] Re: A small proposed change to dictionaries' "get" method In-Reply-To: Your message of "Thu, 03 Aug 2000 12:11:24 +0200." <3989454C.5C9EF39B@lemburg.com> References: <14728.63466.263123.434708@anthem.concentric.net> <3989454C.5C9EF39B@lemburg.com> Message-ID: <200008031256.HAA06107@cj20424-a.reston1.va.home.com> > "Barry A. Warsaw" wrote: > > Good idea, not so good solution. Let's make it more explicit by > > adding a new method instead of a flag. You're learning to channel me. :-) > > I'll use `put' here since this > > seems (in a sense) opposite of get() and my sleep addled brain can't > > think of anything more clever. Let's not argue about the name of this > > method though -- if Guido likes the extension, he'll pick a good name > > and I go on record as agreeing with his name choice, just to avoid a > > protracted war. But I'll need input. My own idea was dict.getput(), but that's ugly as hell; dict.put() doesn't suggest that it also returns the value. Protocol: if you have a suggestion for a name for this function, mail it to me. DON'T MAIL THE LIST. (If you mail it to the list, that name is disqualified.) Don't explain me why the name is good -- if it's good, I'll know, if it needs an explanation, it's not good. From the suggestions I'll pick one if I can, and the first person to suggest it gets a special mention in the implementation. If I can't decide, I'll ask the PythonLabs folks to help. Marc-Andre writes: > The following one-liner already does what you want: > > d[word] = d.get(word, []).append('world') Are you using a patch to the list object so that append() returns the list itself? Or was it just late? For me, this makes d[word] = None. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From akuchlin@mems-exchange.org Thu Aug 3 13:06:49 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Thu, 3 Aug 2000 08:06:49 -0400 Subject: [Python-Dev] Cookies.py in the core In-Reply-To: <002601bffd21$5f23e800$f2a6b5d4@hagrid>; from effbot@telia.com on Thu, Aug 03, 2000 at 10:03:53AM +0200 References: <200008021406.JAA02743@cj20424-a.reston1.va.home.com> <14728.38251.289986.857417@anthem.concentric.net> <200008022246.RAA04405@cj20424-a.reston1.va.home.com> <20000802175553.A30340@kronos.cnri.reston.va.us> <200008022310.SAA04518@cj20424-a.reston1.va.home.com> <20000802224102.A25837@newcnri.cnri.reston.va.us> <002601bffd21$5f23e800$f2a6b5d4@hagrid> Message-ID: <20000803080649.A27333@newcnri.cnri.reston.va.us> On Thu, Aug 03, 2000 at 10:03:53AM +0200, Fredrik Lundh wrote: >readers of the daily URL might have noticed that he posted >a socket timeout wrapper a few days ago: Noted; thanks! I've sent him an e-mail... --amk From akuchlin@mems-exchange.org Thu Aug 3 13:14:56 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Thu, 3 Aug 2000 08:14:56 -0400 Subject: [Python-Dev] (s)re crashing in regrtest In-Reply-To: <39895914.133D52A4@per.dem.csiro.au>; from m.favas@per.dem.csiro.au on Thu, Aug 03, 2000 at 07:35:48PM +0800 References: <39894F4D.FB11F098@per.dem.csiro.au> <00fc01bffd3d$91a36460$f2a6b5d4@hagrid> <39895914.133D52A4@per.dem.csiro.au> Message-ID: <20000803081456.B27333@newcnri.cnri.reston.va.us> On Thu, Aug 03, 2000 at 07:35:48PM +0800, Mark Favas wrote: >The stack overflow is caused by the test (in test_re.py): ># Try nasty case that overflows the straightforward recursive ># implementation of repeated groups. That would be the test I added last night to trip this problem, per GvR's instructions. I'll comment out the test for now, so that it can be restored once the bug is fixed. --amk From mal@lemburg.com Thu Aug 3 13:14:55 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 03 Aug 2000 14:14:55 +0200 Subject: [Python-Dev] Still no new license -- but draft text available References: <200008020409.XAA01355@cj20424-a.reston1.va.home.com> <3987E5E1.A2B20241@lemburg.com> <200008021511.KAA03049@cj20424-a.reston1.va.home.com> <398858BE.15928F47@lemburg.com> <200008022218.RAA04178@cj20424-a.reston1.va.home.com> Message-ID: <3989623F.2AB4C00C@lemburg.com> Guido van Rossum wrote: > > [...] > > > > > Some comments on the new version: > > > > > > > > 2. Subject to the terms and conditions of this License Agreement, CNRI > > > > > hereby grants Licensee a nonexclusive, royalty-free, world-wide > > > > > license to reproduce, analyze, test, perform and/or display publicly, > > > > > prepare derivative works, distribute, and otherwise use Python 1.6b1 > > > > > alone or in any derivative version, provided, however, that CNRI's > > > > > License Agreement is retained in Python 1.6b1, alone or in any > > > > > derivative version prepared by Licensee. > > > > > > > > I don't think the latter (retaining the CNRI license alone) is > > > > possible: you always have to include the CWI license. > > > > > > Wow. I hadn't even noticed this! It seems you can prepare a > > > derivative version of the license. Well, maybe. > > > > I think they mean "derivative version of Python 1.6b1", but in > > court, the above wording could cause serious trouble for CNRI > > You're right of course, I misunderstood you *and* the license. Kahn > explains it this way: > > [Kahn] > | Ok. I take the point being made. The way english works with ellipsis or > | anaphoric references is to link back to the last anchor point. In the above > | case, the last referent is Python 1.6b1. > | > | Thus, the last phrase refers to a derivative version of Python1.6b1 > | prepared by Licensee. There is no permission given to make a derivative > | version of the License. > > > ... it seems 2.0 can reuse the CWI license after all ;-) > > I'm not sure why you think that: 2.0 is a derivative version and is > thus bound by the CNRI license as well as by the license that BeOpen > adds. If you interpret the above wording in the sense of "preparing a derivative version of the License Agreement", BeOpen (or anyone else) could just remove the CNRI License text. I understand that this is not intended (that's why I put the smiley there ;-). > [...] > > > > > > 3. In the event Licensee prepares a derivative work that is based on > > > > > or incorporates Python 1.6b1or any part thereof, and wants to make the > > > > > derivative work available to the public as provided herein, then > > > > > Licensee hereby agrees to indicate in any such work the nature of the > > > > > modifications made to Python 1.6b1. > > > > > > > > In what way would those indications have to be made ? A patch > > > > or just text describing the new features ? > > > > > > Just text. Bob Kahn told me that the list of "what's new" that I > > > always add to a release would be fine. > > > > Ok, should be made explicit in the license though... > > It's hard to specify this precisely -- in fact, the more precise you > specify it the more scary it looks and the more likely they are to be > able to find fault with the details of how you do it. In this case, I > believe (and so do lawyers) that vague is good! If you write "ported > to the Macintosh" and that's what you did, they can hardly argue with > you, can they? True. > > > > What does "make available to the public" mean ? If I embed > > > > Python in an application and make this application available > > > > on the Internet for download would this fit the meaning ? > > > > > > Yes, that's why he doesn't use the word "publish" -- such an action > > > would not be considered publication in the sense of the copyright law > > > (at least not in the US, and probably not according to the Bern > > > convention) but it is clearly making it available to the public. > > > > Ouch. That would mean I'd have to describe all additions, > > i.e. the embedding application, in most details in order not to > > breach the terms of the CNRI license. > > No, additional modules aren't modifications to CNRI's work. A change > to the syntax to support curly braces is. Ok, thanks for clarifying this. (I guess the "vague is good" argument fits here as well.) > > > > > 4. CNRI is making Python 1.6b1 available to Licensee on an "AS IS" > > > > > basis. CNRI MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR > > > > > IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, CNRI MAKES NO AND > > > > > DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS > > > > > FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 1.6b1 WILL NOT > > > > > INFRINGE ANY THIRD PARTY RIGHTS. > > > > > > > > > > 5. CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE > > > > > SOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS > > > > > AS A RESULT OF USING, MODIFYING OR DISTRIBUTING PYTHON 1.6b1, OR ANY > > > > > DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. SOME > > > > > STATES DO NOT ALLOW THE LIMITATION OR EXCLUSION OF LIABILITY SO THE > > > > > ABOVE DISCLAIMER MAY NOT APPLY TO LICENSEE. > > > > > > > > I would make this "...SOME STATES AND COUNTRIES...". E.g. in > > > > Germany the above text would only be valid after an initial > > > > 6 month period after installation, AFAIK (this period is > > > > called "Gewährleistung"). Licenses from other vendors usually > > > > add some extra license text to limit the liability in this period > > > > to the carrier on which the software was received by the licensee, > > > > e.g. the diskettes or CDs. > > > > > > I'll mention this to Kahn. > > His response: > > | Guido, Im not willing to do a study of international law here. If you > | can have the person identify one country other than the US that does > | not allow the above limitation or exclusion of liability and provide a > | copy of the section of their law, ill be happy to change this to read > | ".... SOME STATES OR COUNTRIES MAY NOT ALLOW ...." Otherwise, id just > | leave it alone (i.e. as is) for now. > > Please mail this info directly to Kahn@CNRI.Reston.Va.US if you > believe you have the right information. (You may CC me.) Personally, > I wouldn't worry. If the German law says that part of a license is > illegal, it doesn't make it any more or less illegal whether the > license warns you about this fact. > > I believe that in the US, as a form of consumer protection, some > states not only disallow general disclaimers, but also require that > licenses containing such disclaimers notify the reader that the > disclaimer is not valid in their state, so that's where the language > comes from. I don't know about German law. I haven't found an English version of the German law text, but this is the title of the law which handles German business conditions: "Gesetz zur Regelung des Rechts der Allgemeinen Geschäftsbedingungen AGBG) - Act Governing Standard Business Conditions" The relevant paragraph is no. 11 (10). I'm not a lawyer, but from what I know: terms generally excluding liability are invalid; liability may be limited during the first 6 months after license agreement and excluded after this initial period. Anyway, you're right in that the notice about the paragraph not necessarily applying to the licensee only has informational character and that it doesn't do any harm otherwise. > > > > > 6. This License Agreement will automatically terminate upon a material > > > > > breach of its terms and conditions. > > > > > > > > Immediately ? Other licenses usually include a 30-60 day period > > > > which allows the licensee to take actions. With the above text, > > > > the license will put the Python copy in question into an illegal > > > > state *prior* to having even been identified as conflicting with the > > > > license. > > > > > > Believe it or not, this is necessary to ensure GPL compatibility! An > > > earlier draft had 30-60 days. But the GPL doesn't, so this was deemed > > > incompatible. There's an easy workaround though: you fix your > > > compliance and download a new copy, which gives you all the same > > > rights again. > > > > Hmm, but what about the 100.000 copies of the embedding application > > that have already been downloaded -- I would have to force them > > to redownload the application (or even just a demo of it) in > > order to reestablish the lawfulness of the copy action. > > It's better not to violate the license. But do you really think that > they would go after you immediately if you show good intentions to > rectify? I don't intend to violate the license, but customers of an application embedding Python will have to agree to the Python license to be able to legally use the Python engine embedded in the application -- that is: if the application unintensionally fails to meet the CNRI license terms then the application as a whole would immediately become unusable by the customer. Now just think of an eCommerce application which produces some $100k USD revenue each day... such a customer wouldn't like these license terms at all :-( BTW, I think that section 6. can be removed altogether, if it doesn't include any reference to such a 30-60 day period: the permissions set forth in a license are only valid in case the license terms are adhered to whether it includes such a section or not. > > Not that I want to violate the license in any way, but there > > seem to be quite a few pitfalls in the present text, some of > > which are not clear at all (e.g. the paragraph 3). > > I've warned Kahn about this effect of making the license bigger, but > he simply disagrees (and we agree to disagree). I don't know what > else I could do about it, apart from putting a FAQ about the license > on python.org -- which I intend to do. Good (or bad ? :-() -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From akuchlin@mems-exchange.org Thu Aug 3 13:22:44 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Thu, 3 Aug 2000 08:22:44 -0400 Subject: [Python-Dev] SRE 0.9.8 benchmarks In-Reply-To: <006601bffd24$e25a9360$f2a6b5d4@hagrid>; from effbot@telia.com on Thu, Aug 03, 2000 at 10:27:39AM +0200 References: <015a01bffcc9$ed942bc0$f2a6b5d4@hagrid> <20000802180059.B30340@kronos.cnri.reston.va.us> <006601bffd24$e25a9360$f2a6b5d4@hagrid> Message-ID: <20000803082244.C27333@newcnri.cnri.reston.va.us> On Thu, Aug 03, 2000 at 10:27:39AM +0200, Fredrik Lundh wrote: >if it will ever be translated, that is... I'll agree to take a shot at it (which carries no implication of actually finisihing :) ) post-2.0. It's silly for all of Tcl, Python, Perl to grow their own implementations, when a common implementation could benefit from having 3x the number of eyes looking at it and optimizing it. >fwiw, that pattern isn't portable: No, it isn't; the straightforward implementation of repeated groups is recursive, and fixing this requires jumping through hoops to make it nonrecursive (or adopting Python's solution and only recursing up to some upper limit). re had to get this right because regex didn't crash on this pattern, and neither do recent Perls. The vast bulk of my patches to PCRE were to fix this problem. --amk From guido@beopen.com Thu Aug 3 14:31:16 2000 From: guido@beopen.com (Guido van Rossum) Date: Thu, 03 Aug 2000 08:31:16 -0500 Subject: [Python-Dev] New SRE core dump (was: SRE 0.9.8 benchmarks) In-Reply-To: Your message of "Thu, 03 Aug 2000 10:27:39 +0200." <006601bffd24$e25a9360$f2a6b5d4@hagrid> References: <015a01bffcc9$ed942bc0$f2a6b5d4@hagrid> <20000802180059.B30340@kronos.cnri.reston.va.us> <006601bffd24$e25a9360$f2a6b5d4@hagrid> Message-ID: <200008031331.IAA06319@cj20424-a.reston1.va.home.com> > andrew wrote: > > > Hmm... here's an old problem that's returned (recursion on repeated > > group matches, I expect): > > > > >>> p=re.compile('(x)*') > > >>> p > > > > >>> p.match(500000*'x') > > Segmentation fault (core dumped) Effbot: > fwiw, that pattern isn't portable: Who cares -- it shouldn't dump core! > ...but sure, I will fix that in 0.9.9 (SRE, not Python -- Christian > has already taken care of the other one ;-). but 0.9.9 won't be > out before the 1.6b1 release... I assume you are planning to put the backtracking stack back in, as you mentioned in the checkin message? > (and to avoid scaring the hell out of the beta testers, it's probably > better to leave the test out of the regression suite until the bug is > fixed...) Even better, is it possible to put a limit on the recursion level before 1.6b1 is released (tomorrow if we get final agreement on the license) so at least it won't dump core? Otherwise you'll get reports of this from people who write this by accident... --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From thomas@xs4all.net Thu Aug 3 13:57:14 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 3 Aug 2000 14:57:14 +0200 Subject: [Python-Dev] Buglist Message-ID: <20000803145714.B266@xs4all.nl> Just a little FYI and 'is this okay' message; I've been browsing the buglist the last few days, doing a quick mark & message sweep over the bugs that I can understand. I've mostly been closing bugs that look closed, and assigning them when it's very obvious who it should be assigned to. Should I be doing this already ? Is the bug-importing 'done', or is Jeremy still busy with importing and fixing bug status (stati ?) and such ? Is there something better to use as a guideline than my 'best judgement' ? I think it's a good idea to 'resolve' most of the bugs on the list, because a lot of them are really non-issues or no-longer-issues, and the sheer size of the list prohibits a proper overview of the real issues :P However, it's entirely possible we're going to miss out on a few bugs this way. I'm trying my best to be careful, but I think overlooking a few bugs is better than overlooking all of them because of the size of the list :P -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From gmcm@hypernet.com Thu Aug 3 14:05:07 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Thu, 3 Aug 2000 09:05:07 -0400 Subject: [Python-Dev] Go \x yourself In-Reply-To: Message-ID: <1246814587-81974994@hypernet.com> [Tim sez] > The new rule is ... >... an \x escape is of the form > > \xhh > > where h is a hex digit. That's it. > >>> "\x123465" # \x12 -> \022, "3465" left alone > '\0223465' Hooray! I got bit often enough by that one ('e') that I forced myself to always use the wholly unnatural octal. god-gave-us-sixteen-fingers-for-a-reason-ly y'rs - Gordon From fdrake@beopen.com Thu Aug 3 14:06:51 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Thu, 3 Aug 2000 09:06:51 -0400 (EDT) Subject: [Python-Dev] printing xrange objects Message-ID: <14729.28267.517936.331801@cj42289-a.reston1.va.home.com> At various points, there have been comments that xrange objects should not print as lists but as xrange objects. Taking a look at the implementation, I noticed that if you call repr() (by name or by backtick syntax), you get "the right thing"; the list representation comes up when you print the object on a real file object. The tp_print slot of the xrange type produces the list syntax. There is no tp_str handler, so str(xrange(...)) is the same as repr(xrange(...)). I propose ripping out the tp_print handler completely. (And I've already tested my patch. ;) Comments? -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From Moshe Zadka Thu Aug 3 14:09:40 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 3 Aug 2000 16:09:40 +0300 (IDT) Subject: [Python-Dev] printing xrange objects In-Reply-To: <14729.28267.517936.331801@cj42289-a.reston1.va.home.com> Message-ID: On Thu, 3 Aug 2000, Fred L. Drake, Jr. wrote: > I propose ripping out the tp_print handler completely. (And I've > already tested my patch. ;) > Comments? +1. Like I always say: less code, less bugs. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From mal@lemburg.com Thu Aug 3 14:31:34 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 03 Aug 2000 15:31:34 +0200 Subject: [Python-Dev] SRE 0.9.8 benchmarks References: <015a01bffcc9$ed942bc0$f2a6b5d4@hagrid> <398938BA.CA54A98E@lemburg.com> <00eb01bffd3b$8324fb80$f2a6b5d4@hagrid> Message-ID: <39897436.E42F1C3C@lemburg.com> Fredrik Lundh wrote: > > mal wrote: > > > Just for compares: would you mind running the search > > routines in mxTextTools on the same machine ? > > > > searching for "spam" in a string padded with "spaz" (1000 bytes on > > > each side of the target): > > > > > > string.find 0.112 ms > > texttools.find 0.080 ms > > > > sre8.search 0.059 > > > pre.search 0.122 > > > > > > unicode.find 0.130 > > > sre16.search 0.065 > > > > > > same test, without any false matches (padded with "-"): > > > > > > string.find 0.035 ms > > texttools.find 0.083 ms > > > > sre8.search 0.050 > > > pre.search 0.116 > > > > > > unicode.find 0.031 > > > sre16.search 0.055 > > > > Those results are probably due to the fact that string.find > > does a brute force search. If it would do a last match char > > first search or even Boyer-Moore (this only pays off for long > > search targets) then it should be a lot faster than [s|p]re. > > does the TextTools algorithm work with arbitrary character > set sizes, btw? The find function creates a Boyer-Moore search object for the search string (on every call). It compares 1-1 or using a translation table which is applied to the searched text prior to comparing it to the search string (this enables things like case insensitive search and character sets, but is about 45% slower). Real-life usage would be to create the search objects once per process and then reuse them. The Boyer-Moore table calcuation takes some time... But to answer your question: mxTextTools is 8-bit throughout. A Unicode aware version will follow by the end of this year. Thanks for checking, -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mal@lemburg.com Thu Aug 3 14:40:05 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 03 Aug 2000 15:40:05 +0200 Subject: [Python-Dev] (os.kill (was Fork) on Win32 - was (test_fork1 failing...) References: Message-ID: <39897635.6C9FB82D@lemburg.com> Mark Hammond wrote: > > eek - a bit quick off the mark here ;-] > > > Signals are a bit of a problem on Windows. We can terminate the thread > > mid-execution, but a clean way of terminating a thread isn't obvious. > > thread = process - you get the idea! > > > terminate-without-prejudice option any good? > > really should say > > > terminate-without-prejudice only version any good? Well for one you can use signals for many other things than just terminating a process (e.g. to have it reload its configuration files). That's why os.kill() allows you to specify a signal. The usual way of terminating a process on Unix from the outside is to send it a SIGTERM (and if that doesn't work a SIGKILL). I use this strategy a lot to control runaway client processes and safely shut them down: On Unix you can install a signal handler in the Python program which then translates the SIGTERM signal into a normal Python exception. Sending the signal then causes the same as e.g. hitting Ctrl-C in a program: an exception is raised asynchronously, but it can be handled properly by the Python exception clauses to enable safe shutdown of the process. For background: the client processes in my application server can execute arbitrary Python scripts written by users, i.e. potentially buggy code which could effectively hose the server. To control this, I use client processes which do the actual exec code and watch them using a watchdog process. If the processes don't return anything useful within a certain timeout limit, the watchdog process sends them a SIGTERM and restarts a new client. Threads would not support this type of strategy, so I'm looking for something similar on Windows, Win2k to be more specific. Thanks, -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From guido@beopen.com Thu Aug 3 15:50:26 2000 From: guido@beopen.com (Guido van Rossum) Date: Thu, 03 Aug 2000 09:50:26 -0500 Subject: [Python-Dev] Still no new license -- but draft text available In-Reply-To: Your message of "Thu, 03 Aug 2000 14:14:55 +0200." <3989623F.2AB4C00C@lemburg.com> References: <200008020409.XAA01355@cj20424-a.reston1.va.home.com> <3987E5E1.A2B20241@lemburg.com> <200008021511.KAA03049@cj20424-a.reston1.va.home.com> <398858BE.15928F47@lemburg.com> <200008022218.RAA04178@cj20424-a.reston1.va.home.com> <3989623F.2AB4C00C@lemburg.com> Message-ID: <200008031450.JAA06505@cj20424-a.reston1.va.home.com> > > > ... it seems 2.0 can reuse the CWI license after all ;-) > > > > I'm not sure why you think that: 2.0 is a derivative version and is > > thus bound by the CNRI license as well as by the license that BeOpen > > adds. > > If you interpret the above wording in the sense of "preparing > a derivative version of the License Agreement", BeOpen (or > anyone else) could just remove the CNRI License text. I > understand that this is not intended (that's why I put the smiley > there ;-). Please forget this interpretation! :-) > I haven't found an English version of the German law text, > but this is the title of the law which handles German > business conditions: > > "Gesetz zur Regelung des Rechts der Allgemeinen Geschäftsbedingungen > AGBG) - Act Governing Standard Business Conditions" > > The relevant paragraph is no. 11 (10). > > I'm not a lawyer, but from what I know: > terms generally excluding liability are invalid; liability > may be limited during the first 6 months after license > agreement and excluded after this initial period. > > Anyway, you're right in that the notice about the paragraph > not necessarily applying to the licensee only has informational > character and that it doesn't do any harm otherwise. OK, we'll just let this go. > > It's better not to violate the license. But do you really think that > > they would go after you immediately if you show good intentions to > > rectify? > > I don't intend to violate the license, but customers of > an application embedding Python will have to agree to the > Python license to be able to legally use the Python engine > embedded in the application -- that is: if the application > unintensionally fails to meet the CNRI license terms > then the application as a whole would immediately become > unusable by the customer. > > Now just think of an eCommerce application which produces > some $100k USD revenue each day... such a customer wouldn't > like these license terms at all :-( That depends. Unintentional failure to meet the license terms seems unlikely to me considering that the license doesn't impose a lot of requirments. It's vague in its definitions, but I think that works in your advantage. > BTW, I think that section 6. can be removed altogether, if > it doesn't include any reference to such a 30-60 day period: > the permissions set forth in a license are only valid in case > the license terms are adhered to whether it includes such > a section or not. Try to explain that to a lawyer. :) --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From mal@lemburg.com Thu Aug 3 14:55:28 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 03 Aug 2000 15:55:28 +0200 Subject: [Python-Dev] Re: A small proposed change to dictionaries' "get" method References: <14728.63466.263123.434708@anthem.concentric.net> <3989454C.5C9EF39B@lemburg.com> <200008031256.HAA06107@cj20424-a.reston1.va.home.com> Message-ID: <398979D0.5AF80126@lemburg.com> Guido van Rossum wrote: > > Marc-Andre writes: > > The following one-liner already does what you want: > > > > d[word] = d.get(word, []).append('world') > > Are you using a patch to the list object so that append() returns the > list itself? Or was it just late? For me, this makes d[word] = None. Ouch... looks like I haven't had enough coffee today. I'll fix that immediately ;-) How about making this a method: def inplace(dict, key, default): value = dict.get(key, default) dict[key] = value return value >>> d = {} >>> inplace(d, 'hello', []).append('world') >>> d {'hello': ['world']} >>> inplace(d, 'hello', []).append('world') >>> d {'hello': ['world', 'world']} >>> inplace(d, 'hello', []).append('world') >>> d {'hello': ['world', 'world', 'world']} (Hope I got it right this time ;-) -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From jeremy@beopen.com Thu Aug 3 15:14:13 2000 From: jeremy@beopen.com (Jeremy Hylton) Date: Thu, 3 Aug 2000 10:14:13 -0400 (EDT) Subject: [Python-Dev] Buglist In-Reply-To: <20000803145714.B266@xs4all.nl> References: <20000803145714.B266@xs4all.nl> Message-ID: <14729.32309.807363.345594@bitdiddle.concentric.net> I am done moving old bugs from Jitterbug to SF. There are still some new bugs being submitted to Jitterbug, which I'll need to move one at a time. In principle, it's okay to mark bugs as closed, as long as you are *sure* that the bug has been fixed. If you try to reproduce a bug on your system and can't, it's not clear that it has been fixed. It might be a platform-specific bug, for example. I would prefer it if you only closed bugs where you can point to the CVS checkin that fixed it. Whenever you fix a bug, you should add a test case to the regression test that would have caught the bug. Have you done that for any of the bugs you've marked as closed? You should also add a comment at any bug you're closing explaining why it is closed. It is good to assign bugs to people -- probably even if we end up playing hot potato for a while. If a bug is assigned to you, you should either try to fix it, diagnose it, or assign it to someone else. > I think overlooking a few bugs is better than overlooking all of > them because of the size of the list :P You seem to be arguing that the sheer number of bug reports bothers you and that it's better to have a shorter list of bugs regardless of whether they're actually fixed. Come on! I don't want to overlook any bugs. Jeremy From bwarsaw@beopen.com Thu Aug 3 15:25:20 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Thu, 3 Aug 2000 10:25:20 -0400 (EDT) Subject: [Python-Dev] Go \x yourself References: Message-ID: <14729.32976.819777.292096@anthem.concentric.net> >>>>> "TP" == Tim Peters writes: TP> The new rule is the same as Perl uses for \x escapes in -w TP> mode, except that Python will raise ValueError at compile-time TP> for an invalid \x escape: an \x escape is of the form TP> \xhh TP> where h is a hex digit. That's it. +1 From bwarsaw@beopen.com Thu Aug 3 15:41:10 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Thu, 3 Aug 2000 10:41:10 -0400 (EDT) Subject: [Python-Dev] Re: A small proposed change to dictionaries' "get" method References: <14728.63466.263123.434708@anthem.concentric.net> <3989454C.5C9EF39B@lemburg.com> Message-ID: <14729.33926.145263.296629@anthem.concentric.net> >>>>> "M" == M writes: M> The following one-liner already does what you want: M> d[word] = d.get(word, []).append('world') M> ... and it's in no way more readable than your proposed M> .put() line ;-) Does that mean it's less readable? :) -Barry From mal@lemburg.com Thu Aug 3 15:49:01 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 03 Aug 2000 16:49:01 +0200 Subject: [Python-Dev] Re: A small proposed change to dictionaries' "get" method References: <14728.63466.263123.434708@anthem.concentric.net> <3989454C.5C9EF39B@lemburg.com> <14729.33926.145263.296629@anthem.concentric.net> Message-ID: <3989865D.A52964D6@lemburg.com> "Barry A. Warsaw" wrote: > > >>>>> "M" == M writes: > > M> The following one-liner already does what you want: > > M> d[word] = d.get(word, []).append('world') > > M> ... and it's in no way more readable than your proposed > M> .put() line ;-) > > Does that mean it's less readable? :) I find these .go_home().get_some_cheese().and_eat()... constructions rather obscure. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From thomas@xs4all.net Thu Aug 3 15:49:49 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 3 Aug 2000 16:49:49 +0200 Subject: [Python-Dev] Buglist In-Reply-To: <14729.32309.807363.345594@bitdiddle.concentric.net>; from jeremy@beopen.com on Thu, Aug 03, 2000 at 10:14:13AM -0400 References: <20000803145714.B266@xs4all.nl> <14729.32309.807363.345594@bitdiddle.concentric.net> Message-ID: <20000803164949.D13365@xs4all.nl> On Thu, Aug 03, 2000 at 10:14:13AM -0400, Jeremy Hylton wrote: > In principle, it's okay to mark bugs as closed, as long as you are > *sure* that the bug has been fixed. If you try to reproduce a bug on > your system and can't, it's not clear that it has been fixed. It > might be a platform-specific bug, for example. I would prefer it if > you only closed bugs where you can point to the CVS checkin that fixed > it. This is tricky for some bugreports, as they don't say *anything* about the platform in question. However, I have been conservative, and haven't done anything if I didn't either have the same platform as mentioned and could reproduce the bug with 1.6a2 and/or Python 1.5.2 (very handy to have them lying around) but not with current CVS, OR could find the CVS checkin that fixed them. For instance, the incorrect usage of PyMem_Del() in some modules (bug #110638) *seems* to be fixed, but I can't really test it and the CVS checkin(s) that seem to fix it don't even mention the bug or the reason for the change. > Whenever you fix a bug, you should add a test case to the regression > test that would have caught the bug. Have you done that for any of > the bugs you've marked as closed? No, because all the bugs I've closed so far are 'obviously fixed', by someone other than me. I would write one if I fixed the bug myself, I guess. Also, most of these are more 'issues' rather than 'bugs', like someone complaining about installing Python without Tcl/Tk and Tkinter not working, threads misbehaving on some systems (didn't close that one, just added a remark), etc. > You should also add a comment at any bug you're closing explaining why > it is closed. Of course. I also forward the SF excerpt to the original submittor, since they are not likely to browse the SF buglist and spot their own bug. > It is good to assign bugs to people -- probably even if we end up > playing hot potato for a while. If a bug is assigned to you, you > should either try to fix it, diagnose it, or assign it to someone > else. Hm, I did that for a few, but it's not very easy to find the right person, in some cases. Bugs in the 're' module, should they go to amk or to /F ? XML stuff, should it go to Paul Prescod or some of the other people who seem to be doing something with XML ? A 'capabilities' list would be pretty neat! > > I think overlooking a few bugs is better than overlooking all of > > them because of the size of the list :P > You seem to be arguing that the sheer number of bug reports bothers > you and that it's better to have a shorter list of bugs regardless of > whether they're actually fixed. Come on! I don't want to overlook any > bugs. No, that wasn't what I meant :P It's just that some bugs are vague, and *seem* fixed, but are still an issue on some combination of compiler, libraries, OS, etc. Also, there is the question on whether something is a bug or a feature, or an artifact of compiler, library or design. A quick pass over the bugs will either have to draw a firm line somewhere, or keep most of the bugs and hope someone will look at them. Having 9 out of 10 bugs waiting in the buglist without anyone looking at them because it's too vague and everyone thinks not 'their' field of expertise and expect someone else to look at them, defeats the purpose of the buglist. But closing those bugreports, explaining the problem and even forwarding the excerpt to the submittor *might* result in the original submittor, who still has the bug, to forget about explaining it further, whereas a couple of hours trying to duplicate the bug might locate it. I personally just wouldn't want to be the one doing all that effort ;) Just-trying-to-help-you-do-your-job---not-taking-it-over-ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Thu Aug 3 16:00:03 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 3 Aug 2000 17:00:03 +0200 Subject: [Python-Dev] printing xrange objects In-Reply-To: <14729.28267.517936.331801@cj42289-a.reston1.va.home.com>; from fdrake@beopen.com on Thu, Aug 03, 2000 at 09:06:51AM -0400 References: <14729.28267.517936.331801@cj42289-a.reston1.va.home.com> Message-ID: <20000803170002.C266@xs4all.nl> On Thu, Aug 03, 2000 at 09:06:51AM -0400, Fred L. Drake, Jr. wrote: > There is no tp_str handler, so str(xrange(...)) is the same as > repr(xrange(...)). > I propose ripping out the tp_print handler completely. (And I've > already tested my patch. ;) > Comments? +0... I would say 'swap str and repr', because str(xrange) does what repr(xrange) should do, and the other way 'round: >>> x = xrange(1000) >>> repr(x) (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, ... ... ... ... 998, 999) >>> str(x) '(xrange(0, 1000, 1) * 1)' But I don't really care either way. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From fdrake@beopen.com Thu Aug 3 16:14:57 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Thu, 3 Aug 2000 11:14:57 -0400 (EDT) Subject: [Python-Dev] printing xrange objects In-Reply-To: <20000803170002.C266@xs4all.nl> References: <14729.28267.517936.331801@cj42289-a.reston1.va.home.com> <20000803170002.C266@xs4all.nl> Message-ID: <14729.35953.19610.61905@cj42289-a.reston1.va.home.com> Thomas Wouters writes: > >>> x = xrange(1000) > >>> repr(x) > (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, > ... ... ... > ... 998, 999) > > >>> str(x) > '(xrange(0, 1000, 1) * 1)' What version is this with? 1.5.2 gives me: Python 1.5.2 (#1, May 9 2000, 15:05:56) [GCC 2.95.3 19991030 (prerelease)] on linux-i386 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> x = xrange(2) >>> str(x) '(xrange(0, 2, 1) * 1)' >>> repr(x) '(xrange(0, 2, 1) * 1)' >>> x (0, 1) The 1.6b1 that's getting itself ready says this: Python 1.6b1 (#19, Aug 2 2000, 01:11:29) [GCC 2.95.3 19991030 (prerelease)] on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam Copyright 1995-2000 Corporation for National Research Initiatives (CNRI) Module readline not available. >>> x = xrange(2) >>> str(x) '(xrange(0, 2, 1) * 1)' >>> repr(x) '(xrange(0, 2, 1) * 1)' >>> x (0, 1) What I'm proposing is: Python 2.0b1 (#116, Aug 2 2000, 15:35:35) [GCC 2.95.3 19991030 (prerelease)] on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam Copyright 1995-2000 Corporation for National Research Initiatives (CNRI) >>> x = xrange(2) >>> str(x) 'xrange(0, 2, 1)' >>> repr(x) 'xrange(0, 2, 1)' >>> x xrange(0, 2, 1) (Where the outer (... * n) is added only when n != 1, 'cause I think that's just ugly.) -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From thomas@xs4all.net Thu Aug 3 16:30:23 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 3 Aug 2000 17:30:23 +0200 Subject: [Python-Dev] printing xrange objects In-Reply-To: <14729.35953.19610.61905@cj42289-a.reston1.va.home.com>; from fdrake@beopen.com on Thu, Aug 03, 2000 at 11:14:57AM -0400 References: <14729.28267.517936.331801@cj42289-a.reston1.va.home.com> <20000803170002.C266@xs4all.nl> <14729.35953.19610.61905@cj42289-a.reston1.va.home.com> Message-ID: <20000803173023.D266@xs4all.nl> On Thu, Aug 03, 2000 at 11:14:57AM -0400, Fred L. Drake, Jr. wrote: > Thomas Wouters writes: > > >>> x = xrange(1000) > > >>> repr(x) > > (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, > > ... ... ... > > ... 998, 999) > > > > >>> str(x) > > '(xrange(0, 1000, 1) * 1)' > What version is this with? 1.5.2 gives me: > > Python 1.5.2 (#1, May 9 2000, 15:05:56) [GCC 2.95.3 19991030 (prerelease)] on linux-i386 > Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam > >>> x = xrange(2) > >>> str(x) > '(xrange(0, 2, 1) * 1)' > >>> repr(x) > '(xrange(0, 2, 1) * 1)' > >>> x > (0, 1) Sorry, my bad. I just did 'x', and assumed it called repr(). I guess my newbiehood shows in that I thought 'print x' always called 'str(x)'. Like I replied to Tim this morning, after he caught me in the same kind of ebmarrasing thinko: Sigh, that's what I get for getting up when my GF had to and being at the office at 8am. Don't mind my postings today, they're likely 99% brainfart. Seeing as how 'print "range: %s"%x' did already use the 'str' and 'repr' output, I see no reason not to make 'print x' do the same. So +1. > >>> x > xrange(0, 2, 1) > > (Where the outer (... * n) is added only when n != 1, 'cause I think > that's just ugly.) Why not remove the first and last argument, if they are respectively 0 and 1? >>> xrange(100) xrange(100) >>> xrange(10,100) xrange(10, 100) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From fdrake@beopen.com Thu Aug 3 16:48:28 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Thu, 3 Aug 2000 11:48:28 -0400 (EDT) Subject: [Python-Dev] printing xrange objects In-Reply-To: <20000803173023.D266@xs4all.nl> References: <14729.28267.517936.331801@cj42289-a.reston1.va.home.com> <20000803170002.C266@xs4all.nl> <14729.35953.19610.61905@cj42289-a.reston1.va.home.com> <20000803173023.D266@xs4all.nl> Message-ID: <14729.37964.46818.653202@cj42289-a.reston1.va.home.com> Thomas Wouters writes: > Sorry, my bad. I just did 'x', and assumed it called repr(). I guess my > newbiehood shows in that I thought 'print x' always called 'str(x)'. Like I That's the evil beauty of tp_print -- nobody really expects it because most types don't implement it (and don't need to); I seem to recall Guido saying it was a performance issue for certain types, but don't recall the specifics. > Why not remove the first and last argument, if they are respectively 0 and 1? I agree! In fact, always removing the last argument if it == 1 is a good idea as well. Here's the output from the current patch: >>> xrange(2) xrange(2) >>> xrange(2, 4) xrange(2, 4) >>> x = xrange(10, 4, -1) >>> x xrange(10, 4, -1) >>> x.tolist() [10, 9, 8, 7, 6, 5] >>> x*3 (xrange(10, 4, -1) * 3) -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From jeremy@beopen.com Thu Aug 3 17:26:51 2000 From: jeremy@beopen.com (Jeremy Hylton) Date: Thu, 3 Aug 2000 12:26:51 -0400 (EDT) Subject: [Python-Dev] Buglist In-Reply-To: <20000803164949.D13365@xs4all.nl> References: <20000803145714.B266@xs4all.nl> <14729.32309.807363.345594@bitdiddle.concentric.net> <20000803164949.D13365@xs4all.nl> Message-ID: <14729.40267.557470.612144@bitdiddle.concentric.net> >>>>> "TW" == Thomas Wouters writes: >> It is good to assign bugs to people -- probably even if we end up >> playing hot potato for a while. If a bug is assigned to you, you >> should either try to fix it, diagnose it, or assign it to someone >> else. TW> Hm, I did that for a few, but it's not very easy to find the TW> right person, in some cases. Bugs in the 're' module, should TW> they go to amk or to /F ? XML stuff, should it go to Paul TW> Prescod or some of the other people who seem to be doing TW> something with XML ? A 'capabilities' list would be pretty neat! I had the same problem when I was trying to assign bugs. It is seldom clear who should be assigned a bug. I have used two rules when processing open, uncategorized bugs: * If you have a reasonable guess about who to assign a bug to, it's better to assign to the wrong person than not to assign at all. If the wrong person gets it, she can assign it to someone else. * If you don't know who to assign it to, at least give it a category. That allows someone who feels expert in a category (e.g. a Tkinter guru), to easily scan all the unassigned bugs in that category. >> You seem to be arguing that the sheer number of bug reports >> bothers you and that it's better to have a shorter list of bugs >> regardless of whether they're actually fixed. Come on! I don't >> want to overlook any bugs. TW> No, that wasn't what I meant :P Sorry. I didn't believe you really meant that, but you came off sounding like you did :-). TW> Having 9 out of 10 bugs waiting in the buglist without anyone TW> looking at them because it's too vague and everyone thinks not TW> 'their' field of expertise and expect someone else to look at TW> them, defeats the purpose of the buglist. I still don't agree here. If you're not fairly certain about the bug, keep it on the list. I don't see too much harm in having vague, open bugs on the list. TW> But closing those TW> bugreports, explaining the problem and even forwarding the TW> excerpt to the submittor *might* result in the original TW> submittor, who still has the bug, to forget about explaining it TW> further, whereas a couple of hours trying to duplicate the bug TW> might locate it. I personally just wouldn't want to be the one TW> doing all that effort ;) You can send mail to the person who reported the bug and ask her for more details without closing it. TW> Just-trying-to-help-you-do-your-job---not-taking-it-over-ly And I appreciate the help!! The more bugs we have categorized or assigned, the better. of-course-actually-fixing-real-bugs-is-good-too-ly y'rs, Jeremy From Moshe Zadka Thu Aug 3 17:44:28 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 3 Aug 2000 19:44:28 +0300 (IDT) Subject: [Python-Dev] Breaking Test Cases on Purpose Message-ID: Suppose I'm fixing a bug in the library. I want peer review for my fix, but I need none for my new "would have caught" test cases. Is it considered alright to check-in right away the test case, breaking the test suite, and to upload a patch to SF to fix it? Or should the patch include the new test cases? The XP answer would be "hey, you have to checkin the breaking test case right away", and I'm inclined to agree. I really want to break the standard library, just because I'm a sadist -- but seriously, we need tests that break more often, so bugs will be easier to fix. waiting-for-fellow-sadists-ly y'rs, Z. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From guido@beopen.com Thu Aug 3 18:54:55 2000 From: guido@beopen.com (Guido van Rossum) Date: Thu, 03 Aug 2000 12:54:55 -0500 Subject: [Python-Dev] Breaking Test Cases on Purpose In-Reply-To: Your message of "Thu, 03 Aug 2000 19:44:28 +0300." References: Message-ID: <200008031754.MAA08812@cj20424-a.reston1.va.home.com> > Suppose I'm fixing a bug in the library. I want peer review for my fix, > but I need none for my new "would have caught" test cases. Is it > considered alright to check-in right away the test case, breaking the test > suite, and to upload a patch to SF to fix it? Or should the patch include > the new test cases? > > The XP answer would be "hey, you have to checkin the breaking test case > right away", and I'm inclined to agree. > > I really want to break the standard library, just because I'm a sadist -- > but seriously, we need tests that break more often, so bugs will be easier > to fix. In theory I'm with you. In practice, each time the test suite breaks, we get worried mail from people who aren't following the list closely, did a checkout, and suddenly find that the test suite breaks. That just adds noise to the list. So I'm against it. -1 --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From Moshe Zadka Thu Aug 3 17:55:41 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 3 Aug 2000 19:55:41 +0300 (IDT) Subject: [Python-Dev] Breaking Test Cases on Purpose In-Reply-To: <200008031754.MAA08812@cj20424-a.reston1.va.home.com> Message-ID: On Thu, 3 Aug 2000, Guido van Rossum wrote: > In theory I'm with you. In practice, each time the test suite breaks, > we get worried mail from people who aren't following the list closely, > did a checkout, and suddenly find that the test suite breaks. That > just adds noise to the list. So I'm against it. > > -1 In theory, theory and practice shouldn't differ. In practice, they do. Guido, you're way too much realist <1.6 wink> Oh, well. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From gstein@lyra.org Thu Aug 3 18:04:01 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 3 Aug 2000 10:04:01 -0700 Subject: [Python-Dev] Breaking Test Cases on Purpose In-Reply-To: ; from moshez@math.huji.ac.il on Thu, Aug 03, 2000 at 07:44:28PM +0300 References: Message-ID: <20000803100401.T19525@lyra.org> On Thu, Aug 03, 2000 at 07:44:28PM +0300, Moshe Zadka wrote: > Suppose I'm fixing a bug in the library. I want peer review for my fix, > but I need none for my new "would have caught" test cases. Is it > considered alright to check-in right away the test case, breaking the test > suite, and to upload a patch to SF to fix it? Or should the patch include > the new test cases? If you're fixing a bug, then check in *both* pieces and call explicitly for a peer reviewer (plus the people watching -checkins). If you don't quite fix the bug, then a second checkin can smooth things out. Let's not get too caught up in "process", to the exclusion of being productive about bug fixing. > The XP answer would be "hey, you have to checkin the breaking test case > right away", and I'm inclined to agree. > > I really want to break the standard library, just because I'm a sadist -- > but seriously, we need tests that break more often, so bugs will be easier > to fix. I really want to see less process and discussion, and more code. Cheers, -g -- Greg Stein, http://www.lyra.org/ From Fredrik Lundh" <20000802180059.B30340@kronos.cnri.reston.va.us> <006601bffd24$e25a9360$f2a6b5d4@hagrid> <200008031331.IAA06319@cj20424-a.reston1.va.home.com> Message-ID: <007401bffd6e$ed9bbde0$f2a6b5d4@hagrid> guido wrote: > > ...but sure, I will fix that in 0.9.9 (SRE, not Python -- Christian > > has already taken care of the other one ;-). but 0.9.9 won't be > > out before the 1.6b1 release... >=20 > I assume you are planning to put the backtracking stack back in, as > you mentioned in the checkin message? yup -- but that'll have to wait a few more days... > > (and to avoid scaring the hell out of the beta testers, it's = probably > > better to leave the test out of the regression suite until the bug = is > > fixed...) >=20 > Even better, is it possible to put a limit on the recursion level > before 1.6b1 is released (tomorrow if we get final agreement on the > license) so at least it won't dump core? shouldn't be too hard, given that I added a "recursion level counter" in _sre.c revision 2.30. I just added the necessary if-statement. From gstein@lyra.org Thu Aug 3 19:39:08 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 3 Aug 2000 11:39:08 -0700 Subject: [Python-Dev] Breaking Test Cases on Purpose In-Reply-To: <200008031754.MAA08812@cj20424-a.reston1.va.home.com>; from guido@beopen.com on Thu, Aug 03, 2000 at 12:54:55PM -0500 References: <200008031754.MAA08812@cj20424-a.reston1.va.home.com> Message-ID: <20000803113908.X19525@lyra.org> On Thu, Aug 03, 2000 at 12:54:55PM -0500, Guido van Rossum wrote: > > Suppose I'm fixing a bug in the library. I want peer review for my fix, > > but I need none for my new "would have caught" test cases. Is it > > considered alright to check-in right away the test case, breaking the test > > suite, and to upload a patch to SF to fix it? Or should the patch include > > the new test cases? > > > > The XP answer would be "hey, you have to checkin the breaking test case > > right away", and I'm inclined to agree. > > > > I really want to break the standard library, just because I'm a sadist -- > > but seriously, we need tests that break more often, so bugs will be easier > > to fix. > > In theory I'm with you. In practice, each time the test suite breaks, > we get worried mail from people who aren't following the list closely, > did a checkout, and suddenly find that the test suite breaks. That > just adds noise to the list. So I'm against it. Tell those people to chill out for a few days and not be so jumpy. You're talking about behavior that can easily be remedied. It is a simple statement about the CVS repository: "CVS builds but may not pass the test suite in certain cases" rather than "CVS is perfect" Cheers, -g -- Greg Stein, http://www.lyra.org/ From tim_one@email.msn.com Thu Aug 3 19:49:02 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 3 Aug 2000 14:49:02 -0400 Subject: [Python-Dev] Breaking Test Cases on Purpose In-Reply-To: Message-ID: [Moshe Zadka] > Suppose I'm fixing a bug in the library. I want peer review > for my fix, but I need none for my new "would have caught" > test cases. Is it considered alright to check-in right away > the test case, breaking the test suite, and to upload a patch > to SF to fix it? Or should the patch include the new test cases? > > The XP answer would be "hey, you have to checkin the breaking > test case right away", and I'm inclined to agree. It's abhorrent to me to ever leave the tree in a state where a test is "expected to fail". If it's left in a failing state for a brief period, at best other developers will waste time wondering whether it's due to something they did. If it's left in a failing state longer than that, people quickly stop paying attention to failures at all (the difference between "all passed" and "something failed" is huge, the differences among 1 or 2 or 3 or ... failures get overlooked, and we've seen over and over that when 1 failure is allowed to persist, others soon join it). You can check in an anti-test right away, though: a test that passes so long as the code remains broken . From jeremy@beopen.com Thu Aug 3 19:58:15 2000 From: jeremy@beopen.com (Jeremy Hylton) Date: Thu, 3 Aug 2000 14:58:15 -0400 (EDT) Subject: [Python-Dev] Breaking Test Cases on Purpose In-Reply-To: References: Message-ID: <14729.49351.574550.48521@bitdiddle.concentric.net> I'm Tim on this issue. As officially appointed release manager for 2.0, I set some guidelines for checking in code. One is that no checkin should cause the regression test to fail. If it does, I'll back it out. If you didn't review the contribution guidelines when they were posted on this list, please look at PEP 200 now. Jeremy From jeremy@beopen.com Thu Aug 3 20:00:23 2000 From: jeremy@beopen.com (Jeremy Hylton) Date: Thu, 3 Aug 2000 15:00:23 -0400 (EDT) Subject: [Python-Dev] Breaking Test Cases on Purpose In-Reply-To: <14729.49351.574550.48521@bitdiddle.concentric.net> References: <14729.49351.574550.48521@bitdiddle.concentric.net> Message-ID: <14729.49479.677157.957162@bitdiddle.concentric.net> >>>>> "JH" == Jeremy Hylton writes: JH> I'm Tim on this issue. Make that "I'm with Tim on this issue." I'm sure it would be fun to channel Tim, but I don't have the skills for that. Jeremy From guido@beopen.com Thu Aug 3 21:02:07 2000 From: guido@beopen.com (Guido van Rossum) Date: Thu, 03 Aug 2000 15:02:07 -0500 Subject: [Python-Dev] Breaking Test Cases on Purpose In-Reply-To: Your message of "Thu, 03 Aug 2000 11:39:08 MST." <20000803113908.X19525@lyra.org> References: <200008031754.MAA08812@cj20424-a.reston1.va.home.com> <20000803113908.X19525@lyra.org> Message-ID: <200008032002.PAA17349@cj20424-a.reston1.va.home.com> > Tell those people to chill out for a few days and not be so jumpy. You're > talking about behavior that can easily be remedied. > > It is a simple statement about the CVS repository: "CVS builds but may not > pass the test suite in certain cases" rather than "CVS is perfect" I would agree if it was only the python-dev crowd -- they are easily trained. But there are lots of others who check out the tree, so it would be a continuing education process. I don't see what good it does. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From Fredrik Lundh" Message-ID: <00d501bffd7e$deb6ece0$f2a6b5d4@hagrid> moshe: > > The XP answer would be "hey, you have to checkin the breaking > > test case right away", and I'm inclined to agree. tim: > It's abhorrent to me to ever leave the tree in a state where a test is > "expected to fail". If it's left in a failing state for a brief = period, at > best other developers will waste time wondering whether it's due to > something they did note that we've just seen this in action, in the SRE crash thread. Andrew checked in a test that caused the test suite to bomb, and sent me and Mark F. looking for an non-existing portability bug... > You can check in an anti-test right away, though: a test that passes = so > long as the code remains broken . which is what the new SRE test script does -- the day SRE supports unlimited recursion (soon), the test script will complain... From tim_one@email.msn.com Thu Aug 3 20:06:49 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 3 Aug 2000 15:06:49 -0400 Subject: [Python-Dev] Breaking Test Cases on Purpose In-Reply-To: <14729.49351.574550.48521@bitdiddle.concentric.net> Message-ID: [Jeremy Hylton] > I'm Tim on this issue. Then I'm Jeremy too. Wow! I needed a vacation . From guido@beopen.com Thu Aug 3 21:15:26 2000 From: guido@beopen.com (Guido van Rossum) Date: Thu, 03 Aug 2000 15:15:26 -0500 Subject: [Python-Dev] Breaking Test Cases on Purpose In-Reply-To: Your message of "Thu, 03 Aug 2000 15:00:23 -0400." <14729.49479.677157.957162@bitdiddle.concentric.net> References: <14729.49351.574550.48521@bitdiddle.concentric.net> <14729.49479.677157.957162@bitdiddle.concentric.net> Message-ID: <200008032015.PAA17571@cj20424-a.reston1.va.home.com> > JH> I'm Tim on this issue. > > Make that "I'm with Tim on this issue." I'm sure it would be fun to > channel Tim, but I don't have the skills for that. Actually, in my attic there's a small door that leads to a portal into Tim's brain. Maybe we could get Tim to enter the portal -- it would be fun to see him lying on a piano in a dress reciting a famous aria. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From jeremy@beopen.com Thu Aug 3 20:19:18 2000 From: jeremy@beopen.com (Jeremy Hylton) Date: Thu, 3 Aug 2000 15:19:18 -0400 (EDT) Subject: [Python-Dev] Breaking Test Cases on Purpose In-Reply-To: <200008032015.PAA17571@cj20424-a.reston1.va.home.com> References: <14729.49351.574550.48521@bitdiddle.concentric.net> <14729.49479.677157.957162@bitdiddle.concentric.net> <200008032015.PAA17571@cj20424-a.reston1.va.home.com> Message-ID: <14729.50614.806442.190962@bitdiddle.concentric.net> >>>>> "GvR" == Guido van Rossum writes: JH> I'm Tim on this issue. >> Make that "I'm with Tim on this issue." I'm sure it would be >> fun to channel Tim, but I don't have the skills for that. GvR> Actually, in my attic there's a small door that leads to a GvR> portal into Tim's brain. Maybe we could get Tim to enter the GvR> portal -- it would be fun to see him lying on a piano in a GvR> dress reciting a famous aria. You should have been on the ride from Monterey to the San Jose airport a couple of weeks ago. There was no piano, but it was pretty close. Jeremy From jeremy@beopen.com Thu Aug 3 20:31:50 2000 From: jeremy@beopen.com (Jeremy Hylton) Date: Thu, 3 Aug 2000 15:31:50 -0400 (EDT) Subject: [Python-Dev] tests for standard library modules Message-ID: <14729.51366.391122.131492@bitdiddle.concentric.net> Most of the standard library is untested. There are 148 top-level Python modules in the standard library, plus a few packages that contain 50 or 60 more modules. When we run the regression test, we only touch 48 of those modules. Only 18 of the modules have their own test suite. The other 30 modules at least get imported, though sometimes none of the code gets executed. (The traceback module is an example.) I would like to see much better test coverage in Python 2.0. I would welcome any test case submissions that improve the coverage of the standard library. Skip's trace.py code coverage tool is now available in Tools/script. You can use it to examine how much of a particular module is covered by existing tests. Jeremy From guido@beopen.com Thu Aug 3 21:39:44 2000 From: guido@beopen.com (Guido van Rossum) Date: Thu, 03 Aug 2000 15:39:44 -0500 Subject: [Python-Dev] tests for standard library modules In-Reply-To: Your message of "Thu, 03 Aug 2000 15:31:50 -0400." <14729.51366.391122.131492@bitdiddle.concentric.net> References: <14729.51366.391122.131492@bitdiddle.concentric.net> Message-ID: <200008032039.PAA17852@cj20424-a.reston1.va.home.com> > Most of the standard library is untested. Indeed. I would suggest looking at the Tcl test suite. It's very thorough! When I look at many of the test modules we *do* have, I cringe at how little of the module the test actually covers. Many tests (not the good ones!) seem to be content with checking that all functions in a module *exist*. Much of this dates back to one particular period in 1996-1997 when we (at CNRI) tried to write test suites for all modules -- clearly we were in a hurry! :-( --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From Vladimir.Marangozov@inrialpes.fr Thu Aug 3 23:25:38 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 4 Aug 2000 00:25:38 +0200 (CEST) Subject: [Python-Dev] tests for standard library modules In-Reply-To: <14729.51366.391122.131492@bitdiddle.concentric.net> from "Jeremy Hylton" at Aug 03, 2000 03:31:50 PM Message-ID: <200008032225.AAA27154@python.inrialpes.fr> Jeremy Hylton wrote: > > Skip's trace.py code coverage tool is now available in Tools/script. > You can use it to examine how much of a particular module is covered > by existing tests. Hmm. Glancing quickly at trace.py, I see that half of it is guessing line numbers. The same SET_LINENO problem again. This is unfortunate. But fortunately , here's another piece of code, modeled after its C counterpart, that comes to Skip's rescue and that works with -O. Example: >>> import codeutil >>> co = codeutil.PyCode_Line2Addr.func_code # some code object >>> codeutil.PyCode_GetExecLines(co) [20, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36] >>> codeutil.PyCode_Line2Addr(co, 29) 173 >>> codeutil.PyCode_Addr2Line(co, 173) 29 >>> codeutil.PyCode_Line2Addr(co, 10) Traceback (innermost last): File " ", line 1, in ? File "codeutil.py", line 26, in PyCode_Line2Addr raise IndexError, "line must be in range [%d,%d]" % (line, lastlineno) IndexError: line must be in range [20,36] etc... -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 ------------------------------[ codeutil.py ]------------------------- import types def PyCode_Addr2Line(co, addrq): assert type(co) == types.CodeType, \ "1st arg must be a code object, %s given" % type(co).__name__ if addrq < 0 or addrq > len(co.co_code): raise IndexError, "address must be in range [0,%d]" % len(co.co_code) addr = 0 line = co.co_firstlineno lnotab = co.co_lnotab for i in range(0, len(lnotab), 2): addr_incr = ord(lnotab[i]) line_incr = ord(lnotab[i+1]) addr = addr + addr_incr if (addr > addrq): break line = line + line_incr return line def PyCode_Line2Addr(co, lineq): assert type(co) == types.CodeType, \ "1st arg must be a code object, %s given" % type(co).__name__ line = co.co_firstlineno lastlineno = PyCode_Addr2Line(co, len(co.co_code)) if lineq < line or lineq > lastlineno: raise IndexError, "line must be in range [%d,%d]" % (line, lastlineno) addr = 0 lnotab = co.co_lnotab for i in range(0, len(lnotab), 2): if line >= lineq: break addr_incr = ord(lnotab[i]) line_incr = ord(lnotab[i+1]) addr = addr + addr_incr line = line + line_incr return addr def PyCode_GetExecLines(co): assert type(co) == types.CodeType, \ "arg must be a code object, %s given" % type(co).__name__ lastlineno = PyCode_Addr2Line(co, len(co.co_code)) lines = range(co.co_firstlineno, lastlineno + 1) # remove void lines (w/o opcodes): comments, blank/escaped lines i = len(lines) - 1 while i >= 0: if lines[i] != PyCode_Addr2Line(co, PyCode_Line2Addr(co, lines[i])): lines.pop(i) i = i - 1 return lines From mwh21@cam.ac.uk Thu Aug 3 23:19:51 2000 From: mwh21@cam.ac.uk (Michael Hudson) Date: 03 Aug 2000 23:19:51 +0100 Subject: [Python-Dev] Breaking Test Cases on Purpose In-Reply-To: Moshe Zadka's message of "Thu, 3 Aug 2000 19:44:28 +0300 (IDT)" References: Message-ID: Moshe Zadka writes: > Suppose I'm fixing a bug in the library. I want peer review for my fix, > but I need none for my new "would have caught" test cases. Is it > considered alright to check-in right away the test case, breaking the test > suite, and to upload a patch to SF to fix it? Or should the patch include > the new test cases? > > The XP answer would be "hey, you have to checkin the breaking test case > right away", and I'm inclined to agree. I'm not so sure. I can't find the bit I'm looking for in Beck's book[1], but ISTR that you have two sorts of test, unit tests and functional tests. Unit tests always work, functional tests are more what you want to work in the future, but may not now. What goes in Lib/test is definitely more of the unit test variety, and so if something in there breaks it's a cause for alarm. Checking in a test you know will break just raises blood pressure for no good reason. Also what if you're hacking on some bit of Python, run the test suite and it fails? You worry that you've broken something, when in fact it's nothing to do with you. -1. (like everyone else...) Cheers, M. [1] Found it; p. 118 of "Extreme Programming Explained" -- I'm okay with intellegent buildings, I'm okay with non-sentient buildings. I have serious reservations about stupid buildings. -- Dan Sheppard, ucam.chat (from Owen Dunn's summary of the year) From skip@mojam.com (Skip Montanaro) Thu Aug 3 23:21:04 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 3 Aug 2000 17:21:04 -0500 (CDT) Subject: [Python-Dev] Re: A small proposed change to dictionaries' "get" method In-Reply-To: <398979D0.5AF80126@lemburg.com> References: <14728.63466.263123.434708@anthem.concentric.net> <3989454C.5C9EF39B@lemburg.com> <200008031256.HAA06107@cj20424-a.reston1.va.home.com> <398979D0.5AF80126@lemburg.com> Message-ID: <14729.61520.11958.530601@beluga.mojam.com> >> How about making this a method: >> def inplace(dict, key, default): >> value = dict.get(key, default) >> dict[key] = value >> return value eh... I don't like these do two things at once kind of methods. I see nothing wrong with >>> dict = {} >>> dict['hello'] = dict.get('hello', []) >>> dict['hello'].append('world') >>> print dict {'hello': ['world']} or >>> d = dict['hello'] = dict.get('hello', []) >>> d.insert(0, 'cruel') >>> print dict {'hello': ['cruel', 'world']} for the obsessively efficiency-minded folks. Also, we're talking about a method that would generally only be useful when dictionaries have values which were mutable objects. Irregardless of how useful instances and lists are, I still find that my predominant day-to-day use of dictionaries is with strings as keys and values. Perhaps that's just the nature of my work. In short, I don't think anything needs to be changed. -1 (don't like the concept, so I don't care about the implementation) Skip From mal@lemburg.com Thu Aug 3 23:36:33 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 04 Aug 2000 00:36:33 +0200 Subject: [Python-Dev] Line number tools (tests for standard library modules) References: <200008032225.AAA27154@python.inrialpes.fr> Message-ID: <3989F3F1.162A9766@lemburg.com> Vladimir Marangozov wrote: > > Jeremy Hylton wrote: > > > > Skip's trace.py code coverage tool is now available in Tools/script. > > You can use it to examine how much of a particular module is covered > > by existing tests. > > Hmm. Glancing quickly at trace.py, I see that half of it is guessing > line numbers. The same SET_LINENO problem again. This is unfortunate. > But fortunately , here's another piece of code, modeled after > its C counterpart, that comes to Skip's rescue and that works with -O. > > Example: > > >>> import codeutil > >>> co = codeutil.PyCode_Line2Addr.func_code # some code object > >>> codeutil.PyCode_GetExecLines(co) > [20, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36] > >>> codeutil.PyCode_Line2Addr(co, 29) > 173 > >>> codeutil.PyCode_Addr2Line(co, 173) > 29 > >>> codeutil.PyCode_Line2Addr(co, 10) > Traceback (innermost last): > File " ", line 1, in ? > File "codeutil.py", line 26, in PyCode_Line2Addr > raise IndexError, "line must be in range [%d,%d]" % (line, lastlineno) > IndexError: line must be in range [20,36] > > etc... Cool. With proper Python style names these utilities would be nice additions for e.g. codeop.py or code.py. BTW, I wonder why code.py includes Python console emulations: there seems to be a naming bug there... I would have named the module PythonConsole.py and left code.py what it was previously: a collection of tools dealing with Python code objects. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From skip@mojam.com (Skip Montanaro) Thu Aug 3 23:53:58 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 3 Aug 2000 17:53:58 -0500 (CDT) Subject: [Python-Dev] tests for standard library modules In-Reply-To: <14729.51366.391122.131492@bitdiddle.concentric.net> References: <14729.51366.391122.131492@bitdiddle.concentric.net> Message-ID: <14729.63494.544079.516429@beluga.mojam.com> Jeremy> Skip's trace.py code coverage tool is now available in Jeremy> Tools/script. You can use it to examine how much of a Jeremy> particular module is covered by existing tests. Yes, though note that in the summary stuff on my web site there are obvious bugs that I haven't had time to look at. Sometimes modules are counted twice. Other times a module is listed as untested when right above it there is a test coverage line... Skip From skip@mojam.com (Skip Montanaro) Thu Aug 3 23:59:34 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 3 Aug 2000 17:59:34 -0500 (CDT) Subject: [Python-Dev] tests for standard library modules In-Reply-To: <200008032225.AAA27154@python.inrialpes.fr> References: <14729.51366.391122.131492@bitdiddle.concentric.net> <200008032225.AAA27154@python.inrialpes.fr> Message-ID: <14729.63830.894657.930184@beluga.mojam.com> Vlad> Hmm. Glancing quickly at trace.py, I see that half of it is Vlad> guessing line numbers. The same SET_LINENO problem again. This is Vlad> unfortunate. But fortunately , here's another piece of Vlad> code, modeled after its C counterpart, that comes to Skip's rescue Vlad> and that works with -O. Go ahead and check in any changes you see that need doing. I haven't fiddled with trace.py much in the past couple of years, so there are some places that clearly do things different than currently accepted practice. (I am going to be up to my ass in alligators pretty much from now through Labor Day (early September for the furriners among us), so things I thought I would get to probably will remain undone. The most important thing is to fix the list comprehensions patch to force expression tuples to be parenthesized. Guido says it's an easy fix, and the grammar changes seem trivial, but fixing compile.c is beyond my rusty knowledge at the moment. Someone want to pick this up?) Skip From MarkH@ActiveState.com Fri Aug 4 00:13:06 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Fri, 4 Aug 2000 09:13:06 +1000 Subject: [Python-Dev] (os.kill (was Fork) on Win32 - was (test_fork1 failing...) In-Reply-To: <39897635.6C9FB82D@lemburg.com> Message-ID: [Marc writes] > On Unix you can install a signal > handler in the Python program which then translates the SIGTERM > signal into a normal Python exception. Sending the signal then > causes the same as e.g. hitting Ctrl-C in a program: an > exception is raised asynchronously, but it can be handled > properly by the Python exception clauses to enable safe > shutdown of the process. I understand this. This is why I was skeptical that a "terminate-without-prejudice" only version would be useful. I _think_ this fairly large email is agreeing that it isn't of much use. If so, then I am afraid you are on your own :-( Mark. From Vladimir.Marangozov@inrialpes.fr Fri Aug 4 00:27:39 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 4 Aug 2000 01:27:39 +0200 (CEST) Subject: [Python-Dev] Removing the 16 bit arg limit Message-ID: <200008032327.BAA27362@python.inrialpes.fr> I've looked at this and the best compromise solution I ended up with (before Py3K) is sketched here: opcode.h: #define EXTENDED_ARG 135 /* 16 higher bits of the next opcode arg */ ceval.c: case EXTENDED_ARG: do { oparg <<= 16; op = NEXTOP(); oparg += NEXTARG(); } while (op == EXTENDED_ARG); goto dispatch_opcode; compile.c: static void com_addoparg(struct compiling *c, int op, int arg) { if (arg < 0) { com_error(c, PyExc_SystemError, "com_addoparg: argument out of range"); } if (op == SET_LINENO) { com_set_lineno(c, arg); if (Py_OptimizeFlag) return; } do { int arg2 = arg & 0xffff; arg -= arg2; if (arg > 0) com_addbyte(c, EXTENDED_ARG); else com_addbyte(c, op); com_addint(c, arg2); } while (arg > 0); } But this is only difficulty level 0. Difficulty level 1 is the jumps and their forward refs & backpatching in compile.c. There's no tricky solution to this (due to the absolute jumps). The only reasonable, long-term useful solution I can think of is to build a table of all anchors (delimiting the basic blocks of the code), then make a final pass over the serialized basic blocks and update the anchors (with or without EXTENDED_ARG jumps depending on the need). However, I won't even think about it anymore whithout BDFL & Tim's approval and strong encouragement . -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From gward@python.net Fri Aug 4 02:24:44 2000 From: gward@python.net (Greg Ward) Date: Thu, 3 Aug 2000 21:24:44 -0400 Subject: [Python-Dev] Library pragma in PC/config.h Message-ID: <20000803212444.A1237@beelzebub> Hi all -- for building extensions with non-MS compilers, it sounds like a small change to PC/config.h is needed. Rene Liebscher suggests changing #ifndef USE_DL_EXPORT /* So nobody needs to specify the .lib in their Makefile any more */ #ifdef _DEBUG #pragma comment(lib,"python20_d.lib") #else #pragma comment(lib,"python20.lib") #endif #endif /* USE_DL_EXPORT */ to #if !defined(USE_DL_EXPORT) && defined(_MSC_VER) ... That way, the convenience pragma will still be there for MSVC users, but it won't break building extensions with Borland C++. (As I understand it, Borland C++ understands the pragma, but then tries to use Python's standard python20.lib, which of course is only for MSVC.) Non-MSVC users will have to explicitly supply the library, but that's OK: the Distutils does it for them. (Even with MSVC, because it's too much bother *not* to specify python20.lib explicitly.) Does this look like the right change to everyone? I can check it in (and on the 1.6 branch too) if it looks OK. While I have your attention, Rene also suggests the convention of "bcpp_python20.lib" for the Borland-format lib file, with other compilers (library formats) supported in future similarly. Works for me -- anyone have any problems with that? Greg -- Greg Ward - programmer-at-big gward@python.net http://starship.python.net/~gward/ Know thyself. If you need help, call the CIA. From Moshe Zadka Fri Aug 4 02:38:32 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 4 Aug 2000 04:38:32 +0300 (IDT) Subject: [Python-Dev] Breaking Test Cases on Purpose In-Reply-To: <14729.49351.574550.48521@bitdiddle.concentric.net> Message-ID: On Thu, 3 Aug 2000, Jeremy Hylton wrote: > I'm Tim on this issue. As officially appointed release manager for > 2.0, I set some guidelines for checking in code. One is that no > checkin should cause the regression test to fail. If it does, I'll > back it out. > > If you didn't review the contribution guidelines when they were posted > on this list, please look at PEP 200 now. Actually, I did. The thing is, it seems to me there's a huge difference between breaking code, and manifesting that the code is wrong. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From Moshe Zadka Fri Aug 4 02:41:12 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 4 Aug 2000 04:41:12 +0300 (IDT) Subject: [Python-Dev] Breaking Test Cases on Purpose In-Reply-To: <200008032015.PAA17571@cj20424-a.reston1.va.home.com> Message-ID: On Thu, 3 Aug 2000, Guido van Rossum wrote: > > JH> I'm Tim on this issue. > > > > Make that "I'm with Tim on this issue." I'm sure it would be fun to > > channel Tim, but I don't have the skills for that. > > Actually, in my attic there's a small door that leads to a portal into > Tim's brain. Maybe we could get Tim to enter the portal -- it would > be fun to see him lying on a piano in a dress reciting a famous aria. I think I need to get out more often. I just realized I think it would be fun to. Anybody there has a video camera? -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From Moshe Zadka Fri Aug 4 02:45:52 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 4 Aug 2000 04:45:52 +0300 (IDT) Subject: [Python-Dev] tests for standard library modules In-Reply-To: <200008032039.PAA17852@cj20424-a.reston1.va.home.com> Message-ID: On Thu, 3 Aug 2000, Guido van Rossum wrote: > > Most of the standard library is untested. > > Indeed. I would suggest looking at the Tcl test suite. It's very > thorough! When I look at many of the test modules we *do* have, I > cringe at how little of the module the test actually covers. Many > tests (not the good ones!) seem to be content with checking that all > functions in a module *exist*. Much of this dates back to one > particular period in 1996-1997 when we (at CNRI) tried to write test > suites for all modules -- clearly we were in a hurry! :-( Here's a suggestion for easily getting hints about what test suites to write: go through the list of open bugs, and write a "would have caught" test. At worst, we will actually have to fix some bugs . -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From tim_one@email.msn.com Fri Aug 4 03:23:59 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 3 Aug 2000 22:23:59 -0400 Subject: [Python-Dev] snprintf breaks build Message-ID: Fred checked in a new rangeobject.c with 3 calls to snprintf. That isn't a std C function, and the lack of it breaks the build at least under Windows. In the absence of a checkin supplying snprintf on all platforms within the next hour, I'll just replace the snprintf calls with something that's portable. From MarkH@ActiveState.com Fri Aug 4 03:27:32 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Fri, 4 Aug 2000 12:27:32 +1000 Subject: [Python-Dev] Library pragma in PC/config.h In-Reply-To: <20000803212444.A1237@beelzebub> Message-ID: > Does this look like the right change to everyone? I can check it in > (and on the 1.6 branch too) if it looks OK. I have no problems with this (but am a little confused - see below) > While I have your attention, Rene also suggests the convention of > "bcpp_python20.lib" for the Borland-format lib file, with other > compilers (library formats) supported in future similarly. Works for me > -- anyone have any problems with that? I would prefer python20_bcpp.lib, but that is not an issue. I am a little confused by the intention, tho. Wouldnt it make sense to have Borland builds of the core create a Python20.lib, then we could keep the pragma in too? If people want to use Borland for extensions, can't we ask them to use that same compiler to build the core too? That would seem to make lots of the problems go away? But assuming there are good reasons, I am happy. It wont bother me for some time yet ;-) Sometimes-the-best-things-in-life-arent-free ly, Mark. From MarkH@ActiveState.com Fri Aug 4 03:30:22 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Fri, 4 Aug 2000 12:30:22 +1000 Subject: [Python-Dev] Breaking Test Cases on Purpose In-Reply-To: Message-ID: > Anybody there has a video camera? Eeeuuugghhh - the concept of Tim's last threatened photo-essay turning into a video-essay has made me physically ill ;-) Just-dont-go-there ly, Mark. From fdrake@beopen.com Fri Aug 4 03:34:34 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Thu, 3 Aug 2000 22:34:34 -0400 (EDT) Subject: [Python-Dev] snprintf breaks build In-Reply-To: References: Message-ID: <14730.11194.599976.438416@cj42289-a.reston1.va.home.com> Tim Peters writes: > Fred checked in a new rangeobject.c with 3 calls to snprintf. That isn't a > std C function, and the lack of it breaks the build at least under Windows. > In the absence of a checkin supplying snprintf on all platforms within the > next hour, I'll just replace the snprintf calls with something that's > portable. Hmm. I think the issue with known existing snprintf() implementations with Open Source licenses was that they were at least somewhat contanimating. I'll switch back to sprintf() until we have a portable snprintf() implementation. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From tim_one@email.msn.com Fri Aug 4 03:49:32 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 3 Aug 2000 22:49:32 -0400 Subject: [Python-Dev] snprintf breaks build In-Reply-To: <14730.11194.599976.438416@cj42289-a.reston1.va.home.com> Message-ID: [Fred] > Hmm. I think the issue with known existing snprintf() > implementations with Open Source licenses was that they were at > least somewhat contanimating. I'll switch back to sprintf() > until we have a portable snprintf() implementation. Please don't bother! Clearly, I've already fixed it on my machine so I could make progress. I'll simply check it in. I didn't like the excessive cleverness with the fmt vrbl anyway (your compiler may not complain that you can end up passing more s[n]printf args than the format has specifiers to convert, but it's a no-no anyway) .... From tim_one@email.msn.com Fri Aug 4 03:55:47 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 3 Aug 2000 22:55:47 -0400 Subject: [Python-Dev] Library pragma in PC/config.h In-Reply-To: <20000803212444.A1237@beelzebub> Message-ID: [Greg Ward] > for building extensions with non-MS compilers, it sounds like a small > change to PC/config.h is needed. Rene Liebscher suggests changing > > #ifndef USE_DL_EXPORT > /* So nobody needs to specify the .lib in their Makefile any more */ > #ifdef _DEBUG > #pragma comment(lib,"python20_d.lib") > #else > #pragma comment(lib,"python20.lib") > #endif > #endif /* USE_DL_EXPORT */ > > to > > #if !defined(USE_DL_EXPORT) && defined(_MSC_VER) > ... > > That way, the convenience pragma will still be there for MSVC users, but > it won't break building extensions with Borland C++. OK by me. > ... > While I have your attention, You're pushing your luck, Greg . > Rene also suggests the convention of "bcpp_python20.lib" for > the Borland-format lib file, with other compilers (library > formats) supported in future similarly. Works for me -- anyone > have any problems with that? Nope, but I don't understand anything about how Borland differs from the real <0.5 wink> Windows compiler, so don't know squat about the issues or the goals. If it works for Rene, I give up without a whimper. From nhodgson@bigpond.net.au Fri Aug 4 04:36:12 2000 From: nhodgson@bigpond.net.au (Neil Hodgson) Date: Fri, 4 Aug 2000 13:36:12 +1000 Subject: [Python-Dev] Library pragma in PC/config.h References: Message-ID: <00cf01bffdc5$246867f0$8119fea9@neil> > But assuming there are good reasons, I am happy. It wont bother me for > some time yet ;-) Windows who values their time in more than cents-per-hour would use MSVC, > but deleted it ;-> OK. Better cut my rates. Some people will be pleased ;) Borland C++ isn't that bad. With an optimiser and a decent debugger it'd even be usable as my main compiler. What is good about Borland is that it produces lots of meaningful warnings. I've never regretted ensuring that Scintilla/SciTE build on Windows with each of MSVC, Borland and GCC. It wasn't much work and real problems have been found by the extra checks done by Borland. You-should-try-it-sometime-ly y'rs, Neil From bwarsaw@beopen.com Fri Aug 4 04:46:02 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Thu, 3 Aug 2000 23:46:02 -0400 (EDT) Subject: [Python-Dev] snprintf breaks build References: <14730.11194.599976.438416@cj42289-a.reston1.va.home.com> Message-ID: <14730.15482.216054.249627@anthem.concentric.net> >>>>> "Fred" == Fred L Drake, Jr writes: Fred> Hmm. I think the issue with known existing snprintf() Fred> implementations with Open Source licenses was that they were Fred> at least somewhat contanimating. I'll switch back to Fred> sprintf() until we have a portable snprintf() Fred> implementation. In Mailman, I used the one from GNU screen, which is obviously GPL'd. But Apache also comes with an snprintf implementation which doesn't have the infectious license. I don't feel like searching the archives, but I'd be surprised if Greg Stein /didn't/ suggest this a while back. -Barry From tim_one@email.msn.com Fri Aug 4 04:54:47 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 3 Aug 2000 23:54:47 -0400 Subject: [Python-Dev] Library pragma in PC/config.h In-Reply-To: <00cf01bffdc5$246867f0$8119fea9@neil> Message-ID: [Neil Hodgson] > ... > I've never regretted ensuring that Scintilla/SciTE build on > Windows with each of MSVC, Borland and GCC. It wasn't much work > and real problems have been found by the extra checks done by > Borland. > > You-should-try-it-sometime-ly y'rs, Indeed, the more compilers the better. I've long wished that Guido would leave CNRI, and find some situation in which he could hire people to work on Python full-time. If that ever happens, and he hires me, I'd like to do serious work to free the Windows build config from such total dependence on MSVC. From MarkH@ActiveState.com Fri Aug 4 04:52:58 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Fri, 4 Aug 2000 13:52:58 +1000 Subject: [Python-Dev] Library pragma in PC/config.h In-Reply-To: <00cf01bffdc5$246867f0$8119fea9@neil> Message-ID: > Borland C++ isn't that bad. With an optimiser and a decent > debugger it'd even be usable as my main compiler. > You-should-try-it-sometime-ly y'rs, OK - let me know when it has an optimiser and a decent debugger, and is usable as a main compiler, and I will be happy to ;-) Only-need-one-main-anything ly, Mark. From Moshe Zadka Fri Aug 4 05:30:59 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 4 Aug 2000 07:30:59 +0300 (IDT) Subject: [Python-Dev] snprintf breaks build In-Reply-To: <14730.11194.599976.438416@cj42289-a.reston1.va.home.com> Message-ID: On Thu, 3 Aug 2000, Fred L. Drake, Jr. wrote: > Hmm. I think the issue with known existing snprintf() > implementations with Open Source licenses was that they were at least > somewhat contanimating. I'll switch back to sprintf() until we have a > portable snprintf() implementation. Fred -- in your case, there is no need for sprintf -- a few sizeof(long)s along the way would make sure that your buffers are large enough. (For extreme future-proofing, you might also sizeof() the messages you print) (Tidbit: since sizeof(long) measures in bytes, and %d prints in decimals, then a buffer of length sizeof(long) is enough to hold a decimal represntation of a long). -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From greg@cosc.canterbury.ac.nz Fri Aug 4 05:38:16 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Fri, 04 Aug 2000 16:38:16 +1200 (NZST) Subject: [Python-Dev] snprintf breaks build In-Reply-To: Message-ID: <200008040438.QAA11982@s454.cosc.canterbury.ac.nz> Moshe Zadka: > (Tidbit: since sizeof(long) measures in bytes, and %d prints in decimals, > then a buffer of length sizeof(long) is enough to hold a decimal > represntation of a long). Pardon? I think you're a bit out in your calculation there! 3*sizeof(long) should be enough, though (unless some weird C implementation measures sizes in units of more than 8 bits). Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim_one@email.msn.com Fri Aug 4 06:22:23 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 4 Aug 2000 01:22:23 -0400 Subject: [Python-Dev] snprintf breaks build In-Reply-To: <200008040438.QAA11982@s454.cosc.canterbury.ac.nz> Message-ID: [Moshe Zadka] > (Tidbit: since sizeof(long) measures in bytes, and %d prints in > decimals, then a buffer of length sizeof(long) is enough to hold > a decimal represntation of a long). [Greg Ewing] > Pardon? I think you're a bit out in your calculation there! > > 3*sizeof(long) should be enough, though (unless some weird C > implementation measures sizes in units of more than 8 bits). Getting closer, but the sign bit can consume a character all by itself, so 3*sizeof(long) sitll isn't enough. To do this correctly and minimally requires that we implement an arbitrary-precision log10 function, use the platform MIN/MIX #define's for longs and chars, and malloc the buffers at runtime. Note that instead I boosted the buffer sizes in the module from 80 to 250. That's obviously way more than enough for 64-bit platforms, and "obviously way more" is the correct thing to do for programmers . If one of the principled alternatives is ever checked in (be it an snprintf or /F's custom solution (which I like better)), we can go back and use those instead. From MarkH@ActiveState.com Fri Aug 4 06:58:52 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Fri, 4 Aug 2000 15:58:52 +1000 Subject: [Python-Dev] Preventing 1.5 extensions crashing under 1.6/2.0 Python In-Reply-To: <200008022318.SAA04558@cj20424-a.reston1.va.home.com> Message-ID: [Re forcing all extensions to use PythonExtensionInit_XXX] > I sort-of like this idea -- at least at the +0 level. Since this email there have been some strong objections to this. I too would weigh in at -1 for this, simply for the amount of work it would cost me personally! > Unfortunately we only have two days to get this done for 1.6 -- I plan > to release 1.6b1 this Friday! If you don't get to it, prepare a patch > for 2.0 would be the next best thing. It is now Friday afternoon for me. Regardless of the outcome of this, the patch Fredrik posted recently would still seem reasonable, and not have too much impact on performance (ie, after locating and loading a .dll/.so, one function call isnt too bad!): I've even left his trailing comment, which I agree with too? Shall this be checked in to the 1.6 and 2.0 trees? Mark. Index: Python/modsupport.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/modsupport.c,v retrieving revision 2.48 diff -u -r2.48 modsupport.c --- Python/modsupport.c 2000/07/09 03:09:56 2.48 +++ Python/modsupport.c 2000/07/18 07:55:03 @@ -51,6 +51,8 @@ { PyObject *m, *d, *v; PyMethodDef *ml; + if (!Py_IsInitialized()) + Py_FatalError("Interpreter not initialized (version mismatch?)"); if (module_api_version != PYTHON_API_VERSION) fprintf(stderr, api_version_warning, name, PYTHON_API_VERSION, name, module_api_version); "Fatal Python error: Interpreter not initialized" might not be too helpful, but it's surely better than "PyThreadState_Get: no current thread"... From tim_one@email.msn.com Fri Aug 4 08:06:21 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 4 Aug 2000 03:06:21 -0400 Subject: [Python-Dev] FW: submitting patches against 1.6a2 Message-ID: Anyone competent with urrlib care to check out this fellow's complaint? Thanks! -----Original Message----- From: python-list-admin@python.org [mailto:python-list-admin@python.org]On Behalf Of Paul Schreiber Sent: Friday, August 04, 2000 2:20 AM To: python-list@python.org Subject: submitting patches against 1.6a2 I patched a number of bugs in urllib.py way back when -- in June, I think. That was before the BeOpen announcement. I emailed the patch to patches@python.org. I included the disclaimer. I made the patch into a context diff. I didn't hear back from anyone. Should I resubmit? Where should I send the patch to? Paul -- http://www.python.org/mailman/listinfo/python-list From esr@snark.thyrsus.com Fri Aug 4 08:47:34 2000 From: esr@snark.thyrsus.com (Eric S. Raymond) Date: Fri, 4 Aug 2000 03:47:34 -0400 Subject: [Python-Dev] curses progress Message-ID: <200008040747.DAA02323@snark.thyrsus.com> OK, I've added docs for curses.textpad and curses.wrapper. Did we ever settle on a final location in the distribution tree for the curses HOWTO? -- Eric S. Raymond According to the National Crime Survey administered by the Bureau of the Census and the National Institute of Justice, it was found that only 12 percent of those who use a gun to resist assault are injured, as are 17 percent of those who use a gun to resist robbery. These percentages are 27 and 25 percent, respectively, if they passively comply with the felon's demands. Three times as many were injured if they used other means of resistance. -- G. Kleck, "Policy Lessons from Recent Gun Control Research," From pf@artcom-gmbh.de Fri Aug 4 08:47:17 2000 From: pf@artcom-gmbh.de (Peter Funk) Date: Fri, 4 Aug 2000 09:47:17 +0200 (MEST) Subject: Vladimir's codeutil.py (was Re: [Python-Dev] tests for standard library modules) In-Reply-To: <200008032225.AAA27154@python.inrialpes.fr> from Vladimir Marangozov at "Aug 4, 2000 0:25:38 am" Message-ID: Hi, Vladimir Marangozov: > But fortunately , here's another piece of code, modeled after > its C counterpart, that comes to Skip's rescue and that works with -O. [...] > ------------------------------[ codeutil.py ]------------------------- [...] Neat! This seems to be very useful. I think this could be added to standard library if it were documented. Regards, Peter From thomas@xs4all.net Fri Aug 4 09:14:56 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 4 Aug 2000 10:14:56 +0200 Subject: [Python-Dev] Preventing 1.5 extensions crashing under 1.6/2.0 Python In-Reply-To: ; from MarkH@ActiveState.com on Fri, Aug 04, 2000 at 03:58:52PM +1000 References: <200008022318.SAA04558@cj20424-a.reston1.va.home.com> Message-ID: <20000804101456.H266@xs4all.nl> On Fri, Aug 04, 2000 at 03:58:52PM +1000, Mark Hammond wrote: > It is now Friday afternoon for me. Regardless of the outcome of this, the > patch Fredrik posted recently would still seem reasonable, and not have too > much impact on performance (ie, after locating and loading a .dll/.so, one > function call isnt too bad!): > + if (!Py_IsInitialized()) > + Py_FatalError("Interpreter not initialized (version Wasn't there a problem with this, because the 'Py_FatalError()' would be the one in the uninitialized library and thus result in the same tstate error ? Perhaps it needs a separate error message, that avoids the usual Python cleanup and trickery and just prints the error message and exits ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From MarkH@ActiveState.com Fri Aug 4 09:20:04 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Fri, 4 Aug 2000 18:20:04 +1000 Subject: [Python-Dev] Preventing 1.5 extensions crashing under 1.6/2.0 Python In-Reply-To: <20000804101456.H266@xs4all.nl> Message-ID: > Wasn't there a problem with this, because the 'Py_FatalError()' > would be the > one in the uninitialized library and thus result in the same > tstate error ? > Perhaps it needs a separate error message, that avoids the usual Python > cleanup and trickery and just prints the error message and exits ? I would obviously need to test this, but a cursory look at Py_FatalError() implies it does not touch the thread lock - simply an fprintf, and an abort() (and for debug builds on Windows, an offer to break into the debugger) Regardless, I'm looking for a comment on the concept, and I will make sure that whatever I do actually works ;-) Mark. From Fredrik Lundh" <20000804101456.H266@xs4all.nl> Message-ID: <012b01bffdee$3dadb020$f2a6b5d4@hagrid> thomas wrote: > > + if (!Py_IsInitialized()) > > + Py_FatalError("Interpreter not initialized (version >=20 > Wasn't there a problem with this, because the 'Py_FatalError()' would = be the > one in the uninitialized library and thus result in the same tstate = error ? you mean this one: Py_FatalError("PyThreadState_Get: no current thread"); > Perhaps it needs a separate error message, that avoids the usual = Python > cleanup and trickery and just prints the error message and exits ? void Py_FatalError(char *msg) { fprintf(stderr, "Fatal Python error: %s\n", msg); #ifdef macintosh for (;;); #endif #ifdef MS_WIN32 OutputDebugString("Fatal Python error: "); OutputDebugString(msg); OutputDebugString("\n"); #ifdef _DEBUG DebugBreak(); #endif #endif /* MS_WIN32 */ abort(); } From ping@lfw.org Fri Aug 4 09:38:12 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Fri, 4 Aug 2000 01:38:12 -0700 (PDT) Subject: [Python-Dev] Go \x yourself In-Reply-To: Message-ID: On Thu, 3 Aug 2000, Tim Peters wrote: > > >>> "\x123465" # \x12 -> \022, "3456" left alone > '\0223456' > >>> "\x65" > 'e' > >>> "\x1" > ValueError > >>> "\x\x" > ValueError > >>> I'm quite certain that this should be a SyntaxError, not a ValueError: >>> "\x1" SyntaxError: two hex digits are required after \x >>> "\x\x" SyntaxError: two hex digits are required after \x Otherwise, +1. Sounds great. -- ?!ng From tim_one@email.msn.com Fri Aug 4 10:26:29 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 4 Aug 2000 05:26:29 -0400 Subject: [Python-Dev] Go \x yourself In-Reply-To: Message-ID: [Tim Peters] > >>> "\x123465" # \x12 -> \022, "3456" left alone > '\0223456' > >>> "\x65" > 'e' > >>> "\x1" > ValueError > >>> "\x\x" > ValueError > >>> [?!ng] > I'm quite certain that this should be a SyntaxError, not a > ValueError: > > >>> "\x1" > SyntaxError: two hex digits are required after \x > >>> "\x\x" > SyntaxError: two hex digits are required after \x > > Otherwise, +1. Sounds great. SyntaxError was my original pick too. Guido picked ValueError instead because the corresponding "not enough hex digits" error in Unicode strings for damaged \u1234 escapes raises UnicodeError today, which is a subclass of ValueError. I couldn't care less, and remain +1 either way. On the chance that the BDFL may have changed his mind, I've copied him on this msg, This is your one & only chance to prevail . just-so-long-as-it's-not-XEscapeError-ly y'rs - tim From mal@lemburg.com Fri Aug 4 11:03:49 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 04 Aug 2000 12:03:49 +0200 Subject: [Python-Dev] Go \x yourself References: Message-ID: <398A9505.A88D8F93@lemburg.com> [Wow, 5:26 in the morning and still (or already) up and running...] Tim Peters wrote: > > [Tim Peters] > > >>> "\x123465" # \x12 -> \022, "3456" left alone > > '\0223456' > > >>> "\x65" > > 'e' > > >>> "\x1" > > ValueError > > >>> "\x\x" > > ValueError > > >>> > > [?!ng] > > I'm quite certain that this should be a SyntaxError, not a > > ValueError: > > > > >>> "\x1" > > SyntaxError: two hex digits are required after \x > > >>> "\x\x" > > SyntaxError: two hex digits are required after \x > > > > Otherwise, +1. Sounds great. > > SyntaxError was my original pick too. Guido picked ValueError instead > because the corresponding "not enough hex digits" error in Unicode strings > for damaged \u1234 escapes raises UnicodeError today, which is a subclass of > ValueError. > > I couldn't care less, and remain +1 either way. On the chance that the BDFL > may have changed his mind, I've copied him on this msg, This is your one & > only chance to prevail . The reason for Unicode raising a UnicodeError is that the string is passed through a codec in order to be converted to Unicode. Codecs raise ValueErrors for encoding errors. The "\x..." errors should probably be handled in the same way to assure forward compatibility (they might be passed through codecs as well in some future Python version in order to implement source code encodings). -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From akuchlin@mems-exchange.org Fri Aug 4 13:45:06 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Fri, 4 Aug 2000 08:45:06 -0400 Subject: [Python-Dev] curses progress In-Reply-To: <200008040747.DAA02323@snark.thyrsus.com>; from esr@snark.thyrsus.com on Fri, Aug 04, 2000 at 03:47:34AM -0400 References: <200008040747.DAA02323@snark.thyrsus.com> Message-ID: <20000804084506.B5870@newcnri.cnri.reston.va.us> On Fri, Aug 04, 2000 at 03:47:34AM -0400, Eric S. Raymond wrote: >OK, I've added docs for curses.textpad and curses.wrapper. Did we >ever settle on a final location in the distribution tree for the >curses HOWTO? Fred and GvR thought a separate SF project would be better, so I created http://sourceforge.net/projects/py-howto . You've already been added as a developer, as have Moshe and Fred. Just check out the CVS tree (directory, really) and put it in the Doc/ subdirectory of the Python CVS tree. Preparations for a checkin mailing list are progressing, but still not complete. --amk From thomas@xs4all.net Fri Aug 4 14:01:35 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 4 Aug 2000 15:01:35 +0200 Subject: [Python-Dev] PEP 203 Augmented Assignment In-Reply-To: <200007270559.AAA04753@cj20424-a.reston1.va.home.com>; from guido@beopen.com on Thu, Jul 27, 2000 at 12:59:15AM -0500 References: <20000725230322.N266@xs4all.nl> <200007270559.AAA04753@cj20424-a.reston1.va.home.com> Message-ID: <20000804150134.J266@xs4all.nl> [Don't be scared, I'm revisiting this thread for a purpose -- this isn't a time jump ;-)] On Thu, Jul 27, 2000 at 12:59:15AM -0500, Guido van Rossum wrote: > I'm making up opcodes -- the different variants of LOAD and STORE > don't matter. On the right I'm displaying the stack contents after > execution of the opcode (push appends to the end). I'm writing > 'result' to indicate the result of the += operator. > a[i] += b > > LOAD a [a] > DUP [a, a] > LOAD i [a, a, i] > DUP [a, a, i, i] > ROT3 [a, i, a, i] > GETITEM [a, i, a[i]] > LOAD b [a, i, a[i], b] > AUGADD [a, i, result] > SETITEM [] > > I'm leaving the slice variant out; I'll get to that in a minute. [ And later you gave an example of slicing using slice objects, rather than the *SLICE+x opcodes ] I have two tiny problems with making augmented assignment use the current LOAD/STORE opcodes in the way Guido pointed out, above. One has to do with the order of the arguments, and the other with ROT_FOUR. And they're closely related, too :P The question is in what order the expression x += y is evaluated. x = y evaluates 'y' first, then 'x', but x + y evaluates 'x' first, and then 'y'. x = x + y Would thus evaluate 'x', then 'y', and then 'x' (for storing the result.) (The problem isn't with single-variable expressions like these examples, of course, but with expressions with sideeffects.) I think it makes sense to make '+=' like '+', in that it evaluates the lhs first. However, '+=' is as much '=' as it is '+', so it also makes sense to evaluate the rhs first. There are plenty of arguments both ways, and both sides of my brain have been beating eachother with spiked clubs for the better part of a day now ;) On the other hand, how important is this issue ? Does Python say anything about the order of argument evaluation ? Does it need to ? After making up your mind about the above issue, there's another problem, and that's the generated bytecode. If '+=' should be as '=' and evaluate the rhs first, here's what the bytecode would have to look like for the most complicated case (two-argument slicing.) a[b:c] += i LOAD i [i] LOAD a [i, a] DUP_TOP [i, a, a] LOAD b [i, a, a, b] DUP_TOP [i, a, a, b, b] ROT_THREE [i, a, b, a, b] LOAD c [i, a, b, a, b, c] DUP_TOP [i, a, b, a, b, c, c] ROT_FOUR [i, a, b, c, a, b, c] SLICE+3 [i, a, b, c, a[b:c]] ROT_FIVE [a[b:c], i, a, b, c] ROT_FIVE [c, a[b:c], i, a, b] ROT_FIVE [b, c, a[b:c], i, a] ROT_FIVE [a, b, c, a[b:c], i] INPLACE_ADD [a, b, c, result] STORE_SLICE+3 [] So, *two* new bytecodes, 'ROT_FOUR' and 'ROT_FIVE', just to get the right operands in the right place. On the other hand, if the *left* hand side is evaluated first, it would look like this: a[b:c] += i LOAD a [a] DUP_TOP [a, a] LOAD b [a, a, b] DUP_TOP [a, a, b, b] ROT_THREE [a, b, a, b] LOAD c [a, b, a, b, c] DUP_TOP [a, b, a, b, c, c] ROT_FOUR [a, b, c, a, b, c] SLICE+3 [a, b, c, a[b:c]] LOAD i [a, b, c, a[b:c], i] INPLACE_ADD [a, b, c, result] STORE_SLICE+3 [] A lot shorter, and it only needs ROT_FOUR, not ROT_FIVE. An alternative solution is to drop ROT_FOUR too, and instead use a DUP_TOPX argument-opcode that duplicates the top 'x' items: LOAD a [a] LOAD b [a, b] LOAD c [a, b, c] DUP_TOPX 3 [a, b, c, a, b, c] SLICE+3 [a, b, c, a[b:c]] LOAD i [a, b, c, a[b:c], i] INPLACE_ADD [a, b, c, result] STORE_SLICE+3 [] I think 'DUP_TOPX' makes more sense than ROT_FOUR, as DUP_TOPX could be used in the bytecode for 'a[b] += i' and 'a.b += i' as well. (Guido's example would become something like this: a[b] += i LOAD a [a] LOAD b [a, b] DUP_TOPX 2 [a, b, a, b] BINARY_SUBSCR [a, b, a[b]] LOAD i [a, b, a[b], i] INPLACE_ADD [a, b, result] STORE_SUBSCR [] So, *bytecode* wise, evaluating the lhs of '+=' first is easiest. It requires a lot more hacking of compile.c, but I think I can manage that. However, one part of me is still yelling that '+=' should evaluate its arguments like '=', not '+'. Which part should I lobotomize ? :) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Fri Aug 4 14:08:58 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 4 Aug 2000 15:08:58 +0200 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib/test test_linuxaudiodev.py,1.1,1.2 In-Reply-To: <200008041259.FAA24651@slayer.i.sourceforge.net>; from moshez@users.sourceforge.net on Fri, Aug 04, 2000 at 05:59:43AM -0700 References: <200008041259.FAA24651@slayer.i.sourceforge.net> Message-ID: <20000804150858.K266@xs4all.nl> On Fri, Aug 04, 2000 at 05:59:43AM -0700, Moshe Zadka wrote: > Log Message: > The only error the test suite skips is currently ImportError -- so that's > what we raise. If you see a problem with this patch, say so and I'll > retract. test_support creates a class 'TestSkipped', which has a docstring that suggests it can be used in the same way as ImportError. However, it doesn't work ! Is that intentional ? The easiest fix to make it work is probably making TestSkipped a subclass of ImportError, rather than Error (which it is, now.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From Moshe Zadka Fri Aug 4 14:11:38 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 4 Aug 2000 16:11:38 +0300 (IDT) Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib/test test_linuxaudiodev.py,1.1,1.2 In-Reply-To: <20000804150858.K266@xs4all.nl> Message-ID: On Fri, 4 Aug 2000, Thomas Wouters wrote: > On Fri, Aug 04, 2000 at 05:59:43AM -0700, Moshe Zadka wrote: > > > Log Message: > > The only error the test suite skips is currently ImportError -- so that's > > what we raise. If you see a problem with this patch, say so and I'll > > retract. > > test_support creates a class 'TestSkipped', which has a docstring that > suggests it can be used in the same way as ImportError. However, it doesn't > work ! Is that intentional ? The easiest fix to make it work is probably > making TestSkipped a subclass of ImportError, rather than Error (which it > is, now.) Thanks for the tip, Thomas! I didn't know about it -- but I just read the regrtest.py code, and it seemed to be the only exception it catches. Why not just add test_support.TestSkipped to the exception it catches when it catches the ImportError? -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From thomas@xs4all.net Fri Aug 4 14:19:31 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 4 Aug 2000 15:19:31 +0200 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib/test test_linuxaudiodev.py,1.1,1.2 In-Reply-To: ; from moshez@math.huji.ac.il on Fri, Aug 04, 2000 at 04:11:38PM +0300 References: <20000804150858.K266@xs4all.nl> Message-ID: <20000804151931.L266@xs4all.nl> On Fri, Aug 04, 2000 at 04:11:38PM +0300, Moshe Zadka wrote: > > test_support creates a class 'TestSkipped', which has a docstring that > > suggests it can be used in the same way as ImportError. However, it doesn't > > work ! Is that intentional ? The easiest fix to make it work is probably > > making TestSkipped a subclass of ImportError, rather than Error (which it > > is, now.) > Thanks for the tip, Thomas! I didn't know about it -- but I just read > the regrtest.py code, and it seemed to be the only exception it catches. > Why not just add test_support.TestSkipped to the exception it catches > when it catches the ImportError? Right. Done. Now to update all those tests that raise ImportError when they *mean* 'TestSkipped' :) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From fdrake@beopen.com Fri Aug 4 14:26:53 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Fri, 4 Aug 2000 09:26:53 -0400 (EDT) Subject: [Python-Dev] curses progress In-Reply-To: <200008040747.DAA02323@snark.thyrsus.com> References: <200008040747.DAA02323@snark.thyrsus.com> Message-ID: <14730.50333.391218.736370@cj42289-a.reston1.va.home.com> Eric S. Raymond writes: > OK, I've added docs for curses.textpad and curses.wrapper. Did we > ever settle on a final location in the distribution tree for the > curses HOWTO? Andrew is creating a new project on SourceForge. I think this is the right thing to do. We may want to discuss packaging, to make it easier for users to get to the documentation they need; this will have to wait until after 1.6. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From guido@beopen.com Fri Aug 4 15:59:35 2000 From: guido@beopen.com (Guido van Rossum) Date: Fri, 04 Aug 2000 09:59:35 -0500 Subject: [Python-Dev] Preventing 1.5 extensions crashing under 1.6/2.0 Python In-Reply-To: Your message of "Fri, 04 Aug 2000 15:58:52 +1000." References: Message-ID: <200008041459.JAA01621@cj20424-a.reston1.va.home.com> > [Re forcing all extensions to use PythonExtensionInit_XXX] [GvR] > > I sort-of like this idea -- at least at the +0 level. [MH] > Since this email there have been some strong objections to this. I too > would weigh in at -1 for this, simply for the amount of work it would cost > me personally! OK. Dead it is. -1. > Shall this be checked in to the 1.6 and 2.0 trees? Yes, I'll do so. > "Fatal Python error: Interpreter not initialized" might not be too helpful, > but it's surely better than "PyThreadState_Get: no current thread"... Yes. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From guido@beopen.com Fri Aug 4 16:06:33 2000 From: guido@beopen.com (Guido van Rossum) Date: Fri, 04 Aug 2000 10:06:33 -0500 Subject: [Python-Dev] FW: submitting patches against 1.6a2 In-Reply-To: Your message of "Fri, 04 Aug 2000 03:06:21 -0400." References: Message-ID: <200008041506.KAA01874@cj20424-a.reston1.va.home.com> > Anyone competent with urrlib care to check out this fellow's complaint? It arrived on June 14, so I probably ignored it -- with 1000s of other messages received while I was on vacation. This was before we started using the SF PM. But I still have his email. Someone else please look at this! --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) Subject: [Patches] urllib.py patch From: Paul Schreiber To: patches@python.org Date: Wed, 14 Jun 2000 16:52:02 -0700 Content-Type: multipart/mixed; boundary="------------3EE36A3787159ED881FD3EC3" This is a multi-part message in MIME format. --------------3EE36A3787159ED881FD3EC3 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Patch description ----------------- This addresses four issues: (1) usernames and passwords in urls with special characters are now decoded properly. i.e. http://foo%2C:bar@www.whatever.com/ (2) Basic Auth support has been added to HTTPS, like it was in HTTP. (3) Version 1.92 sent the POSTed data, but did not deal with errors (HTTP responses other than 200) properly. HTTPS now behaves the same way HTTP does. (4) made URL-checking beahve the same way with HTTPS as it does with HTTP (changed == to !=). Paul Schreiber --------------3EE36A3787159ED881FD3EC3 Content-Type: text/plain; charset=us-ascii; name="urllib-diff-2" Content-Disposition: inline; filename="urllib-diff-2" Content-Transfer-Encoding: 7bit *** urllib.old Tue Jun 13 18:27:02 2000 --- urllib.py Tue Jun 13 18:33:27 2000 *************** *** 302,316 **** def open_https(self, url, data=None): """Use HTTPS protocol.""" import httplib if type(url) is type(""): host, selector = splithost(url) ! user_passwd, host = splituser(host) else: host, selector = url urltype, rest = splittype(selector) ! if string.lower(urltype) == 'https': realhost, rest = splithost(rest) ! user_passwd, realhost = splituser(realhost) if user_passwd: selector = "%s://%s%s" % (urltype, realhost, rest) #print "proxy via https:", host, selector --- 302,325 ---- def open_https(self, url, data=None): """Use HTTPS protocol.""" import httplib + user_passwd = None if type(url) is type(""): host, selector = splithost(url) ! if host: ! user_passwd, host = splituser(host) ! host = unquote(host) ! realhost = host else: host, selector = url urltype, rest = splittype(selector) ! url = rest ! user_passwd = None ! if string.lower(urltype) != 'https': ! realhost = None ! else: realhost, rest = splithost(rest) ! if realhost: ! user_passwd, realhost = splituser(realhost) if user_passwd: selector = "%s://%s%s" % (urltype, realhost, rest) #print "proxy via https:", host, selector *************** *** 331,336 **** --- 340,346 ---- else: h.putrequest('GET', selector) if auth: h.putheader('Authorization: Basic %s' % auth) + if realhost: h.putheader('Host', realhost) for args in self.addheaders: apply(h.putheader, args) h.endheaders() if data is not None: *************** *** 340,347 **** if errcode == 200: return addinfourl(fp, headers, url) else: ! return self.http_error(url, fp, errcode, errmsg, headers) ! def open_gopher(self, url): """Use Gopher protocol.""" import gopherlib --- 350,360 ---- if errcode == 200: return addinfourl(fp, headers, url) else: ! if data is None: ! return self.http_error(url, fp, errcode, errmsg, headers) ! else: ! return self.http_error(url, fp, errcode, errmsg, headers, data) ! def open_gopher(self, url): """Use Gopher protocol.""" import gopherlib *************** *** 872,878 **** _userprog = re.compile('^([^@]*)@(.*)$') match = _userprog.match(host) ! if match: return match.group(1, 2) return None, host _passwdprog = None --- 885,891 ---- _userprog = re.compile('^([^@]*)@(.*)$') match = _userprog.match(host) ! if match: return map(unquote, match.group(1, 2)) return None, host _passwdprog = None --------------3EE36A3787159ED881FD3EC3-- _______________________________________________ Patches mailing list Patches@python.org http://www.python.org/mailman/listinfo/patches From guido@beopen.com Fri Aug 4 16:11:03 2000 From: guido@beopen.com (Guido van Rossum) Date: Fri, 04 Aug 2000 10:11:03 -0500 Subject: [Python-Dev] Go \x yourself In-Reply-To: Your message of "Fri, 04 Aug 2000 01:38:12 MST." References: Message-ID: <200008041511.KAA01925@cj20424-a.reston1.va.home.com> > I'm quite certain that this should be a SyntaxError, not a ValueError: > > >>> "\x1" > SyntaxError: two hex digits are required after \x > >>> "\x\x" > SyntaxError: two hex digits are required after \x > > Otherwise, +1. Sounds great. No, problems with literal interpretations traditionally raise "runtime" exceptions rather than syntax errors. E.g. >>> 111111111111111111111111111111111111 OverflowError: integer literal too large >>> u'\u123' UnicodeError: Unicode-Escape decoding error: truncated \uXXXX >>> Note that UnicodeError is a subclass of ValueError. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From Moshe Zadka Fri Aug 4 15:11:00 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 4 Aug 2000 17:11:00 +0300 (IDT) Subject: [Python-Dev] FW: submitting patches against 1.6a2 In-Reply-To: <200008041506.KAA01874@cj20424-a.reston1.va.home.com> Message-ID: On Fri, 4 Aug 2000, Guido van Rossum wrote: > > Anyone competent with urrlib care to check out this fellow's complaint? > > It arrived on June 14, so I probably ignored it -- with 1000s of other > messages received while I was on vacation. This was before we started > using the SF PM. > > But I still have his email. Someone else please look at this! AFAIK, those are the two urllib patches assigned to Jeremy. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From akuchlin@mems-exchange.org Fri Aug 4 15:13:05 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Fri, 4 Aug 2000 10:13:05 -0400 Subject: [Python-Dev] FW: submitting patches against 1.6a2 In-Reply-To: <200008041506.KAA01874@cj20424-a.reston1.va.home.com>; from guido@beopen.com on Fri, Aug 04, 2000 at 10:06:33AM -0500 References: <200008041506.KAA01874@cj20424-a.reston1.va.home.com> Message-ID: <20000804101305.A11929@kronos.cnri.reston.va.us> On Fri, Aug 04, 2000 at 10:06:33AM -0500, Guido van Rossum wrote: >It arrived on June 14, so I probably ignored it -- with 1000s of other >messages received while I was on vacation. This was before we started >using the SF PM. I think this is SF patch#100880 -- I entered it so it wouldn't get lost. --amk From guido@beopen.com Fri Aug 4 16:26:45 2000 From: guido@beopen.com (Guido van Rossum) Date: Fri, 04 Aug 2000 10:26:45 -0500 Subject: [Python-Dev] PEP 203 Augmented Assignment In-Reply-To: Your message of "Fri, 04 Aug 2000 15:01:35 +0200." <20000804150134.J266@xs4all.nl> References: <20000725230322.N266@xs4all.nl> <200007270559.AAA04753@cj20424-a.reston1.va.home.com> <20000804150134.J266@xs4all.nl> Message-ID: <200008041526.KAA02071@cj20424-a.reston1.va.home.com> [Thomas] > The question is in what order the expression > > x += y > > is evaluated. > > x = y > > evaluates 'y' first, then 'x', but > > x + y > > evaluates 'x' first, and then 'y'. > > x = x + y > > Would thus evaluate 'x', then 'y', and then 'x' (for storing the result.) > (The problem isn't with single-variable expressions like these examples, of > course, but with expressions with sideeffects.) Yes. And note that the Python reference manual specifies the execution order (or at least tries to) -- I figured that in a user-friendly interpreted language, predictability is more important than some optimizer being able to speed your code up a tiny bit by rearranging evaluation order. > I think it makes sense to make '+=' like '+', in that it evaluates the lhs > first. However, '+=' is as much '=' as it is '+', so it also makes sense to > evaluate the rhs first. There are plenty of arguments both ways, and both > sides of my brain have been beating eachother with spiked clubs for the > better part of a day now ;) On the other hand, how important is this issue ? > Does Python say anything about the order of argument evaluation ? Does it > need to ? I say that in x += y, x should be evaluated before y. > After making up your mind about the above issue, there's another problem, > and that's the generated bytecode. [...] > A lot shorter, and it only needs ROT_FOUR, not ROT_FIVE. An alternative > solution is to drop ROT_FOUR too, and instead use a DUP_TOPX argument-opcode > that duplicates the top 'x' items: Sure. > However, one part of me is still yelling that '+=' should evaluate its > arguments like '=', not '+'. Which part should I lobotomize ? :) That part. If you see x+=y as shorthand for x=x+y, x gets evaluated before y anyway! We're saving the second evaluation of x, not the first one! --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From guido@beopen.com Fri Aug 4 16:46:57 2000 From: guido@beopen.com (Guido van Rossum) Date: Fri, 04 Aug 2000 10:46:57 -0500 Subject: [Python-Dev] Re: A small proposed change to dictionaries' "get" method In-Reply-To: Your message of "Thu, 03 Aug 2000 17:21:04 EST." <14729.61520.11958.530601@beluga.mojam.com> References: <14728.63466.263123.434708@anthem.concentric.net> <3989454C.5C9EF39B@lemburg.com> <200008031256.HAA06107@cj20424-a.reston1.va.home.com> <398979D0.5AF80126@lemburg.com> <14729.61520.11958.530601@beluga.mojam.com> Message-ID: <200008041546.KAA02168@cj20424-a.reston1.va.home.com> [Skip] > eh... I don't like these do two things at once kind of methods. I see > nothing wrong with > > >>> dict = {} > >>> dict['hello'] = dict.get('hello', []) > >>> dict['hello'].append('world') > >>> print dict > {'hello': ['world']} > > or > > >>> d = dict['hello'] = dict.get('hello', []) > >>> d.insert(0, 'cruel') > >>> print dict > {'hello': ['cruel', 'world']} > > for the obsessively efficiency-minded folks. Good! Two lines instead of three, and only two dict lookups in the latter one. > Also, we're talking about a method that would generally only be useful when > dictionaries have values which were mutable objects. Irregardless of how > useful instances and lists are, I still find that my predominant day-to-day > use of dictionaries is with strings as keys and values. Perhaps that's just > the nature of my work. Must be. I have used the above two idioms many times -- a dict of lists is pretty common. I believe that the fact that you don't need it is the reason why you don't like it. I believe that as long as we agree that dict['hello'] += 1 is clearer (less strain on the reader's brain) than dict['hello'] = dict['hello'] + 1 we might as well look for a clearer way to spell the above idiom. My current proposal (violating my own embargo against posting proposed names to the list :-) would be dict.default('hello', []).append('hello') --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From paul@prescod.net Fri Aug 4 16:52:11 2000 From: paul@prescod.net (Paul Prescod) Date: Fri, 04 Aug 2000 11:52:11 -0400 Subject: [Python-Dev] New winreg module really an improvement? References: <3986794E.ADBB938C@prescod.net> <200008011820.NAA30284@cj20424-a.reston1.va.home.com> <004d01bffc50$522fa2a0$f2a6b5d4@hagrid> Message-ID: <398AE6AB.9D8F943B@prescod.net> Fredrik Lundh wrote: > > ... > > how about letting _winreg export all functions with their > win32 names, and adding a winreg.py which looks some- > thing like this: > > from _winreg import * > > class Key: > .... > > HKEY_CLASSES_ROOT = Key(...) > ... To me, that would defeat the purpose. Have you looked at the "*" exported by _winreg? The whole point is to impose some organization on something that is totally disorganized (because that's how the C module is). -- Paul Prescod - Not encumbered by corporate consensus "I don't want you to describe to me -- not ever -- what you were doing to that poor boy to make him sound like that; but if you ever do it again, please cover his mouth with your hand," Grandmother said. -- John Irving, "A Prayer for Owen Meany" From skip@mojam.com (Skip Montanaro) Fri Aug 4 19:07:28 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Fri, 4 Aug 2000 13:07:28 -0500 (CDT) Subject: [Python-Dev] Re: A small proposed change to dictionaries' "get" method In-Reply-To: <200008041546.KAA02168@cj20424-a.reston1.va.home.com> References: <14728.63466.263123.434708@anthem.concentric.net> <3989454C.5C9EF39B@lemburg.com> <200008031256.HAA06107@cj20424-a.reston1.va.home.com> <398979D0.5AF80126@lemburg.com> <14729.61520.11958.530601@beluga.mojam.com> <200008041546.KAA02168@cj20424-a.reston1.va.home.com> Message-ID: <14731.1632.44037.499807@beluga.mojam.com> >> Also, we're talking about a method that would generally only be >> useful when dictionaries have values which were mutable objects. >> Irregardless of how useful instances and lists are, I still find that >> my predominant day-to-day use of dictionaries is with strings as keys >> and values. Perhaps that's just the nature of my work. Guido> Must be. I have used the above two idioms many times -- a dict Guido> of lists is pretty common. I believe that the fact that you Guido> don't need it is the reason why you don't like it. I do use lists in dicts as well, it's just that it seems to me that using strings as values (especially because I use bsddb a lot and often want to map dictionaries to files) dominates. The two examples I posted are what I've used for a long time. I guess I just don't find them to be big limitations. Skip From barry@scottb.demon.co.uk Sat Aug 5 00:19:52 2000 From: barry@scottb.demon.co.uk (Barry Scott) Date: Sat, 5 Aug 2000 00:19:52 +0100 Subject: [Python-Dev] Preventing 1.5 extensions crashing under 1.6/2.0 Python In-Reply-To: <01d701bffcd7$46a74a00$f2a6b5d4@hagrid> Message-ID: <000d01bffe6a$7e4bab60$060210ac@private> > > Yes indeed once the story of 1.6 and 2.0 is out I expect folks > > will skip 1.6. For example, if your win32 stuff is not ported then > > Python 1.6 is not usable on Windows/NT. > > "not usable"? > > guess you haven't done much cross-platform development lately... True. On Unix I have an ISDN status monitor, it depends on FReeBSD interfaces and PIL. On Windows I have an SCM solution that depends on COM to drive SourceSafe. Without Mark's COM support I cannot run any of my code on Windows. > > Change the init function name to a new name PythonExtensionInit_ say. > > Pass in the API version for the extension writer to check. If the > > version is bad for this extension returns without calling any python > > huh? are you seriously proposing to break every single C extension > ever written -- on each and every platform -- just to trap an error > message caused by extensions linked against 1.5.2 on your favourite > platform? What makes you think that a crash will not happen under Unix when you change the API? You just don't get the Windows crash. As this thread has pointed out you have no intention of checking for binary compatibility on the API as you move up versions. > > Small code change in python core. But need to tell extension writers > > what the new interface is and update all extensions within the python > > CVS tree. > > you mean "update the source code for all extensions ever written." Yes, I'm aware of the impact. > -1 > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://www.python.org/mailman/listinfo/python-dev > From gward@python.net Sat Aug 5 01:53:09 2000 From: gward@python.net (Greg Ward) Date: Fri, 4 Aug 2000 20:53:09 -0400 Subject: [Python-Dev] Library pragma in PC/config.h In-Reply-To: ; from MarkH@ActiveState.com on Fri, Aug 04, 2000 at 12:27:32PM +1000 References: <20000803212444.A1237@beelzebub> Message-ID: <20000804205309.A1013@beelzebub> On 04 August 2000, Mark Hammond said: > I would prefer python20_bcpp.lib, but that is not an issue. Good suggestion: the contents of the library are more important than the format. Rene, can you make this change and include it in your next patch? Or did you have some hidden, subtle reson for "bcpp_python20" as opposed to "python20_bcpp"? > I am a little confused by the intention, tho. Wouldnt it make sense to > have Borland builds of the core create a Python20.lib, then we could keep > the pragma in too? > > If people want to use Borland for extensions, can't we ask them to use that > same compiler to build the core too? That would seem to make lots of the > problems go away? But that requires people to build all of Python from source, which I'm guessing is a bit more bothersome than building an extension or two from source. Especially since Python is already distributed as a very easy-to-use binary installer for Windows, but most extensions are not. Rest assured that we probably won't be making things *completely* painless for those who do not toe Chairman Bill's party line and insist on using "non-standard" Windows compilers. They'll probably have to get python20_bcpp.lib (or python20_gcc.lib, or python20_lcc.lib) on their own -- whether downloaded or generated, I don't know. But the alternative is to include 3 or 4 python20_xxx.lib files in the standard Windows distribution, which I think is silly. > But assuming there are good reasons, I am happy. It wont bother me for > some time yet ;-) Windows who values their time in more than cents-per-hour would use MSVC, > but deleted it ;-> Then I won't even write my "it's not just about money, it's not even about features, it's about the freedom to use the software you want to use no matter what it says in Chairman Bill's book of wisdom" rant. Windows: the Cultural Revolution of the 90s. ;-) Greg -- Greg Ward - geek-at-large gward@python.net http://starship.python.net/~gward/ What happens if you touch these two wires tog-- From guido@beopen.com Sat Aug 5 03:27:59 2000 From: guido@beopen.com (Guido van Rossum) Date: Fri, 04 Aug 2000 21:27:59 -0500 Subject: [Python-Dev] Python 1.6b1 is released! Message-ID: <200008050227.VAA11161@cj20424-a.reston1.va.home.com> Python 1.6b1, with the new CNRI open source license, is released today from the python.org website. Read all about it: http://www.python.org/1.6/ Here's a little background on the new license (also posted on www.pythonlabs.com): CNRI has funded Python development for five years and held copyright, but never placed a CNRI-specific license on the software. In order to clarify the licensing, BeOpen.com has been working with CNRI to produce a new CNRI license. The result of these discussions (which included Eric Raymond, Bruce Perens, Richard Stallman and Python Consortium members) has produced the CNRI Open Source License, under which Python 1.6b1 has been released. Bob Weiner, CTO of BeOpen.com, on the result of the licensing discussions: "Bob Kahn [CNRI's President] worked with us to understand the particular needs of the Open Source community and Python users. The result is a very open license." The new CNRI license was approved by the Python Consortium members, at a meeting of the Python Consortium on Friday, July 21, 2000 in Monterey, California. Eric Raymond, President of the Open Source Initiative (OSI), reports that OSI's Board of Directors voted to certify the new CNRI license [modulo minor editing] as fully Open Source compliant. Richard Stallman, founder of the Free Software Foundation, is in discussion with CNRI about the new license's compatibility with the GPL. We are hopeful that the remaining issues will be resolved in favor of GPL compatibility before the release of Python 1.6 final. We would like to thank all who graciously volunteered their time to help make these results possible: Bob Kahn for traveling out west to discuss these issues in person; Eric Raymond and Bruce Perens for their useful contributions to the discussions; Bob Weiner for taking care of the bulk of the negotiations; Richard Stallman for GNU; and the Python Consortium representatives for making the consortium meeting a success! (And I would personally like to thank Tim Peters for keeping the newsgroup informed and for significant editing of the text above.) --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From akuchlin@mems-exchange.org Sat Aug 5 05:15:22 2000 From: akuchlin@mems-exchange.org (A.M. Kuchling) Date: Sat, 5 Aug 2000 00:15:22 -0400 Subject: [Python-Dev] python-dev summary posted Message-ID: <200008050415.AAA00811@207-172-146-87.s87.tnt3.ann.va.dialup.rcn.com> I've posted the python-dev summary for July 16-31 to comp.lang.python/python-list; interested people can go check it out. --amk From just@letterror.com Sat Aug 5 09:03:33 2000 From: just@letterror.com (Just van Rossum) Date: Sat, 05 Aug 2000 09:03:33 +0100 Subject: [Python-Dev] Re: Python 2.0 and Stackless References: <9b13RLA800i5EwLY@jessikat.fsnet.co.uk> <8mg3au$rtb$1@nnrp1.deja.com> Message-ID: <398BCA4F.17E23309@letterror.com> [ CC-d to python-dev from c.l.py ] Jeremy Hylton wrote: > It is a conservative response. JPython is an implementation of Python, > and compatibility between Python and JPython is important. It's not > required for every language feature, of course; you can't load a Java > class file in C Python. Jeremy, have you ever *looked* at stackless? Even though it requires extensive patches in the eval loop, all additional semantics are nicely hidden in an extension module. The Java argument is a *very* poor one because of this. No, you can't load a Java class in CPython, and yes, "import continuation" fails under JPython. So what? > I'm not sure what you mean by distinguishing between the semantics of > continuations and the implementation of Stackless Python. They are > both issues! In the second half of my earlier message, I observed that > we would never add continuations without a PEP detailing their exact > semantics. I do not believe such a specification currently exists for > stackless Python. That's completely unfair. Stackless has been around *much* longer than those silly PEPs. It seems stackless isn't in the same league as, say, "adding @ to the print statement for something that is almost as conveniently done with a function". I mean, jeez. > The PEP would also need to document the C interface and how it affects > people writing extensions and doing embedded work. Python is a glue > language and the effects on the glue interface are also important. The stackless API is 100% b/w compatible. There are (or could/should be) additional calls for extension writers and embedders that would like to take advantage of stackless features, but full compatibility is *there*. To illustrate this: for windows as well as MacOS, there are DLLs for stackless that you just put in the place if the original Python core DLLs, and *everything* just works. Christian has done an amazing piece of work, and he's gotten much praise from the community. I mean, if you *are* looking for a killer feature to distinguish 1.6 from 2.0, I'd know where to look... Just From mal@lemburg.com Sat Aug 5 10:35:06 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Sat, 05 Aug 2000 11:35:06 +0200 Subject: [Python-Dev] Python 1.6b1 out ?! Message-ID: <398BDFCA.4D5A262D@lemburg.com> Strange: either I missed it or Guido chose to release 1.6b1 in silence, but I haven't seen any official announcement of the release available from http://www.python.org/1.6/. BTW, nice holiday, Guido ;-) -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From tim_one@email.msn.com Sun Aug 6 00:34:43 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 5 Aug 2000 19:34:43 -0400 Subject: [Python-Dev] Python 1.6b1 out ?! In-Reply-To: <398BDFCA.4D5A262D@lemburg.com> Message-ID: [M.-A. Lemburg] > Strange: either I missed it or Guido chose to release 1.6b1 > in silence, but I haven't seen any official announcement of the > release available from http://www.python.org/1.6/. > > BTW, nice holiday, Guido ;-) There's an announcement at the top of http://www.python.org/, though, and an announcement about the new license at http://www.pythonlabs.com/. Guido also posted to comp.lang.python. You probably haven't seen the latter if you use the mailing list gateway, because many mailing lists at python.org coincidentally got hosed at the same time due to a full disk. Else your news server simply hasn't gotten it yet (I saw it come across on netnews.msn.com, but then Microsoft customers get everything first ). From thomas@xs4all.net Sat Aug 5 16:18:30 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sat, 5 Aug 2000 17:18:30 +0200 Subject: [Python-Dev] UNPACK_LIST & UNPACK_TUPLE Message-ID: <20000805171829.N266@xs4all.nl> I'm a tad confused about the 'UNPACK_LIST' and 'UNPACK_TUPLE' opcodes. There doesn't seem to be a difference between the two, yet the way they are compiled is slightly different (but not much.) I can list all the differences I can see, but I just don't understand them, and because of that I'm not sure how to handle them in augmented assignment. UNPACK_LIST just seems so redundant :) Wouldn't it make sense to remove the difference between the two, or better yet, remove UNPACK_LIST (and possibly rename UNPACK_TUPLE to UNPACK_SEQ ?) We already lost bytecode compability anyway! -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From esr@thyrsus.com Sun Aug 6 00:46:00 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Sat, 5 Aug 2000 19:46:00 -0400 Subject: [Python-Dev] Re: Python 2.0 and Stackless In-Reply-To: <398BCA4F.17E23309@letterror.com>; from just@letterror.com on Sat, Aug 05, 2000 at 09:03:33AM +0100 References: <9b13RLA800i5EwLY@jessikat.fsnet.co.uk> <8mg3au$rtb$1@nnrp1.deja.com> <398BCA4F.17E23309@letterror.com> Message-ID: <20000805194600.A7242@thyrsus.com> Just van Rossum : > Christian has done an amazing piece of work, and he's gotten much > praise from the community. I mean, if you *are* looking for a killer > feature to distinguish 1.6 from 2.0, I'd know where to look... I must say I agree. Something pretty similar to Stackless Python is going to have to happen anyway for the language to make its next major advance in capability -- generators, co-routining, and continuations. I also agree that this is a more important debate, and a harder set of decisions, than the PEPs. Which means we should start paying attention to it *now*. -- Eric S. Raymond I don't like the idea that the police department seems bent on keeping a pool of unarmed victims available for the predations of the criminal class. -- David Mohler, 1989, on being denied a carry permit in NYC From bwarsaw@beopen.com Sun Aug 6 00:50:04 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Sat, 5 Aug 2000 19:50:04 -0400 (EDT) Subject: [Python-Dev] Python 1.6b1 out ?! References: <398BDFCA.4D5A262D@lemburg.com> Message-ID: <14732.43052.91330.426211@anthem.concentric.net> >>>>> "TP" == Tim Peters writes: TP> There's an announcement at the top of http://www.python.org/, TP> though, and an announcement about the new license at TP> http://www.pythonlabs.com/. Guido also posted to TP> comp.lang.python. You probably haven't seen the latter if you TP> use the mailing list gateway, because many mailing lists at TP> python.org coincidentally got hosed at the same time due to a TP> full disk. Else your news server simply hasn't gotten it yet TP> (I saw it come across on netnews.msn.com, but then Microsoft TP> customers get everything first ). And you should soon see the announcement if you haven't already. All the mailing lists on py.org should be back on line now. It'll take a while to clear out the queue though. -Barry From bwarsaw@beopen.com Sun Aug 6 00:52:05 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Sat, 5 Aug 2000 19:52:05 -0400 (EDT) Subject: [Python-Dev] UNPACK_LIST & UNPACK_TUPLE References: <20000805171829.N266@xs4all.nl> Message-ID: <14732.43173.634118.381282@anthem.concentric.net> >>>>> "TW" == Thomas Wouters writes: TW> I'm a tad confused about the 'UNPACK_LIST' and 'UNPACK_TUPLE' TW> opcodes. There doesn't seem to be a difference between the TW> two, yet the way they are compiled is slightly different (but TW> not much.) I can list all the differences I can see, but I TW> just don't understand them, and because of that I'm not sure TW> how to handle them in augmented assignment. UNPACK_LIST just TW> seems so redundant :) TW> Wouldn't it make sense to remove the difference between the TW> two, or better yet, remove UNPACK_LIST (and possibly rename TW> UNPACK_TUPLE to UNPACK_SEQ ?) We already lost bytecode TW> compability anyway! This is a historical artifact. I don't remember what version it was, but at one point there was a difference between a, b, c = gimme_a_tuple() and [a, b, c] = gimme_a_list() That difference was removed, and support was added for any sequence unpacking. If changing the bytecode is okay, then there doesn't seem to be any reason to retain the differences. -Barry From jack@oratrix.nl Sat Aug 5 22:14:08 2000 From: jack@oratrix.nl (Jack Jansen) Date: Sat, 05 Aug 2000 23:14:08 +0200 Subject: [Python-Dev] New SRE core dump (was: SRE 0.9.8 benchmarks) In-Reply-To: Message by "Fredrik Lundh" , Thu, 3 Aug 2000 19:19:03 +0200 , <007401bffd6e$ed9bbde0$f2a6b5d4@hagrid> Message-ID: <20000805211413.E1224E2670@oratrix.oratrix.nl> Fredrik, could you add a PyOS_CheckStack() call to the recursive part of sre (within #ifdef USE_STACKCHECK, of course)? I'm getting really really nasty crashes on the Mac if I run the regression tests... -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From jack@oratrix.nl Sat Aug 5 22:41:15 2000 From: jack@oratrix.nl (Jack Jansen) Date: Sat, 05 Aug 2000 23:41:15 +0200 Subject: [Python-Dev] strftime() Message-ID: <20000805214120.A55EEE2670@oratrix.oratrix.nl> The test_strftime regression test has been failing on the Mac for ages, and I finally got round to investigating the problem: the MetroWerks library returns the strings "am" and "pm" for %p but the regression test expects "AM" and "PM". According to the comments in the source of the library (long live vendors who provide it! Yeah!) this is C9X compatibility. I can of course move the %p to the nonstandard expectations, but maybe someone has a better idea? -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++ From bwarsaw@beopen.com Sun Aug 6 01:12:58 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Sat, 5 Aug 2000 20:12:58 -0400 (EDT) Subject: [Python-Dev] Re: Python 2.0 and Stackless References: <9b13RLA800i5EwLY@jessikat.fsnet.co.uk> <8mg3au$rtb$1@nnrp1.deja.com> <398BCA4F.17E23309@letterror.com> <20000805194600.A7242@thyrsus.com> Message-ID: <14732.44426.201651.690336@anthem.concentric.net> >>>>> "ESR" == Eric S Raymond writes: ESR> I must say I agree. Something pretty similar to Stackless ESR> Python is going to have to happen anyway for the language to ESR> make its next major advance in capability -- generators, ESR> co-routining, and continuations. Stackless definitely appeals to me from a coolness factor, though I don't know how much I'd use those new capabilities that it allows. The ability to embed Python on hardware that might otherwise not be possible without Stackless is also an interesting thing to explore. ESR> I also agree that this is a more important debate, and a ESR> harder set of decisions, than the PEPs. Which means we ESR> should start paying attention to it *now*. Maybe a PEP isn't the right venue, but the semantics and externally visible effects of Stackless need to be documented. What if JPython or Python .NET wanted to adopt those same semantics, either by doing their implementation's equivalent of Stackless or by some other means? We can't even think about doing that without a clear and complete specification. Personally, I don't see Stackless making it into 2.0 and possibly not 2.x. But I agree it is something to seriously consider for Py3K. -Barry From tim_one@email.msn.com Sun Aug 6 06:07:27 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 6 Aug 2000 01:07:27 -0400 Subject: [Python-Dev] strftime() In-Reply-To: <20000805214120.A55EEE2670@oratrix.oratrix.nl> Message-ID: [Jack Jansen] > The test_strftime regression test has been failing on the Mac for > ages, and I finally got round to investigating the problem: the > MetroWerks library returns the strings "am" and "pm" for %p but the > regression test expects "AM" and "PM". According to the comments in > the source of the library (long live vendors who provide it! Yeah!) > this is C9X compatibility. My copy of a draft C99 std agrees (7.23.3.5) with MetroWerks on this point (i.e., that %p in the "C" locale becomes "am" or "pm"). > I can of course move the %p to the nonstandard expectations, but maybe > someone has a better idea? Not really. If Python thinks this function is valuable, it "should" offer a platform-independent implementation, but as nobody has time for that ... From MarkH@ActiveState.com Sun Aug 6 06:08:46 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Sun, 6 Aug 2000 15:08:46 +1000 Subject: [Python-Dev] Preventing 1.5 extensions crashing under 1.6/2.0 Python In-Reply-To: <000d01bffe6a$7e4bab60$060210ac@private> Message-ID: [/F] > > huh? are you seriously proposing to break every single C extension > > ever written -- on each and every platform -- just to trap an error > > message caused by extensions linked against 1.5.2 on your favourite > > platform? [Barry] > What makes you think that a crash will not happen under Unix > when you change the API? You just don't get the Windows crash. > > As this thread has pointed out you have no intention of checking > for binary compatibility on the API as you move up versions. I imtimated the following, but did not spell it out, so I will here to clarify. I was -1 on Barry's solution getting into 1.6, given the time frame. I hinted that the solution Guido recently checked in "if (!Py_IsInitialized()) ..." would not be too great an impact even if Barry's solution, or one like it, was eventually adopted. So I think that the adoption of our half-solution (ie, we are really only forcing a better error message - not even getting a traceback to indicate _which_ module fails) need not preclude a better solution when we have more time to implement it... Mark. From Moshe Zadka Sun Aug 6 07:23:48 2000 From: Moshe Zadka (Moshe Zadka) Date: Sun, 6 Aug 2000 09:23:48 +0300 (IDT) Subject: [Python-Dev] Re: Python 2.0 and Stackless In-Reply-To: <20000805194600.A7242@thyrsus.com> Message-ID: On Sat, 5 Aug 2000, Eric S. Raymond wrote: > I must say I agree. Something pretty similar to Stackless Python is > going to have to happen anyway for the language to make its next major > advance in capability -- generators, co-routining, and continuations. > > I also agree that this is a more important debate, and a harder set of > decisions, than the PEPs. Which means we should start paying attention > to it *now*. I tend to disagree. For a while now I'm keeping an eye on the guile interpreter development (a very cool project, but unfortunately limping along. It probably will be the .NET of free software, though). In guile, they were able to implement continuations *without* what we call stacklessness. Sure, it might look inefficient, but for most applications (like co-routines) it's actually quite all right. What all that goes to say is that we should treat stackles exactly like it is -- an implementation detail. Now, that's not putting down Christian's work -- on the contrary, I think the Python implementation is very important. But that alone should indicate there's no need for a PEP. I, for one, am for it, because I happen to think it's a much better implementation. If it also has the effect of making continuationsmodule.c easier to write, well, that's not an issue in this discussion as far as I'm concerned. brain-dumping-ly y'rs, Z. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From mal@lemburg.com Sun Aug 6 09:55:55 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Sun, 06 Aug 2000 10:55:55 +0200 Subject: [Python-Dev] Python 1.6b1 out ?! References: Message-ID: <398D281B.E7F118C0@lemburg.com> Tim Peters wrote: > > [M.-A. Lemburg] > > Strange: either I missed it or Guido chose to release 1.6b1 > > in silence, but I haven't seen any official announcement of the > > release available from http://www.python.org/1.6/. > > > > BTW, nice holiday, Guido ;-) > > There's an announcement at the top of http://www.python.org/, though, and an > announcement about the new license at http://www.pythonlabs.com/. Guido > also posted to comp.lang.python. You probably haven't seen the latter if > you use the mailing list gateway, because many mailing lists at python.org > coincidentally got hosed at the same time due to a full disk. Else your > news server simply hasn't gotten it yet (I saw it come across on > netnews.msn.com, but then Microsoft customers get everything first ). I saw the announcement on www.python.org, thanks. (I already started to miss the usual 100+ Python messages I get into my mailbox every day ;-) -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mal@lemburg.com Sun Aug 6 13:20:56 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Sun, 06 Aug 2000 14:20:56 +0200 Subject: [Python-Dev] Pickling using XML as output format Message-ID: <398D5827.EE8938DD@lemburg.com> Before starting to reinvent the wheel: I need a pickle.py compatible module which essentially works just like pickle.py, but uses XML as output format. I've already looked at xml_pickle.py (see Parnassus), but this doesn't seem to handle object references at all. Also, it depends on xml.dom which I'd rather avoid. My idea was to rewrite the format used by pickle in an XML syntax and then hard-code the DTD into a subclass of the parser in xmllib.py. Now, I'm very new to XML, so I may be missing something here... would this be doable in a fairly sensible way (I'm thinking of closely sticking to the pickle stream format) ? -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From Moshe Zadka Sun Aug 6 13:46:09 2000 From: Moshe Zadka (Moshe Zadka) Date: Sun, 6 Aug 2000 15:46:09 +0300 (IDT) Subject: [Python-Dev] Pickling using XML as output format In-Reply-To: <398D5827.EE8938DD@lemburg.com> Message-ID: On Sun, 6 Aug 2000, M.-A. Lemburg wrote: > Before starting to reinvent the wheel: Ummmm......I'd wait for some DC guy to chime in: I think Zope had something like that. You might want to ask around on the Zope lists or search zope.org. I'm not sure what it has and what it doesn't have, though. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From Moshe Zadka Sun Aug 6 14:22:09 2000 From: Moshe Zadka (Moshe Zadka) Date: Sun, 6 Aug 2000 16:22:09 +0300 (IDT) Subject: [Python-Dev] Warnings on gcc -Wall Message-ID: As those of you with a firm eye on python-checkins noticed, I've been trying to clear the source files (as many of them as I could get to compile on my setup) from warnings. This is only with gcc -Wall: a future project of mine is to enable much more warnings and try to clean them too. There are currently two places where warnings still remain: -- readline.c -- readline/history.h is included only on BeOS, and otherwise prototypes are declared by hand. Does anyone remember why? -- ceval.c, in ceval() gcc -Wall (wrongly) complains about opcode and oparg which might be used before initialized. I've had a look at that code, and I'm certain gcc's flow analysis is simply not good enough. However, I would like to silence the warning, so I can get used to building with -Wall -Werror and make sure to mind any warnings. Does anyone see any problem with putting opcode=0 and oparg=0 near the top? Any comments welcome, of course. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From thomas@xs4all.net Sun Aug 6 15:00:26 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sun, 6 Aug 2000 16:00:26 +0200 Subject: [Python-Dev] Warnings on gcc -Wall In-Reply-To: ; from moshez@math.huji.ac.il on Sun, Aug 06, 2000 at 04:22:09PM +0300 References: Message-ID: <20000806160025.P266@xs4all.nl> On Sun, Aug 06, 2000 at 04:22:09PM +0300, Moshe Zadka wrote: > -- readline.c -- readline/history.h is included only on BeOS, and > otherwise prototypes are declared by hand. Does anyone remember why? Possibly because old versions of readline don't have history.h ? > -- ceval.c, in ceval() gcc -Wall (wrongly) complains about opcode and > oparg which might be used before initialized. I've had a look at that > code, and I'm certain gcc's flow analysis is simply not good enough. > However, I would like to silence the warning, so I can get used to > building with -Wall -Werror and make sure to mind any warnings. Does > anyone see any problem with putting opcode=0 and oparg=0 near the top? Actually, I don't think this is true. 'opcode' and 'oparg' get filled inside the permanent for-loop, but after the check on pending signals and exceptions. I think it's theoretically possible to have 'things_to_do' on the first time through the loop, which end up in an exception, thereby causing the jump to on_error, entering the branch on WHY_EXCEPTION, which uses oparg and opcode. I'm not sure if initializing opcode/oparg is the right thing to do, though, but I'm not sure what is, either :-) As for the checkins, I haven't seen some of the pending checkin-mails pass by (I did some cleaning up of configure.in last night, for instance, after the re-indent and grammar change in compile.c that *did* come through.) Barry (or someone else ;) are those still waiting in the queue, or should we consider them 'lost' ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From Moshe Zadka Sun Aug 6 15:13:10 2000 From: Moshe Zadka (Moshe Zadka) Date: Sun, 6 Aug 2000 17:13:10 +0300 (IDT) Subject: [Python-Dev] Warnings on gcc -Wall In-Reply-To: <20000806160025.P266@xs4all.nl> Message-ID: On Sun, 6 Aug 2000, Thomas Wouters wrote: > On Sun, Aug 06, 2000 at 04:22:09PM +0300, Moshe Zadka wrote: > > > -- readline.c -- readline/history.h is included only on BeOS, and > > otherwise prototypes are declared by hand. Does anyone remember why? > > Possibly because old versions of readline don't have history.h ? And it did have the history functions? If so, maybe we can include unconditionally, and switch on the readline version. If not, I'd just announce support for earlier versions of readline nonexistant and be over and done with it. > 'opcode' and 'oparg' get filled inside > the permanent for-loop, but after the check on pending signals and > exceptions. I think it's theoretically possible to have 'things_to_do' on > the first time through the loop, which end up in an exception, thereby > causing the jump to on_error, entering the branch on WHY_EXCEPTION, which > uses oparg and opcode. I'm not sure if initializing opcode/oparg is the > right thing to do, though, but I'm not sure what is, either :-) Probably initializing them before the "goto no_error" to some dummy value, then checking for this dummy value in the relevant place. You're right, of course, I hadn't noticed the goto. > As for the checkins, I haven't seen some of the pending checkin-mails pass > by (I did some cleaning up of configure.in last night, for instance, after > the re-indent and grammar change in compile.c that *did* come through.) > Barry (or someone else ;) are those still waiting in the queue, or should we > consider them 'lost' ? I got a reject on two e-mails, but I didn't think of saving them....oooops..well, no matter, most of them were trivial stuff. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From tismer@appliedbiometrics.com Sun Aug 6 15:47:26 2000 From: tismer@appliedbiometrics.com (Christian Tismer) Date: Sun, 06 Aug 2000 16:47:26 +0200 Subject: [Python-Dev] Re: Python 2.0 and Stackless References: Message-ID: <398D7A7E.2AB1BDF3@appliedbiometrics.com> Moshe Zadka wrote: > > On Sat, 5 Aug 2000, Eric S. Raymond wrote: > > > I must say I agree. Something pretty similar to Stackless Python is > > going to have to happen anyway for the language to make its next major > > advance in capability -- generators, co-routining, and continuations. > > > > I also agree that this is a more important debate, and a harder set of > > decisions, than the PEPs. Which means we should start paying attention > > to it *now*. > > I tend to disagree. For a while now I'm keeping an eye on the guile > interpreter development (a very cool project, but unfortunately limping > along. It probably will be the .NET of free software, though). In guile, > they were able to implement continuations *without* what we call > stacklessness. Sure, it might look inefficient, but for most applications > (like co-routines) it's actually quite all right. Despite the fact that I consider the Guile implementation a pile of junk code that I would never touch like I did with Python*), you are probably right. Stackless goes a bit too far in a sense, that it implies abilities for other implementations which are hard to achieve. There are in fact other ways to implement coroutines and uthreads. Stackless happens to achieve all of that and a lot more, and to be very efficient. Therefore it would be a waste to go back to a restricted implementation since it exists already. If stackless didn't go so far, it would probably have been successfully integrated, already. I wanted it all and luckily got it all. On the other hand, there is no need to enforce every Python implementation to do the full continuation support. In CPython, continuationmodule.c can be used for such purposes, and it can be used as a basement for coroutine and generator implementations. Using Guile's way to implement these would be a possible path for JPython. The point is to use only parts of the possibilities and don't enforce everything for every environment. There is just no point in shrinking the current implementation down; not even a subset would be helpful in JPython. > What all that goes to > say is that we should treat stackles exactly like it is -- an > implementation detail. Now, that's not putting down Christian's work -- on > the contrary, I think the Python implementation is very important. But > that alone should indicate there's no need for a PEP. I, for one, am for > it, because I happen to think it's a much better implementation. If it > also has the effect of making continuationsmodule.c easier to write, well, > that's not an issue in this discussion as far as I'm concerned. A possible proposal could be this: - incorporate Stackless into CPython, but don't demand it for every implementation - implement coroutines and others with Stackless for CPython try alternative implementations for JPython if there are users - do *not* make continuations a standard language feature until there is a portable way to get it everywhere Still, I can't see the point with Java. There are enough C extension which are not available for JPython, but it is allowed to use them. Same with the continuationmodule, why does it need to exist for Java, in order to allow it for CPython? This is like not implementing new browser features until they can be implemented on my WAP handy. Sonsense. ciao - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF where do you want to jump today? http://www.stackless.com *) sorry, feel free to disagree, but this was my impression when I read the whole code half a year ago. This is exactly what I not want :-) From Moshe Zadka Sun Aug 6 16:11:21 2000 From: Moshe Zadka (Moshe Zadka) Date: Sun, 6 Aug 2000 18:11:21 +0300 (IDT) Subject: [Python-Dev] Re: Python 2.0 and Stackless In-Reply-To: <398D7A7E.2AB1BDF3@appliedbiometrics.com> Message-ID: On Sun, 6 Aug 2000, Christian Tismer wrote: > On the other hand, there is no need to enforce every Python > implementation to do the full continuation support. In CPython, > continuationmodule.c can be used for such purposes, and it can > be used as a basement for coroutine and generator implementations. > Using Guile's way to implement these would be a possible path > for JPython. Actually, you can't use Guile's way for JPython -- the guile folks are doing some low-level semi-portable stuff in C... > - incorporate Stackless into CPython, but don't demand it > for every implementation Again, I want to say I don't think there's a meaning for "for every implementation" -- Stackless is not part of the language definiton, it's part of the implementation. The whole Java/.NET is a red herring. > - implement coroutines and others with Stackless for CPython I think that should be done in a third-party module. But hey, if Guido wants to maintain another module... > - do *not* make continuations a standard language feature until > there is a portable way to get it everywhere I'd got further and say "do *not* make continuations a standard language feature" > Still, I can't see the point with Java. There are enough > C extension which are not available for JPython, but it is > allowed to use them. Same with the continuationmodule, why > does it need to exist for Java, in order to allow it for > CPython? My point exactly. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From tismer@appliedbiometrics.com Sun Aug 6 16:22:39 2000 From: tismer@appliedbiometrics.com (Christian Tismer) Date: Sun, 06 Aug 2000 17:22:39 +0200 Subject: [Python-Dev] Re: Python 2.0 and Stackless References: Message-ID: <398D82BF.85D0E5AB@appliedbiometrics.com> Moshe Zadka wrote: ... > > - implement coroutines and others with Stackless for CPython > > I think that should be done in a third-party module. But hey, if Guido > wants to maintain another module... Right, like now. CPython has the necessary stackless hooks, nuts and bolts, but nothing else, and no speed impact. Then is just happens to be *possible* to write such an extension, and it will be written, but this is no language feature. > > - do *not* make continuations a standard language feature until > > there is a portable way to get it everywhere > > I'd got further and say "do *not* make continuations a standard language > feature" This was my sentence, in the first place, but when reviewing the message, I could not resist to plug that in again <1.5 wink> As discussed in a private thread with Just, some continuation features can only be made "nice" if they are supported by some language extension. I want to use Python in CS classes to teach them continuations, therefore I need a backdoor :-) and-there-will-always-be-a-version-on-my-site-that-goes- -beyond-the-standard - ly y'rs - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF where do you want to jump today? http://www.stackless.com From bwarsaw@beopen.com Sun Aug 6 16:49:07 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Sun, 6 Aug 2000 11:49:07 -0400 (EDT) Subject: [Python-Dev] Re: Python 2.0 and Stackless References: <398D7A7E.2AB1BDF3@appliedbiometrics.com> Message-ID: <14733.35059.53619.98300@anthem.concentric.net> >>>>> "CT" == Christian Tismer writes: CT> Still, I can't see the point with Java. There are enough C CT> extension which are not available for JPython, but it is CT> allowed to use them. Same with the continuationmodule, why CT> does it need to exist for Java, in order to allow it for CT> CPython? This is like not implementing new browser features CT> until they can be implemented on my WAP handy. Sonsense. It's okay if there are some peripheral modules that are available to CPython but not JPython (include Python .NET here too), and vice versa. That'll just be the nature of things. But whatever basic language features Stackless allows one to do /from Python/ must be documented. That's the only way we'll be able to one of these things: - support the feature a different way in a different implementation - agree that the feature is part of the Python language definition, but possibly not (yet) supported by all implementations. - define the feature as implementation dependent (so people writing portable code know to avoid those features). -Barry From guido@beopen.com Sun Aug 6 18:23:52 2000 From: guido@beopen.com (Guido van Rossum) Date: Sun, 06 Aug 2000 12:23:52 -0500 Subject: [Python-Dev] Warnings on gcc -Wall In-Reply-To: Your message of "Sun, 06 Aug 2000 16:22:09 +0300." References: Message-ID: <200008061723.MAA14418@cj20424-a.reston1.va.home.com> > -- readline.c -- readline/history.h is included only on BeOS, and > otherwise prototypes are declared by hand. Does anyone remember why? Please don't touch that module. GNU readline is wacky. > -- ceval.c, in ceval() gcc -Wall (wrongly) complains about opcode and > oparg which might be used before initialized. I've had a look at that > code, and I'm certain gcc's flow analysis is simply not good enough. > However, I would like to silence the warning, so I can get used to > building with -Wall -Werror and make sure to mind any warnings. Does > anyone see any problem with putting opcode=0 and oparg=0 near the top? No problem. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From guido@beopen.com Sun Aug 6 18:34:34 2000 From: guido@beopen.com (Guido van Rossum) Date: Sun, 06 Aug 2000 12:34:34 -0500 Subject: [Python-Dev] strftime() In-Reply-To: Your message of "Sun, 06 Aug 2000 01:07:27 -0400." References: Message-ID: <200008061734.MAA14488@cj20424-a.reston1.va.home.com> > [Jack Jansen] > > The test_strftime regression test has been failing on the Mac for > > ages, and I finally got round to investigating the problem: the > > MetroWerks library returns the strings "am" and "pm" for %p but the > > regression test expects "AM" and "PM". According to the comments in > > the source of the library (long live vendors who provide it! Yeah!) > > this is C9X compatibility. > > My copy of a draft C99 std agrees (7.23.3.5) with MetroWerks on this point > (i.e., that %p in the "C" locale becomes "am" or "pm"). > > > I can of course move the %p to the nonstandard expectations, but maybe > > someone has a better idea? > > Not really. If Python thinks this function is valuable, it "should" offer a > platform-independent implementation, but as nobody has time for that ... No. The test is too strict. It should be fixed. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From just@letterror.com Sun Aug 6 18:59:42 2000 From: just@letterror.com (Just van Rossum) Date: Sun, 6 Aug 2000 18:59:42 +0100 Subject: [Python-Dev] Re: Python 2.0 and Stackless In-Reply-To: <14733.35059.53619.98300@anthem.concentric.net> References: <398D7A7E.2AB1BDF3@appliedbiometrics.com> Message-ID: At 11:49 AM -0400 06-08-2000, Barry A. Warsaw wrote: >It's okay if there are some peripheral modules that are available to >CPython but not JPython (include Python .NET here too), and vice >versa. That'll just be the nature of things. But whatever basic >language features Stackless allows one to do /from Python/ must be >documented. The things stackless offers are no different from: - os.open() - os.popen() - os.system() - os.fork() - threading (!!!) These things are all doable /from Python/, yet their non-portability seems hardly an issue for the Python Standard Library. >That's the only way we'll be able to one of these things: > >- support the feature a different way in a different implementation >- agree that the feature is part of the Python language definition, > but possibly not (yet) supported by all implementations. Honest (but possibly stupid) question: are extension modules part of the language definition? >- define the feature as implementation dependent (so people writing > portable code know to avoid those features). It's an optional extension module, so this should be obvious. (As it happens, it depends on a new and improved implementation of ceval.c, but this is really beside the point.) Just PS: thanks to everybody who kept CC-ing me in this thread; it's much appreciated as I'm not on python-dev. From jeremy@alum.mit.edu Sun Aug 6 19:54:56 2000 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Sun, 6 Aug 2000 14:54:56 -0400 Subject: [Python-Dev] Re: Python 2.0 and Stackless In-Reply-To: <20000805194600.A7242@thyrsus.com> Message-ID: Eric S. Raymond writes: >Just van Rossum : >> Christian has done an amazing piece of work, and he's gotten much >> praise from the community. I mean, if you *are* looking for a killer >> feature to distinguish 1.6 from 2.0, I'd know where to look... > >I must say I agree. Something pretty similar to Stackless Python is >going to have to happen anyway for the language to make its next major >advance in capability -- generators, co-routining, and continuations. > >I also agree that this is a more important debate, and a harder set of >decisions, than the PEPs. Which means we should start paying attention >to it *now*. The PEPs exist as a way to formalize important debates and hard decisions. Without a PEP that offers a formal description of the changes, it is hard to have a reasonable debate. I would not be comfortable with the specification for any feature from stackless being the implementation. Given the current release schedule for Python 2.0, I don't see any possibility of getting a PEP accepted in time. The schedule, from PEP 200, is: Tentative Release Schedule Aug. 14: All 2.0 PEPs finished / feature freeze Aug. 28: 2.0 beta 1 Sep. 29: 2.0 final Jeremy From guido@beopen.com Sun Aug 6 22:17:33 2000 From: guido@beopen.com (Guido van Rossum) Date: Sun, 06 Aug 2000 16:17:33 -0500 Subject: [Python-Dev] math.rint bites the dust Message-ID: <200008062117.QAA15501@cj20424-a.reston1.va.home.com> After a brief consult with Tim, I've decided to drop math.rint() -- it's not standard C, can't be implemented in portable C, and its naive (non-IEEE-754-aware) effect can easily be had in other ways. If you disagree, speak up now! --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From thomas@xs4all.net Sun Aug 6 21:25:03 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sun, 6 Aug 2000 22:25:03 +0200 Subject: [Python-Dev] math.rint bites the dust In-Reply-To: <200008062117.QAA15501@cj20424-a.reston1.va.home.com>; from guido@beopen.com on Sun, Aug 06, 2000 at 04:17:33PM -0500 References: <200008062117.QAA15501@cj20424-a.reston1.va.home.com> Message-ID: <20000806222502.S266@xs4all.nl> On Sun, Aug 06, 2000 at 04:17:33PM -0500, Guido van Rossum wrote: > After a brief consult with Tim, I've decided to drop math.rint() -- > it's not standard C, can't be implemented in portable C, and its > naive (non-IEEE-754-aware) effect can easily be had in other ways. I don't particularly disagree, since I hardly do anything with floating point numbers, but how can something both not be implementable in portable C *and* it's effect easily be had in other ways ? I also recall someone who was implementing rint() on platforms that didnt have it... Or did that idea get trashed because it wasn't portable enough ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From nowonder@nowonder.de Sun Aug 6 23:49:06 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Sun, 06 Aug 2000 22:49:06 +0000 Subject: [Python-Dev] bug-fixes in cnri-16-start branch Message-ID: <398DEB62.789B4C9C@nowonder.de> I have a question on the right procedure for fixing a simple bug in the 1.6 release branch. Bug #111162 appeared because the tests for math.rint() are already contained in the cnri-16-start revision of test_math.py while the "try: ... except AttributeError: ..." construct which was checked in shortly after was not. Now the correct bugfix is already known (and has been applied to the main branch). I have updated the test_math.py file in my working version with "-r cnri-16-start" and made the changes. Now I probably should just commit, close the patch (with an appropriate follow-up) and be happy. did-I-get-that-right-or-does-something-else-have-to-be-done-ly y'rs Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From tim_one@email.msn.com Sun Aug 6 21:54:02 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 6 Aug 2000 16:54:02 -0400 Subject: [Python-Dev] math.rint bites the dust In-Reply-To: <20000806222502.S266@xs4all.nl> Message-ID: [Guido] > After a brief consult with Tim, I've decided to drop math.rint() -- > it's not standard C, can't be implemented in portable C, and its > naive (non-IEEE-754-aware) effect can easily be had in other ways. [Thomas Wouters] > I don't particularly disagree, since I hardly do anything with floating > point numbers, but how can something both not be implementable in > portable C *and* it's effect easily be had in other ways ? Can't. rint is not standard C, but is standard C99, where a conforming implementation requires paying attention to all the details of the 754 fp model. It's a *non* 754-aware version of rint that can be easily had in other ways (e.g., you easily write a rint in Python that always rounds to nearest/even, by building on math.floor and checking the sign bit, but ignoring the possibilities of infinities, NaNs, current 754 rounding mode, and correct treatment of (at least) the 754 inexact and underflow flags -- Python gives no way to get at any of those now, neither does current C, and a correct rint from the C99 point of view has to deal with all of them). This is a case where I'm unwilling to support a function at all before it can be supported correctly; I see no value in exposing current platforms' divergent and incorrect implementations of rint, not in the math module. Code that uses them will fail to work at all on some platforms (since rint is not in today's C, some platfroms don't have it), and change meaning over time as the other platforms move toward C99 compliance. > I also recall someone who was implementing rint() on platforms > that didnt have it... Or did that idea get trashed because it wasn't > portable enough ? Bingo. everyone's-welcome-to-right-their-own-incorrect-version -ly y'rs - tim From jack@oratrix.nl Sun Aug 6 21:56:48 2000 From: jack@oratrix.nl (Jack Jansen) Date: Sun, 06 Aug 2000 22:56:48 +0200 Subject: [Python-Dev] Stackless Python - Pros and Cons Message-ID: <20000806205653.B0341E2670@oratrix.oratrix.nl> Could the defenders of Stackless Python please explain _why_ this is such a great idea? Just and Christian seem to swear by it, but I'd like to hear of some simple examples of programming tasks that will be programmable in 50% less code with it (or 50% more understandable code, for that matter). And, similarly, could the detractors of Stackless Python explain why it is such a bad idea. A lot of core-pythoneers seem to have misgivings about it, even though issues of compatability and efficiency have been countered many times here by its champions (at least, it seems that way to a clueless bystander like myself). I'd really like to be able to take a firm standpoint myself, that's part of my personality, but I really don't know which firm standpoint at the moment:-) -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++ From tim_one@email.msn.com Sun Aug 6 22:03:23 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 6 Aug 2000 17:03:23 -0400 Subject: [Python-Dev] Stackless Python - Pros and Cons In-Reply-To: <20000806205653.B0341E2670@oratrix.oratrix.nl> Message-ID: [Jack Jansen] > Could the defenders of Stackless Python please explain _why_ this is > such a great idea? ... But they already have, and many times. That's why it needs a PEP: so we don't have to endure the exact same heated discussions multiple times every year for eternity. > ... > And, similarly, could the detractors of Stackless Python explain why > it is such a bad idea. Ditto. if-anyone-hasn't-yet-noticed-98%-of-advocacy-posts-go-straight- into-a-black-hole-ly y'rs - tim From thomas@xs4all.net Sun Aug 6 22:05:45 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sun, 6 Aug 2000 23:05:45 +0200 Subject: [Python-Dev] Stackless Python - Pros and Cons In-Reply-To: <20000806205653.B0341E2670@oratrix.oratrix.nl>; from jack@oratrix.nl on Sun, Aug 06, 2000 at 10:56:48PM +0200 References: <20000806205653.B0341E2670@oratrix.oratrix.nl> Message-ID: <20000806230545.T266@xs4all.nl> On Sun, Aug 06, 2000 at 10:56:48PM +0200, Jack Jansen wrote: > Could the defenders of Stackless Python please explain _why_ this is > such a great idea? Just and Christian seem to swear by it, but I'd > like to hear of some simple examples of programming tasks that will be > programmable in 50% less code with it (or 50% more understandable > code, for that matter). That's *continuations*, not Stackless. Stackless itself is just a way of implementing the Python bytecode eval loop with minimized use of the C stack. It doesn't change any functionality except the internal dependance on the C stack (which is limited on some platforms.) Stackless also makes a number of things possible, like continuations. Continuations can certainly reduce code, if used properly, and they can make it a lot more readable if the choice is between continuations or threaded spaghetti-code. It can, however, make code a lot less readable too, if used improperly, or when viewed by someone who doesn't grok continuations. I'm +1 on Stackless, +0 on continuations. (Continuations are cool, and Pythonic in one sense (stackframes become even firster class citizens ;) but not easy to learn or get used to.) > And, similarly, could the detractors of Stackless Python explain why > it is such a bad idea. I think my main reservation towards Stackless is the change to ceval.c, which is likely to be involved (I haven't looked at it, yet) -- but ceval.c isn't a childrens' book now, and I think the added complexity (if any) is worth the loss of some of the dependancies on the C stack. fl.0,02-ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From nowonder@nowonder.de Mon Aug 7 00:18:22 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Sun, 06 Aug 2000 23:18:22 +0000 Subject: [Python-Dev] math.rint bites the dust References: <200008062117.QAA15501@cj20424-a.reston1.va.home.com> Message-ID: <398DF23E.D1DDE196@nowonder.de> Guido van Rossum wrote: > > After a brief consult with Tim, I've decided to drop math.rint() -- > it's not standard C, can't be implemented in portable C, and its > naive (non-IEEE-754-aware) effect can easily be had in other ways. If this is because of Bug #111162, things can be fixed easily. (as I said in another post just some minutes ago, I just need to recommit the changes made after cnri-16-start.) I wouldn't be terribly concerned about its (maybe temporary) death, though. After I learned more about it I am sure I want to use round() rather than math.rint(). floating-disap-point-ed-ly y'rs Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From esr@thyrsus.com Sun Aug 6 22:59:35 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Sun, 6 Aug 2000 17:59:35 -0400 Subject: [Python-Dev] Stackless Python - Pros and Cons In-Reply-To: <20000806205653.B0341E2670@oratrix.oratrix.nl>; from jack@oratrix.nl on Sun, Aug 06, 2000 at 10:56:48PM +0200 References: <20000806205653.B0341E2670@oratrix.oratrix.nl> Message-ID: <20000806175935.A14138@thyrsus.com> Jack Jansen : > Could the defenders of Stackless Python please explain _why_ this is > such a great idea? Just and Christian seem to swear by it, but I'd > like to hear of some simple examples of programming tasks that will be > programmable in 50% less code with it (or 50% more understandable > code, for that matter). My interest in Stackless is that I want to see Icon-style generators, co-routining, and first-class continuations in Python. Generators and co-routining are wrappers around continuations. Something functionally equivalent to the Stackless mods is needed to get us there, because using the processor stack makes it very hard to do continuations properly. In their full generality, first-class continuations are hard to think about and to explain clearly, and I'm not going to try here. A large part of Guido's reluctance to introduce them is precisely because they are so hard to think about; he thinks it's a recipe for trouble stuff in the language that *he* has trouble understanding, let alone other people. He has a point, and part of the debate going on in the group that has been tracking this stuff (Guido, Barry Warsaw, Jeremy Hylton, Fred Drake, Eric Tiedemann and myself) is whether Python should expose support for first-class continuations or only "safer" packagings such as coroutining and generators. So for the moment just think of continuations as the necessary primitive to implement coroutining and generators. You can think of a generator as a function that, internally, is coded as a special kind of loop. Let's say, for example, that you want a function that returns successive entries in the list "squares of integers". In Python-with-generators, that would look something like this. def square_generator(): i = 1 while 1: yield i**2 i = i + 1 Calling this function five times in succession would return 1, 4, 9, 16, 25. Now what would be going on under the hood is that the new primitive "yield" says "return the given value, and save a continuation of this function to be run next time the function is called". The continuation saves the program counter and the state of automatic variables (the stack) to be restored on the next call -- thus, execution effectively resumes just after the yield statement. This example probably does not look very interesting. It's a very trivial use of the facility. But now suppose you had an analogous function implemented by a code loop that gets an X event and yields the event data! Suddenly, X programs don't have to look like a monster loop with all the rest of the code hanging off of them. Instead, any function in the program that needs to do stateful input parsing can just say "give me the next event" and get it. In general, what generators let you do is invert control hierarchies based on stateful loops or recursions. This is extremely nice for things like state machines and tree traversals -- you can bundle away the control loop away in a generator, interrupt it, and restart it without losing your place. I want this feature a lot. Guido has agreed in principle that we ought to have generators, but there is not yet a well-defined path forward to them. Stackless may be the most promising route. I was going to explain coroutines separately, but I realized while writing this that the semantics of "yield" proposed above actually gives full coroutining. Two functions can ping-pong control back and forth among themselves while retaining their individual stack states as a pair of continuations. -- Eric S. Raymond "This country, with its institutions, belongs to the people who inhabit it. Whenever they shall grow weary of the existing government, they can exercise their constitutional right of amending it or their revolutionary right to dismember it or overthrow it." -- Abraham Lincoln, 4 April 1861 From tim_one@email.msn.com Sun Aug 6 23:07:45 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 6 Aug 2000 18:07:45 -0400 Subject: [Python-Dev] Stackless Python - Pros and Cons In-Reply-To: <20000806175935.A14138@thyrsus.com> Message-ID: [ Eric S. Raymond] > ... > I want this feature [generators] a lot. Guido has agreed in principle > that we ought to have generators, but there is not yet a well-defined > path forward to them. Stackless may be the most promising route. Actually, if we had a PEP , it would have recorded for all time that Guido gave a detailed description of how to implement generators with minor changes to the current code. It would also record that Steven Majewski had already done so some 5 or 6 years ago. IMO, the real reason we don't have generators already is that they keep getting hijacked by continuations (indeed, Steven gave up on his patches as soon as he realized he couldn't extend his approach to continuations). > I was going to explain coroutines separately, but I realized while > writing this that the semantics of "yield" proposed above actually > gives full coroutining. Well, the Icon semantics for "suspend"-- which are sufficient for Icon's generators --are not sufficient for Icon's coroutines. It's for that very reason that Icon supports generators on all platforms (including JCon, their moral equivalent of JPython), but supports coroutines only on platforms that have the magical blob of platform-dependent machine-language cruft needed to trick out the C stack at coroutine context switches (excepting JCon, where coroutines are implemented as Java threads). Coroutines are plain harder. Generators are just semi-coroutines (suspend/yield *always* return "to the caller", and that makes life 100x easier in a conventional eval loop like Python's -- it's still "stack-like", and the only novel thing needed is a way to resume a suspended frame but still in call-like fashion). and-if-we-had-a-pep-every-word-of-this-reply-would-have-been- in-it-too -ly y'rs - tim From esr@thyrsus.com Sun Aug 6 23:51:59 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Sun, 6 Aug 2000 18:51:59 -0400 Subject: [Python-Dev] Stackless Python - Pros and Cons In-Reply-To: ; from tim_one@email.msn.com on Sun, Aug 06, 2000 at 06:07:45PM -0400 References: <20000806175935.A14138@thyrsus.com> Message-ID: <20000806185159.A14259@thyrsus.com> Tim Peters : > [ Eric S. Raymond] > > ... > > I want this feature [generators] a lot. Guido has agreed in principle > > that we ought to have generators, but there is not yet a well-defined > > path forward to them. Stackless may be the most promising route. > > Actually, if we had a PEP , it would have recorded for all time that > Guido gave a detailed description of how to implement generators with minor > changes to the current code. It would also record that Steven Majewski had > already done so some 5 or 6 years ago. Christian Tismer, over to you. I am not going to presume to initiate the continuations PEP when there's someone with a Python implementation and extensive usage experience on the list. However, I will help with editing and critiques based on my experience with other languages that have similar features, if you want. > IMO, the real reason we don't have > generators already is that they keep getting hijacked by continuations > (indeed, Steven gave up on his patches as soon as he realized he couldn't > extend his approach to continuations). This report of repeated "hijacking" doesn't surprise me a bit. In fact, if I'd thought about it I'd have *expected* it. We know from experience with other languages (notably Scheme) that call-with-current-continuation is the simplest orthogonal primitive that this whole cluster of concepts can be based on. Implementors with good design taste are going to keep finding their way back to it, and they're going to feel incompleteness and pressure if they can't get there. This is why I'm holding out for continuation objects and call-with-continuation to be an explicit Python builtin. We're going to get there anyway; best to do it cleanly right away. -- Eric S. Raymond "Taking my gun away because I might shoot someone is like cutting my tongue out because I might yell `Fire!' in a crowded theater." -- Peter Venetoklis From esr@snark.thyrsus.com Mon Aug 7 00:18:35 2000 From: esr@snark.thyrsus.com (Eric S. Raymond) Date: Sun, 6 Aug 2000 19:18:35 -0400 Subject: [Python-Dev] Adding a new class to the library? Message-ID: <200008062318.TAA14335@snark.thyrsus.com> I have a candidate for admission to the Python class library. It's a framework class for writing things like menu trees and object browsers. What's the correct approval procedure for such things? In more detail, it supports manipulating a stack of sequence objects. Each sequence object has an associated selection point (the cirrently selected sequence member) and an associated viewport around it (a range of indices or sequence members that are considered `visible'. There are methods to manipulate the object stack. More importantly, there are functions which move the selection point in the current object around, and drag the viewport with it. (This sort of thing sounds simple, but is tricky for the same reason BitBlt is tricky -- lots of funky boundary cases.) I've used this as the framework for implementing the curses menu interface for CML2. It is well-tested and stable. It might also be useful for implementing other kinds of data browsers in any situation where the concept of limited visibility around a selection point makes sense. Symbolic debuggers is an example that leaps to mind. I am, of course, willing to fully document it. -- Eric S. Raymond "One of the ordinary modes, by which tyrants accomplish their purposes without resistance, is, by disarming the people, and making it an offense to keep arms." -- Constitutional scholar and Supreme Court Justice Joseph Story, 1840 From gmcm@hypernet.com Mon Aug 7 00:34:44 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Sun, 6 Aug 2000 19:34:44 -0400 Subject: [Python-Dev] Stackless Python - Pros and Cons In-Reply-To: <20000806205653.B0341E2670@oratrix.oratrix.nl> Message-ID: <1246517606-99838203@hypernet.com> Jack Jansen wrote: > Could the defenders of Stackless Python please explain _why_ this > is such a great idea? Just and Christian seem to swear by it, but > I'd like to hear of some simple examples of programming tasks > that will be programmable in 50% less code with it (or 50% more > understandable code, for that matter). Here's the complete code for the download of a file (the data connection of an FTP server): def _doDnStream(self, binary=0): mode = 'r' if binary: mode = mode + 'b' f = open(self.cmdconn.filename, mode) if self.cmdconn.filepos: #XXX check length of file f.seek(self.cmdconn.filepos, 0) while 1: if self.abort: break data = f.read(8192) sz = len(data) if sz: if not binary: data = '\r\n'.join(data.split('\n')) self.write(data) if sz < 8192: break [from the base class] def write(self, msg): while msg: sent = self.dispatcher.write(self.sock, msg) if sent == 0: raise IOError, "unexpected EOF" msg = msg[sent:] Looks like blocking sockets, right? Wrong. That's a fully multiplexed socket. About a dozen lines of code (hidden in that dispatcher object) mean that I can write async without using a state machine. stackless-forever-ly y'rs - Gordon From guido@beopen.com Mon Aug 7 02:32:59 2000 From: guido@beopen.com (Guido van Rossum) Date: Sun, 06 Aug 2000 20:32:59 -0500 Subject: [Python-Dev] Adding a new class to the library? In-Reply-To: Your message of "Sun, 06 Aug 2000 19:18:35 -0400." <200008062318.TAA14335@snark.thyrsus.com> References: <200008062318.TAA14335@snark.thyrsus.com> Message-ID: <200008070132.UAA16111@cj20424-a.reston1.va.home.com> > I have a candidate for admission to the Python class library. It's a > framework class for writing things like menu trees and object > browsers. What's the correct approval procedure for such things? > > In more detail, it supports manipulating a stack of sequence objects. > Each sequence object has an associated selection point (the cirrently > selected sequence member) and an associated viewport around it (a > range of indices or sequence members that are considered `visible'. > > There are methods to manipulate the object stack. More importantly, > there are functions which move the selection point in the current > object around, and drag the viewport with it. (This sort of > thing sounds simple, but is tricky for the same reason BitBlt is > tricky -- lots of funky boundary cases.) > > I've used this as the framework for implementing the curses menu > interface for CML2. It is well-tested and stable. It might also > be useful for implementing other kinds of data browsers in any > situation where the concept of limited visibility around a selection > point makes sense. Symbolic debuggers is an example that leaps to mind. > > I am, of course, willing to fully document it. Have a look at the tree widget in IDLE. That's Tk specific, but I believe there's a lot of GUI independent concepts in there. IDLE's path and object browsers are built on it. How does this compare? --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From esr@thyrsus.com Mon Aug 7 01:52:53 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Sun, 6 Aug 2000 20:52:53 -0400 Subject: [Python-Dev] Adding a new class to the library? In-Reply-To: <200008070132.UAA16111@cj20424-a.reston1.va.home.com>; from guido@beopen.com on Sun, Aug 06, 2000 at 08:32:59PM -0500 References: <200008062318.TAA14335@snark.thyrsus.com> <200008070132.UAA16111@cj20424-a.reston1.va.home.com> Message-ID: <20000806205253.B14423@thyrsus.com> Guido van Rossum : > Have a look at the tree widget in IDLE. That's Tk specific, but I > believe there's a lot of GUI independent concepts in there. IDLE's > path and object browsers are built on it. How does this compare? Where is this in the CVS tree? I groveled for it but without success. -- Eric S. Raymond To make inexpensive guns impossible to get is to say that you're putting a money test on getting a gun. It's racism in its worst form. -- Roy Innis, president of the Congress of Racial Equality (CORE), 1988 From greg@cosc.canterbury.ac.nz Mon Aug 7 02:04:27 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Mon, 07 Aug 2000 13:04:27 +1200 (NZST) Subject: [Python-Dev] Go \x yourself In-Reply-To: <200008041511.KAA01925@cj20424-a.reston1.va.home.com> Message-ID: <200008070104.NAA12334@s454.cosc.canterbury.ac.nz> BDFL: > No, problems with literal interpretations traditionally raise > "runtime" exceptions rather than syntax errors. E.g. What about using an exception that's a subclass of *both* ValueError and SyntaxError? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim_one@email.msn.com Mon Aug 7 02:16:44 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 6 Aug 2000 21:16:44 -0400 Subject: [Python-Dev] Stackless Python - Pros and Cons In-Reply-To: <20000806185159.A14259@thyrsus.com> Message-ID: [Tim] > IMO, the real reason we don't have generators already is that > they keep getting hijacked by continuations (indeed, Steven gave > up on his patches as soon as he realized he couldn't extend his > approach to continuations). [esr] > This report of repeated "hijacking" doesn't surprise me a bit. In > fact, if I'd thought about it I'd have *expected* it. We know from > experience with other languages (notably Scheme) that call-with- > current-continuation is the simplest orthogonal primitive that this > whole cluster of concepts can be based on. Implementors with good > design taste are going to keep finding their way back to it, and > they're going to feel incompleteness and pressure if they can't get > there. On the one hand, I don't think I know of a language *not* based on Scheme that has call/cc (or a moral equivalent). REBOL did at first, but after Joe Marshal left, Carl Sassenrath ripped it out in favor of a more conventional implementation. Even the massive Common Lisp declined to adopt call/cc, the reasons for which Kent Pitman has posted eloquently and often on comp.lang.lisp (basically summarized by that continuations are, in Kent's view, "a semantic mess" in the way Scheme exposed them -- btw, people should look his stuff up, as he has good ideas for cleaning that mess w/o sacrificing the power (and so the Lisp world splinters yet again?)). So call/cc remains "a Scheme thing" to me after all these years, and even there by far the most common warning in the release notes for a new implementation is that call/cc doesn't work correctly yet or at all (but, in the meantime, here are 3 obscure variations that will work in hard-to-explain special cases ...). So, ya, we *do* have experience with this stuff, and it sure ain't all good. On the other hand, what implementors other than Schemeheads *do* keep rediscovering is that generators are darned useful and can be implemented easily without exotic views of the world. CLU, Icon and Sather all fit in that box, and their designers wouldn't touch continuations with a 10-foot thick condom . > This is why I'm holding out for continuation objects and > call-with-continuation to be an explicit Python builtin. We're > going to get there anyway; best to do it cleanly right away. This can get sorted out in the PEP. As I'm sure someone else has screamed by now (because it's all been screamed before), Stackless and the continuation module are distinct beasts (although the latter relies on the former). It would be a shame if the fact that it makes continuations *possible* were to be held against Stackless. It makes all sorts of things possible, some of which Guido would even like if people stopped throwing continuations in his face long enough for him to see beyond them <0.5 wink -- but he doesn't like continuations, and probably never will>. From jeremy@alum.mit.edu Mon Aug 7 02:39:46 2000 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Sun, 6 Aug 2000 21:39:46 -0400 Subject: [Python-Dev] Stackless Python - Pros and Cons In-Reply-To: <20000806205653.B0341E2670@oratrix.oratrix.nl> Message-ID: If someone is going to write a PEP, I hope they will explain how the implementation deals with the various Python C API calls that can call back into Python. In the stackless implementation, builtin_apply is a thin wrapper around builtin_apply_nr. The wrapper checks the return value from builtin_apply_nr for Py_UnwindToken. If Py_UnwindToken is found, it calls PyEval_Frame_Dispatch. In this case, builtin_apply returns whatever PyEval_Frame_Dispatch returns; the frame dispatcher just executes stack frames until it is ready to return. How does this control flow at the C level interact with a Python API call like PySequence_Tuple or PyObject_Compare that can start executing Python code again? Say there is a Python function call which in turn calls PySequence_Tuple, which in turn calls a __getitem__ method on some Python object, which in turn uses a continuation to transfer control. After the continuation is called, the Python function will never return and the PySquence_Tuple call is no longer necessary, but there is still a call to PySequence_Tuple on the C stack. How does stackless deal with the return through this function? I expect that any C function that may cause Python code to be executed must be wrapped the way apply was wrapper. So in the example, PySequence_Tuple may return Py_UnwindToken. This adds an extra return condition that every caller of PySequence_Tuple must check. Currently, the caller must check for NULL/exception in addition to a normal return. With stackless, I assume the caller would also need to check for "unwinding." Is this analysis correct? Or is there something I'm missing? I see that the current source release of stackless does not do anything special to deal with C API calls that execute Python code. For example, PyDict_GetItem calls PyObject_Hash, which could in theory lead to a call on a continuation, but neither caller nor callee does anything special to account for the possibility. Is there some other part of the implementation that prevents this from being a problem? Jeremy From greg@cosc.canterbury.ac.nz Mon Aug 7 02:50:32 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Mon, 07 Aug 2000 13:50:32 +1200 (NZST) Subject: [Python-Dev] Re: A small proposed change to dictionaries' "get" method In-Reply-To: <200008041546.KAA02168@cj20424-a.reston1.va.home.com> Message-ID: <200008070150.NAA12345@s454.cosc.canterbury.ac.nz> > dict.default('hello', []).append('hello') Is this new method going to apply to dictionaries only, or is it to be considered part of the standard mapping interface? If the latter, I wonder whether it would be better to provide a builtin function instead. The more methods are added to the mapping interface, the more complicated it becomes to implement an object which fully complies with the mapping interface. Operations which can be carried out through the basic interface are perhaps best kept "outside" the object, in a function or wrapper object. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From bwarsaw@beopen.com Mon Aug 7 03:25:54 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Sun, 6 Aug 2000 22:25:54 -0400 (EDT) Subject: [Python-Dev] Re: A small proposed change to dictionaries' "get" method References: <200008041546.KAA02168@cj20424-a.reston1.va.home.com> <200008070150.NAA12345@s454.cosc.canterbury.ac.nz> Message-ID: <14734.7730.698860.642851@anthem.concentric.net> >>>>> "GE" == Greg Ewing writes: >> dict.default('hello', []).append('hello') GE> Is this new method going to apply to dictionaries only, GE> or is it to be considered part of the standard mapping GE> interface? I think we've settled on setdefault(), which is more descriptive, even if it's a little longer. I have posted SF patch #101102 which adds setdefault() to both the dictionary object and UserDict (along with the requisite test suite and doco changes). -Barry From pf@artcom-gmbh.de Mon Aug 7 09:32:00 2000 From: pf@artcom-gmbh.de (Peter Funk) Date: Mon, 7 Aug 2000 10:32:00 +0200 (MEST) Subject: [Python-Dev] Who is the author of lib-tk/Tkdnd.py? Message-ID: Hi, I've some ideas (already implemented <0.5 wink>) for generic Drag'n'Drop in Python/Tkinter applications. Before bothering the list here I would like to discuss this with the original author of Tkdnd.py. Thank you for your attention, Peter -- Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260 After all, Python is a programming language, not a psychic hotline. --Tim Peters From mal@lemburg.com Mon Aug 7 09:57:01 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 07 Aug 2000 10:57:01 +0200 Subject: [Python-Dev] Pickling using XML as output format References: Message-ID: <398E79DD.3EB21D3A@lemburg.com> Moshe Zadka wrote: > > On Sun, 6 Aug 2000, M.-A. Lemburg wrote: > > > Before starting to reinvent the wheel: > > Ummmm......I'd wait for some DC guy to chime in: I think Zope had > something like that. You might want to ask around on the Zope lists > or search zope.org. > > I'm not sure what it has and what it doesn't have, though. I'll download the latest beta and check this out. Thanks for the tip, -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mal@lemburg.com Mon Aug 7 10:15:08 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 07 Aug 2000 11:15:08 +0200 Subject: [Python-Dev] Go \x yourself References: <200008070104.NAA12334@s454.cosc.canterbury.ac.nz> Message-ID: <398E7E1C.84D28EA5@lemburg.com> Greg Ewing wrote: > > BDFL: > > > No, problems with literal interpretations traditionally raise > > "runtime" exceptions rather than syntax errors. E.g. > > What about using an exception that's a subclass of *both* > ValueError and SyntaxError? What would this buy you ? Note that the contents of a literal string don't really have anything to do with syntax. The \x escape sequences are details of the codecs used for converting those literal strings to Python string objects. Perhaps we need a CodecError which is subclass of ValueError and then make the UnicodeError a subclass of this CodecError ?! -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From artcom0!pf@artcom-gmbh.de Mon Aug 7 09:14:54 2000 From: artcom0!pf@artcom-gmbh.de (artcom0!pf@artcom-gmbh.de) Date: Mon, 7 Aug 2000 10:14:54 +0200 (MEST) Subject: dict.setdefault() (Patch#101102) (was: Re: [Python-Dev] Re: A small proposed change to dictionaries' "get" method...) In-Reply-To: <14734.7730.698860.642851@anthem.concentric.net> from "Barry A. Warsaw" at "Aug 6, 2000 10:25:54 pm" Message-ID: Hi, Guido: > >> dict.default('hello', []).append('hello') Greg Ewing : > GE> Is this new method going to apply to dictionaries only, > GE> or is it to be considered part of the standard mapping > GE> interface? Barry A. Warsaw: > I think we've settled on setdefault(), which is more descriptive, even > if it's a little longer. I have posted SF patch #101102 which adds > setdefault() to both the dictionary object and UserDict (along with > the requisite test suite and doco changes). This didn't answer the question raised by Greg Ewing. AFAI have seen, the patch doesn't touch 'dbm', 'shelve' and so on. So from the patch the answer is "applies to dictionaries only". What is with the other external mapping types already in the core, like 'dbm', 'shelve' and so on? If the patch doesn't add this new method to these other mapping types, this fact should at least be documented similar to the methods 'items()' and 'values' that are already unimplemented in 'dbm': """Dbm objects behave like mappings (dictionaries), except that keys and values are always strings. Printing a dbm object doesn't print the keys and values, and the items() and values() methods are not supported.""" I'm still -1 on the name: Nobody would expect, that a method called 'setdefault()' will actually return something useful. May be it would be better to invent an absolutely obfuscuated new name, so that everybody is forced to actually *READ* the documentation of this method or nobody will guess, what it is supposed to do or even worse: how to make clever use of it. At least it would be a lot more likely, that someone becomes curious, what a method called 'grezelbatz()' is suppoed to do, than that someone will actually lookup the documentation of a method called 'setdefault()'. If the average Python programmer would ever start to use this method at all, then I believe it is very likely that we will see him/her coding: dict.setdefault('key', []) dict['key'].append('bar') So I'm also still -1 on the concept. I'm +0 on Gregs proposal, that it would be better to make this a builtin function, that can be applied to all mapping types. Maybe it would be even better to delay this until in Python 3000 builtin types may have become real classes, so that this method may be inherited by all mapping types from an abstract mapping base class. Regards, Peter -- Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260 office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen) From mal@lemburg.com Mon Aug 7 11:07:09 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 07 Aug 2000 12:07:09 +0200 Subject: [Python-Dev] Pickling using XML as output format References: <398E79DD.3EB21D3A@lemburg.com> Message-ID: <398E8A4D.CAA87E02@lemburg.com> "M.-A. Lemburg" wrote: > > Moshe Zadka wrote: > > > > On Sun, 6 Aug 2000, M.-A. Lemburg wrote: > > > > > Before starting to reinvent the wheel: > > > > Ummmm......I'd wait for some DC guy to chime in: I think Zope had > > something like that. You might want to ask around on the Zope lists > > or search zope.org. > > > > I'm not sure what it has and what it doesn't have, though. > > I'll download the latest beta and check this out. Ok, Zope has something called ppml.py which aims at converting Python pickles to XML. It doesn't really pickle directly to XML though and e.g. uses the Python encoding for various objects. I guess, I'll start hacking away at my own xmlpickle.py implementation with the goal of making Python pickles editable using a XML editor. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From tismer@appliedbiometrics.com Mon Aug 7 11:48:19 2000 From: tismer@appliedbiometrics.com (Christian Tismer) Date: Mon, 07 Aug 2000 12:48:19 +0200 Subject: [Python-Dev] Stackless Python - Pros and Cons References: Message-ID: <398E93F3.374B585A@appliedbiometrics.com> Jeremy Hylton wrote: > > If someone is going to write a PEP, I hope they will explain how the > implementation deals with the various Python C API calls that can call back > into Python. He will. > In the stackless implementation, builtin_apply is a thin wrapper around > builtin_apply_nr. The wrapper checks the return value from builtin_apply_nr > for Py_UnwindToken. If Py_UnwindToken is found, it calls > PyEval_Frame_Dispatch. In this case, builtin_apply returns whatever > PyEval_Frame_Dispatch returns; the frame dispatcher just executes stack > frames until it is ready to return. Correct. > How does this control flow at the C level interact with a Python API call > like PySequence_Tuple or PyObject_Compare that can start executing Python > code again? Say there is a Python function call which in turn calls > PySequence_Tuple, which in turn calls a __getitem__ method on some Python > object, which in turn uses a continuation to transfer control. After the > continuation is called, the Python function will never return and the > PySquence_Tuple call is no longer necessary, but there is still a call to > PySequence_Tuple on the C stack. How does stackless deal with the return > through this function? Right. What you see here is the incompleteness of Stackless. In order to get this "right", I would have to change many parts of the implementation, in order to allow for continuations in every (probably even unwanted) place. I could not do this. Instead, the situation of these still occouring recursions are handled differently. continuationmodule guarantees, that in the context of recursive interpreter calls, the given stack order of execution is obeyed. Violations of this cause simply an exception. > I expect that any C function that may cause Python code to be executed must > be wrapped the way apply was wrapper. So in the example, PySequence_Tuple > may return Py_UnwindToken. This adds an extra return condition that every > caller of PySequence_Tuple must check. Currently, the caller must check for > NULL/exception in addition to a normal return. With stackless, I assume the > caller would also need to check for "unwinding." No, nobody else is allowed to return Py_UnwindToken but the few functions in the builtins implementation and in ceval. The continuationmodule may produce it since it knows the context where it is called. eval_code is supposed to be the main instance who checks for this special value. As said, allowing this in any context would have been a huge change to the whole implementation, and would probably also have broken existing extensions which do not expect that a standard function wants to do a callback. > Is this analysis correct? Or is there something I'm missing? > > I see that the current source release of stackless does not do anything > special to deal with C API calls that execute Python code. For example, > PyDict_GetItem calls PyObject_Hash, which could in theory lead to a call on > a continuation, but neither caller nor callee does anything special to > account for the possibility. Is there some other part of the implementation > that prevents this from being a problem? This problem is no problem for itself, since inside the stackless modification for Python, there are no places where unexpected Py_UnwindTokens or continuations are produced. This is a closed system insofar. But with the continuation extension, it is of course a major problem. The final solution to the recursive interpreter/continuation problem was found long after my paper was presented. The idea is simple, solves everything, and shortened my implementation substantially: Whenever a recursive interpreter call takes place, the calling frame gets a lock flag set. This flag says "this frame is wrapped in a suspended eval_code call and cannot be a continuation". continuationmodule always obeys this flag and prevends the creation of continuations for such frames by raising an exception. In other words: Stack-like behavior is enforced in situations where the C stack is involved. So, a builtin or an extension *can* call a continuation, but finally, it will have to come back to the calling point. If not, then one of the locked frames will be touched, finally, in the wrong C stack order. But by reference counting, this touching will cause the attempt to create a continuation, and what I said above will raise an exception. Probably the wrong place to explain this in more detail here, but it doesn't apply to the stackless core at all which is just responsible for the necessary support machinery. ciao - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF where do you want to jump today? http://www.stackless.com From paul@prescod.net Mon Aug 7 13:18:43 2000 From: paul@prescod.net (Paul Prescod) Date: Mon, 07 Aug 2000 08:18:43 -0400 Subject: [Python-Dev] New winreg module really an improvement? References: <39866F8D.FCFA85CB@prescod.net> <1246975873-72274187@hypernet.com> Message-ID: <398EA923.E5400D2B@prescod.net> Gordon McMillan wrote: > > ... > > As a practical matter, it looks to me like winreg (under any but > the most well-informed usage) may well leak handles. If so, > that would be a disaster. But I don't have time to check it out. I would be very surprised if that was the case. Perhaps you can outline your thinking so that *I* can check it out. I claim that: _winreg never leaks Windows handles as long _winreg handle objects are destroyed. winreg is written entirely in Python and destroys _winreg handles as long as winreg key objects are destroyed. winreg key objects are destroyed as long as there is no cycle. winreg does not create cycles. Therefore, winreg does not leak handles. I'm 98% confident of each assertion...for a total confidence of 92%. -- Paul Prescod - Not encumbered by corporate consensus "I don't want you to describe to me -- not ever -- what you were doing to that poor boy to make him sound like that; but if you ever do it again, please cover his mouth with your hand," Grandmother said. -- John Irving, "A Prayer for Owen Meany" From guido@beopen.com Mon Aug 7 13:38:11 2000 From: guido@beopen.com (Guido van Rossum) Date: Mon, 07 Aug 2000 07:38:11 -0500 Subject: [Python-Dev] Re: A small proposed change to dictionaries' "get" method In-Reply-To: Your message of "Mon, 07 Aug 2000 13:50:32 +1200." <200008070150.NAA12345@s454.cosc.canterbury.ac.nz> References: <200008070150.NAA12345@s454.cosc.canterbury.ac.nz> Message-ID: <200008071238.HAA18076@cj20424-a.reston1.va.home.com> > > dict.default('hello', []).append('hello') > > Is this new method going to apply to dictionaries only, > or is it to be considered part of the standard mapping > interface? > > If the latter, I wonder whether it would be better to > provide a builtin function instead. The more methods > are added to the mapping interface, the more complicated > it becomes to implement an object which fully complies > with the mapping interface. Operations which can be > carried out through the basic interface are perhaps > best kept "outside" the object, in a function or > wrapper object. The "mapping interface" has no firm definition. You're free to implement something without a default() method and call it a mapping. In Python 3000, where classes and built-in types will be unified, of course this will be fixed: there will be a "mapping" base class that implements get() and default() in terms of other, more primitive operations. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From Moshe Zadka Mon Aug 7 12:45:45 2000 From: Moshe Zadka (Moshe Zadka) Date: Mon, 7 Aug 2000 14:45:45 +0300 (IDT) Subject: [Python-Dev] Minor compilation problem on HP-UX (1.6b1) (fwd) Message-ID: I've answered him personally about the first part -- but the second part is interesting (and even troubling) -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez ---------- Forwarded message ---------- Date: Mon, 7 Aug 2000 08:59:30 +0000 (UTC) From: Eddy De Greef To: python-list@python.org Newsgroups: comp.lang.python Subject: Minor compilation problem on HP-UX (1.6b1) Hi, when I compile version 1.6b1 on HP-UX-10, I get a few compilation errors in Python/getargs.c (undefined UCHAR_MAX etc). The following patch fixes this: ------------------------------------------------------------------------------ *** Python/getargs.c.orig Mon Aug 7 10:19:55 2000 --- Python/getargs.c Mon Aug 7 10:20:21 2000 *************** *** 8,13 **** --- 8,14 ---- #include "Python.h" #include + #include int PyArg_Parse Py_PROTO((PyObject *, char *, ...)); ------------------------------------------------------------------------------ I also have a suggestion to improve the speed on the HP-UX platform. By tuning the memory allocation algorithm (see the patch below), it is possible to obtain a speed improvement of up to 22% on non-trivial Python scripts, especially when lots of (small) objects have to be created. I'm aware that platform-specific features are undesirable for a multi-platform application such as Python, but 22% is quite a lot for such a small modification ... Maybe similar tricks can be used on other platforms too. ------------------------------------------------------------------------------ *** Modules/main.c.orig Mon Aug 7 10:02:09 2000 --- Modules/main.c Mon Aug 7 10:02:37 2000 *************** *** 83,88 **** --- 83,92 ---- orig_argc = argc; /* For Py_GetArgcArgv() */ orig_argv = argv; + #ifdef __hpux + mallopt (M_MXFAST, 512); + #endif /* __hpux */ + if ((p = getenv("PYTHONINSPECT")) && *p != '\0') inspect = 1; if ((p = getenv("PYTHONUNBUFFERED")) && *p != '\0') ------------------------------------------------------------------------------ Regards, Eddy -- http://www.python.org/mailman/listinfo/python-list From gmcm@hypernet.com Mon Aug 7 13:00:10 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Mon, 7 Aug 2000 08:00:10 -0400 Subject: [Python-Dev] New winreg module really an improvement? In-Reply-To: <398EA923.E5400D2B@prescod.net> Message-ID: <1246472883-102528128@hypernet.com> Paul Prescod wrote: > Gordon McMillan wrote: > > > > ... > > > > As a practical matter, it looks to me like winreg (under any > > but the most well-informed usage) may well leak handles. If so, > > that would be a disaster. But I don't have time to check it > > out. > > I would be very surprised if that was the case. Perhaps you can > outline your thinking so that *I* can check it out. Well, I saw RegKey.close nowhere referenced. I saw the method it calls in _winreg not getting triggered elsewhere. I missed that _winreg closes them another way on dealloc. BTW, not all your hive names exist on every Windows platform (or build of _winreg). - Gordon From jack@oratrix.nl Mon Aug 7 13:27:59 2000 From: jack@oratrix.nl (Jack Jansen) Date: Mon, 07 Aug 2000 14:27:59 +0200 Subject: [Python-Dev] Minor compilation problem on HP-UX (1.6b1) (fwd) In-Reply-To: Message by Moshe Zadka , Mon, 7 Aug 2000 14:45:45 +0300 (IDT) , Message-ID: <20000807122800.8D0B1303181@snelboot.oratrix.nl> > + #ifdef __hpux > + mallopt (M_MXFAST, 512); > + #endif /* __hpux */ > + After reading this I went off and actually _read_ the mallopt manpage for the first time in my life, and it seems there's quite a few parameters there we might want to experiment with. Besides the M_MXFAST there's also M_GRAIN, M_BLKSIZ, M_MXCHK and M_FREEHD that could have significant impact on Python performance. I know that all the tweaks and tricks I did in the MacPython malloc implementation resulted in a speedup of 20% or more (including the cache-aligment code in dictobject.c). -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From Vladimir.Marangozov@inrialpes.fr Mon Aug 7 13:59:49 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Mon, 7 Aug 2000 14:59:49 +0200 (CEST) Subject: mallopt (Re: [Python-Dev] Minor compilation problem on HP-UX (1.6b1) (fwd)) In-Reply-To: <20000807122800.8D0B1303181@snelboot.oratrix.nl> from "Jack Jansen" at Aug 07, 2000 02:27:59 PM Message-ID: <200008071259.OAA22446@python.inrialpes.fr> Jack Jansen wrote: > > > > + #ifdef __hpux > > + mallopt (M_MXFAST, 512); > > + #endif /* __hpux */ > > + > > After reading this I went off and actually _read_ the mallopt manpage for the > first time in my life, and it seems there's quite a few parameters there we > might want to experiment with. Besides the M_MXFAST there's also M_GRAIN, > M_BLKSIZ, M_MXCHK and M_FREEHD that could have significant impact on Python > performance. I know that all the tweaks and tricks I did in the MacPython > malloc implementation resulted in a speedup of 20% or more (including the > cache-aligment code in dictobject.c). To start with, try the optional object malloc I uploaded yestedray at SF. [Patch #101104] Tweaking mallopt and getting 20% speedup for some scripts is no surprise at all. For me . It is not portable though. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From jeremy@beopen.com Mon Aug 7 14:05:20 2000 From: jeremy@beopen.com (Jeremy Hylton) Date: Mon, 7 Aug 2000 09:05:20 -0400 (EDT) Subject: [Python-Dev] Stackless Python - Pros and Cons In-Reply-To: References: <20000806185159.A14259@thyrsus.com> Message-ID: <14734.46096.366920.827786@bitdiddle.concentric.net> >>>>> "TP" == Tim Peters writes: TP> On the one hand, I don't think I know of a language *not* based TP> on Scheme that has call/cc (or a moral equivalent). ML also has call/cc, at least the Concurrent ML variant. Jeremy From jeremy@beopen.com Mon Aug 7 14:10:14 2000 From: jeremy@beopen.com (Jeremy Hylton) Date: Mon, 7 Aug 2000 09:10:14 -0400 (EDT) Subject: [Python-Dev] Stackless Python - Pros and Cons In-Reply-To: <398E93F3.374B585A@appliedbiometrics.com> References: <398E93F3.374B585A@appliedbiometrics.com> Message-ID: <14734.46390.190481.441065@bitdiddle.concentric.net> >>>>> "CT" == Christian Tismer writes: >> If someone is going to write a PEP, I hope they will explain how >> the implementation deals with the various Python C API calls that >> can call back into Python. CT> He will. Good! You'll write a PEP. >> How does this control flow at the C level interact with a Python >> API call like PySequence_Tuple or PyObject_Compare that can start >> executing Python code again? Say there is a Python function call >> which in turn calls PySequence_Tuple, which in turn calls a >> __getitem__ method on some Python object, which in turn uses a >> continuation to transfer control. After the continuation is >> called, the Python function will never return and the >> PySquence_Tuple call is no longer necessary, but there is still a >> call to PySequence_Tuple on the C stack. How does stackless deal >> with the return through this function? CT> Right. What you see here is the incompleteness of Stackless. In CT> order to get this "right", I would have to change many parts of CT> the implementation, in order to allow for continuations in every CT> (probably even unwanted) place. I could not do this. CT> Instead, the situation of these still occouring recursions are CT> handled differently. continuationmodule guarantees, that in the CT> context of recursive interpreter calls, the given stack order of CT> execution is obeyed. Violations of this cause simply an CT> exception. Let me make sure I understand: If I invoke a continuation when there are extra C stack frames between the mainloop invocation that captured the continuation and the call of the continuation, the interpreter raises an exception? If so, continuations don't sound like they would mix well with C extension modules and callbacks. I guess it also could not be used inside methods that implement operator overloading. Is there a simple set of rules that describe the situtations where they will not work? Jeremy From thomas@xs4all.net Mon Aug 7 14:07:11 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 7 Aug 2000 15:07:11 +0200 Subject: [Python-Dev] augmented assignment Message-ID: <20000807150711.W266@xs4all.nl> --0OAP2g/MAC+5xKAE Content-Type: text/plain; charset=us-ascii I 'finished' the new augmented assignment patch yesterday, following the suggestions made by Guido about using INPLACE_* bytecodes rather than special GETSET_* opcodes. I ended up with 13 new opcodes: INPLACE_* opcodes for the 11 binary operation opcodes, DUP_TOPX which duplicates a number of stack items instead of just the topmost item, and ROT_FOUR. I thought I didn't need ROT_FOUR if we had DUP_TOPX but I hadn't realized assignment needs the new value at the bottom of the 'stack', and the objects that are used in the assignment above that. So ROT_FOUR is necessary in the case of slice-assignment: a[b:c] += i LOAD a [a] LOAD b [a, b] LOAD c [a, b, c] DUP_TOPX 3 [a, b, c, a, b, c] SLICE+3 [a, b, c, a[b:c]] LOAD i [a, b, c, a[b:c], i] INPLACE_ADD [a, b, c, result] ROT_FOUR [result, a, b, c] STORE_SLICE+3 [] When (and if) the *SLICE opcodes are removed, ROT_FOUR can, too :) The patch is 'done' in my opinion, except for two tiny things: - PyNumber_InPlacePower() takes just two arguments, not three. Three argument power() does such 'orrible things to coerce all the arguments, and you can't do augmented-assignment-three-argument-power anyway. If it's added it would be for the API only, and I'm not sure if it's worth it :P - I still don't like the '_ab_' names :) I think __inplace_add__ or __iadd__ is better, but that's just me. The PEP is also 'done'. Feedback is more than welcome, including spelling fixes and the like. I've attached the PEP to this mail, for convenience. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! --0OAP2g/MAC+5xKAE Content-Type: text/plain Content-Disposition: attachment; filename="pep-0203.txt" PEP: 203 Title: Augmented Assignments Version: $Revision: 1.4 $ Owner: thomas@xs4all.net (Thomas Wouters) Python-Version: 2.0 Status: Draft Introduction This PEP describes the `augmented assignment' proposal for Python 2.0. This PEP tracks the status and ownership of this feature, slated for introduction in Python 2.0. It contains a description of the feature and outlines changes necessary to support the feature. This PEP summarizes discussions held in mailing list forums, and provides URLs for further information, where appropriate. The CVS revision history of this file contains the definitive historical record. Proposed semantics The proposed patch that adds augmented assignment to Python introduces the following new operators: += -= *= /= %= **= <<= >>= &= ^= |= They implement the same operator as their normal binary form, with the exception that the operation is done `in-place' whenever possible. They truly behave as augmented assignment, in that they perform all of the normal load and store operations, in addition to the binary operation they are intended to do. So, given the expression: x += y The object `x' is loaded, then added with 1, and the resulting object is stored back in the original place. The precise action performed on the two arguments depends on the type of `x', and possibly of `y'. The idea behind augmented assignment in Python is that it isn't just an easier way to write the common practice of storing the result of a binary operation in its left-hand operand, but also a way for the left-hand operand in question to know that it should operate 'on itself', rather than creating a modified copy of itself. To make this possible, a number of new `hooks' are added to Python classes and C extention types, which are called when the object in question is used as the left hand side of an augmented assignment operation. If the class or type does not implement the `in-place' hooks, the normal hooks for the particular binary operation are used. So, given an instance object `x', the expression x += y tries to call x.__add_ab__(y), which is the 'in-place' variant of __add__. If __add_ab__ is not present, x.__add__(y) is attempted, and finally y.__radd__(x) if __add__ is missing too. There is no `right-hand-side' variant of __add_ab__, because that would require for `y' to know how to in-place modify `x', which is an unsafe assumption. The __add_ab__ hook should behave exactly like __add__, returning the result of the operation (which could be `self') which is to be stored in the variable `x'. For C extention types, the `hooks' are members of the PyNumberMethods and PySequenceMethods structures, and are called in exactly the same manner as the existing non-inplace operations, including argument coercion. C methods should also take care to return a new reference to the result object, whether it's the same object or a new one. So if the original object is returned, it should be INCREF()'d appropriately. New methods The proposed implementation adds the following 11 possible `hooks' which Python classes can implement to overload the augmented assignment operations: __add_ab__ __sub_ab__ __mul_ab__ __div_ab__ __mod_ab__ __pow_ab__ __lshift_ab__ __rshift_ab__ __and_ab__ __xor_ab__ __or_ab__ The `__add_ab__' name is one proposed by Guido[1], and stands for `and becomes'. Other proposed names include '__iadd__', `__add_in__' `__inplace_add__' For C extention types, the following struct members are added: To PyNumberMethods: binaryfunc nb_inplace_add; binaryfunc nb_inplace_subtract; binaryfunc nb_inplace_multiply; binaryfunc nb_inplace_divide; binaryfunc nb_inplace_remainder; binaryfunc nb_inplace_power; binaryfunc nb_inplace_lshift; binaryfunc nb_inplace_rshift; binaryfunc nb_inplace_and; binaryfunc nb_inplace_xor; binaryfunc nb_inplace_or; To PySequenceMethods: binaryfunc sq_inplace_concat; intargfunc sq_inplace_repeat; In order to keep binary compatibility, the tp_flags TypeObject member is used to determine whether the TypeObject in question has allocated room for these slots. Until a clean break in binary compatibility is made (which may or may not happen before 2.0) code that wants to use one of the new struct members must first check that they are available with the 'PyType_HasFeature()' macro: if (PyType_HasFeature(x->ob_type, Py_TPFLAGS_HAVE_INPLACE_OPS) && x->ob_type->tp_as_number && x->ob_type->tp_as_number->nb_inplace_add) { /* ... */ This check must be made even before testing the method slots for NULL values! The macro only tests whether the slots are available, not whether they are filled with methods or not. Implementation The current implementation of augmented assignment[2] adds, in addition to the methods and slots alread covered, 13 new bytecodes and 13 new API functions. The API functions are simply in-place versions of the current binary-operation API functions: PyNumber_InPlaceAdd(PyObject *o1, PyObject *o2); PyNumber_InPlaceSubtract(PyObject *o1, PyObject *o2); PyNumber_InPlaceMultiply(PyObject *o1, PyObject *o2); PyNumber_InPlaceDivide(PyObject *o1, PyObject *o2); PyNumber_InPlaceRemainder(PyObject *o1, PyObject *o2); PyNumber_InPlacePower(PyObject *o1, PyObject *o2); PyNumber_InPlaceLshift(PyObject *o1, PyObject *o2); PyNumber_InPlaceRshift(PyObject *o1, PyObject *o2); PyNumber_InPlaceAnd(PyObject *o1, PyObject *o2); PyNumber_InPlaceXor(PyObject *o1, PyObject *o2); PyNumber_InPlaceOr(PyObject *o1, PyObject *o2); PySequence_InPlaceConcat(PyObject *o1, PyObject *o2); PySequence_InPlaceRepeat(PyObject *o, int count); They call either the Python class hooks (if either of the objects is a Python class instance) or the C type's number or sequence methods. The new bytecodes are: INPLACE_ADD INPLACE_SUBTRACT INPLACE_MULTIPLY INPLACE_DIVIDE INPLACE_REMAINDER INPLACE_POWER INPLACE_LEFTSHIFT INPLACE_RIGHTSHIFT INPLACE_AND INPLACE_XOR INPLACE_OR ROT_FOUR DUP_TOPX The INPLACE_* bytecodes mirror the BINARY_* bytecodes, except that they are implemented as calls to the 'InPlace' API functions. The other two bytecodes are 'utility' bytecodes: ROT_FOUR behaves like ROT_THREE except that the four topmost stack items are rotated. DUP_TOPX is a bytecode that takes a single argument, which should be an integer between 1 and 5 (inclusive) which is the number of items to duplicate in one block. Given a stack like this (where the left side of the list is the 'top' of the stack): [a, b, c, d, e, f, g] "DUP_TOPX 3" would duplicate the top 3 items, resulting in this stack: [a, b, c, d, e, f, g, e, f, g] DUP_TOPX with an argument of 1 is the same as DUP_TOP. The limit of 5 is purely an implementation limit. The implementation of augmented assignment requires only DUP_TOPX with an argument of 2 and 3, and could do without this new opcode at the cost of a fair number of DUP_TOP and ROT_*. Copyright This document has been placed in the public domain. References [1] http://www.python.org/pipermail/python-list/2000-June/059556.html [2] http://sourceforge.net/patch?func=detailpatch&patch_id=100699&group_id=5470 Local Variables: mode: indented-text indent-tabs-mode: nil End: --0OAP2g/MAC+5xKAE-- From guido@beopen.com Mon Aug 7 15:11:52 2000 From: guido@beopen.com (Guido van Rossum) Date: Mon, 07 Aug 2000 09:11:52 -0500 Subject: dict.setdefault() (Patch#101102) (was: Re: [Python-Dev] Re: A small proposed change to dictionaries' "get" method...) In-Reply-To: Your message of "Mon, 07 Aug 2000 10:14:54 +0200." References: Message-ID: <200008071411.JAA18437@cj20424-a.reston1.va.home.com> > Guido: > > >> dict.default('hello', []).append('hello') > > Greg Ewing : > > GE> Is this new method going to apply to dictionaries only, > > GE> or is it to be considered part of the standard mapping > > GE> interface? > > Barry A. Warsaw: > > I think we've settled on setdefault(), which is more descriptive, even > > if it's a little longer. I have posted SF patch #101102 which adds > > setdefault() to both the dictionary object and UserDict (along with > > the requisite test suite and doco changes). PF: > This didn't answer the question raised by Greg Ewing. AFAI have seen, > the patch doesn't touch 'dbm', 'shelve' and so on. So from the patch > the answer is "applies to dictionaries only". I replied to Greg Ewing already: it's not part of the required mapping protocol. > What is with the other external mapping types already in the core, > like 'dbm', 'shelve' and so on? > > If the patch doesn't add this new method to these other mapping types, > this fact should at least be documented similar to the methods 'items()' > and 'values' that are already unimplemented in 'dbm': > """Dbm objects behave like mappings (dictionaries), except that > keys and values are always strings. Printing a dbm object > doesn't print the keys and values, and the items() and values() > methods are not supported.""" Good point. > I'm still -1 on the name: Nobody would expect, that a method > called 'setdefault()' will actually return something useful. May be > it would be better to invent an absolutely obfuscuated new name, so > that everybody is forced to actually *READ* the documentation of this > method or nobody will guess, what it is supposed to do or even > worse: how to make clever use of it. I don't get your point. Since when is it a requirement for a method to convey its full meaning by just its name? As long as the name doesn't intuitively contradict the actual meaning it should be fine. If you read code that does: dict.setdefault('key', []) dict['key'].append('bar') you will have no problem understanding this. There's no need for the reader to know that this is suboptimal. (Of course, if you're an experienced Python user doing a code review, you might know that. But it's not needed to understand what goes on.) Likewise, if you read code like this: dict.setdefault('key', []).append('bar') it doesn't seem hard to guess what it does (under the assumption that you already know the program works). After all, there are at most three things that setdefault() could *possibly* return: 1. None -- but then the append() would't work 2. dict -- but append() is not a dict method so wouldn't work either 3. dict['key'] -- this is the only one that makes sense > At least it would be a lot more likely, that someone becomes curious, > what a method called 'grezelbatz()' is suppoed to do, than that someone > will actually lookup the documentation of a method called 'setdefault()'. Bogus. This would argue that we should give all methods obscure names. > If the average Python programmer would ever start to use this method > at all, then I believe it is very likely that we will see him/her > coding: > dict.setdefault('key', []) > dict['key'].append('bar') And I have no problem with that. It's still less writing than the currently common idioms to deal with this! > So I'm also still -1 on the concept. I'm +0 on Gregs proposal, that > it would be better to make this a builtin function, that can be applied > to all mapping types. Yes, and let's also make values(), items(), has_key() and get() builtins instead of methods. Come on! Python is an OO language. > Maybe it would be even better to delay this until in Python 3000 > builtin types may have become real classes, so that this method may > be inherited by all mapping types from an abstract mapping base class. Sure, but that's not an argument for not adding it to the dictionary type today! --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From jack@oratrix.nl Mon Aug 7 14:26:40 2000 From: jack@oratrix.nl (Jack Jansen) Date: Mon, 07 Aug 2000 15:26:40 +0200 Subject: mallopt (Re: [Python-Dev] Minor compilation problem on HP-UX (1.6b1) (fwd)) In-Reply-To: Message by Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) , Mon, 7 Aug 2000 14:59:49 +0200 (CEST) , <200008071259.OAA22446@python.inrialpes.fr> Message-ID: <20000807132641.A60E6303181@snelboot.oratrix.nl> Don't worry, Vladimir, I hadn't forgotten your malloc stuff:-) Its just that if mallopt is available in the standard C library this may be a way to squeeze out a couple of extra percent of performance that the admin who installs Python needn't be aware of. And I don't think your allocator can be dropped in to the standard distribution, because it has the potential problem of fragmenting the heap due to multiple malloc packages in one address space (at least, that was the problem when I last looked at it, which is admittedly more than a year ago). And about mallopt not being portable: right, but I would assume that something like #ifdef M_MXFAST mallopt(M_MXFAST, xxxx); #endif shouldn't do any harm if we set xxxx to be a size that will cause 80% or so of the python objects to fall into the M_MXFAST category (sizeof(PyObject)+sizeof(void *), maybe?). This doesn't sound platform-dependent... Similarly, M_FREEHD sounds like it could speed up Python allocation, but this would need to be measured. Python allocation patterns shouldn't be influenced too much by platform, so again if this is good on one platform it is probably good on all. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From mark@per.dem.csiro.au Mon Aug 7 22:34:42 2000 From: mark@per.dem.csiro.au (Mark Favas) Date: Mon, 7 Aug 0 21:34:42 WST Subject: [Python-Dev] mallopt (Was: Minor compilation problem on HP-UX (1.6b1)) Message-ID: <200008071334.VAA15707@demperth.per.dem.csiro.au> To add to Vladimir Marangozov's comments about mallopt, in terms of both portability and utility (before too much time is expended)... From the Tru64 Unix man page: NOTES The mallopt() and mallinfo() functions are not supported for multithreaded applications. The mallopt() and mallinfo() functions are designed for tuning a specific algorithm. The Tru64 UNIX operating system uses a new, more efficient algorithm. The mallopt() and mallinfo() functions are provided for System V compatibility only and should not be used by new applications. The behavior of the current malloc() and free() functions is not be affected by calls to mallopt(). The structure returned by the mallinfo() function might not contain any useful information. -- Mark Favas From tismer@appliedbiometrics.com Mon Aug 7 14:47:39 2000 From: tismer@appliedbiometrics.com (Christian Tismer) Date: Mon, 07 Aug 2000 15:47:39 +0200 Subject: [Python-Dev] Stackless Python - Pros and Cons References: <398E93F3.374B585A@appliedbiometrics.com> <14734.46390.190481.441065@bitdiddle.concentric.net> Message-ID: <398EBDFB.4ED9FAE7@appliedbiometrics.com> [about recursion and continuations] > CT> Right. What you see here is the incompleteness of Stackless. In > CT> order to get this "right", I would have to change many parts of > CT> the implementation, in order to allow for continuations in every > CT> (probably even unwanted) place. I could not do this. > > CT> Instead, the situation of these still occouring recursions are > CT> handled differently. continuationmodule guarantees, that in the > CT> context of recursive interpreter calls, the given stack order of > CT> execution is obeyed. Violations of this cause simply an > CT> exception. > > Let me make sure I understand: If I invoke a continuation when there > are extra C stack frames between the mainloop invocation that captured > the continuation and the call of the continuation, the interpreter > raises an exception? Not always. Frames which are not currently bound by an interpreter acting on them can always be jump targets. Only those frames which are currently in the middle of an opcode are forbidden. > If so, continuations don't sound like they would mix well with C > extension modules and callbacks. I guess it also could not be used > inside methods that implement operator overloading. Is there a simple > set of rules that describe the situtations where they will not work? Right. In order to have good mixing with C callbacks, extra work is necessary. The C extension module must then play the same frame dribbling game as the eval_loop does. An example can be found in stackless map. If the C extension does not do so, it restricts execution order in the way I explained. This is not always needed, and it is no new requirement for C developers to do so. Only if they want to support free continuation switching, they have to implement it. The simple set of rules where continuations will not work at the moment is: Generally it does not work across interpreter recursions. At least restrictions appear: - you cannot run an import and jump off to the caller's frame + but you can save a continuation in your import and use it later, when this recursive interpreter is gone. - all special class functions are restricted. + but you can for instance save a continuation in __init__ and use it later, when the init recursion has gone. Reducing all these restrictions is a major task, and there are situations where it looks impossible without an extra subinterpreter language. If you look into the implementation of operators like __add__, you will see that there are repeated method calls which all may cause other interpreters to show up. I tried to find a way to roll these functions out in a restartable way, but it is quite a mess. The clean way to do it would be to have microcodes, and to allow for continuations to be caught between them. this-is-a-stackless-3000-feature - ly y'rs - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF where do you want to jump today? http://www.stackless.com From Vladimir.Marangozov@inrialpes.fr Mon Aug 7 15:00:08 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Mon, 7 Aug 2000 16:00:08 +0200 (CEST) Subject: mallopt (Re: [Python-Dev] Minor compilation problem on HP-UX In-Reply-To: <20000807132641.A60E6303181@snelboot.oratrix.nl> from "Jack Jansen" at Aug 07, 2000 03:26:40 PM Message-ID: <200008071400.QAA22652@python.inrialpes.fr> Jack Jansen wrote: > > Don't worry, Vladimir, I hadn't forgotten your malloc stuff:-) Me? worried about mallocs? :-) > if mallopt is available in the standard C library this may be a way > to squeeze out a couple of extra percent of performance that the admin > who installs Python needn't be aware of. As long as you're maintaining a Mac-specific port of Python, you can do this without pbs on the Mac port. > And I don't think your allocator can be dropped in > to the standard distribution, because it has the potential problem of > fragmenting the heap due to multiple malloc packages in one address > space (at least, that was the problem when I last looked at it, which > is admittedly more than a year ago). Things have changed since then. Mainly on the Python side. Have a look again. > > And about mallopt not being portable: right, but I would assume that > something like > #ifdef M_MXFAST > mallopt(M_MXFAST, xxxx); > #endif > shouldn't do any harm if we set xxxx to be a size that will cause 80% > or so of the python objects to fall into the M_MXFAST category Which is exactly what pymalloc does, except that this applies for > 95% of all allocations. > (sizeof(PyObject)+sizeof(void *), maybe?). This doesn't sound > platform-dependent... Indeed, I also use this trick to tune automatically the object allocator for 64-bit platforms. I haven't tested it on such machines as I don't have access to them, though. But it should work. > Similarly, M_FREEHD sounds like it could speed up Python allocation, > but this would need to be measured. Python allocation patterns shouldn't > be influenced too much by platform, so again if this is good on one > platform it is probably good on all. I am against any guesses in this domain. Measures and profiling evidence: that's it. Being able to make lazy decisions about Python's mallocs is our main advantage. Anything else is wild hype <0.3 wink>. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From gmcm@hypernet.com Mon Aug 7 15:20:50 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Mon, 7 Aug 2000 10:20:50 -0400 Subject: [Python-Dev] Stackless Python - Pros and Cons In-Reply-To: <14734.46390.190481.441065@bitdiddle.concentric.net> References: <398E93F3.374B585A@appliedbiometrics.com> Message-ID: <1246464444-103035791@hypernet.com> Jeremy wrote: > >>>>> "CT" == Christian Tismer > >>>>> writes: > > >> If someone is going to write a PEP, I hope they will explain > how >> the implementation deals with the various Python C API > calls that >> can call back into Python. > > CT> He will. > > Good! You'll write a PEP. Actually, "He" is me. While I speak terrible German, my Tismerish is pretty good (Tismerish to English is a *huge* jump ). But I can't figure out what the h*ll is being PEPed. We know that continuations / coroutines / generators have great value. We know that stackless is not continuations; it's some mods (mostly to ceval.c) that enables continuation.c. But the questions you're asking (after protesting that you want a formal spec, not a reference implementation) are all about Christian's implementation of continuation.c. (Well, OK, it's whether the stackless mods are enough to allow a perfect continuations implementation.) Assuming that stackless can get along with GC, ceval.c and grammar changes (or Christian can make it so), it seems to me the PEPable issue is whether the value this can add is worth the price of a less linear implementation. still-a-no-brainer-to-me-ly y'rs - Gordon From jack@oratrix.nl Mon Aug 7 15:23:14 2000 From: jack@oratrix.nl (Jack Jansen) Date: Mon, 07 Aug 2000 16:23:14 +0200 Subject: [Python-Dev] Stackless Python - Pros and Cons In-Reply-To: Message by Christian Tismer , Mon, 07 Aug 2000 15:47:39 +0200 , <398EBDFB.4ED9FAE7@appliedbiometrics.com> Message-ID: <20000807142314.C0186303181@snelboot.oratrix.nl> > > Let me make sure I understand: If I invoke a continuation when there > > are extra C stack frames between the mainloop invocation that captured > > the continuation and the call of the continuation, the interpreter > > raises an exception? > > Not always. Frames which are not currently bound by an > interpreter acting on them can always be jump targets. > Only those frames which are currently in the middle of > an opcode are forbidden. And how about the reverse? If I'm inside a Python callback from C code, will the Python code be able to use continuations? This is important, because there are a lot of GUI applications where almost all code is executed within a C callback. I'm pretty sure (and otherwise I'll be corrected within milliseconds:-) that this is the case for MacPython IDE and PythonWin (don't know about Idle). -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From jeremy@beopen.com Mon Aug 7 15:32:35 2000 From: jeremy@beopen.com (Jeremy Hylton) Date: Mon, 7 Aug 2000 10:32:35 -0400 (EDT) Subject: [Python-Dev] Stackless Python - Pros and Cons In-Reply-To: <1246464444-103035791@hypernet.com> References: <398E93F3.374B585A@appliedbiometrics.com> <1246464444-103035791@hypernet.com> Message-ID: <14734.51331.820955.54653@bitdiddle.concentric.net> Gordon, Thanks for channeling Christian, if that's what writing a PEP on this entails :-). I am also a little puzzled about the subject of the PEP. I think you should hash it out with Barry "PEPmeister" Warsaw. There are two different issues -- the stackless implementation and the new control structure exposed to programmers (e.g. continuations, coroutines, iterators, generators, etc.). It seems plausible to address these in two different PEPs, possibly in competing PEPs (e.g. coroutines vs. continuations). Jeremy From Vladimir.Marangozov@inrialpes.fr Mon Aug 7 15:38:32 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Mon, 7 Aug 2000 16:38:32 +0200 (CEST) Subject: [Python-Dev] Stackless Python - Pros and Cons In-Reply-To: <1246464444-103035791@hypernet.com> from "Gordon McMillan" at Aug 07, 2000 10:20:50 AM Message-ID: <200008071438.QAA22748@python.inrialpes.fr> Gordon McMillan wrote: > > But I can't figure out what the h*ll is being PEPed. > ... > Assuming that stackless can get along with GC, As long as frames are not considered for GC, don't worry about GC. > ceval.c and grammar changes (or Christian can make it so), it seems to > me the PEPable issue is whether the value this can add is > worth the price of a less linear implementation. There's an essay + paper available, slides and an implementation. What's the problem about formalizing this in a PEP and addressing the controversial issues + explaining how they are dealt with? I mean, if you're a convinced long-time Stackless user and everything is obvious for you, this PEP should try to convince the rest of us -- so write it down and ask no more . -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From tismer@appliedbiometrics.com Mon Aug 7 15:50:42 2000 From: tismer@appliedbiometrics.com (Christian Tismer) Date: Mon, 07 Aug 2000 16:50:42 +0200 Subject: [Python-Dev] Stackless Python - Pros and Cons References: <20000807142314.C0186303181@snelboot.oratrix.nl> Message-ID: <398ECCC2.957A9F67@appliedbiometrics.com> Jack Jansen wrote: > > > > Let me make sure I understand: If I invoke a continuation when there > > > are extra C stack frames between the mainloop invocation that captured > > > the continuation and the call of the continuation, the interpreter > > > raises an exception? > > > > Not always. Frames which are not currently bound by an > > interpreter acting on them can always be jump targets. > > Only those frames which are currently in the middle of > > an opcode are forbidden. > > And how about the reverse? If I'm inside a Python callback from C code, will > the Python code be able to use continuations? This is important, because there > are a lot of GUI applications where almost all code is executed within a C > callback. I'm pretty sure (and otherwise I'll be corrected within > milliseconds:-) that this is the case for MacPython IDE and PythonWin (don't > know about Idle). Without extra effort, this will be problematic. If C calls back into Python, not by the trampoline scheme that stackless uses, but by causing an interpreter recursion, then this interpreter will be limited. It can jump to any other frame that is not held by an interpreter on the C stack, but the calling frame of the C extension for instance is locked. Touching it causes an exception. This need not necessarily be a problem. Assume you have one or a couple of frames sitting around, caught as a continuation. Your Python callback from C jumps to that continuation and does something. Afterwards, it returns to the C callback. Performing some cycles of an idle task may be a use of such a thing. But as soon as you want to leave the complete calling chain, be able to modify it, return to a level above your callback and such, you need to implement your callback in a different way. The scheme is rather simple and can be seen in the stackless map implementation: You need to be able to store your complete state information in a frame, and you need to provide an execute function for your frame. Then you return the magic Py_UnwindToken, and your prepared frame will be scheduled like any pure Python function frame. Summary: By default, C extensions are restricted to stackful behaviror. By giving them a stackless interface, you can enable it completely for all continuation stuff. cheers - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF where do you want to jump today? http://www.stackless.com From gmcm@hypernet.com Mon Aug 7 16:28:01 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Mon, 7 Aug 2000 11:28:01 -0400 Subject: [Python-Dev] Stackless Python - Pros and Cons In-Reply-To: <200008071438.QAA22748@python.inrialpes.fr> References: <1246464444-103035791@hypernet.com> from "Gordon McMillan" at Aug 07, 2000 10:20:50 AM Message-ID: <1246460413-103278281@hypernet.com> Vladimir Marangozov wrote: > Gordon McMillan wrote: > > > > But I can't figure out what the h*ll is being PEPed. > > ... ... > > > ceval.c and grammar changes (or Christian can make it so), it > > seems to me the PEPable issue is whether the value this can add > > is worth the price of a less linear implementation. > > There's an essay + paper available, slides and an implementation. Of which the most up to date is the implementation. The slides / docs describe an earlier, more complex scheme. > What's the problem about formalizing this in a PEP and addressing > the controversial issues + explaining how they are dealt with? That's sort of what I was asking. As far as I can tell, what's controversial is "continuations". That's not in scope. I would like to know what controversial issues there are that *are* in scope. > I mean, if you're a convinced long-time Stackless user and > everything is obvious for you, this PEP should try to convince > the rest of us -- so write it down and ask no more . That's exactly wrong. If that were the case, I would be forced to vote -1 on any addition / enhancement to Python that I personally didn't plan on using. - Gordon From Vladimir.Marangozov@inrialpes.fr Mon Aug 7 16:53:15 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Mon, 7 Aug 2000 17:53:15 +0200 (CEST) Subject: [Python-Dev] Stackless Python - Pros and Cons In-Reply-To: <1246460413-103278281@hypernet.com> from "Gordon McMillan" at Aug 07, 2000 11:28:01 AM Message-ID: <200008071553.RAA22891@python.inrialpes.fr> Gordon McMillan wrote: > > > What's the problem about formalizing this in a PEP and addressing > > the controversial issues + explaining how they are dealt with? > > That's sort of what I was asking. As far as I can tell, what's > controversial is "continuations". That's not in scope. I would > like to know what controversial issues there are that *are* in > scope. Here's the context that might help you figure out what I'd like to see in this PEP. I haven't been at the last conference, I have read the source and the essay as of years ago and have no idea that the most up to date thing is the implementation, which I refuse to look at again, btw, without a clear summary of what this code does, refreshing my memory on the whole subject. I'd like to see an overview of the changes, their expected impact on the core, the extensions, and whatever else you judge worthy to write about. I'd like to see a summary of the reactions that have been emitted and what issues are non-issues for you, and which ones are. I'd like to see a first draft giving me a horizontal view on the subject in its entirety. Code examples are welcome, too. I can then start thinking about it in a more structured way on this basis. I don't have such a basis right now, because there's no an up to date document in plain English that allows me to do that. And without such a document, I won't do it. it's-simple- 'ly y'rs -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From sjoerd@oratrix.nl Mon Aug 7 17:19:59 2000 From: sjoerd@oratrix.nl (Sjoerd Mullender) Date: Mon, 07 Aug 2000 18:19:59 +0200 Subject: [Python-Dev] SRE incompatibility In-Reply-To: Your message of Wed, 05 Jul 2000 01:46:07 +0200. <002601bfe612$06e90ec0$f2a6b5d4@hagrid> References: <20000704095542.8697B31047C@bireme.oratrix.nl> <002601bfe612$06e90ec0$f2a6b5d4@hagrid> Message-ID: <20000807162000.5190631047C@bireme.oratrix.nl> Is this problem ever going to be solved or is it too hard? If it's too hard, I can fix xmllib to not depend on this. This incompatibility is the only reason I'm still not using sre. In case you don't remember, the regexp that is referred to is regexp = '(([a-z]+):)?([a-z]+)$' On Wed, Jul 5 2000 "Fredrik Lundh" wrote: > sjoerd wrote: > > > >>> re.match(regexp, 'smil').group(0,1,2,3) > > ('smil', None, 's', 'smil') > > >>> import pre > > >>> pre.match(regexp, 'smil').group(0,1,2,3) > > ('smil', None, None, 'smil') > > > > Needless to say, I am relying on the third value being None... > > I've confirmed this (last night's fix should have solved this, > but it didn't). I'll post patches as soon as I have them... > > > > -- Sjoerd Mullender From pf@artcom-gmbh.de Mon Aug 7 09:14:54 2000 From: pf@artcom-gmbh.de (Peter Funk) Date: Mon, 7 Aug 2000 10:14:54 +0200 (MEST) Subject: dict.setdefault() (Patch#101102) (was: Re: [Python-Dev] Re: A small proposed change to dictionaries' "get" method...) In-Reply-To: <14734.7730.698860.642851@anthem.concentric.net> from "Barry A. Warsaw" at "Aug 6, 2000 10:25:54 pm" Message-ID: Hi, Guido: > >> dict.default('hello', []).append('hello') Greg Ewing : > GE> Is this new method going to apply to dictionaries only, > GE> or is it to be considered part of the standard mapping > GE> interface? Barry A. Warsaw: > I think we've settled on setdefault(), which is more descriptive, even > if it's a little longer. I have posted SF patch #101102 which adds > setdefault() to both the dictionary object and UserDict (along with > the requisite test suite and doco changes). This didn't answer the question raised by Greg Ewing. AFAI have seen, the patch doesn't touch 'dbm', 'shelve' and so on. So from the patch the answer is "applies to dictionaries only". What is with the other external mapping types already in the core, like 'dbm', 'shelve' and so on? If the patch doesn't add this new method to these other mapping types, this fact should at least be documented similar to the methods 'items()' and 'values' that are already unimplemented in 'dbm': """Dbm objects behave like mappings (dictionaries), except that keys and values are always strings. Printing a dbm object doesn't print the keys and values, and the items() and values() methods are not supported.""" I'm still -1 on the name: Nobody would expect, that a method called 'setdefault()' will actually return something useful. May be it would be better to invent an absolutely obfuscuated new name, so that everybody is forced to actually *READ* the documentation of this method or nobody will guess, what it is supposed to do or even worse: how to make clever use of it. At least it would be a lot more likely, that someone becomes curious, what a method called 'grezelbatz()' is suppoed to do, than that someone will actually lookup the documentation of a method called 'setdefault()'. If the average Python programmer would ever start to use this method at all, then I believe it is very likely that we will see him/her coding: dict.setdefault('key', []) dict['key'].append('bar') So I'm also still -1 on the concept. I'm +0 on Gregs proposal, that it would be better to make this a builtin function, that can be applied to all mapping types. Maybe it would be even better to delay this until in Python 3000 builtin types may have become real classes, so that this method may be inherited by all mapping types from an abstract mapping base class. Regards, Peter -- Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260 office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen) From tim_one@email.msn.com Mon Aug 7 22:52:18 2000 From: tim_one@email.msn.com (Tim Peters) Date: Mon, 7 Aug 2000 17:52:18 -0400 Subject: Fun with call/cc (was RE: [Python-Dev] Stackless Python - Pros and Cons) In-Reply-To: <14734.46096.366920.827786@bitdiddle.concentric.net> Message-ID: [Tim] > On the one hand, I don't think I know of a language *not* based > on Scheme that has call/cc (or a moral equivalent). [Jeremy Hylton] > ML also has call/cc, at least the Concurrent ML variant. So it does! I've found 3 language lines that have full-blown call/cc (not counting the early versions of REBOL, since they took it out later), and at least one web page claiming "that's all, folks": 1. Scheme + derivatives (but not including most Lisps). 2. Standard ML + derivatives (but almost unique among truly functional languages): http://cm.bell-labs.com/cm/cs/what/smlnj/doc/SMLofNJ/pages/cont.html That page is pretty much incomprehensible on its own. Besides callcc (no "/"), SML-NJ also has related "throw", "isolate", "capture" and "escape" functions. At least some of them *appear* to be addressing Kent Pitman's specific complaints about the excruciating interactions between call/cc and unwind-protect in Scheme. 3. Unlambda. This one is a hoot! Don't know why I haven't bumped into it before: http://www.eleves.ens.fr:8080/home/madore/programs/unlambda/ "Your Functional Programming Language Nightmares Come True" Unlambda is a deliberately obfuscated functional programming language, whose only data type is function and whose only syntax is function application: no lambdas (or other "special forms"), no integers, no lists, no variables, no if/then/else, ... call/cc is spelled with the single letter "c" in Unlambda, and the docs note "expressions including c function calls tend to be hopelessly difficult to track down. This was, of course, the reason for including it in the language in the first place". Not all frivolous, though! The page goes on to point out that writing an interpreter for Unlambda in something other than Scheme exposes many of the difficult issues (like implementing call/cc in a language that doesn't have any such notion -- which is, after all, almost all languages), in a language that's otherwise relentlessly simple-minded so doesn't bog you down with accidental complexities. Doesn't mean call/cc sucks, but language designers *have* been avoiding it in vast numbers -- despite that the Scheme folks have been pushing it (& pushing it, & pushing it) in every real language they flee to . BTW, lest anyone get the wrong idea, I'm (mostly) in favor of it! It can't possibly be sold on any grounds other than that "it works, for real Python programmers with real programming problems they can't solve in other ways", though. Christian has been doing a wonderful (if slow-motion ) job of building that critical base of real-life users. From guido@beopen.com Tue Aug 8 00:03:46 2000 From: guido@beopen.com (Guido van Rossum) Date: Mon, 07 Aug 2000 18:03:46 -0500 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib sre.py,1.23,1.24 sre_compile.py,1.29,1.30 sre_parse.py,1.29,1.30 In-Reply-To: Your message of "Mon, 07 Aug 2000 13:59:08 MST." <200008072059.NAA11904@slayer.i.sourceforge.net> References: <200008072059.NAA11904@slayer.i.sourceforge.net> Message-ID: <200008072303.SAA31635@cj20424-a.reston1.va.home.com> > -- reset marks if repeat_one tail doesn't match > (this should fix Sjoerd's xmllib problem) Somebody please add a test case for this to test_re.py !!! --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From esr@thyrsus.com Mon Aug 7 23:13:02 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Mon, 7 Aug 2000 18:13:02 -0400 Subject: [Python-Dev] Adding library modules to the core Message-ID: <20000807181302.A27463@thyrsus.com> A few days ago I asked about the procedure for adding a module to the Python core library. I have a framework class for things like menu systems and symbolic debuggers I'd like to add. Guido asked if this was similar to the TreeWidget class in IDLE. I investigated and discovered that it is not, and told him so. I am left with a couple of related questions: 1. Has anybody got a vote on the menubrowser framwork facility I described? 1. Do we have a procedure for vetting modules for inclusion in the stock distribution? If not, should be institute one? 2. I am willing to do a pass through the Vaults of Parnassus and other sources for modules that seem both sufficiently useful and sufficiently mature to be added. I have in mind things like mimectl, PIL, and Vladimir's shared-memory module. Now, assuming I do 3, would I need to go through the vote process on each of these, or can I get a ukase from the BDFL authorizing me to fold in stuff? I realize I'm raising questions for which there are no easy answers. But Python is growing. The Python social machine needs to adapt to make such decisions in a more timely and less ad-hoc fashion. I'm not attached to being the point person in this process, but somebody's gotta be. -- Eric S. Raymond Question with boldness even the existence of a God; because, if there be one, he must more approve the homage of reason, than that of blindfolded fear.... Do not be frightened from this inquiry from any fear of its consequences. If it ends in the belief that there is no God, you will find incitements to virtue in the comfort and pleasantness you feel in its exercise... -- Thomas Jefferson, in a 1787 letter to his nephew From esr@thyrsus.com Mon Aug 7 23:24:03 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Mon, 7 Aug 2000 18:24:03 -0400 Subject: Fun with call/cc (was RE: [Python-Dev] Stackless Python - Pros and Cons) In-Reply-To: ; from tim_one@email.msn.com on Mon, Aug 07, 2000 at 05:52:18PM -0400 References: <14734.46096.366920.827786@bitdiddle.concentric.net> Message-ID: <20000807182403.A27485@thyrsus.com> Tim Peters : > Doesn't mean call/cc sucks, but language designers *have* been avoiding it > in vast numbers -- despite that the Scheme folks have been pushing it (& > pushing it, & pushing it) in every real language they flee to . Yes, we have. I haven't participated in conspiratorial hugggermugger with other ex-Schemers, but I suspect we'd all answer pretty much the same way. Lots of people have been avoiding call/cc not because it sucks but but because the whole area is very hard to think about even if you have the right set of primitives. > BTW, lest anyone get the wrong idea, I'm (mostly) in favor of it! It can't > possibly be sold on any grounds other than that "it works, for real Python > programmers with real programming problems they can't solve in other ways", > though. Christian has been doing a wonderful (if slow-motion ) job of > building that critical base of real-life users. And it's now Christian's job to do the next stop, supplying up-to-date documentation on his patch and proposal as a PEP. Suggestion: In order to satisfy the BDFL's conservative instincts, perhaps it would be better to break the Stackless patch into two pieces -- one that de-stack-izes ceval, and one that implements new language features. That way we can build a firm base for later exploration. -- Eric S. Raymond "Government is not reason, it is not eloquence, it is force; like fire, a troublesome servant and a fearful master. Never for a moment should it be left to irresponsible action." -- George Washington, in a speech of January 7, 1790 From thomas@xs4all.net Mon Aug 7 23:23:35 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Tue, 8 Aug 2000 00:23:35 +0200 Subject: [Python-Dev] Adding library modules to the core In-Reply-To: <20000807181302.A27463@thyrsus.com>; from esr@thyrsus.com on Mon, Aug 07, 2000 at 06:13:02PM -0400 References: <20000807181302.A27463@thyrsus.com> Message-ID: <20000808002335.A266@xs4all.nl> On Mon, Aug 07, 2000 at 06:13:02PM -0400, Eric S. Raymond wrote: [ You didn't ask for votes on all these, but the best thing I can do is vote :-] > 1. Has anybody got a vote on the menubrowser framwork facility I described? +0. I don't see any harm in adding it, but I can't envision a use for it, myself. > I have in mind things like mimectl, +1. A nice complement to the current mime and message handling routines. > PIL, +0. The main reason I don't compile PIL myself is because it's such a hassle to do it each time, so I think adding it would be nice. However, I'm not sure if it's doable to add, whether it would present a lot of problems for 'strange' platforms and the like. > and Vladimir's shared-memory module. +1. Fits very nicely with the mmapmodule, even if it's supported on less platforms. But perhaps all this falls in the 'batteries included' PEP ? Or perhaps a new PEP, 'enriching the Standard Library' ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From esr@thyrsus.com Mon Aug 7 23:39:30 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Mon, 7 Aug 2000 18:39:30 -0400 Subject: [Python-Dev] Adding library modules to the core In-Reply-To: <20000808002335.A266@xs4all.nl>; from thomas@xs4all.net on Tue, Aug 08, 2000 at 12:23:35AM +0200 References: <20000807181302.A27463@thyrsus.com> <20000808002335.A266@xs4all.nl> Message-ID: <20000807183930.A27556@thyrsus.com> Thomas Wouters : > But perhaps all this falls in the 'batteries included' PEP ? Or perhaps a > new PEP, 'enriching the Standard Library' ? I think that leads in a sub-optimal direction. Adding suitable modules shouldn't be a one-shot or episodic event but a continuous process of incorporating the best work the community has done. -- Eric S. Raymond "Taking my gun away because I might shoot someone is like cutting my tongue out because I might yell `Fire!' in a crowded theater." -- Peter Venetoklis From esr@thyrsus.com Mon Aug 7 23:42:24 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Mon, 7 Aug 2000 18:42:24 -0400 Subject: [Python-Dev] Adding library modules to the core In-Reply-To: <20000808002335.A266@xs4all.nl>; from thomas@xs4all.net on Tue, Aug 08, 2000 at 12:23:35AM +0200 References: <20000807181302.A27463@thyrsus.com> <20000808002335.A266@xs4all.nl> Message-ID: <20000807184224.B27556@thyrsus.com> Thomas Wouters : > On Mon, Aug 07, 2000 at 06:13:02PM -0400, Eric S. Raymond wrote: > > [ You didn't ask for votes on all these, but the best thing I can do is > vote :-] > > > 1. Has anybody got a vote on the menubrowser framwork facility I described? > > +0. I don't see any harm in adding it, but I can't envision a use for it, > myself. I'll cheerfully admit that I think it's kind of borderline myself. It works, but it teeters on the edge of being too specialized for the core library. I might only +0 it myself :-). -- Eric S. Raymond As with the Christian religion, the worst advertisement for Socialism is its adherents. -- George Orwell From thomas@xs4all.net Mon Aug 7 23:38:39 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Tue, 8 Aug 2000 00:38:39 +0200 Subject: [Python-Dev] Adding library modules to the core In-Reply-To: <20000807183930.A27556@thyrsus.com>; from esr@thyrsus.com on Mon, Aug 07, 2000 at 06:39:30PM -0400 References: <20000807181302.A27463@thyrsus.com> <20000808002335.A266@xs4all.nl> <20000807183930.A27556@thyrsus.com> Message-ID: <20000808003839.Q13365@xs4all.nl> On Mon, Aug 07, 2000 at 06:39:30PM -0400, Eric S. Raymond wrote: > Thomas Wouters : > > But perhaps all this falls in the 'batteries included' PEP ? Or perhaps a > > new PEP, 'enriching the Standard Library' ? > I think that leads in a sub-optimal direction. Adding suitable modules > shouldn't be a one-shot or episodic event but a continuous process of > incorporating the best work the community has done. That depends on what the PEP does. PEPs can do two things (according to the PEP that covers PEPs :): argue for a new feature/addition to the Python language, or describe a standard or procedure of some sort. This PEP could perhaps do both: describe a standard procedure for proposing and accepting a new module in the library (and probably also removal, though that's a lot trickier) AND do some catching-up on that process to get a few good modules into the stdlib before 2.0 goes into a feature freeze (which is next week, by the way.) As for the procedure to add a new module, I think someone volunteering to 'adopt' the module and perhaps a few people reviewing it would about do it, for the average module. Giving people a chance to say 'no!' of course. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From esr@thyrsus.com Mon Aug 7 23:59:54 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Mon, 7 Aug 2000 18:59:54 -0400 Subject: [Python-Dev] Adding library modules to the core In-Reply-To: <20000808003839.Q13365@xs4all.nl>; from thomas@xs4all.net on Tue, Aug 08, 2000 at 12:38:39AM +0200 References: <20000807181302.A27463@thyrsus.com> <20000808002335.A266@xs4all.nl> <20000807183930.A27556@thyrsus.com> <20000808003839.Q13365@xs4all.nl> Message-ID: <20000807185954.B27636@thyrsus.com> Thomas Wouters : > That depends on what the PEP does. PEPs can do two things (according to the > PEP that covers PEPs :): argue for a new feature/addition to the Python > language, or describe a standard or procedure of some sort. This PEP could > perhaps do both: describe a standard procedure for proposing and accepting a > new module in the library (and probably also removal, though that's a lot > trickier) AND do some catching-up on that process to get a few good modules > into the stdlib before 2.0 goes into a feature freeze (which is next week, > by the way.) > > As for the procedure to add a new module, I think someone volunteering to > 'adopt' the module and perhaps a few people reviewing it would about do it, > for the average module. Giving people a chance to say 'no!' of course. Sounds like my cue to write a PEP. What's the URL for the PEP on PEPs again? -- Eric S. Raymond See, when the GOVERNMENT spends money, it creates jobs; whereas when the money is left in the hands of TAXPAYERS, God only knows what they do with it. Bake it into pies, probably. Anything to avoid creating jobs. -- Dave Barry From bwarsaw@beopen.com Mon Aug 7 23:58:42 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Mon, 7 Aug 2000 18:58:42 -0400 (EDT) Subject: [Python-Dev] Adding library modules to the core References: <20000807181302.A27463@thyrsus.com> Message-ID: <14735.16162.275037.583897@anthem.concentric.net> >>>>> "ESR" == Eric S Raymond writes: ESR> 1. Do we have a procedure for vetting modules for inclusion ESR> in the stock distribution? If not, should be institute one? Is there any way to use the SourceForge machinery to help here? The first step would be to upload a patch so at least the new stuff doesn't get forgotten, and it's always easy to find the latest version of the changes. Also SF has a polling or voting tool, doesn't it? I know nothing about it, but perhaps there's some way to leverage it to test the pulse of the community for any new module (with BDFL veto of course). -Barry From esr@thyrsus.com Tue Aug 8 00:09:39 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Mon, 7 Aug 2000 19:09:39 -0400 Subject: [Python-Dev] Adding library modules to the core In-Reply-To: <14735.16162.275037.583897@anthem.concentric.net>; from bwarsaw@beopen.com on Mon, Aug 07, 2000 at 06:58:42PM -0400 References: <20000807181302.A27463@thyrsus.com> <14735.16162.275037.583897@anthem.concentric.net> Message-ID: <20000807190939.A27730@thyrsus.com> Barry A. Warsaw : > Is there any way to use the SourceForge machinery to help here? The > first step would be to upload a patch so at least the new stuff > doesn't get forgotten, and it's always easy to find the latest version > of the changes. Patch? Eh? In most cases, adding a library module will consist of adding one .py and one .tex, with no changes to existing code. -- Eric S. Raymond The price of liberty is, always has been, and always will be blood. The person who is not willing to die for his liberty has already lost it to the first scoundrel who is willing to risk dying to violate that person's liberty. Are you free? -- Andrew Ford From bwarsaw@beopen.com Tue Aug 8 00:04:39 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Mon, 7 Aug 2000 19:04:39 -0400 (EDT) Subject: [Python-Dev] Adding library modules to the core References: <20000807181302.A27463@thyrsus.com> <20000808002335.A266@xs4all.nl> <20000807183930.A27556@thyrsus.com> <20000808003839.Q13365@xs4all.nl> <20000807185954.B27636@thyrsus.com> Message-ID: <14735.16519.185236.794662@anthem.concentric.net> >>>>> "ESR" == Eric S Raymond writes: ESR> Sounds like my cue to write a PEP. What's the URL for the ESR> PEP on PEPs again? http://python.sourceforge.net/peps/pep-0001.html -Barry From bwarsaw@beopen.com Tue Aug 8 00:06:21 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Mon, 7 Aug 2000 19:06:21 -0400 (EDT) Subject: [Python-Dev] Adding library modules to the core References: <20000807181302.A27463@thyrsus.com> <14735.16162.275037.583897@anthem.concentric.net> <20000807190939.A27730@thyrsus.com> Message-ID: <14735.16621.369206.564320@anthem.concentric.net> >>>>> "ESR" == Eric S Raymond writes: ESR> Patch? Eh? In most cases, adding a library module will ESR> consist of adding one .py and one .tex, with no changes to ESR> existing code. And there's no good way to put those into SF? If the Patch Manager isn't appropriate, what about the Task Manager (I dunno, I've never looked at it). The cool thing about using SF is that there's less of a chance that this stuff will get buried in an inbox. -Barry From guido@beopen.com Tue Aug 8 01:21:43 2000 From: guido@beopen.com (Guido van Rossum) Date: Mon, 07 Aug 2000 19:21:43 -0500 Subject: [Python-Dev] bug-fixes in cnri-16-start branch In-Reply-To: Your message of "Sun, 06 Aug 2000 22:49:06 GMT." <398DEB62.789B4C9C@nowonder.de> References: <398DEB62.789B4C9C@nowonder.de> Message-ID: <200008080021.TAA31766@cj20424-a.reston1.va.home.com> > I have a question on the right procedure for fixing a simple > bug in the 1.6 release branch. > > Bug #111162 appeared because the tests for math.rint() are > already contained in the cnri-16-start revision of test_math.py > while the "try: ... except AttributeError: ..." construct which > was checked in shortly after was not. > > Now the correct bugfix is already known (and has been > applied to the main branch). I have updated the test_math.py > file in my working version with "-r cnri-16-start" and > made the changes. > > Now I probably should just commit, close the patch > (with an appropriate follow-up) and be happy. That would work, except that I prefer to remove math.rint altogether, as explained by Tim Peters. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From esr@snark.thyrsus.com Tue Aug 8 00:31:21 2000 From: esr@snark.thyrsus.com (Eric S. Raymond) Date: Mon, 7 Aug 2000 19:31:21 -0400 Subject: [Python-Dev] Request for PEP number Message-ID: <200008072331.TAA27825@snark.thyrsus.com> In accordance with the procedures in PEP 1, I am applying to initiate PEP 2. Proposed title: Procedure for incorporating new modules into the core. Abstract: This PEP will describes review and voting procedures for incorporating candidate modules and extensions into the Python core. Barry, could I get you to create a pep2@python.org mailing list for this one? -- Eric S. Raymond That the said Constitution shall never be construed to authorize Congress to infringe the just liberty of the press or the rights of conscience; or to prevent the people of the United states who are peaceable citizens from keeping their own arms... -- Samuel Adams, in "Phila. Independent Gazetteer", August 20, 1789 From guido@beopen.com Tue Aug 8 01:42:40 2000 From: guido@beopen.com (Guido van Rossum) Date: Mon, 07 Aug 2000 19:42:40 -0500 Subject: [Python-Dev] Adding library modules to the core In-Reply-To: Your message of "Mon, 07 Aug 2000 18:13:02 -0400." <20000807181302.A27463@thyrsus.com> References: <20000807181302.A27463@thyrsus.com> Message-ID: <200008080042.TAA31856@cj20424-a.reston1.va.home.com> [ESR] > 1. Has anybody got a vote on the menubrowser framwork facility I described? Eric, as far as I can tell you haven't shown the code or given a pointer to it. I explained to you that your description left me in the dark as to what it does. Or did I miss a pointer? It seems your module doesn't even have a name! This is a bad way to start a discussion about the admission procedure. Nothing has ever been accepted into Python before the code was written and shown. > 1. Do we have a procedure for vetting modules for inclusion in the stock > distribution? If not, should be institute one? Typically, modules get accepted after extensive lobbying and agreement from multiple developers. The definition of "developer" is vague, and I can't give a good rule -- not everybody who has been admitted to the python-dev list has enough standing to make his opinion count! Basically, I rely a lot on what various people say, but I have my own bias about who I trust in what area. I don't think I'll have to publish a list of this bias, but one thing is clear: I;m not counting votes! Proposals and ideas get approved based on merit, not on how many people argue for (or against) it. I want Python to keep its typical Guido-flavored style, and (apart from the occasional succesful channeling by TP) there's only one way to do that: let me be the final arbiter. I'm willing to be the bottleneck, it gives Python the typical slow-flowing evolution that has served it well over the past ten years. (At the same time, I can't read all messages in every thread on python-dev any more -- that's why substantial ideas need a PEP to summarize the discussion.) > 2. I am willing to do a pass through the Vaults of Parnassus and other > sources for modules that seem both sufficiently useful and sufficiently > mature to be added. I have in mind things like mimectl, PIL, and Vladimir's > shared-memory module. I don't know mimectl or Vladimir's module (how does it compare to mmap?). Regarding PIL, I believe the problem there is that it is a large body of code maintained by a third party. It should become part of a SUMO release and of binary releases, but I see no advantage in carrying it along in the core source distribution. > Now, assuming I do 3, would I need to go through the vote process > on each of these, or can I get a ukase from the BDFL authorizing me to > fold in stuff? Sorry, I don't write blank checks. > I realize I'm raising questions for which there are no easy answers. > But Python is growing. The Python social machine needs to adapt to > make such decisions in a more timely and less ad-hoc fashion. I'm > not attached to being the point person in this process, but > somebody's gotta be. Watch out though: if we open the floodgates now we may seriously deteriorate the quality of the standard library, without doing much good. I'd much rather see an improved Vaults of Parnassus (where every module uses distutils and installation becomes trivial) than a fast-track process for including new code in the core. That said, I think writing a bunch of thoughts up as a PEP makes a lot of sense! --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From esr@thyrsus.com Tue Aug 8 02:23:34 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Mon, 7 Aug 2000 21:23:34 -0400 Subject: [Python-Dev] Adding library modules to the core In-Reply-To: <200008080042.TAA31856@cj20424-a.reston1.va.home.com>; from guido@beopen.com on Mon, Aug 07, 2000 at 07:42:40PM -0500 References: <20000807181302.A27463@thyrsus.com> <200008080042.TAA31856@cj20424-a.reston1.va.home.com> Message-ID: <20000807212333.A27996@thyrsus.com> --EVF5PPMfhYS0aIcm Content-Type: text/plain; charset=us-ascii Guido van Rossum : > [ESR] > > 1. Has anybody got a vote on the menubrowser framwork facility I described? > > Eric, as far as I can tell you haven't shown the code or given a > pointer to it. I explained to you that your description left me in > the dark as to what it does. Or did I miss a pointer? It seems your > module doesn't even have a name! This is a bad way to start a > discussion about the admission procedure. Nothing has ever been > accepted into Python before the code was written and shown. Easily fixed. Code's in an enclosure. > > 1. Do we have a procedure for vetting modules for inclusion in the stock > > distribution? If not, should be institute one? > > Typically, modules get accepted after extensive lobbying and agreement > from multiple developers. The definition of "developer" is vague, and > I can't give a good rule -- not everybody who has been admitted to the > python-dev list has enough standing to make his opinion count! Understood, and I assume one of those insufficient-standing people is *me*, given my short tenure on the list, and I cheerfully accept that. The real problem I'm going after here is that this vague rule won't scale well. > Basically, I rely a lot on what various people say, but I have my own > bias about who I trust in what area. I don't think I'll have to > publish a list of this bias, but one thing is clear: I;m not counting > votes! I wasn't necessarily expecting you to. I can't imagine writing a procedure in which the BDFL doesn't retain a veto. > I don't know mimectl or Vladimir's module (how does it compare to > mmap?). Different, as Thomas Wouters has already observed. Vladimir's module is more oriented towards supporting semaphores and exclusion. At one point many months ago, before Vladimir was on the list, I looked into it as a way to do exclusion locking for shared shelves. Vladimir and I even negotiated a license change with INRIA so Python could use it. That was my first pass at sharable shelves; it foundered on problems with the BSDDB 1.85 API. But shm would still be worth having in the core librariy, IMO. The mimecntl module supports classes for representing MIME objects that include MIME-structure-sensitive mutation operations. Very strong candidate for inclusion, IMO. > > Now, assuming I do 3, would I need to go through the vote process > > on each of these, or can I get a ukase from the BDFL authorizing me to > > fold in stuff? > > Sorry, I don't write blank checks. And I wasn't expecting one. I'll write up some thoughts about this in the PEP. > > I realize I'm raising questions for which there are no easy answers. > > But Python is growing. The Python social machine needs to adapt to > > make such decisions in a more timely and less ad-hoc fashion. I'm > > not attached to being the point person in this process, but > > somebody's gotta be. > > Watch out though: if we open the floodgates now we may seriously > deteriorate the quality of the standard library, without doing much > good. The alternative is to end up with a Perl-like Case of the Missing Modules, where lots of things Python writers should be able to count on as standard builtins can't realistically be used, because the users they deliver to aren't going to want to go through a download step. > I'd much rather see an improved Vaults of Parnassus (where every > module uses distutils and installation becomes trivial) than a > fast-track process for including new code in the core. The trouble is that I flat don't believe in this solution. It works OK for developers, who will be willing to do extra download steps -- but it won't fly with end-user types. > That said, I think writing a bunch of thoughts up as a PEP makes a lot > of sense! I've applied to initiate PEP 2. -- Eric S. Raymond Hoplophobia (n.): The irrational fear of weapons, correctly described by Freud as "a sign of emotional and sexual immaturity". Hoplophobia, like homophobia, is a displacement symptom; hoplophobes fear their own "forbidden" feelings and urges to commit violence. This would be harmless, except that they project these feelings onto others. The sequelae of this neurosis include irrational and dangerous behaviors such as passing "gun-control" laws and trashing the Constitution. --EVF5PPMfhYS0aIcm Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="menubrowser.py" # menubrowser.py -- framework class for abstract browser objects from sys import stderr class MenuBrowser: "Support abstract browser operations on a stack of indexable objects." def __init__(self, debug=0, errout=stderr): self.page_stack = [] self.selection_stack = [] self.viewbase_stack = [] self.viewport_height = 0 self.debug = debug self.errout = errout def match(self, a, b): "Browseable-object comparison." return a == b def push(self, browseable, selected=None): "Push a browseable object onto the location stack." if self.debug: self.errout.write("menubrowser.push(): pushing %s=@%d, selection=%s\n" % (browseable, id(browseable), `selected`)) selnum = 0 if selected == None: if self.debug: self.errout.write("menubrowser.push(): selection defaulted\n") else: for i in range(len(browseable)): selnum = len(browseable) - i - 1 if self.match(browseable[selnum], selected): break if self.debug: self.errout.write("menubrowser.push(): selection set to %d\n" % (selnum)) self.page_stack.append(browseable) self.selection_stack.append(selnum) self.viewbase_stack.append(selnum - selnum % self.viewport_height) if self.debug: object = self.page_stack[-1] selection = self.selection_stack[-1] viewbase = self.viewbase_stack[-1] self.errout.write("menubrowser.push(): pushed %s=@%d->%d, selection=%d, viewbase=%d\n" % (object, id(object), len(self.page_stack), selection, viewbase)) def pop(self): "Pop a browseable object off the location stack." if not self.page_stack: if self.debug: self.errout.write("menubrowser.pop(): stack empty\n") return None else: item = self.page_stack[-1] self.page_stack = self.page_stack[:-1] self.selection_stack = self.selection_stack[:-1] self.viewbase_stack = self.viewbase_stack[:-1] if self.debug: if len(self.page_stack) == 0: self.errout.write("menubrowser.pop(): stack is empty.") else: self.errout.write("menubrowser.pop(): new level %d, object=@%d, selection=%d, viewbase=%d\n" % (len(self.page_stack), id(self.page_stack[-1]), self.selection_stack[-1], self.viewbase_stack[-1])) return item def stackdepth(self): "Return the current stack depth." return len(self.page_stack) def list(self): "Return all elements of the current object that ought to be visible." if not self.page_stack: return None object = self.page_stack[-1] selection = self.selection_stack[-1] viewbase = self.viewbase_stack[-1] if self.debug: self.errout.write("menubrowser.list(): stack level %d. object @%d, listing %s\n" % (len(self.page_stack)-1, id(object), object[viewbase:viewbase+self.viewport_height])) # This requires a slice method return object[viewbase:viewbase+self.viewport_height] def top(self): "Return the top-of-stack menu" if self.debug >= 2: self.errout.write("menubrowser.top(): level=%d, @%d\n" % (len(self.page_stack)-1,id(self.page_stack[-1]))) return self.page_stack[-1] def selected(self): "Return the currently selected element in the top menu." object = self.page_stack[-1] selection = self.selection_stack[-1] if self.debug: self.errout.write("menubrowser.selected(): at %d, object=@%d, %s\n" % (len(self.page_stack)-1, id(object), self.selection_stack[-1])) return object[selection] def viewbase(self): "Return the viewport base of the current menu." object = self.page_stack[-1] selection = self.selection_stack[-1] base = self.viewbase_stack[-1] if self.debug: self.errout.write("menubrowser.viewbase(): at level=%d, object=@%d, %d\n" % (len(self.page_stack)-1, id(object), base,)) return base def thumb(self): "Return top and bottom boundaries of a thumb scaled to the viewport." object = self.page_stack[-1] windowscale = float(self.viewport_height) / float(len(object)) thumb_top = self.viewbase() * windowscale thumb_bottom = thumb_top + windowscale * self.viewport_height - 1 return (thumb_top, thumb_bottom) def move(self, delta=1, wrap=0): "Move the selection on the current item downward." if delta == 0: return object = self.page_stack[-1] oldloc = self.selection_stack[-1] # Change the selection. Requires a length method if oldloc + delta in range(len(object)): newloc = oldloc + delta elif wrap: newloc = (oldloc + delta) % len(object) elif delta > 0: newloc = len(object) - 1 else: newloc = 0 self.selection_stack[-1] = newloc # When the selection is moved out of the viewport, move the viewbase # just part enough to track it. oldbase = self.viewbase_stack[-1] if newloc in range(oldbase, oldbase + self.viewport_height): pass elif newloc < oldbase: self.viewbase_stack[-1] = newloc else: self.scroll(newloc - (oldbase + self.viewport_height) + 1) if self.debug: self.errout.write("menubrowser.down(): at level=%d, object=@%d, old selection=%d, new selection = %d, new base = %d\n" % (len(self.page_stack)-1, id(object), oldloc, newloc, self.viewbase_stack[-1])) return (oldloc != newloc) def scroll(self, delta=1, wrap=0): "Scroll the viewport up or down in the current option." print "delta:", delta object = self.page_stack[-1] if not wrap: oldbase = self.viewbase_stack[-1] if delta > 0 and oldbase+delta > len(object)-self.viewport_height: return elif delta < 0 and oldbase + delta < 0: return self.viewbase_stack[-1] = (self.viewbase_stack[-1] + delta) % len(object) def dump(self): "Dump the whole stack of objects." self.errout.write("Viewport height: %d\n" % (self.viewport_height,)) for i in range(len(self.page_stack)): self.errout.write("Page: %d\n" % (i,)) self.errout.write("Selection: %d\n" % (self.selection_stack[i],)) self.errout.write(`self.page_stack[i]` + "\n"); def next(self, wrap=0): return self.move(1, wrap) def previous(self, wrap=0): return self.move(-1, wrap) def page_down(self): return self.move(2*self.viewport_height-1) def page_up(self): return self.move(-(2*self.viewport_height-1)) if __name__ == '__main__': import cmd, string, readline def itemgen(prefix, count): return map(lambda x, pre=prefix: pre + `x`, range(count)) testobject = menubrowser() testobject.viewport_height = 6 testobject.push(itemgen("A", 11)) class browser(cmd.Cmd): def __init__(self): self.wrap = 0 self.prompt = "browser> " def preloop(self): print "%d: %s (%d) in %s" % (testobject.stackdepth(), testobject.selected(), testobject.viewbase(), testobject.list()) def postloop(self): print "Goodbye." def postcmd(self, stop, line): self.preloop() return stop def do_quit(self, line): return 1 def do_exit(self, line): return 1 def do_EOF(self, line): return 1 def do_list(self, line): testobject.dump() def do_n(self, line): testobject.next() def do_p(self, line): testobject.previous() def do_pu(self, line): testobject.page_up() def do_pd(self, line): testobject.page_down() def do_up(self, line): if string.strip(line): n = string.atoi(line) else: n = 1 testobject.move(-n, self.wrap) def do_down(self, line): if string.strip(line): n = string.atoi(line) else: n = 1 testobject.move(n, self.wrap) def do_s(self, line): if string.strip(line): n = string.atoi(line) else: n = 1 testobject.scroll(n, self.wrap) def do_pop(self, line): testobject.pop() def do_gen(self, line): tokens = string.split(line) testobject.push(itemgen(tokens[0], string.atoi(tokens[1]))) def do_dump(self, line): testobject.dump() def do_wrap(self, line): self.wrap = 1 - self.wrap if self.wrap: print "Wrap is now on." else: print "Wrap is now off." def emptyline(self): pass browser().cmdloop() --EVF5PPMfhYS0aIcm-- From MarkH@ActiveState.com Tue Aug 8 02:36:24 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Tue, 8 Aug 2000 11:36:24 +1000 Subject: [Python-Dev] Adding library modules to the core In-Reply-To: <20000807181302.A27463@thyrsus.com> Message-ID: > Guido asked if this was similar to the TreeWidget class in IDLE. I > investigated and discovered that it is not, and told him so. I am left > with a couple of related questions: > > 1. Has anybody got a vote on the menubrowser framwork facility I > described? I would have to give it a -1. It probably should only be a -0, but I dropped it a level in the interests of keeping the library small and relevant. In a nutshell, it is proposed as a "framework class for abstract browser objects", but I don't see how. It looks like a reasonable framework for a particular kind of browser built for a text based system. I can not see how a GUI browser could take advantage of it. For example: * How does a "page" concept make sense in a high-res GUI? Why do we have a stack of pages? * What is a "viewport height" - is that a measure of pixels? If not, what font are you assuming? (sorry - obviously rhetorical, given my "text only" comments above.) * How does a "thumb position" relate to scroll bars that existing GUI widgets almost certainly have. etc. While I am sure you find it useful, I don't see how it helps anyone else, so I dont see how it qualifies as a standard module. If it is designed as part of a "curses" package, then I would be +0 - I would happily defer to your (or someone elses) judgement regarding its relevance in that domain. Obviously, there is a reasonable chance I am missing the point.... Mark. From bwarsaw@beopen.com Tue Aug 8 03:34:18 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Mon, 7 Aug 2000 22:34:18 -0400 (EDT) Subject: [Python-Dev] Request for PEP number References: <200008072331.TAA27825@snark.thyrsus.com> Message-ID: <14735.29098.168698.86981@anthem.concentric.net> >>>>> "ESR" == Eric S Raymond writes: ESR> In accordance with the procedures in PEP 1, I am applying to ESR> initiate PEP 2. ESR> Proposed title: Procedure for incorporating new modules into ESR> the core. ESR> Abstract: This PEP will describes review and voting ESR> procedures for incorporating candidate modules and extensions ESR> into the Python core. Done. ESR> Barry, could I get you to create a pep2@python.org mailing ESR> list for this one? We decided not to create separate mailing lists for each PEP. -Barry From greg@cosc.canterbury.ac.nz Tue Aug 8 04:08:48 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 08 Aug 2000 15:08:48 +1200 (NZST) Subject: dict.setdefault() (Patch#101102) (was: Re: [Python-Dev] Re: A small proposed change to dictionaries' "get" method...) In-Reply-To: Message-ID: <200008080308.PAA12740@s454.cosc.canterbury.ac.nz> artcom0!pf@artcom-gmbh.de: > dict.setdefault('key', []) > dict['key'].append('bar') I would agree with this more if it said dict.setdefault([]) dict['key'].append('bar') But I have a problem with all of these proposals: they require implicitly making a copy of the default value, which violates the principle that Python never copies anything unless you tell it to. The default "value" should really be a thunk, not a value, e.g. dict.setdefault(lambda: []) dict['key'].append('bar') or dict.get_or_add('key', lambda: []).append('bar') But I don't really like that, either, because lambdas look ugly to me, and I don't want to see any more builtin constructs that more-or-less require their use. I keep thinking that the solution to this lies somewhere in the direction of short-circuit evaluation techniques and/or augmented assignment, but I can't quite see how yet. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From esr@thyrsus.com Tue Aug 8 04:30:03 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Mon, 7 Aug 2000 23:30:03 -0400 Subject: [Python-Dev] Request for PEP number In-Reply-To: <14735.29098.168698.86981@anthem.concentric.net>; from bwarsaw@beopen.com on Mon, Aug 07, 2000 at 10:34:18PM -0400 References: <200008072331.TAA27825@snark.thyrsus.com> <14735.29098.168698.86981@anthem.concentric.net> Message-ID: <20000807233003.A28267@thyrsus.com> Barry A. Warsaw : > ESR> Barry, could I get you to create a pep2@python.org mailing > ESR> list for this one? > > We decided not to create separate mailing lists for each PEP. OK, where should discussion take place? -- Eric S. Raymond A ``decay in the social contract'' is detectable; there is a growing feeling, particularly among middle-income taxpayers, that they are not getting back, from society and government, their money's worth for taxes paid. The tendency is for taxpayers to try to take more control of their finances .. -- IRS Strategic Plan, (May 1984) From tim_one@email.msn.com Tue Aug 8 04:44:05 2000 From: tim_one@email.msn.com (Tim Peters) Date: Mon, 7 Aug 2000 23:44:05 -0400 Subject: dict.setdefault() (Patch#101102) (was: Re: [Python-Dev] Re: A small proposed change to dictionaries' "get" method...) In-Reply-To: <200008080308.PAA12740@s454.cosc.canterbury.ac.nz> Message-ID: > artcom0!pf@artcom-gmbh.de: > > dict.setdefault('key', []) > > dict['key'].append('bar') > [Greg Ewing] > I would agree with this more if it said > > dict.setdefault([]) > dict['key'].append('bar') Ha! I *told* Guido people would think that's the proper use of something named setdefault <0.9 wink>. > But I have a problem with all of these proposals: they require > implicitly making a copy of the default value, which violates > the principle that Python never copies anything unless you > tell it to. But they don't. The occurrence of an, e.g., [] literal in Python source *always* leads to a fresh list being created whenever the line of code containing it is executed. That behavior is guaranteed by the Reference Manual. In that respect dict.get('hi', []) or dict.getorset('hi', []).append(42) # getorset is my favorite is exactly the same as x = [] No copy of anything is made; the real irritation is that because arguments are always evaluated, we end up mucking around allocating an empty list regardless of whether it's needed; which you partly get away from via your: The default "value" should really be a thunk, not > a value, e.g. > > dict.setdefault(lambda: []) > dict['key'].append('bar') > > or > > dict.get_or_add('key', lambda: []).append('bar') except that lambda is also an executable expression and so now we end up creating an anonymous function dynamically regardless of whether it's needed. > But I don't really like that, either, because lambdas look > ugly to me, and I don't want to see any more builtin > constructs that more-or-less require their use. Ditto. > I keep thinking that the solution to this lies somewhere > in the direction of short-circuit evaluation techniques and/or > augmented assignment, but I can't quite see how yet. If new *syntax* were added, the compiler could generate short-circuiting code. Guido will never go for this , but to make it concrete, e.g., dict['key']||[].append('bar') count[word]||0 += 1 I found that dict.get(...) already confused my brain at times because my *eyes* want to stop at "[]" when scanning code for dict references. ".get()" just doesn't stick out as much; setdefault/default/getorset won't either. can't-win-ly y'rs - tim From esr@thyrsus.com Tue Aug 8 04:55:14 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Mon, 7 Aug 2000 23:55:14 -0400 Subject: [Python-Dev] Adding library modules to the core In-Reply-To: ; from MarkH@ActiveState.com on Tue, Aug 08, 2000 at 11:36:24AM +1000 References: <20000807181302.A27463@thyrsus.com> Message-ID: <20000807235514.C28267@thyrsus.com> Mark Hammond : > For example: > * How does a "page" concept make sense in a high-res GUI? Why do we have a > stack of pages? > * What is a "viewport height" - is that a measure of pixels? If not, what > font are you assuming? (sorry - obviously rhetorical, given my "text only" > comments above.) > * How does a "thumb position" relate to scroll bars that existing GUI > widgets almost certainly have. It's not designed for use with graphical browsers. Here are three contexts that could use it: * A menu tree being presented through a window or viewport (this is how it's being used now). * A symbolic debugger that can browse text around a current line. * A database browser for a sequential record-based file format. -- Eric S. Raymond Under democracy one party always devotes its chief energies to trying to prove that the other party is unfit to rule--and both commonly succeed, and are right... The United States has never developed an aristocracy really disinterested or an intelligentsia really intelligent. Its history is simply a record of vacillations between two gangs of frauds. --- H. L. Mencken From tim_one@email.msn.com Tue Aug 8 05:52:20 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 8 Aug 2000 00:52:20 -0400 Subject: [Python-Dev] Adding library modules to the core In-Reply-To: <200008080042.TAA31856@cj20424-a.reston1.va.home.com> Message-ID: [Guido] > ... > Nothing has ever been accepted into Python before the code > was written and shown. C'mon, admit it: you were sooooo appalled by the thread that lead to the creation of tabnanny.py that you decided at once it would end up in the distribution, just so you could justify skipping all the dozens of tedious long messages in which The Community developed The General Theory of Tab-Space Equivalence ab initio. It was just too much of a stupid-yet-difficult hack to resist . > ... > I want Python to keep its typical Guido-flavored style, So do most of us, most of the time. Paradoxically, it may be easier to stick to that as Python's popularity zooms beyond the point where it's even *conceivable* "votes" make any sense. > and (apart from the occasional succesful channeling by TP) there's > only one way to do that: let me be the final arbiter. Well, there's only one *obvious* way to do it. That's what keeps it Pythonic. > I'm willing to be the bottleneck, it gives Python the typical slow- > flowing evolution that has served it well over the past ten years. Except presumably for 2.0, where we decided at the last second to change large patches from "postponed" to "gotta have it". Consistency is the hobgoblin ... but-that's-pythonic-too-ly y'rs - tim From Moshe Zadka Tue Aug 8 06:42:30 2000 From: Moshe Zadka (Moshe Zadka) Date: Tue, 8 Aug 2000 08:42:30 +0300 (IDT) Subject: [Python-Dev] Adding library modules to the core In-Reply-To: <20000808002335.A266@xs4all.nl> Message-ID: On Tue, 8 Aug 2000, Thomas Wouters wrote: > > PIL, > > +0. The main reason I don't compile PIL myself is because it's such a hassle > to do it each time, so I think adding it would be nice. However, I'm not > sure if it's doable to add, whether it would present a lot of problems for > 'strange' platforms and the like. > > > and Vladimir's shared-memory module. > > +1. Fits very nicely with the mmapmodule, even if it's supported on less > platforms. > > But perhaps all this falls in the 'batteries included' PEP ? Or perhaps a > new PEP, 'enriching the Standard Library' ? PIL is definitely in the 206 PEP. The others are not yet there. Please note that a central database of "useful modules" which are in distutils' .tar.gz (or maybe .zip, now that Python has zipfile.py") + a simple tool to download and install. The main reason for the "batteries included" PEP is reliance on external libraries, which do not mesh as well with the distutils. expect-a-change-of-direction-in-the-pep-ly y'rs, Z. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From pf@artcom-gmbh.de Tue Aug 8 09:00:29 2000 From: pf@artcom-gmbh.de (Peter Funk) Date: Tue, 8 Aug 2000 10:00:29 +0200 (MEST) Subject: dict.setdefault() (Patch#101102) (was: Re: [Python-Dev] Re: A small proposed change to dictionaries' "get" method...) In-Reply-To: from Tim Peters at "Aug 7, 2000 11:44: 5 pm" Message-ID: Hi Tim! Tim Peters: [...] > Ha! I *told* Guido people would think that's the proper use of something > named setdefault <0.9 wink>. [...] > dict.getorset('hi', []).append(42) # getorset is my favorite 'getorset' is a *MUCH* better name compared to 'default' or 'setdefault'. Regards, Peter From R.Liebscher@gmx.de Tue Aug 8 10:26:47 2000 From: R.Liebscher@gmx.de (Rene Liebscher) Date: Tue, 08 Aug 2000 11:26:47 +0200 Subject: [Python-Dev] Library pragma in PC/config.h References: <20000803212444.A1237@beelzebub> <20000804205309.A1013@beelzebub> Message-ID: <398FD257.CFDC3B74@gmx.de> Greg Ward wrote: > > On 04 August 2000, Mark Hammond said: > > I would prefer python20_bcpp.lib, but that is not an issue. > > Good suggestion: the contents of the library are more important than the > format. Rene, can you make this change and include it in your next > patch? Or did you have some hidden, subtle reson for "bcpp_python20" as > opposed to "python20_bcpp"? OK, it is no problem to change it. > > > I am a little confused by the intention, tho. Wouldnt it make sense to > > have Borland builds of the core create a Python20.lib, then we could keep > > the pragma in too? > > > > If people want to use Borland for extensions, can't we ask them to use that > > same compiler to build the core too? That would seem to make lots of the > > problems go away? > > But that requires people to build all of Python from source, which I'm > guessing is a bit more bothersome than building an extension or two from > source. Especially since Python is already distributed as a very > easy-to-use binary installer for Windows, but most extensions are not. > > Rest assured that we probably won't be making things *completely* > painless for those who do not toe Chairman Bill's party line and insist > on using "non-standard" Windows compilers. They'll probably have to get > python20_bcpp.lib (or python20_gcc.lib, or python20_lcc.lib) on their > own -- whether downloaded or generated, I don't know. But the > alternative is to include 3 or 4 python20_xxx.lib files in the standard > Windows distribution, which I think is silly. (GCC uses libpython20.a) It is not necessary to include the libraries for all compilers. The only thing what is necessary is a def-file for the library. Every compiler I know has a program to create a import library from a def-file. BCC55 can even convert python20.lib in its own format. (The program is called "coff2omf". BCC55 uses the OMF format for its libraries which is different of MSVC's COFF format. (This answers your question, Tim?)) Maybe it should be there a file in the distribution, which explains what to do if someone wants to use another compiler, especially how to build a import library for this compiler or at least some general information what you need to do. ( or should it be included in the 'Ext' documentation. ) kind regards Rene Liebscher From Vladimir.Marangozov@inrialpes.fr Tue Aug 8 11:00:35 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Tue, 8 Aug 2000 12:00:35 +0200 (CEST) Subject: [Python-Dev] Adding library modules to the core In-Reply-To: <200008080042.TAA31856@cj20424-a.reston1.va.home.com> from "Guido van Rossum" at Aug 07, 2000 07:42:40 PM Message-ID: <200008081000.MAA29344@python.inrialpes.fr> Guido van Rossum wrote: > > I don't know mimectl or Vladimir's module (how does it compare to > mmap?). To complement ESR: - written 3 years ago - exports a file-like interface, defines 2 object types: shm & sem - resembles buffer but lacks the slice interface. - has all sysV shared memory bells & whistles + native semaphore support http://sirac.inrialpes.fr/~marangoz/python/shm Technically, mmap is often built on top of shared memory OS facilities. Adding slices + Windows code for shared mem & semaphores + a simplified unified interface might be a plan. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From mal@lemburg.com Tue Aug 8 11:46:25 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 08 Aug 2000 12:46:25 +0200 Subject: [Python-Dev] Adding library modules to the core References: <200008081000.MAA29344@python.inrialpes.fr> Message-ID: <398FE501.FFF09FAE@lemburg.com> Vladimir Marangozov wrote: > > Guido van Rossum wrote: > > > > I don't know mimectl or Vladimir's module (how does it compare to > > mmap?). > > To complement ESR: > > - written 3 years ago > - exports a file-like interface, defines 2 object types: shm & sem > - resembles buffer but lacks the slice interface. > - has all sysV shared memory bells & whistles + native semaphore support > > http://sirac.inrialpes.fr/~marangoz/python/shm > > Technically, mmap is often built on top of shared memory OS facilities. > Adding slices + Windows code for shared mem & semaphores + a simplified > unified interface might be a plan. I would be +1 if you could get it to work on Windows, +0 otherwise. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From R.Liebscher@gmx.de Tue Aug 8 12:41:12 2000 From: R.Liebscher@gmx.de (Rene Liebscher) Date: Tue, 08 Aug 2000 13:41:12 +0200 Subject: [Python-Dev] Library pragma in PC/config.h References: <20000803212444.A1237@beelzebub> <20000804205309.A1013@beelzebub> <398FD257.CFDC3B74@gmx.de> Message-ID: <398FF1D8.A91A8C02@gmx.de> Rene Liebscher wrote: > > Greg Ward wrote: > > > > On 04 August 2000, Mark Hammond said: > > > I would prefer python20_bcpp.lib, but that is not an issue. > > > > Good suggestion: the contents of the library are more important than the > > format. Rene, can you make this change and include it in your next > > patch? Or did you have some hidden, subtle reson for "bcpp_python20" as > > opposed to "python20_bcpp"? > OK, it is no problem to change it. I forgot to ask which name you would like for debug libraries "python20_bcpp_d.lib" or "python20_d_bcpp.lib" may be we should use "bcpp_python20_d.lib", and use the name schema which I suggested first. kind regards Rene Liebscher From skip@mojam.com (Skip Montanaro) Tue Aug 8 14:24:06 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Tue, 8 Aug 2000 08:24:06 -0500 (CDT) Subject: dict.setdefault() (Patch#101102) (was: Re: [Python-Dev] Re: A small proposed change to dictionaries' "get" method...) In-Reply-To: References: Message-ID: <14736.2550.586217.758500@beluga.mojam.com> >> dict.getorset('hi', []).append(42) # getorset is my favorite Peter> 'getorset' is a *MUCH* better name compared to 'default' or Peter> 'setdefault'. Shouldn't that be getorsetandget? After all, it doesn't just set or get it gets, but if it's undefined, it sets, then gets. I know I'll be shouted down, but I still vote against a method that both sets and gets dict values. I don't think the abbreviation in the source is worth the obfuscation of the code. Skip From akuchlin@mems-exchange.org Tue Aug 8 14:31:29 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Tue, 8 Aug 2000 09:31:29 -0400 Subject: [Python-Dev] Re: dict.setdefault() (Patch#101102) In-Reply-To: <14736.2550.586217.758500@beluga.mojam.com>; from skip@mojam.com on Tue, Aug 08, 2000 at 08:24:06AM -0500 References: <14736.2550.586217.758500@beluga.mojam.com> Message-ID: <20000808093129.A18519@kronos.cnri.reston.va.us> On Tue, Aug 08, 2000 at 08:24:06AM -0500, Skip Montanaro wrote: >I know I'll be shouted down, but I still vote against a method that both >sets and gets dict values. I don't think the abbreviation in the source is >worth the obfuscation of the code. -1 from me, too. A shortcut that only saves a line or two of code isn't worth the obscurity of the name. ("Ohhh, I get it. Back on that old minimalism kick, Andrew?" "Not back on it. Still on it.") --amk From Fredrik Lundh" Message-ID: <00c901c0014a$cc7c9be0$f2a6b5d4@hagrid> mark wrote: > So I think that the adoption of our half-solution (ie, we are really = only > forcing a better error message - not even getting a traceback to = indicate > _which_ module fails) note that the module name is available to the Py_InitModule4 module (for obvious reasons ;-), so it's not that difficult to improve the error message. how about: ... static char not_initialized_error[] =3D "ERROR: Module %.200s loaded an uninitialized interpreter!\n\ This Python has API version %d, module %.200s has version %d.\n"; ... if (!Py_IsInitialized()) { char message[500]; sprintf(message, not_initialized_error, name, = PYTHON_API_VERSION, name, module_api_version) Py_FatalError(message); } From pf@artcom-gmbh.de Tue Aug 8 15:48:32 2000 From: pf@artcom-gmbh.de (Peter Funk) Date: Tue, 8 Aug 2000 16:48:32 +0200 (MEST) Subject: [Python-Dev] Re: dict.setdefault() (Patch#101102) In-Reply-To: <14736.2550.586217.758500@beluga.mojam.com> from Skip Montanaro at "Aug 8, 2000 8:24: 6 am" Message-ID: Hi, Tim Peters: > >> dict.getorset('hi', []).append(42) # getorset is my favorite > > Peter> 'getorset' is a *MUCH* better name compared to 'default' or > Peter> 'setdefault'. Skip Montanaro: > Shouldn't that be getorsetandget? After all, it doesn't just set or get it > gets, but if it's undefined, it sets, then gets. That would defeat the main purpose of this method: abbreviation. This name is simply too long. > I know I'll be shouted down, but I still vote against a method that both > sets and gets dict values. I don't think the abbreviation in the source is > worth the obfuscation of the code. Yes. But I got the impression that Patch#101102 can't be avoided any more. So in this situation Tims '.getorset()' is the lesser of two evils compared to '.default()' or '.setdefault()'. BTW: I think the "informal" mapping interface should get a more explicit documentation. The language reference only mentions the 'len()' builtin method and indexing. But the section about mappings contains the sentence: "The extension modules dbm, gdbm, bsddb provide additional examples of mapping types." On the other hand section "2.1.6 Mapping Types" of the library reference says: "The following operations are defined on mappings ..." and than lists all methods including 'get()', 'update()', 'copy()' ... Unfortunately only a small subset of these methods actually works on a dbm mapping: >>> import dbm >>> d = dbm.open("piff", "c") >>> d.get('foo', []) Traceback (innermost last): File " ", line 1, in ? AttributeError: get >>> d.copy() Traceback (innermost last): File " ", line 1, in ? AttributeError: copy That should be documented. Regards, Peter From trentm@ActiveState.com Tue Aug 8 16:18:12 2000 From: trentm@ActiveState.com (Trent Mick) Date: Tue, 8 Aug 2000 08:18:12 -0700 Subject: [Python-Dev] Library pragma in PC/config.h In-Reply-To: <398FF1D8.A91A8C02@gmx.de>; from R.Liebscher@gmx.de on Tue, Aug 08, 2000 at 01:41:12PM +0200 References: <20000803212444.A1237@beelzebub> <20000804205309.A1013@beelzebub> <398FD257.CFDC3B74@gmx.de> <398FF1D8.A91A8C02@gmx.de> Message-ID: <20000808081811.A10965@ActiveState.com> On Tue, Aug 08, 2000 at 01:41:12PM +0200, Rene Liebscher wrote: > I forgot to ask which name you would like for debug libraries > > "python20_bcpp_d.lib" or "python20_d_bcpp.lib" > > may be we should use "bcpp_python20_d.lib", and use the name schema > which > I suggested first. Python20 is most important so it should go first. Then I suppose it is debatable whether 'd' or 'bcpp' should come first. My preference is "python20_bcpp_d.lib" because this would maintain the pattern that the basename of debug-built libs, etc. end in "_d". Generally speaking this would give a name spec of python (_ )*(_d)?.lib Trent -- Trent Mick TrentM@ActiveState.com From thomas@xs4all.net Tue Aug 8 16:22:17 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Tue, 8 Aug 2000 17:22:17 +0200 Subject: [Python-Dev] Library pragma in PC/config.h In-Reply-To: <20000808081811.A10965@ActiveState.com>; from trentm@ActiveState.com on Tue, Aug 08, 2000 at 08:18:12AM -0700 References: <20000803212444.A1237@beelzebub> <20000804205309.A1013@beelzebub> <398FD257.CFDC3B74@gmx.de> <398FF1D8.A91A8C02@gmx.de> <20000808081811.A10965@ActiveState.com> Message-ID: <20000808172217.G266@xs4all.nl> On Tue, Aug 08, 2000 at 08:18:12AM -0700, Trent Mick wrote: > On Tue, Aug 08, 2000 at 01:41:12PM +0200, Rene Liebscher wrote: > > I forgot to ask which name you would like for debug libraries > > "python20_bcpp_d.lib" or "python20_d_bcpp.lib" > > may be we should use "bcpp_python20_d.lib", and use the name schema > > which I suggested first. > Python20 is most important so it should go first. To clarify something Rene said earlier (I appear to have deleted that mail eventhough I had intended to reply to it :P) 'gcc' names its libraries 'libpython .{so,a}' because that's the UNIX convention: libraries are named 'lib . ', where libtype is '.a' for static libraries and '.so' for dynamic (ELF, in any case) ones, and you link with -l , without the 'lib' in front of it. The 'lib' is UNIX-imposed, not something gcc or Guido made up. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From trentm@ActiveState.com Tue Aug 8 16:26:03 2000 From: trentm@ActiveState.com (Trent Mick) Date: Tue, 8 Aug 2000 08:26:03 -0700 Subject: [Python-Dev] Library pragma in PC/config.h In-Reply-To: <20000808172217.G266@xs4all.nl>; from thomas@xs4all.net on Tue, Aug 08, 2000 at 05:22:17PM +0200 References: <20000803212444.A1237@beelzebub> <20000804205309.A1013@beelzebub> <398FD257.CFDC3B74@gmx.de> <398FF1D8.A91A8C02@gmx.de> <20000808081811.A10965@ActiveState.com> <20000808172217.G266@xs4all.nl> Message-ID: <20000808082603.B10965@ActiveState.com> On Tue, Aug 08, 2000 at 05:22:17PM +0200, Thomas Wouters wrote: > On Tue, Aug 08, 2000 at 08:18:12AM -0700, Trent Mick wrote: > > On Tue, Aug 08, 2000 at 01:41:12PM +0200, Rene Liebscher wrote: > > > I forgot to ask which name you would like for debug libraries > > > > "python20_bcpp_d.lib" or "python20_d_bcpp.lib" > > > > may be we should use "bcpp_python20_d.lib", and use the name schema > > > which I suggested first. > > > Python20 is most important so it should go first. > > To clarify something Rene said earlier (I appear to have deleted that mail > eventhough I had intended to reply to it :P) 'gcc' names its libraries > 'libpython .{so,a}' because that's the UNIX convention: libraries > are named 'lib . ', where libtype is '.a' for static libraries > and '.so' for dynamic (ELF, in any case) ones, and you link with -l , > without the 'lib' in front of it. The 'lib' is UNIX-imposed, not something > gcc or Guido made up. > Yes, you are right. I was being a Windows bigot there for an email. :) Trent -- Trent Mick TrentM@ActiveState.com From thomas@xs4all.net Tue Aug 8 16:35:24 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Tue, 8 Aug 2000 17:35:24 +0200 Subject: [Python-Dev] Library pragma in PC/config.h In-Reply-To: <20000808082603.B10965@ActiveState.com>; from trentm@ActiveState.com on Tue, Aug 08, 2000 at 08:26:03AM -0700 References: <20000803212444.A1237@beelzebub> <20000804205309.A1013@beelzebub> <398FD257.CFDC3B74@gmx.de> <398FF1D8.A91A8C02@gmx.de> <20000808081811.A10965@ActiveState.com> <20000808172217.G266@xs4all.nl> <20000808082603.B10965@ActiveState.com> Message-ID: <20000808173524.H266@xs4all.nl> On Tue, Aug 08, 2000 at 08:26:03AM -0700, Trent Mick wrote: [ Discussion about what to call the Borland version of python20.dll: bcpp_python20.dll or python20_bcpp.dll. Rene brought up that gcc calls "its" library libpython.so, and Thomas points out that that isn't Python's decision. ] > Yes, you are right. I was being a Windows bigot there for an email. :) And rightly so ! :) I think the 'python20_bcpp' name is more Windows-like, and if there is some area in which Python should try to stay as platform specific as possible, it's platform specifics such as libraries :) Would Windows users(*) when seeing 'bcpp_python20.dll' be thinking "this is a bcpp specific library of python20", or would they be thinking "this is a bcpp library for use with python20" ? I'm more inclined to think the second, myself :-) *) And the 'user' in this context is the extention-writer and python-embedder, of course. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From esr@thyrsus.com Tue Aug 8 16:46:55 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Tue, 8 Aug 2000 11:46:55 -0400 Subject: [Python-Dev] Adding library modules to the core In-Reply-To: <200008081000.MAA29344@python.inrialpes.fr>; from Vladimir.Marangozov@inrialpes.fr on Tue, Aug 08, 2000 at 12:00:35PM +0200 References: <200008080042.TAA31856@cj20424-a.reston1.va.home.com> <200008081000.MAA29344@python.inrialpes.fr> Message-ID: <20000808114655.C29686@thyrsus.com> Vladimir Marangozov : > Guido van Rossum wrote: > > > > I don't know mimectl or Vladimir's module (how does it compare to > > mmap?). > > To complement ESR: > > - written 3 years ago > - exports a file-like interface, defines 2 object types: shm & sem > - resembles buffer but lacks the slice interface. > - has all sysV shared memory bells & whistles + native semaphore support > > http://sirac.inrialpes.fr/~marangoz/python/shm > > Technically, mmap is often built on top of shared memory OS facilities. > Adding slices + Windows code for shared mem & semaphores + a simplified > unified interface might be a plan. Vladimir, I suggest that the most useful thing you could do to advance the process at this point would be to document shm in core-library style. At the moment, core Python has nothing (with the weak and nonportable exception of open(..., O_EXCL)) that can do semaphores properly. Thus shm would address a real gap in the language. -- Eric S. Raymond Are we at last brought to such a humiliating and debasing degradation, that we cannot be trusted with arms for our own defence? Where is the difference between having our arms in our own possession and under our own direction, and having them under the management of Congress? If our defence be the *real* object of having those arms, in whose hands can they be trusted with more propriety, or equal safety to us, as in our own hands? -- Patrick Henry, speech of June 9 1788 From tim_one@email.msn.com Tue Aug 8 16:46:00 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 8 Aug 2000 11:46:00 -0400 Subject: dict.setdefault() (Patch#101102) (was: Re: [Python-Dev] Re: A small proposed change to dictionaries' "get" method...) In-Reply-To: <14736.2550.586217.758500@beluga.mojam.com> Message-ID: [Skip Montanaro, on .getorset] > Shouldn't that be getorsetandget? After all, it doesn't just set > or get it gets, but if it's undefined, it sets, then gets. It's mnemonic enough for me. You can take comfort in that Guido seems to like "default" better, and is merely incensed by arguments about names . > I know I'll be shouted down, but I still vote against a method that both > sets and gets dict values. I don't think the abbreviation in the > source is worth the obfuscation of the code. So this is at least your second vote, while I haven't voted at all? I protest. +1 from me. I'd use it a lot. Yes, I'm one of those who probably has more dicts mapping to lists than to strings, and if dict.has_key(thing): dict[thing].append(newvalue) else: dict[thing] = [newvalue] litters my code -- talk about obfuscated! Of course I know shorter ways to spell that, but I find them even more obscure than the above. Easing a common operation is valuable, firmly in the tradition of the list.extend(), list.pop(), dict.get(), 3-arg getattr() and no-arg "raise" extensions. The *semantics* are clear and non-controversial and frequently desired, they're simply clumsy to spell now. The usual ploy in cases like this is to add the new gimmick and call it "experimental". Phooey. Add it or don't. for-that-matter-i'm-a-fan-of-"from-m-import-x-as-y"-too-ly y'rs - tim From guido@beopen.com Tue Aug 8 17:51:27 2000 From: guido@beopen.com (Guido van Rossum) Date: Tue, 08 Aug 2000 11:51:27 -0500 Subject: [Python-Dev] Adding library modules to the core In-Reply-To: Your message of "Tue, 08 Aug 2000 11:46:55 -0400." <20000808114655.C29686@thyrsus.com> References: <200008080042.TAA31856@cj20424-a.reston1.va.home.com> <200008081000.MAA29344@python.inrialpes.fr> <20000808114655.C29686@thyrsus.com> Message-ID: <200008081651.LAA01319@cj20424-a.reston1.va.home.com> > At the moment, core Python has nothing (with the weak and nonportable > exception of open(..., O_EXCL)) that can do semaphores properly. Thus > shm would address a real gap in the language. If it also works on Windows. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From Vladimir.Marangozov@inrialpes.fr Tue Aug 8 16:58:27 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Tue, 8 Aug 2000 17:58:27 +0200 (CEST) Subject: [Python-Dev] Adding library modules to the core In-Reply-To: <20000808114655.C29686@thyrsus.com> from "Eric S. Raymond" at Aug 08, 2000 11:46:55 AM Message-ID: <200008081558.RAA30190@python.inrialpes.fr> Eric S. Raymond wrote: > > At the moment, core Python has nothing (with the weak and nonportable > exception of open(..., O_EXCL)) that can do semaphores properly. Thus > shm would address a real gap in the language. There's a Semaphore class in Lib/threading.py. Are there any problems with it? I haven't used it, but threading.py has thread mutexes and semaphores on top of them, so as long as you don't need IPC, they should be fine. Or am I missing something? -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From esr@thyrsus.com Tue Aug 8 17:07:15 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Tue, 8 Aug 2000 12:07:15 -0400 Subject: [Python-Dev] Adding library modules to the core In-Reply-To: <200008081558.RAA30190@python.inrialpes.fr>; from Vladimir.Marangozov@inrialpes.fr on Tue, Aug 08, 2000 at 05:58:27PM +0200 References: <20000808114655.C29686@thyrsus.com> <200008081558.RAA30190@python.inrialpes.fr> Message-ID: <20000808120715.A29873@thyrsus.com> Vladimir Marangozov : > Eric S. Raymond wrote: > > > > At the moment, core Python has nothing (with the weak and nonportable > > exception of open(..., O_EXCL)) that can do semaphores properly. Thus > > shm would address a real gap in the language. > > There's a Semaphore class in Lib/threading.py. Are there any problems > with it? I haven't used it, but threading.py has thread mutexes and > semaphores on top of them, so as long as you don't need IPC, they should > be fine. Or am I missing something? If I'm not mistaken, that's semaphores across a thread bundle within a single process. It's semaphores visible across processes that I don't think we currently have a facility for. -- Eric S. Raymond The people cannot delegate to government the power to do anything which would be unlawful for them to do themselves. -- John Locke, "A Treatise Concerning Civil Government" From esr@thyrsus.com Tue Aug 8 17:07:58 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Tue, 8 Aug 2000 12:07:58 -0400 Subject: [Python-Dev] Adding library modules to the core In-Reply-To: <200008081651.LAA01319@cj20424-a.reston1.va.home.com>; from guido@beopen.com on Tue, Aug 08, 2000 at 11:51:27AM -0500 References: <200008080042.TAA31856@cj20424-a.reston1.va.home.com> <200008081000.MAA29344@python.inrialpes.fr> <20000808114655.C29686@thyrsus.com> <200008081651.LAA01319@cj20424-a.reston1.va.home.com> Message-ID: <20000808120758.B29873@thyrsus.com> Guido van Rossum : > > At the moment, core Python has nothing (with the weak and nonportable > > exception of open(..., O_EXCL)) that can do semaphores properly. Thus > > shm would address a real gap in the language. > > If it also works on Windows. As usual, I expect Unix to lead and Windows to follow. -- Eric S. Raymond Government is actually the worst failure of civilized man. There has never been a really good one, and even those that are most tolerable are arbitrary, cruel, grasping and unintelligent. -- H. L. Mencken From guido@beopen.com Tue Aug 8 18:18:49 2000 From: guido@beopen.com (Guido van Rossum) Date: Tue, 08 Aug 2000 12:18:49 -0500 Subject: [Python-Dev] Re: dict.setdefault() (Patch#101102) In-Reply-To: Your message of "Tue, 08 Aug 2000 11:46:00 -0400." References: Message-ID: <200008081718.MAA01681@cj20424-a.reston1.va.home.com> Enough said. I've checked it in and closed Barry's patch. Since 'default' is a Java reserved word, I decided that that would not be a good name for it after all, so I stuck with setdefault(). > for-that-matter-i'm-a-fan-of-"from-m-import-x-as-y"-too-ly y'rs - tim Hm. Predictably, I'm worried about adding 'as' as a reserved word. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From Fredrik Lundh" <14735.16162.275037.583897@anthem.concentric.net><20000807190939.A27730@thyrsus.com> <14735.16621.369206.564320@anthem.concentric.net> Message-ID: <001101c00155$cc86ad00$f2a6b5d4@hagrid> barry wrote: > And there's no good way to put those into SF? If the Patch Manager > isn't appropriate, what about the Task Manager (I dunno, I've never > looked at it). The cool thing about using SF is that there's less of > a chance that this stuff will get buried in an inbox. why not just switch it on, and see what happens. I'd prefer to get a concise TODO list on the login page, rather than having to look in various strange places (like PEP-160 and PEP-200 ;-) From gmcm@hypernet.com Tue Aug 8 18:51:51 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Tue, 8 Aug 2000 13:51:51 -0400 Subject: [Python-Dev] Adding library modules to the core In-Reply-To: <20000808120715.A29873@thyrsus.com> References: <200008081558.RAA30190@python.inrialpes.fr>; from Vladimir.Marangozov@inrialpes.fr on Tue, Aug 08, 2000 at 05:58:27PM +0200 Message-ID: <1246365382-108994225@hypernet.com> Eric Raymond wrote: > Vladimir Marangozov : > > There's a Semaphore class in Lib/threading.py. Are there any > > problems with it? I haven't used it, but threading.py has > > thread mutexes and semaphores on top of them, so as long as you > > don't need IPC, they should be fine. Or am I missing something? > > If I'm not mistaken, that's semaphores across a thread bundle > within a single process. It's semaphores visible across processes > that I don't think we currently have a facility for. There's the interprocess semaphore / mutex stuff in win32event... oh, never mind... - Gordon From ping@lfw.org Tue Aug 8 21:29:52 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Tue, 8 Aug 2000 13:29:52 -0700 (PDT) Subject: [Python-Dev] Re: dict.setdefault() (Patch#101102) In-Reply-To: Message-ID: On Tue, 8 Aug 2000, Peter Funk wrote: > > Unfortunately only a small subset of these methods actually works on > a dbm mapping: > > >>> import dbm > >>> d = dbm.open("piff", "c") > >>> d.get('foo', []) > Traceback (innermost last): > File " ", line 1, in ? > AttributeError: get I just got burned (again!) because neither the cgi.FieldStorage() nor the cgi.FormContentDict() support .get(). I've submitted a patch that adds FieldStorage.get() and makes FormContentDict a subclass of UserDict (the latter nicely eliminates almost all of the code in FormContentDict). (I know it says we're supposed to use FieldStorage, but i rarely if ever need to use file-upload forms, so SvFormContentDict() is still by far the most useful to me of the 17 different form implementations in the cgi module, i don't care what anyone says...) By the way, when/why did all of the documentation at the top of cgi.py get blown away? -- ?!ng "All models are wrong; some models are useful." -- George Box From Fredrik Lundh" Message-ID: <015901c00179$b718cba0$f2a6b5d4@hagrid> ping wrote: > By the way, when/why did all of the documentation at the top of > cgi.py get blown away? Date: Thu, 3 Aug 2000 13:57:47 -0700 From: Jeremy Hylton To: python-checkins@python.org Subject: [Python-checkins] CVS: python/dist/src/Lib cgi.py,1.48,1.49 Update of /cvsroot/python/python/dist/src/Lib In directory slayer.i.sourceforge.net:/tmp/cvs-serv2916 Modified Files: cgi.py=20 Log Message: Remove very long doc string (it's all in the docs) Modify parse_qsl to interpret 'a=3Db=3Dc' as key 'a' and value = 'b=3Dc' (which matches Perl's CGI.pm)=20 From tim_one@email.msn.com Wed Aug 9 05:57:02 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 9 Aug 2000 00:57:02 -0400 Subject: [Python-Dev] Task Manager on SourceForge Message-ID: Under the "what the heck" theory, I enabled the Task Manager on the Python project -- beware the 6-hour delay! Created two "subprojects" in it, P1.6 and P2, for tasks generally related to finishing the Python 1.6 and 2.0 releases, respectively. Don't know anything more about it. It appears you can set up a web of tasks under a "subproject", with fields for who's assigned, percent complete, status, hours of work, priority, start & end dates, and a list of tasks each task depends on. If anyone can think of a use for it, be my guest . I *suspect* everyone already has admin privileges for the Task Manager, but can't be sure. Today I couldn't fool either Netscape or IE5 into displaying the user-permissions Admin page correctly. Everyone down to "lemburg" does have admin privs for TaskMan, but from the middle of MAL's line on on down it's all empty space for me. From greg@cosc.canterbury.ac.nz Wed Aug 9 06:27:24 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Wed, 09 Aug 2000 17:27:24 +1200 (NZST) Subject: [Python-Dev] Lockstep iteration - eureka! Message-ID: <200008090527.RAA13669@s454.cosc.canterbury.ac.nz> I think I've actually found a syntax for lockstep iteration that looks reasonable (or at least not completely unreasonable) and is backward compatible: for (x in a, y in b): ... Not sure what the implications are for the parser yet. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From MarkH@ActiveState.com Wed Aug 9 07:39:30 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Wed, 9 Aug 2000 16:39:30 +1000 Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: <200008090527.RAA13669@s454.cosc.canterbury.ac.nz> Message-ID: > for (x in a, y in b): > ... Hmmm. Until someone smarter than me shoots it down for some obvious reason , it certainly appeals to me. My immediate reaction _is_ lockstep iteration, and that is the first time I can say that. Part of the reason is that it looks like a tuple unpack, which I think of as a "lockstep/parallel/atomic" operation... Mark. From jack@oratrix.nl Wed Aug 9 09:31:27 2000 From: jack@oratrix.nl (Jack Jansen) Date: Wed, 09 Aug 2000 10:31:27 +0200 Subject: [Python-Dev] A question for the Python Secret Police Message-ID: <20000809083127.7FFF6303181@snelboot.oratrix.nl> A question for the Python Secret Police (or P. Inquisition, or whoever else:-). Is the following morally allowed: package1/mod.py: class Foo: def method1(self): ... package2/mod.py: from package1.mod import * class Foo(Foo): def method2(self): ... (The background is that the modules are machine-generated and contain AppleEvent classes. There's a large set of standard classes, such as Standard_Suite, and applications can signal that they implement Standard_Suite with a couple of extensions to it. So, in the Application-X Standard_Suite I'd like to import everything from the standard Standard_Suite and override/add those methods that are specific to Application X) -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++ From tim_one@email.msn.com Wed Aug 9 08:15:02 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 9 Aug 2000 03:15:02 -0400 Subject: [Python-Dev] Re: dict.setdefault() (Patch#101102) In-Reply-To: <200008081718.MAA01681@cj20424-a.reston1.va.home.com> Message-ID: [Tim] >> for-that-matter-i'm-a-fan-of-"from-m-import-x-as-y"-too-ly y'rs - tim [Guido] > Hm. Predictably, I'm worried about adding 'as' as a reserved word. But it doesn't need to be, right? That is, change the stuff following 'import' in 'from' dotted_name 'import' ('*' | NAME (',' NAME)*) to ('*' | NAME [NAME NAME] (',' NAME [NAME NAME])*) and verify that whenever the 3-NAME form triggers that the middle of the NAMEs is exactly "as". The grammar in the Reference Manual can still advertise it as a syntactic constraint; if a particular implementation happens to need to treat it as a semantic constraint due to parser limitations (and CPython specifically would), the user will never know it. It doesn't interfere with using "as" a regular NAME elsewhere. Anyone pointing out that the line from as import as as as would then be legal will be shot. Fortran had no reserved words of any kind, and nobody abused that in practice. Users may be idiots, but they're not infants . From thomas@xs4all.net Wed Aug 9 09:42:32 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 9 Aug 2000 10:42:32 +0200 Subject: [Python-Dev] Task Manager on SourceForge In-Reply-To: ; from tim_one@email.msn.com on Wed, Aug 09, 2000 at 12:57:02AM -0400 References: Message-ID: <20000809104232.I266@xs4all.nl> On Wed, Aug 09, 2000 at 12:57:02AM -0400, Tim Peters wrote: > Don't know anything more about it. It appears you can set up a web of tasks > under a "subproject", with fields for who's assigned, percent complete, > status, hours of work, priority, start & end dates, and a list of tasks each > task depends on. Well, it seems mildly useful... It's missing some things that would make it fairly useful (per-subtask and per-project todo-list, where you an say 'I need help with this' and such things, the ability to attach patches to subtasks (which would be useful for 'my' task of adding augmented assignment ;) and probably more) but I can imagine why SF didn't include all that (yet) -- it's a lot of work to do right, and I'm not sure if SF has much projects of the size that needs a project manager like this ;) But unless Guido and the rest of the PyLab team want to keep an overview of what us overseas or at least other-state lazy bums are doing by trusting us to keep a webpage up to date rather than informing the mailing list, I don't think we'll see much use for it. If you *do* want such an overview, it might be useful. In which case I'll send out some RFE's on my wishes for the project manager ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From ping@lfw.org Wed Aug 9 10:37:07 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Wed, 9 Aug 2000 02:37:07 -0700 (PDT) Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: <200008090527.RAA13669@s454.cosc.canterbury.ac.nz> Message-ID: On Wed, 9 Aug 2000, Greg Ewing wrote: > > for (x in a, y in b): > ... It looks nice, but i'm pretty sure it won't fly. (x in a, y in b) is a perfectly valid expression. For compatibility the parser must also accept for (x, y) in list_of_pairs: and since the thing after the open-paren can be arbitrarily long, how is the parser to know whether the lockstep form has been invoked? Besides, i think Guido has Pronounced quite firmly on zip(). I would much rather petition now to get indices() and irange() into the built-ins... please pretty please? -- ?!ng "All models are wrong; some models are useful." -- George Box From thomas@xs4all.net Wed Aug 9 12:06:45 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 9 Aug 2000 13:06:45 +0200 Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: ; from MarkH@ActiveState.com on Wed, Aug 09, 2000 at 04:39:30PM +1000 References: <200008090527.RAA13669@s454.cosc.canterbury.ac.nz> Message-ID: <20000809130645.J266@xs4all.nl> On Wed, Aug 09, 2000 at 04:39:30PM +1000, Mark Hammond wrote: > > for (x in a, y in b): > > ... > Hmmm. Until someone smarter than me shoots it down for some obvious reason > , it certainly appeals to me. The only objection I can bring up is that parentheses are almost always optional, in Python, and this kind of violates it. Suddenly the presence of parentheses changes the entire expression, not just the grouping of it. Oh, and there is the question of whether 'for (x in a):' is allowed, too (it isn't, currently.) I'm not entirely sure that the parser will swallow this, however, because 'for (x in a, y in b) in z:' *is* valid syntax... so it might be ambiguous. Then again, it can probably be worked around. It might not be too pretty, but it can be worked around ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Wed Aug 9 12:29:13 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 9 Aug 2000 13:29:13 +0200 Subject: [Python-Dev] Re: dict.setdefault() (Patch#101102) In-Reply-To: ; from tim_one@email.msn.com on Wed, Aug 09, 2000 at 03:15:02AM -0400 References: <200008081718.MAA01681@cj20424-a.reston1.va.home.com> Message-ID: <20000809132913.K266@xs4all.nl> [Tim] > for-that-matter-i'm-a-fan-of-"from-m-import-x-as-y"-too-ly y'rs - tim [Guido] > Hm. Predictably, I'm worried about adding 'as' as a reserved word. [Tim] > But it doesn't need to be, right? That is, change the stuff following > 'import' in > 'from' dotted_name 'import' ('*' | NAME (',' NAME)*) > to > ('*' | NAME [NAME NAME] (',' NAME [NAME NAME])*) I'm very, very much +1 on this. the fact that (for example) 'from' is a reserved word bothers me no end. If noone is going to comment anymore on range literals or augmented assignment, I might just tackle this ;) > Anyone pointing out that the line > from as import as as as > would then be legal will be shot. "Cool, that would make 'from from import as as as' a legal sta" Damned American gun laws ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From guido@beopen.com Wed Aug 9 13:30:43 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 09 Aug 2000 07:30:43 -0500 Subject: [Python-Dev] Task Manager on SourceForge In-Reply-To: Your message of "Wed, 09 Aug 2000 00:57:02 -0400." References: Message-ID: <200008091230.HAA23379@cj20424-a.reston1.va.home.com> > Under the "what the heck" theory, I enabled the Task Manager on the Python > project -- beware the 6-hour delay! Created two "subprojects" in it, P1.6 > and P2, for tasks generally related to finishing the Python 1.6 and 2.0 > releases, respectively. Beauuuutiful! > Don't know anything more about it. It appears you can set up a web of tasks > under a "subproject", with fields for who's assigned, percent complete, > status, hours of work, priority, start & end dates, and a list of tasks each > task depends on. > > If anyone can think of a use for it, be my guest . I played with it a bit. I added three tasks under 1.6 that need to be done. > I *suspect* everyone already has admin privileges for the Task Manager, but > can't be sure. Today I couldn't fool either Netscape or IE5 into displaying > the user-permissions Admin page correctly. Everyone down to "lemburg" does > have admin privs for TaskMan, but from the middle of MAL's line on on down > it's all empty space for me. That must be a Windows limitation on how many popup menus you can have. Stupid Windows :-) ! This looks fine on Linux in Netscape (is there any other browser :-) ? and indeed the permissions are set correctly. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From guido@beopen.com Wed Aug 9 13:42:49 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 09 Aug 2000 07:42:49 -0500 Subject: [Python-Dev] A question for the Python Secret Police In-Reply-To: Your message of "Wed, 09 Aug 2000 10:31:27 +0200." <20000809083127.7FFF6303181@snelboot.oratrix.nl> References: <20000809083127.7FFF6303181@snelboot.oratrix.nl> Message-ID: <200008091242.HAA23451@cj20424-a.reston1.va.home.com> > A question for the Python Secret Police (or P. Inquisition, or whoever > else:-). That would be the Namespace Police in this case. > Is the following morally allowed: > > package1/mod.py: > class Foo: > def method1(self): > ... > > package2/mod.py: > from package1.mod import * > > class Foo(Foo): > def method2(self): > ... I see no problem with this. It's totally well-defined and I don't expect I'll ever have a reason to disallow it. Future picky compilers or IDEs might warn about a redefined name, but I suppose you can live with that given that it's machine-generated. > (The background is that the modules are machine-generated and contain > AppleEvent classes. There's a large set of standard classes, such as > Standard_Suite, and applications can signal that they implement > Standard_Suite with a couple of extensions to it. So, in the > Application-X Standard_Suite I'd like to import everything from the > standard Standard_Suite and override/add those methods that are > specific to Application X) That actually looks like a *good* reason to do exactly what you propose. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From guido@beopen.com Wed Aug 9 13:49:43 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 09 Aug 2000 07:49:43 -0500 Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: Your message of "Wed, 09 Aug 2000 02:37:07 MST." References: Message-ID: <200008091249.HAA23481@cj20424-a.reston1.va.home.com> > On Wed, 9 Aug 2000, Greg Ewing wrote: > > > > for (x in a, y in b): > > ... No, for exactly the reasons Ping explained. Let's give this a rest okay? > I would much rather petition now to get indices() and irange() into > the built-ins... please pretty please? I forget what indices() was -- is it the moreal equivalent of keys()? That's range(len(s)), I don't see a need for a new function. In fact I think indices() would reduce readability because you have to guess what it means. Everybody knows range() and len(); not everybody will know indices() because it's not needed that often. If irange(s) is zip(range(len(s)), s), I see how that's a bit unwieldy. In the past there were syntax proposals, e.g. ``for i indexing s''. Maybe you and Just can draft a PEP? --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From mal@lemburg.com Wed Aug 9 13:58:00 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 09 Aug 2000 14:58:00 +0200 Subject: [Python-Dev] Lockstep iteration - eureka! References: <200008091249.HAA23481@cj20424-a.reston1.va.home.com> Message-ID: <39915558.A68D7792@lemburg.com> Guido van Rossum wrote: > > > On Wed, 9 Aug 2000, Greg Ewing wrote: > > > > > > for (x in a, y in b): > > > ... > > No, for exactly the reasons Ping explained. Let's give this a rest okay? > > > I would much rather petition now to get indices() and irange() into > > the built-ins... please pretty please? > > I forget what indices() was -- is it the moreal equivalent of keys()? indices() and irange() are both builtins which originated from mx.Tools. See: http://starship.python.net/crew/lemburg/mxTools.html * indices(object) is the same as tuple(range(len(object))) - only faster and using a more intuitive and less convoluted name. * irange(object[,indices]) (in its mx.Tools version) creates a tuple of tuples (index, object[index]). indices defaults to indices(object) if not given, otherwise, only the indexes found in indices are used to create the mentioned tuple -- and this even works with arbitrary keys, since the PyObject_GetItem() API is used. Typical use is: for i,value in irange(sequence): sequence[i] = value + 1 In practice I found that I could always use irange() where indices() would have been used, since I typically need the indexed sequence object anyway. > That's range(len(s)), I don't see a need for a new function. In fact > I think indices() would reduce readability because you have to guess > what it means. Everybody knows range() and len(); not everybody will > know indices() because it's not needed that often. > > If irange(s) is zip(range(len(s)), s), I see how that's a bit > unwieldy. In the past there were syntax proposals, e.g. ``for i > indexing s''. Maybe you and Just can draft a PEP? -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From nowonder@nowonder.de Wed Aug 9 16:19:02 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Wed, 09 Aug 2000 15:19:02 +0000 Subject: [Python-Dev] Re: dict.setdefault() (Patch#101102) References: Message-ID: <39917666.87C823E9@nowonder.de> Tim Peters wrote: > > But it doesn't need to be, right? That is, change the stuff following > 'import' in > > 'from' dotted_name 'import' ('*' | NAME (',' NAME)*) > > to > > ('*' | NAME [NAME NAME] (',' NAME [NAME NAME])*) What about doing the same for the regular import? import_stmt: 'import' dotted_name [NAME NAME] (',' dotted_name [NAME NAME])* | 'from' dotted_name 'import' ('*' | NAME (',' NAME)*) "import as as as"-isn't-that-impressive-though-ly y'rs Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From just@letterror.com Wed Aug 9 16:01:18 2000 From: just@letterror.com (Just van Rossum) Date: Wed, 9 Aug 2000 16:01:18 +0100 Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: <200008091249.HAA23481@cj20424-a.reston1.va.home.com> References: Your message of "Wed, 09 Aug 2000 02:37:07 MST." Message-ID: At 7:49 AM -0500 09-08-2000, Guido van Rossum wrote: >In the past there were syntax proposals, e.g. ``for i >indexing s''. Maybe you and Just can draft a PEP? PEP: 1716099-3 Title: Index-enhanced sequence iteration Version: $Revision: 1.1 $ Owner: Someone-with-commit-rights Python-Version: 2.0 Status: Incomplete Introduction This PEP proposes a way to more conveniently iterate over a sequence and its indices. Features It adds an optional clause to the 'for' statement: for indexing in : ... This is equivalent to (see the zip() PEP): for , in zip(range(len(seq)), seq): ... Except no new list is created. Mechanism The index of the current element in a for-loop already exists in the implementation, however, it is not reachable from Python. The new 'indexing' keyword merely exposes the internal counter. Implementation Implementation should be trivial for anyone named Guido, Tim or Thomas. Justs better not try. Advantages: Less code needed for this common operation, which is currently most often written as: for index in range(len(seq)): element = seq[i] ... Disadvantages: It will break that one person's code that uses "indexing" as a variable name. Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil End: From thomas@xs4all.net Wed Aug 9 17:15:39 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 9 Aug 2000 18:15:39 +0200 Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: ; from just@letterror.com on Wed, Aug 09, 2000 at 04:01:18PM +0100 References: <200008091249.HAA23481@cj20424-a.reston1.va.home.com> Message-ID: <20000809181539.M266@xs4all.nl> On Wed, Aug 09, 2000 at 04:01:18PM +0100, Just van Rossum wrote: > PEP: 1716099-3 > Title: Index-enhanced sequence iteration > Version: $Revision: 1.1 $ > Owner: Someone-with-commit-rights I'd be willing to adopt this PEP, if the other two PEPs on my name don't need extensive rewrites anymore. > Features > > It adds an optional clause to the 'for' statement: > > for indexing in : Ever since I saw the implementation of FOR_LOOP I've wanted this, but I never could think up a backwards compatible and readable syntax for it ;P > Disadvantages: > It will break that one person's code that uses "indexing" > as a variable name. This needn't be true, if it's done in the same way as Tim proposed the 'form from import as as as' syntax change ;) for_stmt: 'for' exprlist [NAME exprlist] 'in' testlist ':' suite ['else' ':' suite] If the 5th subnode of the expression is 'in', the 3rd should be 'indexing' and the 4th would be the variable to assign the index number to. If it's ':', the loop is index-less. (this is just a quick and dirty example; 'exprlist' is probably not the right subnode for the indexing variable, because it can't be a tuple or anything like that.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From skip@mojam.com (Skip Montanaro) Wed Aug 9 17:40:27 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 9 Aug 2000 11:40:27 -0500 (CDT) Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: <20000809181539.M266@xs4all.nl> References: <200008091249.HAA23481@cj20424-a.reston1.va.home.com> <20000809181539.M266@xs4all.nl> Message-ID: <14737.35195.31385.867664@beluga.mojam.com> >> Disadvantages: >> It will break that one person's code that uses "indexing" as a >> variable name. Thomas> This needn't be true, if it's done in the same way as Tim Thomas> proposed the 'form from import as as as' syntax change ;) Could this be extended to many/most/all current instances of keywords in Python? As Tim pointed out, Fortran has no keywords. It annoys me that I (for example) can't define a method named "print". Skip From nowonder@nowonder.de Wed Aug 9 19:49:53 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Wed, 09 Aug 2000 18:49:53 +0000 Subject: [Python-Dev] cannot commit 1.6 changes Message-ID: <3991A7D0.4D2479C7@nowonder.de> I have taken care of removing all occurences of math.rint() from the 1.6 sources. The commit worked fine for the Doc, Include and Module directory, but cvs won't let me commit the changes to config.h.in, configure.in, configure: cvs server: sticky tag `cnri-16-start' for file `config.h.in' is not a branch cvs server: sticky tag `cnri-16-start' for file `configure' is not a branch cvs server: sticky tag `cnri-16-start' for file `configure.in' is not a branch cvs [server aborted]: correct above errors first! What am I missing? confused-ly y'rs Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From esr@thyrsus.com Wed Aug 9 19:03:21 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Wed, 9 Aug 2000 14:03:21 -0400 Subject: [Python-Dev] Un-stalling Berkeley DB support Message-ID: <20000809140321.A836@thyrsus.com> I'm still interested in getting support for the version 3 Berkeley DB into the core. This is one of my top three Python priorities currently, along with drafting PEP2 and overhauling the curses HOWTO. (I'd sure like to see shm get in, too, but that's blocked on Vladimir writing suitable documentation, too. I'd like to get the necessary C extension in before 2.0 freeze, if possible. I've copied its author. Again, the motivation here is to make shelving transactional, with useful read-many/write-once guarantees. Thousands of CGI programmers would thank us for this. When we last discussed this subject, there was general support for the functionality, but a couple of people went "bletch!" about SWIG-generated code (there was unhappiness about pointers being treated as strings). Somebody said something about having SWIG patches to address this. Is this the only real issue with SWIG-generated code? If so, we can pursue two paths: (1) Hand Greg a patched SWIG so he can release a 2.1.2 version of the DB extension that meets our cleanliness criteria, and (2) press the SWIG guy to incorporate these patches in his next release. -- Eric S. Raymond "The best we can hope for concerning the people at large is that they be properly armed." -- Alexander Hamilton, The Federalist Papers at 184-188 From akuchlin@mems-exchange.org Wed Aug 9 19:09:55 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Wed, 9 Aug 2000 14:09:55 -0400 Subject: [Python-Dev] py-howto project now operational Message-ID: <20000809140955.C4838@kronos.cnri.reston.va.us> I've just gotten around to setting up the checkin list for the Python HOWTO project on SourceForge (py-howto.sourceforge.net), so the project is now fully operational. People who want to update the HOWTOs, such as ESR and the curses HOWTO, can now go ahead and make changes. And this is the last you'll hear about the HOWTOs on python-dev; please use the Doc-SIG mailing list (doc-sig@python.org) for further discussion of the HOWTOs. --amk From thomas@xs4all.net Wed Aug 9 19:28:54 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 9 Aug 2000 20:28:54 +0200 Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: <14737.35195.31385.867664@beluga.mojam.com>; from skip@mojam.com on Wed, Aug 09, 2000 at 11:40:27AM -0500 References: <200008091249.HAA23481@cj20424-a.reston1.va.home.com> <20000809181539.M266@xs4all.nl> <14737.35195.31385.867664@beluga.mojam.com> Message-ID: <20000809202854.N266@xs4all.nl> On Wed, Aug 09, 2000 at 11:40:27AM -0500, Skip Montanaro wrote: > >> Disadvantages: > >> It will break that one person's code that uses "indexing" as a > >> variable name. > Thomas> This needn't be true, if it's done in the same way as Tim > Thomas> proposed the 'form from import as as as' syntax change ;) > Could this be extended to many/most/all current instances of keywords in > Python? As Tim pointed out, Fortran has no keywords. It annoys me that I > (for example) can't define a method named "print". No. I just (in the trainride from work to home ;) wrote a patch that adds 'from x import y as z' and 'import foo as fee', and came to the conclusion that we can't make 'from' a non-reserved word, for instance. Because if we change 'from' dotted_name 'import' NAME* into NAME dotted_name 'import' NAME* the parser won't know how to parse other expressions that start with NAME, like 'NAME = expr' or 'NAME is expr'. I know this because I tried it and it didn't work :-) So we can probably make most names that are *part* of a statement non-reserved words, but not those that uniquely identify a statement. That doesn't leave much words, except perhaps for the 'in' in 'for' -- but 'in' is already a reserved word for other purposes ;) As for the patch that adds 'as' (as a non-reserved word) to both imports, I'll upload it to SF after I rewrite it a bit ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From bckfnn@worldonline.dk Wed Aug 9 20:43:58 2000 From: bckfnn@worldonline.dk (Finn Bock) Date: Wed, 09 Aug 2000 19:43:58 GMT Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: <20000809202854.N266@xs4all.nl> References: <200008091249.HAA23481@cj20424-a.reston1.va.home.com> <20000809181539.M266@xs4all.nl> <14737.35195.31385.867664@beluga.mojam.com> <20000809202854.N266@xs4all.nl> Message-ID: <3991acc4.10990753@smtp.worldonline.dk> [Skip Montanaro] > Could this be extended to many/most/all current instances of keywords in > Python? As Tim pointed out, Fortran has no keywords. It annoys me that I > (for example) can't define a method named "print". [Thomas Wouters] >No. I just (in the trainride from work to home ;) wrote a patch that adds >'from x import y as z' and 'import foo as fee', and came to the conclusion >that we can't make 'from' a non-reserved word, for instance. Because if we >change > >'from' dotted_name 'import' NAME* > >into > >NAME dotted_name 'import' NAME* > >the parser won't know how to parse other expressions that start with NAME, >like 'NAME = expr' or 'NAME is expr'. I know this because I tried it and it >didn't work :-) So we can probably make most names that are *part* of a >statement non-reserved words, but not those that uniquely identify a >statement. That doesn't leave much words, except perhaps for the 'in' in >'for' -- but 'in' is already a reserved word for other purposes ;) Just a datapoint. JPython goes a bit further in its attempt to unreserve reserved words in certain cases: - after "def" - after a dot "." - after "import" - after "from" (in an import stmt) - and as argument names This allow JPython to do: from from import x def def(): pass x.exec(from=1, to=2) This feature was added to ease JPython's integration to existing java libraries. IIRC it was remarked that CPython could also make use of such a feature when integrating to f.ex Tk or COM. regards, finn From nascheme@enme.ucalgary.ca Wed Aug 9 21:11:04 2000 From: nascheme@enme.ucalgary.ca (Neil Schemenauer) Date: Wed, 9 Aug 2000 14:11:04 -0600 Subject: [Python-Dev] test_fork1 on SMP? (was Re: [Python Dev] test_fork1 failing --with-threads (for some people)...) In-Reply-To: ; from Tim Peters on Mon, Jul 31, 2000 at 04:42:50AM -0400 References: <14724.22554.818853.722906@bitdiddle.concentric.net> Message-ID: <20000809141104.A10805@keymaster.enme.ucalgary.ca> On Mon, Jul 31, 2000 at 04:42:50AM -0400, Tim Peters wrote: > It's a baffler! AFAIK, nobody yet has thought of a way that a fork can > screw up the state of the locks in the *parent* process (it must be easy to > see how they can get screwed up in a child, because two of us already did > ). If I add Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS around fork() in posixmodule then the child is the process which always seems to hang. The child is hanging at: #0 0x4006d58b in __sigsuspend (set=0xbf7ffac4) at ../sysdeps/unix/sysv/linux/sigsuspend.c:48 #1 0x4001f1a0 in pthread_cond_wait (cond=0x8264e1c, mutex=0x8264e28) at restart.h:49 #2 0x806f3c3 in PyThread_acquire_lock (lock=0x8264e18, waitflag=1) at thread_pthread.h:311 #3 0x80564a8 in PyEval_RestoreThread (tstate=0x8265a78) at ceval.c:178 #4 0x80bf274 in posix_fork (self=0x0, args=0x8226ccc) at ./posixmodule.c:1659 #5 0x8059460 in call_builtin (func=0x82380e0, arg=0x8226ccc, kw=0x0) at ceval.c:2376 #6 0x8059378 in PyEval_CallObjectWithKeywords (func=0x82380e0, arg=0x8226ccc, kw=0x0) at ceval.c:2344 #7 0x80584f2 in eval_code2 (co=0x8265e98, globals=0x822755c, locals=0x0, args=0x8226cd8, argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, owner=0x0) at ceval.c:1682 #8 0x805974b in call_function (func=0x8264ddc, arg=0x8226ccc, kw=0x0) at ceval.c:2498 #9 0x805936b in PyEval_CallObjectWithKeywords (func=0x8264ddc, arg=0x8226ccc, kw=0x0) at ceval.c:2342 #10 0x80af26a in t_bootstrap (boot_raw=0x8264e00) at ./threadmodule.c:199 #11 0x4001feca in pthread_start_thread (arg=0xbf7ffe60) at manager.c:213 Since there is only one thread in the child this should not be happening. Can someone explain this? I have tested this both a SMP Linux machine and a UP Linux machine. Neil From thomas@xs4all.net Wed Aug 9 21:27:50 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 9 Aug 2000 22:27:50 +0200 Subject: [Python-Dev] [Patch #101135] 'import x as y' and 'from x import y as z' (fwd) Message-ID: <20000809222749.O266@xs4all.nl> For those of you not on the patches list, here's the summary of the patch I just uploaded to SF. In short, it adds "import x as y" and "from module import x as y", in the way Tim proposed this morning. (Probably late last night for most of you.) ----- Forwarded message from noreply@sourceforge.net ----- This patch adds the oft-proposed 'import as' syntax, to both 'import module' and 'from module import ...', but without making 'as' a reserved word (by using the technique Tim Peters proposed on python-dev.) 'import spam as egg' is a very simple patch to compile.c, which doesn't need changes to the VM, but 'from spam import dog as meat' needs a new bytecode, which this patch calls 'FROM_IMPORT_AS'. The bytecode loads an object from a module onto the stack, so a STORE_NAME can store it later. This can't be done by the normal FROM_IMPORT opcode, because it needs to take the special case of '*' into account. Also, because it uses 'STORE_NAME', it's now possible to mix 'import' and 'global', like so: global X from foo import X as X The patch still generates the old code for from foo import X (without 'as') mostly to save on bytecode size, and for the 'compatibility' with mixing 'global' and 'from .. import'... I'm not sure what's the best thing to do. The patch doesn't include a test suite or documentation, yet. ------------------------------------------------------- For more info, visit: http://sourceforge.net/patch/?func=detailpatch&patch_id=101135&group_id=5470 ----- End forwarded message ----- -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From greg@mad-scientist.com Wed Aug 9 21:27:33 2000 From: greg@mad-scientist.com (Gregory P . Smith) Date: Wed, 9 Aug 2000 13:27:33 -0700 Subject: [Python-Dev] Re: Un-stalling Berkeley DB support In-Reply-To: <20000809140321.A836@thyrsus.com>; from esr@thyrsus.com on Wed, Aug 09, 2000 at 02:03:21PM -0400 References: <20000809140321.A836@thyrsus.com> Message-ID: <20000809132733.C2019@mad-scientist.com> On Wed, Aug 09, 2000 at 02:03:21PM -0400, Eric S. Raymond wrote: > > When we last discussed this subject, there was general support for the > functionality, but a couple of people went "bletch!" about SWIG-generated > code (there was unhappiness about pointers being treated as strings). > > Somebody said something about having SWIG patches to address this. Is this > the only real issue with SWIG-generated code? If so, we can pursue two paths: > (1) Hand Greg a patched SWIG so he can release a 2.1.2 version of the DB > extension that meets our cleanliness criteria, and (2) press the SWIG guy > to incorporate these patches in his next release. I'm not surprised to see the "bletch!" for SWIG's string/pointer things, they are technically gross. Anyone know what SWIG v1.3a3 does (v1.3 is a total rewrite from v1.1)? py-bsddb3 as distributed was build using SWIG v1.1-883. In the meantime, if someone knows of a version of SWIG that does this better, try using it to build bsddb3 (just pass a SWIG=/usr/spam/eggs/bin/swig to the Makefile). If you run into problems, send them and a copy of that swig my way. I'll take a quick look at SWIG v1.3alpha3 here and see what that does. Greg From skip@mojam.com (Skip Montanaro) Wed Aug 9 21:41:57 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 9 Aug 2000 15:41:57 -0500 (CDT) Subject: [Python-Dev] Re: Un-stalling Berkeley DB support In-Reply-To: <20000809132733.C2019@mad-scientist.com> References: <20000809140321.A836@thyrsus.com> <20000809132733.C2019@mad-scientist.com> Message-ID: <14737.49685.902542.576229@beluga.mojam.com> >>>>> "Greg" == Gregory P Smith writes: Greg> On Wed, Aug 09, 2000 at 02:03:21PM -0400, Eric S. Raymond wrote: >> >> When we last discussed this subject, there was general support for >> the functionality, but a couple of people went "bletch!" about >> SWIG-generated code (there was unhappiness about pointers being >> treated as strings). ... Greg> I'm not surprised to see the "bletch!" for SWIG's string/pointer Greg> things, they are technically gross. We're talking about a wrapper around a single smallish library (probably < 20 exposed functions), right? Seems to me that SWIG is the wrong tool to use here. It's for wrapping massive libraries automatically. Why not just recode the current SWIG-generated module manually? What am I missing? -- Skip Montanaro, skip@mojam.com, http://www.mojam.com/, http://www.musi-cal.com/ "To get what you want you must commit yourself for sometime" - fortune cookie From nascheme@enme.ucalgary.ca Wed Aug 9 21:49:25 2000 From: nascheme@enme.ucalgary.ca (Neil Schemenauer) Date: Wed, 9 Aug 2000 14:49:25 -0600 Subject: [Python-Dev] Re: [Patches] [Patch #101135] 'import x as y' and 'from x import y as z' In-Reply-To: <200008092014.NAA08040@delerium.i.sourceforge.net>; from noreply@sourceforge.net on Wed, Aug 09, 2000 at 01:14:52PM -0700 References: <200008092014.NAA08040@delerium.i.sourceforge.net> Message-ID: <20000809144925.A11242@keymaster.enme.ucalgary.ca> On Wed, Aug 09, 2000 at 01:14:52PM -0700, noreply@sourceforge.net wrote: > Patch #101135 has been updated. > > Project: > Category: core (C code) > Status: Open > Summary: 'import x as y' and 'from x import y as z' +1. This is much more useful and clear than setdefault (which I was -1 on, not that it matters). Neil From esr@thyrsus.com Wed Aug 9 22:03:51 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Wed, 9 Aug 2000 17:03:51 -0400 Subject: [Python-Dev] Re: [Patches] [Patch #101135] 'import x as y' and 'from x import y as z' In-Reply-To: <20000809144925.A11242@keymaster.enme.ucalgary.ca>; from nascheme@enme.ucalgary.ca on Wed, Aug 09, 2000 at 02:49:25PM -0600 References: <200008092014.NAA08040@delerium.i.sourceforge.net> <20000809144925.A11242@keymaster.enme.ucalgary.ca> Message-ID: <20000809170351.A1550@thyrsus.com> Neil Schemenauer : > On Wed, Aug 09, 2000 at 01:14:52PM -0700, noreply@sourceforge.net wrote: > > Patch #101135 has been updated. > > > > Project: > > Category: core (C code) > > Status: Open > > Summary: 'import x as y' and 'from x import y as z' > > +1. This is much more useful and clear than setdefault (which I was -1 > on, not that it matters). I'm +0 on this. -- Eric S. Raymond The most foolish mistake we could possibly make would be to permit the conquered Eastern peoples to have arms. History teaches that all conquerors who have allowed their subject races to carry arms have prepared their own downfall by doing so. -- Hitler, April 11 1942, revealing the real agenda of "gun control" From greg@mad-scientist.com Wed Aug 9 22:16:39 2000 From: greg@mad-scientist.com (Gregory P . Smith) Date: Wed, 9 Aug 2000 14:16:39 -0700 Subject: [Python-Dev] Re: Un-stalling Berkeley DB support In-Reply-To: <20000809140321.A836@thyrsus.com>; from esr@thyrsus.com on Wed, Aug 09, 2000 at 02:03:21PM -0400 References: <20000809140321.A836@thyrsus.com> Message-ID: <20000809141639.D2019@mad-scientist.com> On Wed, Aug 09, 2000 at 02:03:21PM -0400, Eric S. Raymond wrote: > > When we last discussed this subject, there was general support for the > functionality, but a couple of people went "bletch!" about SWIG-generated > code (there was unhappiness about pointers being treated as strings). > > Somebody said something about having SWIG patches to address this. Is this > the only real issue with SWIG-generated code? If so, we can pursue two paths: > (1) Hand Greg a patched SWIG so he can release a 2.1.2 version of the DB > extension that meets our cleanliness criteria, and (2) press the SWIG guy > to incorporate these patches in his next release. Out of curiosity, I just made a version of py-bsddb3 that uses SWIG v1.3alpha3 instead of SWIG v1.1-883. It looks like 1.3a3 is still using strings for pointerish things. One thing to note that may calm some peoples sense of "eww gross, pointer strings" is that programmers should never see them. They are "hidden" behind the python shadow class. The pointer strings are only contained within the shadow objects "this" member. example: >>> from bsddb3.db import * >>> e = DbEnv() >>> e >>> e.this '_807eea8_MyDB_ENV_p' Anyways, the update if anyone is curious about a version using the more recent swig is on the py-bsddb3 web site: http://electricrain.com/greg/python/bsddb3/ Greg From guido@beopen.com Wed Aug 9 23:29:58 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 09 Aug 2000 17:29:58 -0500 Subject: [Python-Dev] cannot commit 1.6 changes In-Reply-To: Your message of "Wed, 09 Aug 2000 18:49:53 GMT." <3991A7D0.4D2479C7@nowonder.de> References: <3991A7D0.4D2479C7@nowonder.de> Message-ID: <200008092229.RAA24802@cj20424-a.reston1.va.home.com> > I have taken care of removing all occurences of math.rint() > from the 1.6 sources. The commit worked fine for the Doc, > Include and Module directory, but cvs won't let me commit > the changes to config.h.in, configure.in, configure: > > cvs server: sticky tag `cnri-16-start' for file `config.h.in' is not a > branch > cvs server: sticky tag `cnri-16-start' for file `configure' is not a > branch > cvs server: sticky tag `cnri-16-start' for file `configure.in' is not a > branch > cvs [server aborted]: correct above errors first! > > What am I missing? The error message is right. Somehow whoever set those tags on those files did not make them branch tags. I don't know why -- I think it was Fred, I don't know why he did that. The quickest way to fix this is to issue the command cvs tag -F -b -r cnri-16-start for each file, where is the revision where the tag should be and is the file. Note that -F means "force" (otherwise you get a complaint because the tag is already defined) and -b means "branch" which makes the tag a branch tag. I *believe* that branch tags are recognized because they have the form . .0. but I'm not sure this is documented. I alread did this for you for these three files! --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From guido@beopen.com Wed Aug 9 23:43:35 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 09 Aug 2000 17:43:35 -0500 Subject: [Python-Dev] test_fork1 on SMP? (was Re: [Python Dev] test_fork1 failing --with-threads (for some people)...) In-Reply-To: Your message of "Wed, 09 Aug 2000 14:11:04 CST." <20000809141104.A10805@keymaster.enme.ucalgary.ca> References: <14724.22554.818853.722906@bitdiddle.concentric.net> <20000809141104.A10805@keymaster.enme.ucalgary.ca> Message-ID: <200008092243.RAA24914@cj20424-a.reston1.va.home.com> > If I add Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS around fork() > in posixmodule then the child is the process which always seems to hang. I first thought that the lock should be released around the fork too, but later I realized that that was exactly wrong: if you release the lock before you fork, another thread will likely grab the lock before you fork; then in the child the lock is held by that other thread but that thread doesn't exist, so when the main thread tries to get the lock back it hangs in the Py_END_ALLOW_THREADS. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From ping@lfw.org Wed Aug 9 23:06:15 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Wed, 9 Aug 2000 15:06:15 -0700 (PDT) Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: <200008091249.HAA23481@cj20424-a.reston1.va.home.com> Message-ID: On Wed, 9 Aug 2000, Guido van Rossum wrote: > I forget what indices() was -- is it the moreal equivalent of keys()? Yes, it's range(len(s)). > If irange(s) is zip(range(len(s)), s), I see how that's a bit > unwieldy. In the past there were syntax proposals, e.g. ``for i > indexing s''. Maybe you and Just can draft a PEP? In the same vein as zip(), i think it's much easier to just toss in a couple of built-ins than try to settle on a new syntax. (I already uploaded a patch to add indices() and irange() to the built-ins, immediately after i posted my first message on this thread.) Surely a PEP isn't required for a couple of built-in functions that are simple and well understood? You can just call thumbs-up or thumbs-down and be done with it. -- ?!ng "All models are wrong; some models are useful." -- George Box From klm@digicool.com Wed Aug 9 23:05:57 2000 From: klm@digicool.com (Ken Manheimer) Date: Wed, 9 Aug 2000 18:05:57 -0400 (EDT) Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: Message-ID: On Wed, 9 Aug 2000, Just van Rossum wrote: > PEP: 1716099-3 > Title: Index-enhanced sequence iteration > [...] > It adds an optional clause to the 'for' statement: > > for indexing in : > ... > [...] > Disadvantages: > > It will break that one person's code that uses "indexing" > as a variable name. It creates a new 'for' variant, increasing challenge for beginners (and the befuddled, like me) of tracking the correct syntax. I could see that disadvantage being justified by a more significant change - lockstep iteration would qualify, for me (though it's circumventing this drawback with zip()). List comprehensions have that weight, and analogize elegantly against the existing slice syntax. I don't think the 'indexing' benefits are of that order, not enough so to double the number of 'for' forms, even if there are some performance gains over the (syntactically equivalent) zip(), so, sorry, but i'm -1. Ken klm@digicool.com From klm@digicool.com Wed Aug 9 23:13:37 2000 From: klm@digicool.com (Ken Manheimer) Date: Wed, 9 Aug 2000 18:13:37 -0400 (EDT) Subject: [Python-Dev] [Patch #101135] 'import x as y' and 'from x import y as z' (fwd) In-Reply-To: <20000809222749.O266@xs4all.nl> Message-ID: On Wed, 9 Aug 2000, Thomas Wouters wrote: > For those of you not on the patches list, here's the summary of the patch I > just uploaded to SF. In short, it adds "import x as y" and "from module > import x as y", in the way Tim proposed this morning. (Probably late last > night for most of you.) I guess the criteria i used in my thumbs down on 'indexing' is very subjective, because i would say the added functionality of 'import x as y' *does* satisfy my added-functionality test, and i'd be +1. (I think the determining thing is the ability to avoid name collisions without any gross shuffle.) I also really like the non-keyword basis for the, um, keyword. Ken klm@digicool.com From guido@beopen.com Thu Aug 10 00:14:19 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 09 Aug 2000 18:14:19 -0500 Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: Your message of "Wed, 09 Aug 2000 15:06:15 MST." References: Message-ID: <200008092314.SAA25157@cj20424-a.reston1.va.home.com> > On Wed, 9 Aug 2000, Guido van Rossum wrote: > > I forget what indices() was -- is it the moreal equivalent of keys()? [Ping] > Yes, it's range(len(s)). > > > If irange(s) is zip(range(len(s)), s), I see how that's a bit > > unwieldy. In the past there were syntax proposals, e.g. ``for i > > indexing s''. Maybe you and Just can draft a PEP? > > In the same vein as zip(), i think it's much easier to just toss in > a couple of built-ins than try to settle on a new syntax. (I already > uploaded a patch to add indices() and irange() to the built-ins, > immediately after i posted my first message on this thread.) > > Surely a PEP isn't required for a couple of built-in functions that > are simple and well understood? You can just call thumbs-up or > thumbs-down and be done with it. -1 for indices -0 for irange --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From thomas@xs4all.net Wed Aug 9 23:15:10 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 10 Aug 2000 00:15:10 +0200 Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: ; from just@letterror.com on Wed, Aug 09, 2000 at 04:01:18PM +0100 References: <200008091249.HAA23481@cj20424-a.reston1.va.home.com> Message-ID: <20000810001510.P266@xs4all.nl> On Wed, Aug 09, 2000 at 04:01:18PM +0100, Just van Rossum wrote: > Features > It adds an optional clause to the 'for' statement: > > for indexing in : > ... > Implementation > > Implementation should be trivial for anyone named Guido, > Tim or Thomas. Well, to justify that vote of confidence <0.4 wink> I wrote a quick hack that adds this to Python for loops. It can be found on SF, patch #101138. It's small, but it works. I'll iron out any bugs if there's enough positive feelings towards this kind of syntax change. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Wed Aug 9 23:22:55 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 10 Aug 2000 00:22:55 +0200 Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: <200008092314.SAA25157@cj20424-a.reston1.va.home.com>; from guido@beopen.com on Wed, Aug 09, 2000 at 06:14:19PM -0500 References: <200008092314.SAA25157@cj20424-a.reston1.va.home.com> Message-ID: <20000810002255.Q266@xs4all.nl> On Wed, Aug 09, 2000 at 06:14:19PM -0500, Guido van Rossum wrote: > -1 for indices > > -0 for irange The same for me, though I prefer 'for i indexing x in l' over 'irange()'. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From beazley@schlitz.cs.uchicago.edu Wed Aug 9 23:34:16 2000 From: beazley@schlitz.cs.uchicago.edu (David M. Beazley) Date: Wed, 9 Aug 2000 17:34:16 -0500 (CDT) Subject: [Python-Dev] Python-Dev digest, Vol 1 #737 - 17 msgs In-Reply-To: <20000809221115.AC4E61D182@dinsdale.python.org> References: <20000809221115.AC4E61D182@dinsdale.python.org> Message-ID: <14737.55249.87871.538988@schlitz.cs.uchicago.edu> python-dev-request@python.org writes: > > I'd like to get the necessary C extension in before 2.0 freeze, if > possible. I've copied its author. Again, the motivation here is to make > shelving transactional, with useful read-many/write-once guarantees. > Thousands of CGI programmers would thank us for this. > > When we last discussed this subject, there was general support for the > functionality, but a couple of people went "bletch!" about SWIG-generated > code (there was unhappiness about pointers being treated as strings). > > Somebody said something about having SWIG patches to address this. Is this > the only real issue with SWIG-generated code? If so, we can pursue > two paths: Well, as the guilty party on the SWIG front, I can say that the current development version of SWIG is using CObjects instead of strings (well, actually I lie---you have to compile the wrappers with -DSWIG_COBJECT_TYPES to turn that feature on). Just as a general aside on this topic, I did a number of experiments comparing the performance of using CObjects vs.the gross string-pointer hack about 6 months ago. Strangely enough, there was virtually no-difference in runtime performance and if recall correctly, the string hack might have even been just a little bit faster. Go figure :-). Overall, the main difference between SWIG1.3 and SWIG1.1 is in runtime performance of the wrappers as well as various changes to reduce the amount of wrapper code. However, 1.3 is also very much an alpha release right now---if you're going to use that, make sure you thoroughly test everything. On the subject of the Berkeley DB module, I would definitely like to see a module for this. If there is anything I can do to either modify the behavior of SWIG or to build an extension module by hand, let me know. Cheers, Dave From MarkH@ActiveState.com Thu Aug 10 00:03:19 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Thu, 10 Aug 2000 09:03:19 +1000 Subject: [Python-Dev] Python keywords (was Lockstep iteration - eureka!) In-Reply-To: <14737.35195.31385.867664@beluga.mojam.com> Message-ID: [Skip laments...] > Could this be extended to many/most/all current instances of > keywords in Python? As Tim pointed out, Fortran has no > keywords. It annoys me that I (for example) can't define > a method named "print". Sometimes it is worse than annoying! In the COM and CORBA worlds, it can be a showstopper - if an external object happens to expose a method or property named after a Python keyword, then you simply can not use it! This has lead to COM support having to check _every_ attribute name it sees externally, and mangle it if a keyword. A bigger support exists for .NET. The .NET framework explicitly dictates that a compliant language _must_ have a way of overriding its own keywords when calling external methods (it was either that, or try and dictate a union of reserved words they can ban) Eg, C# allows you to surround a keyword with brackets. ie, I believe something like: object.[if] Would work in C# to provide access to an attribute named "if" Unfortunately, Python COM is a layer ontop of CPython, and Python .NET still uses the CPython parser - so in neither of these cases is there a simple hack I can use to work around it at the parser level. Needless to say, as this affects the 2 major technologies I work with currently, I would like an official way to work around Python keywords! Mark. From guido@beopen.com Thu Aug 10 01:12:59 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 09 Aug 2000 19:12:59 -0500 Subject: [Python-Dev] Python keywords (was Lockstep iteration - eureka!) In-Reply-To: Your message of "Thu, 10 Aug 2000 09:03:19 +1000." References: Message-ID: <200008100012.TAA25968@cj20424-a.reston1.va.home.com> > [Skip laments...] > > Could this be extended to many/most/all current instances of > > keywords in Python? As Tim pointed out, Fortran has no > > keywords. It annoys me that I (for example) can't define > > a method named "print". > > Sometimes it is worse than annoying! > > In the COM and CORBA worlds, it can be a showstopper - if an external > object happens to expose a method or property named after a Python keyword, > then you simply can not use it! > > This has lead to COM support having to check _every_ attribute name it sees > externally, and mangle it if a keyword. > > A bigger support exists for .NET. The .NET framework explicitly dictates > that a compliant language _must_ have a way of overriding its own keywords > when calling external methods (it was either that, or try and dictate a > union of reserved words they can ban) > > Eg, C# allows you to surround a keyword with brackets. ie, I believe > something like: > > object.[if] > > Would work in C# to provide access to an attribute named "if" > > Unfortunately, Python COM is a layer ontop of CPython, and Python .NET > still uses the CPython parser - so in neither of these cases is there a > simple hack I can use to work around it at the parser level. > > Needless to say, as this affects the 2 major technologies I work with > currently, I would like an official way to work around Python keywords! The JPython approach should be added to CPython. This effectively turns off keywords directly after ".", "def" and in a few other places. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From MarkH@ActiveState.com Thu Aug 10 00:17:35 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Thu, 10 Aug 2000 09:17:35 +1000 Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: <200008092314.SAA25157@cj20424-a.reston1.va.home.com> Message-ID: Guido commented yesterday that he doesnt tally votes (yay), but obviously he still issues them! It made me think of a Dutch Crocodile Dundee on a visit to New York, muttering to his harassers as he whips something out from under his clothing... > -1 for indices "You call that a -1, _this_ is a -1" :-) [Apologies to anyone who hasnt seen the knife scene in the forementioned movie ;-] Mark. From MarkH@ActiveState.com Thu Aug 10 00:21:33 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Thu, 10 Aug 2000 09:21:33 +1000 Subject: [Python-Dev] Python keywords (was Lockstep iteration - eureka!) In-Reply-To: <200008100012.TAA25968@cj20424-a.reston1.va.home.com> Message-ID: [Guido] > The JPython approach should be added to CPython. This effectively > turns off keywords directly after ".", "def" and in a few other > places. Excellent. I saw a reference to this after I sent my mail. I'd be happy to help, in a long, drawn out, when-I-find-time kind of way. What is the general strategy - is it simply to maintain a state in the parser? Where would I start to look into? Mark. From guido@beopen.com Thu Aug 10 01:36:30 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 09 Aug 2000 19:36:30 -0500 Subject: [Python-Dev] Python keywords (was Lockstep iteration - eureka!) In-Reply-To: Your message of "Thu, 10 Aug 2000 09:21:33 +1000." References: Message-ID: <200008100036.TAA26235@cj20424-a.reston1.va.home.com> > [Guido] > > The JPython approach should be added to CPython. This effectively > > turns off keywords directly after ".", "def" and in a few other > > places. > > Excellent. I saw a reference to this after I sent my mail. > > I'd be happy to help, in a long, drawn out, when-I-find-time kind of way. > What is the general strategy - is it simply to maintain a state in the > parser? Where would I start to look into? > > Mark. Alas, I'm not sure how easy it will be. The parser generator will probably have to be changed to allow you to indicate not to do a resword lookup at certain points in the grammar. I don't know where to start. :-( --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From Moshe Zadka Thu Aug 10 02:12:59 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 10 Aug 2000 04:12:59 +0300 (IDT) Subject: [Python-Dev] Re: Un-stalling Berkeley DB support In-Reply-To: <20000809141639.D2019@mad-scientist.com> Message-ID: On Wed, 9 Aug 2000, Gregory P . Smith wrote: > Out of curiosity, I just made a version of py-bsddb3 that uses SWIG > v1.3alpha3 instead of SWIG v1.1-883. It looks like 1.3a3 is still > using strings for pointerish things. One thing to note that may calm > some peoples sense of "eww gross, pointer strings" is that programmers > should never see them. They are "hidden" behind the python shadow class. > The pointer strings are only contained within the shadow objects "this" > member. It's not "ewww gross", it's "dangerous!". This makes Python "not safe", since users can access random memory location. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From Moshe Zadka Thu Aug 10 02:28:00 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 10 Aug 2000 04:28:00 +0300 (IDT) Subject: [Python-Dev] Python keywords (was Lockstep iteration - eureka!) In-Reply-To: Message-ID: On Thu, 10 Aug 2000, Mark Hammond wrote: > Sometimes it is worse than annoying! > > In the COM and CORBA worlds, it can be a showstopper - if an external > object happens to expose a method or property named after a Python keyword, > then you simply can not use it! How about this (simple, but relatively unannoying) convention: To COM name: - remove last "_", if any From COM name: - add "_" is it's a keyword - add "_" if last character is "_" -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From greg@cosc.canterbury.ac.nz Thu Aug 10 02:29:38 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 10 Aug 2000 13:29:38 +1200 (NZST) Subject: [Python-Dev] A question for the Python Secret Police In-Reply-To: <20000809083127.7FFF6303181@snelboot.oratrix.nl> Message-ID: <200008100129.NAA13775@s454.cosc.canterbury.ac.nz> Jack Jansen : > Is the following morally allowed: > class Foo(Foo): Well, the standard admonitions against 'import *' apply. Whether using 'import *' or not, though, in the interests of clarity I think I would write it as class Foo(package1.mod.Foo): On the other hand, the funkiness factor of it does have a certain appeal! Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Thu Aug 10 02:56:55 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 10 Aug 2000 13:56:55 +1200 (NZST) Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: Message-ID: <200008100156.NAA13782@s454.cosc.canterbury.ac.nz> > It looks nice, but i'm pretty sure it won't fly. It will! Try it: >>> for (x in a, y in b): File " ", line 1 for (x in a, y in b): ^ SyntaxError: invalid syntax > how is the parser to know whether the lockstep form has been > invoked? The parser doesn't have to know as long as the compiler can tell, and clearly one of them can. > Besides, i think Guido has Pronounced quite firmly on zip(). That won't stop me from gently trying to change his mind one day. The existence of zip() doesn't preclude something more elegant being adopted in a future version. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Thu Aug 10 03:12:08 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 10 Aug 2000 14:12:08 +1200 (NZST) Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: <20000809130645.J266@xs4all.nl> Message-ID: <200008100212.OAA13789@s454.cosc.canterbury.ac.nz> Thomas Wouters : > The only objection I can bring up is that parentheses are almost always > optional, in Python, and this kind of violates it. They're optional around tuple constructors, but this is not a tuple constructor. The parentheses around function arguments aren't optional either, and nobody complains about that. > 'for (x in a, y in b) in z:' *is* valid syntax... But it's not valid Python: >>> for (x in a, y in b) in z: ... print x,y ... SyntaxError: can't assign to operator > It might not be too pretty, but it can be worked around ;) It wouldn't be any uglier than what's currently done with the LHS of an assignment, which is parsed as a general expression and treated specially later on. There's-more-to-the-Python-syntax-than-what-it-says-in- the-Grammar-file-ly, Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Thu Aug 10 03:19:32 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 10 Aug 2000 14:19:32 +1200 (NZST) Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: Message-ID: <200008100219.OAA13793@s454.cosc.canterbury.ac.nz> Just van Rossum : > for indexing in : Then idea is good, but I don't like this particular syntax much. It seems to be trying to do too much at once, giving you both an index and an element. Also, the wording reminds me unpleasantly of COBOL for some reason. Some time ago I suggested for over : as a way of getting hold of the index, and as a direct replacement for 'for i in range(len(blarg))' constructs. It could also be used for lockstep iteration applications, e.g. for i over a: frobulate(a[i], b[i]) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Thu Aug 10 03:23:50 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 10 Aug 2000 14:23:50 +1200 (NZST) Subject: [Python-Dev] Python keywords (was Lockstep iteration - eureka!) In-Reply-To: <200008100036.TAA26235@cj20424-a.reston1.va.home.com> Message-ID: <200008100223.OAA13796@s454.cosc.canterbury.ac.nz> BDFL: > The parser generator will probably have to be changed to allow you to > indicate not to do a resword lookup at certain points in the grammar. Isn't it the scanner which recognises reserved words? In that case, just make it not do that for the next token after certain tokens. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From billtut@microsoft.com Thu Aug 10 04:24:11 2000 From: billtut@microsoft.com (Bill Tutt) Date: Wed, 9 Aug 2000 20:24:11 -0700 Subject: [Python-Dev] Python keywords (was Lockstep iteration - eureka !) Message-ID: <58C671173DB6174A93E9ED88DCB0883D0A611B@red-msg-07.redmond.corp.microsoft.com> The parser actually recognizes keywords atm. We could change that so that each keyword is a token. Then you would have something like: keyword_allowed_name: KEY1 | KEY2 | KEY3 | ... | KEYN | NAME and then tweak func_def like so: func_def: DEF keyword_allowed_name parameters ':' suite I haven't pondered whether or not this would cause the DFA to fall into a degenerate case or not. Wondering where the metagrammer source file went to, Bill -----Original Message----- From: Greg Ewing [mailto:greg@cosc.canterbury.ac.nz] Sent: Wednesday, August 09, 2000 7:24 PM To: python-dev@python.org Subject: Re: [Python-Dev] Python keywords (was Lockstep iteration - eureka!) BDFL: > The parser generator will probably have to be changed to allow you to > indicate not to do a resword lookup at certain points in the grammar. Isn't it the scanner which recognises reserved words? In that case, just make it not do that for the next token after certain tokens. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://www.python.org/mailman/listinfo/python-dev From guido@beopen.com Thu Aug 10 05:44:45 2000 From: guido@beopen.com (Guido van Rossum) Date: Wed, 09 Aug 2000 23:44:45 -0500 Subject: [Python-Dev] Python keywords (was Lockstep iteration - eureka !) In-Reply-To: Your message of "Wed, 09 Aug 2000 20:24:11 MST." <58C671173DB6174A93E9ED88DCB0883D0A611B@red-msg-07.redmond.corp.microsoft.com> References: <58C671173DB6174A93E9ED88DCB0883D0A611B@red-msg-07.redmond.corp.microsoft.com> Message-ID: <200008100444.XAA27348@cj20424-a.reston1.va.home.com> > The parser actually recognizes keywords atm. > > We could change that so that each keyword is a token. > Then you would have something like: > > keyword_allowed_name: KEY1 | KEY2 | KEY3 | ... | KEYN | NAME > and then tweak func_def like so: > func_def: DEF keyword_allowed_name parameters ':' suite > > I haven't pondered whether or not this would cause the DFA to fall into a > degenerate case or not. This would be a good and simple approach. > Wondering where the metagrammer source file went to, It may not have existed; I may have handcrafted the metagrammar.c file. I believe the metagrammar was something like this: MSTART: RULE* RULE: NAME ':' RHS RHS: ITEM ('|' ITEM)* ITEM: (ATOM ['*' | '?'])+ ATOM: NAME | STRING | '(' RHS ')' | '[' RHS ']' --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From nowonder@nowonder.de Thu Aug 10 08:02:12 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Thu, 10 Aug 2000 07:02:12 +0000 Subject: 'import x as y' stops the show (was: Re: [Python-Dev] [Patch #101135] 'import x as y' and 'from x import y as z' (fwd)) References: <20000809222749.O266@xs4all.nl> Message-ID: <39925374.59D974FA@nowonder.de> Thomas Wouters wrote: > > For those of you not on the patches list, here's the summary of the patch I > just uploaded to SF. In short, it adds "import x as y" and "from module > import x as y", in the way Tim proposed this morning. (Probably late last > night for most of you.) -1 on the implementation. Although it looked okay on a first visual inspection, it builds a segfaulting python executable on linux: make distclean && ./configure && make test segfaults when first time starting python to run regrtest.py. Reversing the patch and doing a simple 'make test' has everything running again. +1 on the idea, though. It just seems sooo natural. My first reaction before applying the patch was testing if Python did not already do this <0.25 wink - really did it> Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From nowonder@nowonder.de Thu Aug 10 08:21:13 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Thu, 10 Aug 2000 07:21:13 +0000 Subject: 'import x as y' stops the show (was: Re: [Python-Dev] [Patch #101135] 'import x as y' and 'from x import y as z' (fwd)) References: <20000809222749.O266@xs4all.nl> <39925374.59D974FA@nowonder.de> Message-ID: <399257E9.E399D52D@nowonder.de> Peter Schneider-Kamp wrote: > > -1 on the implementation. Although it looked okay on a first visual > inspection, it builds a segfaulting python executable on linux: > make distclean && ./configure && make test > segfaults when first time starting python to run regrtest.py. > Reversing the patch and doing a simple 'make test' has everything > running again. Also note the following problems: nowonder@mobility:~/python/python/dist/src > ./python Python 2.0b1 (#12, Aug 10 2000, 07:17:46) [GCC 2.95.2 19991024 (release)] on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam Copyright 1995-2000 Corporation for National Research Initiatives (CNRI) >>> from string import join Speicherzugriffsfehler nowonder@mobility:~/python/python/dist/src > ./python Python 2.0b1 (#12, Aug 10 2000, 07:17:46) [GCC 2.95.2 19991024 (release)] on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam Copyright 1995-2000 Corporation for National Research Initiatives (CNRI) >>> from string import join as j File " ", line 1 from string import join as j ^ SyntaxError: invalid syntax >>> I think the problem is in compile.c, but that's just my bet. Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From thomas@xs4all.net Thu Aug 10 06:24:19 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 10 Aug 2000 07:24:19 +0200 Subject: 'import x as y' stops the show (was: Re: [Python-Dev] [Patch #101135] 'import x as y' and 'from x import y as z' (fwd)) In-Reply-To: <39925374.59D974FA@nowonder.de>; from nowonder@nowonder.de on Thu, Aug 10, 2000 at 07:02:12AM +0000 References: <20000809222749.O266@xs4all.nl> <39925374.59D974FA@nowonder.de> Message-ID: <20000810072419.A17171@xs4all.nl> On Thu, Aug 10, 2000 at 07:02:12AM +0000, Peter Schneider-Kamp wrote: > Thomas Wouters wrote: > > > > For those of you not on the patches list, here's the summary of the patch I > > just uploaded to SF. In short, it adds "import x as y" and "from module > > import x as y", in the way Tim proposed this morning. (Probably late last > > night for most of you.) > -1 on the implementation. Although it looked okay on a first visual > inspection, it builds a segfaulting python executable on linux: > make distclean && ./configure && make test > segfaults when first time starting python to run regrtest.py. > Reversing the patch and doing a simple 'make test' has everything > running again. Try running 'make' in 'Grammar/' first. None of my patches that touch Grammar include the changes to graminit.h and graminit.c, because they can be quite lengthy (in the order of several thousand lines, in this case, if I'm not mistaken.) So the same goes for the 'indexing for', 'range literal' and 'augmented assignment' patches ;) If it still goes crashy crashy after you re-make the grammar, I'll, well, I'll, I'll make Baldrick eat one of his own dirty socks ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From nowonder@nowonder.de Thu Aug 10 08:37:44 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Thu, 10 Aug 2000 07:37:44 +0000 Subject: 'import x as y' stops the show (was: Re: [Python-Dev] [Patch #101135] 'import x as y' and 'from x import y as z' (fwd)) References: <20000809222749.O266@xs4all.nl> <39925374.59D974FA@nowonder.de> <20000810072419.A17171@xs4all.nl> Message-ID: <39925BC8.17CD051@nowonder.de> Thomas Wouters wrote: > > If it still goes crashy crashy after you re-make the grammar, I'll, well, > I'll, I'll make Baldrick eat one of his own dirty socks ;) I just found that out for myself. The syntaxerror in the second examples lead my way ... Sorry for the hassle, but next time please remind me that I have to remake the grammar. +1 on the implementation now. perversely-minded-note: What about 'from string import *, join as j'? I think that would make sense, but as we are not fond of the star in any case maybe we don't need that. Peter P.S.: I'd like to see Baldrick do that. What the heck is a Baldrick? I am longing for breakfast, so I hope I can eat it. Mjam. -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From thomas@xs4all.net Thu Aug 10 06:55:10 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 10 Aug 2000 07:55:10 +0200 Subject: 'import x as y' stops the show (was: Re: [Python-Dev] [Patch #101135] 'import x as y' and 'from x import y as z' (fwd)) In-Reply-To: <39925BC8.17CD051@nowonder.de>; from nowonder@nowonder.de on Thu, Aug 10, 2000 at 07:37:44AM +0000 References: <20000809222749.O266@xs4all.nl> <39925374.59D974FA@nowonder.de> <20000810072419.A17171@xs4all.nl> <39925BC8.17CD051@nowonder.de> Message-ID: <20000810075510.B17171@xs4all.nl> On Thu, Aug 10, 2000 at 07:37:44AM +0000, Peter Schneider-Kamp wrote: > Thomas Wouters wrote: > > If it still goes crashy crashy after you re-make the grammar, I'll, well, > > I'll, I'll make Baldrick eat one of his own dirty socks ;) > I just found that out for myself. The syntaxerror in the > second examples lead my way ... > Sorry for the hassle, but next time please remind me that > I have to remake the grammar. It was late, last night, and I have to force myself not to write essays when submitting a patch in the first place ;-P How about we fix the dependencies so that the grammar gets re-made when necessary ? Or is there a good reason not to do that ? > perversely-minded-note: > What about 'from string import *, join as j'? > I think that would make sense, but as we are not fond of > the star in any case maybe we don't need that. 'join as j' ? What would it do ? Import all symbols from 'string' into a new namespace 'j' ? How about you do 'import string as j' instead ? It means you will still be able to do 'j._somevar', which you probably wouldn't in your example, but I don't think that's enough reason :P > P.S.: I'd like to see Baldrick do that. What the heck is > a Baldrick? I am longing for breakfast, so I hope > I can eat it. Mjam. Sorry :) They've been doing re-runs of Blackadder (1st through 4th, they're nearly done) on one of the belgian channels, and it happens to be one of my favorite comedy shows ;) It's a damned sight funnier than Crocodile Dundee, hey, Mark ? :) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From nowonder@nowonder.de Thu Aug 10 09:10:13 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Thu, 10 Aug 2000 08:10:13 +0000 Subject: 'import x as y' stops the show (was: Re: [Python-Dev] [Patch #101135] 'import x as y' and 'from x import y as z' (fwd)) References: <20000809222749.O266@xs4all.nl> <39925374.59D974FA@nowonder.de> <20000810072419.A17171@xs4all.nl> <39925BC8.17CD051@nowonder.de> <20000810075510.B17171@xs4all.nl> Message-ID: <39926365.909B2835@nowonder.de> Thomas Wouters wrote: > > 'join as j' ? What would it do ? Import all symbols from 'string' into a > new namespace 'j' ? How about you do 'import string as j' instead ? It means > you will still be able to do 'j._somevar', which you probably wouldn't in > your example, but I don't think that's enough reason :P Okay, your misunderstanding of the semantics I had in mind are reason enough <0.5 wink>. from string import *, join as j (or equivalently) from string import join as j, * would (in my book) import all "public" symbols from string and assign j = join. Assuming we have a Tkinter app (where all the tutorials do a 'from Tkinter import *') and we don't like 'createtimerhandle'. Then the following would give us tk_timer instead while still importing all the stuff from Tkinter with their regular names: from Tkinter import *, createtimerhandle as tk_timer An even better way of doing this were if it would not only give you another name but if it would not import the original one. In this example our expression would import all the symbols from Tkinter but would rename createtimerhandle as tk_timer. In this way you could still use * if you have a namespace clash. E.g.: from Tkinter import *, mainloop as tk_mainloop def mainloop(): if __name__ == '__main__': mainloop() Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From thomas@xs4all.net Thu Aug 10 07:23:16 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 10 Aug 2000 08:23:16 +0200 Subject: 'import x as y' stops the show (was: Re: [Python-Dev] [Patch #101135] 'import x as y' and 'from x import y as z' (fwd)) In-Reply-To: <39926365.909B2835@nowonder.de>; from nowonder@nowonder.de on Thu, Aug 10, 2000 at 08:10:13AM +0000 References: <20000809222749.O266@xs4all.nl> <39925374.59D974FA@nowonder.de> <20000810072419.A17171@xs4all.nl> <39925BC8.17CD051@nowonder.de> <20000810075510.B17171@xs4all.nl> <39926365.909B2835@nowonder.de> Message-ID: <20000810082316.C17171@xs4all.nl> On Thu, Aug 10, 2000 at 08:10:13AM +0000, Peter Schneider-Kamp wrote: > Thomas Wouters wrote: > > 'join as j' ? What would it do ? Import all symbols from 'string' into a > > new namespace 'j' ? How about you do 'import string as j' instead ? It means > > you will still be able to do 'j._somevar', which you probably wouldn't in > > your example, but I don't think that's enough reason :P > Okay, your misunderstanding of the semantics I had in mind are > reason enough <0.5 wink>. > from string import *, join as j > (or equivalently) > from string import join as j, * Ahh, like that :) Well, I'd say 'no'. "from module import *" has only one legitimate use, as far as I'm concerned, and that's taking over all symbols without prejudice, to encapsulate another module. It shouldn't be used in code that attempts to stay readable, so 'import join as j' is insanity ;-) If you really must do the above, do it in two import statements. > An even better way of doing this were if it would not > only give you another name but if it would not import > the original one. In this example our expression > would import all the symbols from Tkinter but would > rename createtimerhandle as tk_timer. In this way you > could still use * if you have a namespace clash. E.g.: No, that isn't possible. You can't pass a list of names to 'FROM_IMPORT *' to omit loading them. (That's also the reason the patch needs a new opcode, because you can't pass both the name to be imported from a module and the name it should be stored at, to the FROM_IMPORT bytecode :) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From nowonder@nowonder.de Thu Aug 10 09:52:31 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Thu, 10 Aug 2000 08:52:31 +0000 Subject: 'import x as y' stops the show (was: Re: [Python-Dev] [Patch #101135] 'import x as y' and 'from x import y as z' (fwd)) References: <20000809222749.O266@xs4all.nl> <39925374.59D974FA@nowonder.de> <20000810072419.A17171@xs4all.nl> <39925BC8.17CD051@nowonder.de> <20000810075510.B17171@xs4all.nl> <39926365.909B2835@nowonder.de> <20000810082316.C17171@xs4all.nl> Message-ID: <39926D4F.83CAE9C2@nowonder.de> Thomas Wouters wrote: > > On Thu, Aug 10, 2000 at 08:10:13AM +0000, Peter Schneider-Kamp wrote: > > An even better way of doing this were if it would not > > only give you another name but if it would not import > > the original one. In this example our expression > > would import all the symbols from Tkinter but would > > rename createtimerhandle as tk_timer. In this way you > > could still use * if you have a namespace clash. E.g.: > > No, that isn't possible. You can't pass a list of names to 'FROM_IMPORT *' > to omit loading them. (That's also the reason the patch needs a new opcode, > because you can't pass both the name to be imported from a module and the > name it should be stored at, to the FROM_IMPORT bytecode :) Yes, it is possible. But as you correctly point out, not without some serious changes to compile.c and ceval.c. As we both agree (trying to channel you) it is not worth it to make 'from import *' more usable, I think we should stop this discussion before somebody thinks we seriously want to do this. Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From mal@lemburg.com Thu Aug 10 09:36:07 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 10 Aug 2000 10:36:07 +0200 Subject: [Python-Dev] Un-stalling Berkeley DB support References: <20000809140321.A836@thyrsus.com> Message-ID: <39926977.F8495AAD@lemburg.com> "Eric S. Raymond" wrote: > [Berkeley DB 3] > When we last discussed this subject, there was general support for the > functionality, but a couple of people went "bletch!" about SWIG-generated > code (there was unhappiness about pointers being treated as strings). AFAIK, recent versions of SWIG now make proper use of PyCObjects to store pointers. Don't know how well this works though: I've had a report that the new support can cause core dumps. > Somebody said something about having SWIG patches to address this. Is this > the only real issue with SWIG-generated code? If so, we can pursue two paths: > (1) Hand Greg a patched SWIG so he can release a 2.1.2 version of the DB > extension that meets our cleanliness criteria, and (2) press the SWIG guy > to incorporate these patches in his next release. Perhaps these patches are what I was talking about above ?! -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From sjoerd@oratrix.nl Thu Aug 10 11:59:06 2000 From: sjoerd@oratrix.nl (Sjoerd Mullender) Date: Thu, 10 Aug 2000 12:59:06 +0200 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib mailbox.py,1.20,1.21 In-Reply-To: Your message of Wed, 09 Aug 2000 20:05:30 -0700. <200008100305.UAA05018@slayer.i.sourceforge.net> References: <200008100305.UAA05018@slayer.i.sourceforge.net> Message-ID: <20000810105907.713B331047C@bireme.oratrix.nl> On Wed, Aug 9 2000 Guido van Rossum wrote: > files = os.listdir(self.dirname) > ! list = [] > for f in files: > if pat.match(f): > ! list.append(f) > ! list = map(long, list) > ! list.sort() Isn't this just: list = os.listdir(self.dirname) list = filter(pat.match, list) list = map(long, list) list.sort() Or even shorter: list = map(long, filter(pat.match, os.listdir(self.dirname))) list.sort() (Although I can and do see the advantage of the slightly longer version.) -- Sjoerd Mullender From gward@mems-exchange.org Thu Aug 10 13:38:02 2000 From: gward@mems-exchange.org (Greg Ward) Date: Thu, 10 Aug 2000 08:38:02 -0400 Subject: [Python-Dev] Adding library modules to the core Message-ID: <20000810083802.A7912@ludwig.cnri.reston.va.us> [hmmm, this bounced 'cause the root partition on python.org was full... let's try again, shall we?] On 07 August 2000, Eric S. Raymond said: > A few days ago I asked about the procedure for adding a module to the > Python core library. I have a framework class for things like menu systems > and symbolic debuggers I'd like to add. > > Guido asked if this was similar to the TreeWidget class in IDLE. I > investigated and discovered that it is not, and told him so. I am left > with a couple of related questions: Well, I just ploughed through this entire thread, and no one came up with an idea I've been toying with for a while: the Python Advanced Library. This would be the place for well-known, useful, popular, tested, robust, stable, documented module collections that are just too big or too specialized to go in the core. Examples: PIL, mxDateTime, mxTextTools, mxODBC, ExtensionClass, ZODB, and anything else that I use in my daily work and wish that we didn't have maintain separate builds of. ;-) Obviously this would be most useful as an RPM/Debian package/Windows installer/etc., so that non-developers could be told, "You need to install Python 1.6 and the Python Advanced Library 1.0 from ..." and that's *it*. Thoughts? Candidates for admission? Proposed requirements for admission? Greg -- Greg Ward - software developer gward@mems-exchange.org MEMS Exchange / CNRI voice: +1-703-262-5376 Reston, Virginia, USA fax: +1-703-262-5367 From gward@mems-exchange.org Thu Aug 10 14:47:48 2000 From: gward@mems-exchange.org (Greg Ward) Date: Thu, 10 Aug 2000 09:47:48 -0400 Subject: [Python-Dev] Adding library modules to the core In-Reply-To: ; from moshez@math.huji.ac.il on Thu, Aug 10, 2000 at 04:00:51PM +0300 References: <20000810083802.A7912@ludwig.cnri.reston.va.us> Message-ID: <20000810094747.C7912@ludwig.cnri.reston.va.us> [cc'd to python-dev, since I think this belongs out in the open: Moshe, if you really meant to keep this private, go ahead and slap my wrist] On 10 August 2000, Moshe Zadka said: > Greg, this sounds very close to PEP-206. Please let me know if you see > any useful collaboration with it. They're definitely related, and I think we're trying to address the same problem -- but in a different way. If I read the PEP (http://python.sourceforge.net/peps/pep-0206.html) correctly, you want to fatten the standard Python distribution considerably, first by adding lots of third-party C libraries to it, and second by adding lots of third-party Python libraries ("module distributions") to it. This has the advantage of making all these goodies immediately available in a typical Python installation. But it has a couple of serious disadvantages: * makes Python even harder to build and install; why should I have to build half a dozen major C libraries just to get a basic Python installation working? * all these libraries are redundant on modern free Unices -- at least the Linux distributions that I have experience with all include zlib, Tcl/Tk, libjpeg, and ncurses out of the box. Including copies of them with throws out one of the advantages of having all these installed as shared libraries, namely that there only has to be one copy of each in memory. * tell me again: what was the point of the Distutils if we just throw "everything useful" into the standard distribution? Anyways, my idea -- the Python Advanced Library -- is to make all of these goodies available as a single download, *separate* from Python itself. It could well be at the the Advanced Library would be larger than the Python distribution. (Especially if Tcl/Tk migrates from the standard Windows installer to the Advanced Library.) Advantages: * keeps the standard distribution relatively small and focussed; IMHO the "big framework" libraries (PIL, NumPy, etc.) don't belong in the standard library. (I think there could someday be a strong case for moving Tkinter out of the standard library if the Advanced Library really takes off.) * relieves licensing problems in the Python distribution; if something can't be included with Python for licence reasons, then put it in the Advanced Library * can have variations on the PAL for different platforms. Eg. could have an RPM or Debian package that just requires libjpeg, libncurses, libtcl, libtk etc. for the various Linuces, and an enormous installer with separate of absolutely everything for Windows * excellent test case for the Distutils ;-) * great acronym: the Python Advanced Library is your PAL. Sounds worth a PEP to me; I think it should be distinct from (and in competition with!) PEP 206. Greg -- Greg Ward - software developer gward@mems-exchange.org MEMS Exchange / CNRI voice: +1-703-262-5376 Reston, Virginia, USA fax: +1-703-262-5367 From Moshe Zadka Thu Aug 10 15:09:23 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 10 Aug 2000 17:09:23 +0300 (IDT) Subject: [Python-Dev] Adding library modules to the core In-Reply-To: <20000810094747.C7912@ludwig.cnri.reston.va.us> Message-ID: On Thu, 10 Aug 2000, Greg Ward wrote: > Sounds worth a PEP to me; I think it should be distinct from (and in > competition with!) PEP 206. That's sort of why I wanted to keep this off Python-Dev: I don't think so (I don't really want competing PEPs), I'd rather we hashed out our differences in private and come up with a unified PEP to save everyone on Python-Dev a lot of time. So let's keep the conversation off python-dev until we either reach a consensus or agree to disagree. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From mal@lemburg.com Thu Aug 10 15:28:34 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 10 Aug 2000 16:28:34 +0200 Subject: [Python-Dev] Adding library modules to the core References: Message-ID: <3992BC12.BFA16AAC@lemburg.com> Moshe Zadka wrote: > > On Thu, 10 Aug 2000, Greg Ward wrote: > > > Sounds worth a PEP to me; I think it should be distinct from (and in > > competition with!) PEP 206. > > That's sort of why I wanted to keep this off Python-Dev: I don't think > so (I don't really want competing PEPs), I'd rather we hashed out our > differences in private and come up with a unified PEP to save everyone > on Python-Dev a lot of time. > > So let's keep the conversation off python-dev until we either reach > a consensus or agree to disagree. Just a side note: As I recall Guido is not willing to include all these third party tools to the core distribution, but rather to a SUMO Python distribution, which then includes Python + all those nice goodies available to the Python Community. Maintaining this SUMO distribution should, IMHO, be left to a commercial entity like e.g. ActiveState or BeOpen to insure quality and robustness -- this is not an easy job, believe me. I've tried something like this before: it was called Python PowerTools and should still be available at: http://starship.python.net/crew/lemburg/PowerTools-0.2.zip I never got far, though, due to the complexity of getting all that Good Stuff under one umbrella. Perhaps you ought to retarget you PEP206, Moshe ?! -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From Moshe Zadka Thu Aug 10 15:30:40 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 10 Aug 2000 17:30:40 +0300 (IDT) Subject: [Python-Dev] Adding library modules to the core In-Reply-To: <3992BC12.BFA16AAC@lemburg.com> Message-ID: On Thu, 10 Aug 2000, M.-A. Lemburg wrote: > Just a side note: As I recall Guido is not willing to include > all these third party tools to the core distribution, but rather > to a SUMO Python distribution, which then includes Python + > all those nice goodies available to the Python Community. Yes, that's correct. > Maintaining this SUMO distribution should, IMHO, be left to > a commercial entity like e.g. ActiveState or BeOpen to insure > quality and robustness -- this is not an easy job, believe me. Well, I'm hoping that distutils will make this easier. > Perhaps you ought to retarget you PEP206, Moshe ?! I'm sorry -- I'm too foolhardy. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From nowonder@nowonder.de Thu Aug 10 18:00:14 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Thu, 10 Aug 2000 17:00:14 +0000 Subject: [Python-Dev] 2nd thought: fully qualified host names Message-ID: <3992DF9E.BF5A080C@nowonder.de> Hi Guido! After submitting the patch to smtplib, I got a bad feeling about only trying to get the FQDN for the localhost case. Shouldn't _get_fdqn_hostname() try to get the FQDN for every argument passed? Currently it does so only for len(name) == 0 I think (but couldn't immediately find a reference) it is required by some RFC. There is at least an internet draft by the the ietf that says it is required and a lot of references (mostly from postfix) to some RFC, too. Of course, automatically trying to get the fully qualified domain name would mean that the programmer looses some flexibility (by loosing responsibility). If that is a problem I would make _get_fqdn_hostname a public function (and choose a better name). helo() and ehlo() could still call it for the local host case. or-should-I-just-leave-things-as-they-are-ly y'rs Peter P.S.: I am cc'ing the list so everyone and Thomas can rush in and provide their RFC knowledge. -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From guido@beopen.com Thu Aug 10 17:14:20 2000 From: guido@beopen.com (Guido van Rossum) Date: Thu, 10 Aug 2000 11:14:20 -0500 Subject: [Python-Dev] 2nd thought: fully qualified host names In-Reply-To: Your message of "Thu, 10 Aug 2000 17:00:14 GMT." <3992DF9E.BF5A080C@nowonder.de> References: <3992DF9E.BF5A080C@nowonder.de> Message-ID: <200008101614.LAA28785@cj20424-a.reston1.va.home.com> > Hi Guido! > > After submitting the patch to smtplib, I got a bad feeling > about only trying to get the FQDN for the localhost case. > > Shouldn't _get_fdqn_hostname() try to get the FQDN > for every argument passed? Currently it does so only > for len(name) == 0 > > I think (but couldn't immediately find a reference) it > is required by some RFC. There is at least an internet > draft by the the ietf that says it is required > and a lot of references (mostly from postfix) to some > RFC, too. > > Of course, automatically trying to get the fully > qualified domain name would mean that the programmer > looses some flexibility (by loosing responsibility). > > If that is a problem I would make _get_fqdn_hostname > a public function (and choose a better name). helo() > and ehlo() could still call it for the local host case. > > or-should-I-just-leave-things-as-they-are-ly y'rs > Peter > > P.S.: I am cc'ing the list so everyone and Thomas can > rush in and provide their RFC knowledge. Good idea -- I don't know anything about SMTP! --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From thomas@xs4all.net Thu Aug 10 16:40:26 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 10 Aug 2000 17:40:26 +0200 Subject: [Python-Dev] 2nd thought: fully qualified host names In-Reply-To: <200008101614.LAA28785@cj20424-a.reston1.va.home.com>; from guido@beopen.com on Thu, Aug 10, 2000 at 11:14:20AM -0500 References: <3992DF9E.BF5A080C@nowonder.de> <200008101614.LAA28785@cj20424-a.reston1.va.home.com> Message-ID: <20000810174026.D17171@xs4all.nl> On Thu, Aug 10, 2000 at 11:14:20AM -0500, Guido van Rossum wrote: > > After submitting the patch to smtplib, I got a bad feeling > > about only trying to get the FQDN for the localhost case. > > for len(name) == 0 > > I think (but couldn't immediately find a reference) it > > is required by some RFC. There is at least an internet > > draft by the the ietf that says it is required > > and a lot of references (mostly from postfix) to some > > RFC, too. If this is for helo() and ehlo(), screw it. No sane mailer, technician or abuse desk employee pays any attention what so ever to the HELO message, except possibly for debugging. The only use I've ever had for the HELO message is with clients that setup a WinGate or similar braindead port-forwarding service on their dail-in machine, and then buy one of our products, batched-SMTP. They then get their mail passed to them via SMTP when they dial in... except that these *cough*users*cough* redirect their SMTP port to *our* smtp server, creating a confusing mail loop. We first noticed that because their server connected to our server using *our* HELO message ;) > > If that is a problem I would make _get_fqdn_hostname > > a public function (and choose a better name). helo() > > and ehlo() could still call it for the local host case. I don't think this is worth the trouble. Assembling a FQDN is tricky at best, and it's not needed in that many cases. (Sometimes you can break something by trying to FQDN a name and getting it wrong ;) Where would this function be used ? In SMTP chats ? Not necessary. A 'best guess' is enough -- the remote SMTP server won't listen to you, anyway, and provide the ipaddress and it's reverse DNS entry in the mail logs. Mailers that rely on the HELO message are (rightly!) considered insecure, spam-magnets, and are a thankfully dying race. Of course, if anyone else needs a FQDN, it might be worth exposing this algorithm.... but smtplib doesn't seem like the proper place ;P -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From nowonder@nowonder.de Thu Aug 10 19:13:04 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Thu, 10 Aug 2000 18:13:04 +0000 Subject: [Python-Dev] open 'Accepted' patches Message-ID: <3992F0B0.C8CBF85B@nowonder.de> Changing the patch view at sf to 'Accepted' in order to find my patch, I was surprised by the amount of patches that have been accepted and are still lying around. In an insane attack of self-destructiveness I decided to bring up the issue . I know there can be a lot of issues with patches relative to another patch etc., but letting them rot won't improve the situation. "Checked in they should be." If there are still problems with them or the have already been checked in, the status should at least be 'Postponed', 'Out of Date', 'Rejected', 'Open' or 'Closed'. Here is a list of the open 'Accepted' patches that have had no comment for more than a week and which are not obviously checked in yet (those that are, I have closed): patch# | summary | last comment -------+-------------------------------------+-------------- 100510 | largefile support for Win64 (and...)| 2000-Jul-31 100511 | test largefile support (test_lar...)| 2000-Jul-31 100851 | traceback.py, with unicode except...| 2000-Aug-01 100874 | Better error message with Unbound...| 2000-Jul-26 100955 | ptags, eptags: regex->re, 4 char ...| 2000-Jul-26 100978 | Minor updates for BeOS R5 | 2000-Jul-25 100994 | Allow JPython to use more tests | 2000-Jul-27 If I should review, adapt and/or check in some of these, please tell me which ones. Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From thomas@xs4all.net Thu Aug 10 17:30:10 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 10 Aug 2000 18:30:10 +0200 Subject: [Python-Dev] 2nd thought: fully qualified host names In-Reply-To: <200008101614.LAA28785@cj20424-a.reston1.va.home.com>; from guido@beopen.com on Thu, Aug 10, 2000 at 11:14:20AM -0500 References: <3992DF9E.BF5A080C@nowonder.de> <200008101614.LAA28785@cj20424-a.reston1.va.home.com> Message-ID: <20000810183010.E17171@xs4all.nl> On Thu, Aug 10, 2000 at 11:14:20AM -0500, Guido van Rossum wrote: > > P.S.: I am cc'ing the list so everyone and Thomas can > > rush in and provide their RFC knowledge. Oh, I forgot to point out: I have some RFC knowledge, but decided not to use it in the case of the HELO message ;) I do have a lot of hands-on experience with SMTP, and I know for a fact very little MUA that talk SMTP send a FQDN in the HELO message. I think that sending the FQDN when we can (like we do, now) is a good idea, but I don't see a reason to force the HELO message to be a FQDN. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From Moshe Zadka Thu Aug 10 17:43:41 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 10 Aug 2000 19:43:41 +0300 (IDT) Subject: [Python-Dev] open 'Accepted' patches In-Reply-To: <3992F0B0.C8CBF85B@nowonder.de> Message-ID: (Meta: seems every now and again, a developer has a fit of neurosa. I think this is a good thing) On Thu, 10 Aug 2000, Peter Schneider-Kamp wrote: > patch# | summary | last comment > -------+-------------------------------------+-------------- ... > 100955 | ptags, eptags: regex->re, 4 char ...| 2000-Jul-26 This is the only one I actually know about: Jeremy, Guido has approved it, I assigned it to you for final eyeballing -- shouldn't be *too* hard to check it in... -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From DavidA@ActiveState.com Thu Aug 10 17:47:54 2000 From: DavidA@ActiveState.com (David Ascher) Date: Thu, 10 Aug 2000 09:47:54 -0700 (Pacific Daylight Time) Subject: [Python-Dev] FYI: Software Carpentry winners announced Message-ID: I wanted to make sure that everyone here knew that the Software Carpentry winners were announced, and that our very own Ping won in the Track category. Winners in the Config and Build category were Linday Todd (SapCat) and Steven Knight (sccons) respectively. Congrats to all. --david http://software-carpentry.codesourcery.com/entries/second-round/results.html From trentm@ActiveState.com Thu Aug 10 17:50:15 2000 From: trentm@ActiveState.com (Trent Mick) Date: Thu, 10 Aug 2000 09:50:15 -0700 Subject: [Python-Dev] open 'Accepted' patches In-Reply-To: <3992F0B0.C8CBF85B@nowonder.de>; from nowonder@nowonder.de on Thu, Aug 10, 2000 at 06:13:04PM +0000 References: <3992F0B0.C8CBF85B@nowonder.de> Message-ID: <20000810095015.A28562@ActiveState.com> On Thu, Aug 10, 2000 at 06:13:04PM +0000, Peter Schneider-Kamp wrote: > > Here is a list of the open 'Accepted' patches that have had > no comment for more than a week and which are not obviously > checked in yet (those that are, I have closed): > > patch# | summary | last comment > -------+-------------------------------------+-------------- > 100510 | largefile support for Win64 (and...)| 2000-Jul-31 > 100511 | test largefile support (test_lar...)| 2000-Jul-31 These two are mine. For a while I just thought that they had been checked in. Guido poked me to check them in a week or so ago and I will this week. Trent -- Trent Mick TrentM@ActiveState.com From nowonder@nowonder.de Fri Aug 11 00:29:28 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Thu, 10 Aug 2000 23:29:28 +0000 Subject: [Python-Dev] 2nd thought: fully qualified host names References: <3992DF9E.BF5A080C@nowonder.de> <200008101614.LAA28785@cj20424-a.reston1.va.home.com> <20000810174026.D17171@xs4all.nl> Message-ID: <39933AD8.B8EF5D59@nowonder.de> Thomas Wouters wrote: > > If this is for helo() and ehlo(), screw it. No sane mailer, technician or > abuse desk employee pays any attention what so ever to the HELO message, > except possibly for debugging. Well, there are some MTAs (like Postfix) that seem to care. Postfix has an option called "reject_non_fqdn_hostname" with the following description: """ Reject the request when the hostname in the client HELO (EHLO) command is not in fully-qualified domain form, as required by the RFC. The non_fqdn_reject_code specifies the response code to rejected requests (default: 504).""" The submittor of the bug which was addressed by the patch I checked in had a problem with mailman and a postfix program that seemed to have this option turned on. What I am proposing for smtplib is to send every name given to helo (or ehlo) through the guessing framework of gethostbyaddr() if possible. Could this hurt anything? > Of course, if anyone else needs a FQDN, it might be worth exposing this > algorithm.... but smtplib doesn't seem like the proper place ;P Agreed. Where could it go? Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From nowonder@nowonder.de Fri Aug 11 00:34:38 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Thu, 10 Aug 2000 23:34:38 +0000 Subject: [Python-Dev] 2nd thought: fully qualified host names References: <3992DF9E.BF5A080C@nowonder.de> <200008101614.LAA28785@cj20424-a.reston1.va.home.com> <20000810183010.E17171@xs4all.nl> Message-ID: <39933C0E.7A84D6E2@nowonder.de> Thomas Wouters wrote: > > Oh, I forgot to point out: I have some RFC knowledge, but decided not to use > it in the case of the HELO message ;) I do have a lot of hands-on experience > with SMTP, and I know for a fact very little MUA that talk SMTP send a FQDN > in the HELO message. I think that sending the FQDN when we can (like we do, > now) is a good idea, but I don't see a reason to force the HELO message to > be a FQDN. I don't want to force anything. I think it's time for some code to speak for itself, rather than me trying to speak for it <0.8 wink>: def _get_fqdn_hostname(name): name = string.strip(name) if len(name) == 0: name = socket.gethostname() try: hostname, aliases, ipaddrs = socket.gethostbyaddr(name) except socket.error: pass else: aliases.insert(0, hostname) for name in aliases: if '.' in name: break else: name = hostname return name This is the same function as the one I checked into smtplib.py with the exception of executing the try-block also for names with len(name) != 0. Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From bckfnn@worldonline.dk Thu Aug 10 23:17:47 2000 From: bckfnn@worldonline.dk (Finn Bock) Date: Thu, 10 Aug 2000 22:17:47 GMT Subject: [Python-Dev] Freezing unicode codecs. Message-ID: <3993287a.1852013@smtp.worldonline.dk> While porting the unicode API and the encoding modules to JPython I came across a problem which may also (or maybe not) exists in CPython. jpythonc is a compiler for jpython which try to track dependencies between modules in an attempt to detect which modules an application or applet uses. I have the impression that some of the freeze tools for CPython does something similar. A call to unicode("abc", "cp1250") and "abc".encode("cp1250") will cause the encoding.cp1250 module to be loaded as a side effect. The freeze tools will have a hard time figuring this out by scanning the python source. For JPython I'm leaning towards making it a requirement that the encodings must be loading explicit from somewhere in application. Adding import encoding.cp1250 somewhere in the application will allow jpythonc to include this python module in the frozen application. How does CPython solve this? PS. The latest release of the JPython errata have full unicode support and includes the "sre" module and unicode codecs. http://sourceforge.net/project/filelist.php?group_id=1842 regards, finn From thomas@xs4all.net Thu Aug 10 23:50:13 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 11 Aug 2000 00:50:13 +0200 Subject: [Python-Dev] 2nd thought: fully qualified host names In-Reply-To: <39933AD8.B8EF5D59@nowonder.de>; from nowonder@nowonder.de on Thu, Aug 10, 2000 at 11:29:28PM +0000 References: <3992DF9E.BF5A080C@nowonder.de> <200008101614.LAA28785@cj20424-a.reston1.va.home.com> <20000810174026.D17171@xs4all.nl> <39933AD8.B8EF5D59@nowonder.de> Message-ID: <20000811005013.F17171@xs4all.nl> On Thu, Aug 10, 2000 at 11:29:28PM +0000, Peter Schneider-Kamp wrote: > Thomas Wouters wrote: > > If this is for helo() and ehlo(), screw it. No sane mailer, technician or > > abuse desk employee pays any attention what so ever to the HELO message, > > except possibly for debugging. > Well, there are some MTAs (like Postfix) that seem to care. Postfix has > an option called "reject_non_fqdn_hostname" with the following description: > """ > Reject the request when the hostname in the client HELO (EHLO) command is not in > fully-qualified domain form, as required by the RFC. The non_fqdn_reject_code > specifies the response code to rejected requests (default: 504).""" > The submittor of the bug which was addressed by the patch I checked in had > a problem with mailman and a postfix program that seemed to have this option > turned on. Fine, the patch addresses that. When the hostname passed to smtplib is "" (which is the default), it should be turned into a FQDN. I agree. However, if someone passed in a name, we do not know if they even *want* the name turned into a FQDN. In the face of ambiguity, refuse the temptation to guess. Turning on this Postfix feature (which is completely along the lines of Postfix, and I applaud Wietse(*) for supplying it ;) is a tricky decision at best. Like I said in the other email, there are a *lot* of MUAs and MTAs and other throw-away-programs-gone-commercial that don't speak proper SMTP, and don't even pretend to send a FQDN. Most Windows clients send the machine's netbios name, for crying out loud. Turning this on would break all those clients, and more. I'm not too worried about it breaking Python scripts that are explicitly setting the HELO response -- those scripts are probably doing it for a reason. To note, I haven't seen software that uses smtplib that does supply their own HELO message, except for a little script I saw that was *explicitly* setting the HELO message in order to test the SMTP server on the other end. That instance would certainly have been broken by rewriting the name into a FQDN. > > Of course, if anyone else needs a FQDN, it might be worth exposing this > > algorithm.... but smtplib doesn't seem like the proper place ;P > Agreed. Where could it go? On second though, I can't imagine anyone needing such a function outside of smtplib. FQDN's are nice for reporting URIs to the outside world, but for connecting to a certain service you simply pass the hostname you got (which can be an ipaddress) through to the OS-provided network layer. Kind of like not doing type checking on the objects passed to your function, but instead assuming it conforms to an interface and will work correctly or fail obviously when attempted to be used as an object of a certain type. So, make it an exposed function on smtplib, for those people who don't want to set the HELO message to "", but do want it to be rewritten into a FQDN. (*) Amazing how all good software came to be through Dutch people. Even Linux: if it wasn't for Tanenbaum, it wouldn't be what it is today :-) PS: I'm talking as a sysadmin for a large ISP here, not as a user-friendly library-implementor. We won't be able to turn on this postfix feature for many, many years, and I wouldn't advise anyone who expects mail to be sent from the internet to a postfix machine to enable it, either. But if your mailserver is internal-only, or with fixed entrypoints that are running reliable software, I can imagine people turning it on. It would please me no end if we could turn this on ! I spend on average an hour a day closing customer-accounts and helping them find out why their mailserver sucks. And I still discover new mailserver software and new ways for them to suck, it's really amazing ;) that-PS-was-a-bit-long-for-a-signoff-ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From tim_one@email.msn.com Fri Aug 11 01:44:06 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 10 Aug 2000 20:44:06 -0400 Subject: Keyword abuse (was RE: [Python-Dev] Lockstep iteration - eureka!) In-Reply-To: <14737.35195.31385.867664@beluga.mojam.com> Message-ID: [Skip Montanaro] > Could this be extended to many/most/all current instances of > keywords in Python? As Tim pointed out, Fortran has no keywords. > It annoys me that I (for example) can't define a method named "print". This wasn't accidental in Fortran, though: X3J3 spent many tedious hours fiddling the grammar to guarantee it was always possible. Python wasn't designed with this in mind, and e.g. there's no meaningful way to figure out whether raise is an expression or a "raise stmt" in the absence of keywords. Fortran is very careful to make sure such ambiguities can't arise. A *reasonable* thing is to restrict global keywords to special tokens that can begin a line. There's real human and machine parsing value in being able to figure out what *kind* of stmt a line represents from its first token. So letting "print" be a variable name too would, IMO, really suck. But after that, I don't think users have any problem understanding that different stmt types can have different syntax. For example, if "@" has a special meaning in "print" statments, big deal. Nobody splits a spleen over seeing a b, c, d when "a" happens to be "exec" or "print" today, despite that most stmts don't allow that syntax, and even between "exec" and "print" it has very different meanings. Toss in "global", "del" and "import" too for other twists on what the "b, c, d" part can look like and mean. As far as I'm concerned, each stmt type can have any syntax it damn well likes! Identifiers with special meaning *after* a keyword-introduced stmt can usually be anything at all without making them global keywords (be it "as" after "import", or "indexing" after "for", or ...). The only thing Python is missing then is a lisp stmt : lisp (setq a (+ a 1)) Other than that, the JPython hack looks cool too. Note that SSKs (stmt-specific keywords) present a new problem to colorizers (or moral equivalents like font-lock), and to other tools that do more than a trivial parse. the-road-to-p3k-has-toll-booths-ly y'rs - tim From tim_one@email.msn.com Fri Aug 11 01:44:08 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 10 Aug 2000 20:44:08 -0400 Subject: PEP praise (was RE: [Python-Dev] Lockstep iteration - eureka!) In-Reply-To: Message-ID: [Ka-Ping Yee] > ... > Surely a PEP isn't required for a couple of built-in functions that > are simple and well understood? You can just call thumbs-up or > thumbs-down and be done with it. Only half of that is true, and even then only partially: if the verdict is thumbs-up, *almost* cool, except that newcomers delight in pestering "but how come it wasn't done *my* way instead?". You did a bit of that yourself in your day, you know . We're hoping the stream of newcomers never ends, but the group of old-timers willing and able to take an hour or two to explain the past in detail is actually dwindling (heck, you can count the Python-Dev members chipping in on Python-Help with a couple of fingers, and if anything fewer still active on c.l.py). If it's thumbs-down, in the absence of a PEP it's much worse: it will just come back again, and again, and again, and again. The sheer repetition in these endlessly recycled arguments all but guarantees that most old-timers ignore these threads completely. A prime purpose of the PEPs is to be the community's collective memory, pro or con, so I don't have to be . You surely can't believe this is the first time these particular functions have been pushed for core adoption!? If not, why do we need to have the same arguments all over again? It's not because we're assholes, and neither because there's anything truly new here, it's simply because a mailing list has no coherent memory. Not so much as a comma gets changed in an ANSI or ISO std without an elaborate pile of proposal paperwork and formal reviews. PEPs are a very lightweight mechanism compared to that. And it would take you less time to write a PEP for this than I alone spent reading the 21 msgs waiting for me in this thread today. Multiply the savings by billions . world-domination-has-some-scary-aspects-ly y'rs - tim From Vladimir.Marangozov@inrialpes.fr Fri Aug 11 02:59:30 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 11 Aug 2000 03:59:30 +0200 (CEST) Subject: [Python-Dev] Preventing recursion core dumps Message-ID: <200008110159.DAA09540@python.inrialpes.fr> I'm looking at preventing core dumps due to recursive calls. With simple nested call counters for every function in object.c, limited to 500 levels deep recursions, I think this works okay for repr, str and print. It solves most of the complaints, like: class Crasher: def __str__(self): print self print Crasher() With such protection, instead of a core dump, we'll get an exception: RuntimeError: Recursion too deep So far, so good. 500 nested calls to repr, str or print are likely to be programming bugs. Now I wonder whether it's a good idea to do the same thing for getattr and setattr, to avoid crashes like: class Crasher: def __getattr__(self, x): return self.x Crasher().bonk Solving this the same way is likely to slow things down a bit, but would prevent the crash. OTOH, in a complex object hierarchy with tons of delegation and/or lookup dispatching, 500 nested calls is probably not enough. Or am I wondering too much? Opinions? -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From bwarsaw@beopen.com Fri Aug 11 04:00:32 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Thu, 10 Aug 2000 23:00:32 -0400 (EDT) Subject: [Python-Dev] Python keywords (was Lockstep iteration - eureka!) References: <200008100036.TAA26235@cj20424-a.reston1.va.home.com> Message-ID: <14739.27728.960099.342321@anthem.concentric.net> >>>>> "GvR" == Guido van Rossum writes: GvR> Alas, I'm not sure how easy it will be. The parser generator GvR> will probably have to be changed to allow you to indicate not GvR> to do a resword lookup at certain points in the grammar. I GvR> don't know where to start. :-( Yet another reason why it would be nice to (eventually) merge the parsing technology in CPython and JPython. i-don't-wanna-work-i-jes-wanna-bang-on-my-drum-all-day-ly y'rs, -Barry From MarkH@ActiveState.com Fri Aug 11 07:15:00 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Fri, 11 Aug 2000 16:15:00 +1000 Subject: [Python-Dev] Patches and checkins for 1.6 Message-ID: I would like a little guidance on how to handle patches during this 1.6 episode. My understanding of CVS tells me that 1.6 has forked from the main development tree. Any work done in the 1.6 branch will need to also be done in the main branch. Is this correct? If so, it means that all patches assigned to me need to be applied and tested twice, which involves completely refetching the entire tree, and rebuilding the world? Given that 1.6 appears to be mainly an exercise in posturing by CNRI, is it reasonable that I hold some patches off while I'm working with 1.6, and check them in when I move back to the main branch? Surely no one will stick with 1.6 in the long (or even medium) term, once all active development of that code ceases? Of course, this wouldn't include critical bugs, but no one is mad enough to assign them to me anyway Confused-and-in-need-of-a-machine-upgrade ly, Mark. From tim_one@email.msn.com Fri Aug 11 07:48:56 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 11 Aug 2000 02:48:56 -0400 Subject: [Python-Dev] Patches and checkins for 1.6 In-Reply-To: Message-ID: [Mark Hammond] > I would like a little guidance on how to handle patches during this > 1.6 episode. > > My understanding of CVS tells me that 1.6 has forked from the > main development tree. Any work done in the 1.6 branch will need > to also be done in the main branch. Is this correct? Don't look at me -- I first ran screaming in terror from CVS tricks more than a decade ago, and haven't looked back. OTOH, I don't know of *any* work done in the 1.6 branch yet that needs also to be done in the 2.0 branch. Most of what Fred Drake has been doing is in the other direction, and the rest has been fixing buglets unique to 1.6. > If so, it means that all patches assigned to me need to be applied > and tested twice, which involves completely refetching the entire > tree, and rebuilding the world? Patches with new features should *not* go into the 1.6 branch at all! 1.6 is meant to reflect only work that CNRI has clear claims to, plus whatever bugfixes are needed to make that a good release. Actual cash dollars for Unicode development were funneled through CNRI, and that's why the Unicode features are getting backstitched into it. They're unique, though. > Given that 1.6 appears to be mainly an exercise in posturing by > CNRI, Speaking on behalf of BeOpen PythonLabs, 1.6 is a milestone in Python development, worthy of honor, praise and repeated downloading by all. We at BeOpen PythonLabs regret the unfortunate misconceptions that have arisen about its true nature, and fully support CNRI's wise decision to force a release of Python 1.6 in the public interest. > is it reasonable that I hold some patches off while I'm working > with 1.6, and check them in when I move back to the main branch? I really don't know what you're doing. If you find a bug in 1.6 that's also a bug in 2.0, it should go without saying that we'd like that fixed ASAP in 2.0 as well. But since that went without saying, and you seem to be saying something else, I'm not sure what you're asking. If you're asking whether you're allowed to maximize your own efficiency, well, only Guido can force you to do something self-damaging . > Surely no one will stick with 1.6 in the long (or even > medium) term, once all active development of that code ceases? Active development of the 1.6 code has already ceased, far as I can tell. Maybe some more Unicode patches? Other than that, just bugfixes as needed. It's down to a trickle. We're aiming for a quick beta cycle on 1.6b1, and-- last I heard, and barring scads of fresh bug reports --intending to release 1.6 final next. Then bugs opened against 1.6 will be answered by "fixed in 2.0". > Of course, this wouldn't include critical bugs, but no one is mad > enough to assign them to me anyway > > Confused-and-in-need-of-a-machine-upgrade ly, And we'll buy you one, too, if you promise to use it to fix the test_fork1 family of bugs on SMP Linux boxes! don't-forget-that-patches-to-1.6-still-need-cnri-release-forms!- and-that-should-clarify-everything-ly y'rs - tim From gstein@lyra.org Fri Aug 11 08:07:29 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 11 Aug 2000 00:07:29 -0700 Subject: [Python-Dev] Patches and checkins for 1.6 In-Reply-To: ; from MarkH@ActiveState.com on Fri, Aug 11, 2000 at 04:15:00PM +1000 References: Message-ID: <20000811000729.M19525@lyra.org> On Fri, Aug 11, 2000 at 04:15:00PM +1000, Mark Hammond wrote: >... > If so, it means that all patches assigned to me need to be applied and > tested twice, which involves completely refetching the entire tree, and > rebuilding the world? Just fetch two trees. c:\src16 c:\src20 Cheers, -g -- Greg Stein, http://www.lyra.org/ From mal@lemburg.com Fri Aug 11 09:04:48 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 11 Aug 2000 10:04:48 +0200 Subject: [Python-Dev] Freezing unicode codecs. References: <3993287a.1852013@smtp.worldonline.dk> Message-ID: <3993B3A0.28500B22@lemburg.com> Finn Bock wrote: > > While porting the unicode API and the encoding modules to JPython I came > across a problem which may also (or maybe not) exists in CPython. > > jpythonc is a compiler for jpython which try to track dependencies > between modules in an attempt to detect which modules an application or > applet uses. I have the impression that some of the freeze tools for > CPython does something similar. > > A call to unicode("abc", "cp1250") and "abc".encode("cp1250") will cause > the encoding.cp1250 module to be loaded as a side effect. The freeze > tools will have a hard time figuring this out by scanning the python > source. > > For JPython I'm leaning towards making it a requirement that the > encodings must be loading explicit from somewhere in application. Adding > > import encoding.cp1250 > > somewhere in the application will allow jpythonc to include this python > module in the frozen application. > > How does CPython solve this? It doesn't. The design of the codec registry is such that it uses search functions which then locate and load the codecs. These search function can implement whatever scheme they desire for the lookup and also w/r to loading the codec, e.g. they could get the data from a ZIP archive. This design was chosen to allow drop-in configuration of the Python codecs. Applications can easily add new codecs to the registry by registering a new search function (and without having to copy files into the encodings Lib subdir). When it comes to making an application freezable, I'd suggest adding explicit imports to some freeze support module in the application. There are other occasions where this is needed too, e.g. for packages using lazy import of modules such as mx.DateTime. This module would then make sure freeze.py finds the right modules to include in its output. > PS. The latest release of the JPython errata have full unicode support > and includes the "sre" module and unicode codecs. > > http://sourceforge.net/project/filelist.php?group_id=1842 Cool :-) > regards, > finn > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://www.python.org/mailman/listinfo/python-dev -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From nowonder@nowonder.de Fri Aug 11 11:29:04 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Fri, 11 Aug 2000 10:29:04 +0000 Subject: [Python-Dev] xxx.get_fqdn() for the standard lib References: <3992DF9E.BF5A080C@nowonder.de> <200008101614.LAA28785@cj20424-a.reston1.va.home.com> <20000810174026.D17171@xs4all.nl> Message-ID: <3993D570.7578FE71@nowonder.de> After sleeping over it, I noticed that at least BaseHTTPServer and ftplib also use a similar algorithm to get a fully qualified domain name. Together with smtplib there are four occurences of the algorithm (2 in BaseHTTPServer). I think it would be good not to have four, but one implementation. First I thought it could be socket.get_fqdn(), but it seems a bit troublesome to write it in C. Should this go somewhere? If yes, where should it go? I'll happily prepare a patch as soon as I know where to put it. Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From Moshe Zadka Fri Aug 11 09:40:08 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 11 Aug 2000 11:40:08 +0300 (IDT) Subject: [Python-Dev] xxx.get_fqdn() for the standard lib In-Reply-To: <3993D570.7578FE71@nowonder.de> Message-ID: On Fri, 11 Aug 2000, Peter Schneider-Kamp wrote: > First I thought it could be socket.get_fqdn(), > but it seems a bit troublesome to write it in C. > > Should this go somewhere? Yes. We need some OnceAndOnlyOnce mentality here... > If yes, where should > it go? Good question. You'll notice that SimpleHTTPServer imports shutil for copyfileobj, because I had no good answer to a similar question. GS seems to think "put it somewhere" is a good enough answer. I think I might agree. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From barry@scottb.demon.co.uk Fri Aug 11 12:42:11 2000 From: barry@scottb.demon.co.uk (Barry Scott) Date: Fri, 11 Aug 2000 12:42:11 +0100 Subject: [Python-Dev] Preventing recursion core dumps In-Reply-To: <200008110159.DAA09540@python.inrialpes.fr> Message-ID: <000401c00389$2fa577b0$060210ac@private> Why not set a limit in the intepreter? Fixing this for every call in object.c seems a lots of hard work and will always leave holes. For embedding Python being able to control the recursion depth of the intepreter is very useful. I would want to be able to set, from C, the max call depth limit and the current call depth limit. I'd expect Python to set a min call depth limit. BArry > -----Original Message----- > From: python-dev-admin@python.org [mailto:python-dev-admin@python.org]On > Behalf Of Vladimir Marangozov > Sent: 11 August 2000 03:00 > To: Python core developers > Subject: [Python-Dev] Preventing recursion core dumps > > > > I'm looking at preventing core dumps due to recursive calls. With > simple nested call counters for every function in object.c, limited to > 500 levels deep recursions, I think this works okay for repr, str and > print. It solves most of the complaints, like: > > class Crasher: > def __str__(self): print self > > print Crasher() > > With such protection, instead of a core dump, we'll get an exception: > > RuntimeError: Recursion too deep > > > So far, so good. 500 nested calls to repr, str or print are likely > to be programming bugs. Now I wonder whether it's a good idea to do > the same thing for getattr and setattr, to avoid crashes like: > > class Crasher: > def __getattr__(self, x): return self.x > > Crasher().bonk > > Solving this the same way is likely to slow things down a bit, but > would prevent the crash. OTOH, in a complex object hierarchy with > tons of delegation and/or lookup dispatching, 500 nested calls is > probably not enough. Or am I wondering too much? Opinions? > > -- > Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr > http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://www.python.org/mailman/listinfo/python-dev > From guido@beopen.com Fri Aug 11 13:47:09 2000 From: guido@beopen.com (Guido van Rossum) Date: Fri, 11 Aug 2000 07:47:09 -0500 Subject: [Python-Dev] Preventing recursion core dumps In-Reply-To: Your message of "Fri, 11 Aug 2000 03:59:30 +0200." <200008110159.DAA09540@python.inrialpes.fr> References: <200008110159.DAA09540@python.inrialpes.fr> Message-ID: <200008111247.HAA03687@cj20424-a.reston1.va.home.com> > I'm looking at preventing core dumps due to recursive calls. With > simple nested call counters for every function in object.c, limited to > 500 levels deep recursions, I think this works okay for repr, str and > print. It solves most of the complaints, like: > > class Crasher: > def __str__(self): print self > > print Crasher() > > With such protection, instead of a core dump, we'll get an exception: > > RuntimeError: Recursion too deep > > > So far, so good. 500 nested calls to repr, str or print are likely > to be programming bugs. Now I wonder whether it's a good idea to do > the same thing for getattr and setattr, to avoid crashes like: > > class Crasher: > def __getattr__(self, x): return self.x > > Crasher().bonk > > Solving this the same way is likely to slow things down a bit, but > would prevent the crash. OTOH, in a complex object hierarchy with > tons of delegation and/or lookup dispatching, 500 nested calls is > probably not enough. Or am I wondering too much? Opinions? In your examples there's recursive Python code involved. There's *already* a generic recursion check for that, but the limit is too high (the latter example segfaults for me too, while a simple def f(): f() gives a RuntimeError). It seems better to tune the generic check than to special-case str, repr, and getattr. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From guido@beopen.com Fri Aug 11 13:55:29 2000 From: guido@beopen.com (Guido van Rossum) Date: Fri, 11 Aug 2000 07:55:29 -0500 Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of data sent. Message-ID: <200008111255.HAA03735@cj20424-a.reston1.va.home.com> I just noticed this. Is this true? Shouldn't we change send() to raise an error instead of returning a small number? (The number of bytes written can be an attribute of the exception.) Don't look at me for implementing this, sorry, no time... --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) ------- Forwarded Message Date: Thu, 10 Aug 2000 16:39:48 -0700 From: noreply@sourceforge.net To: scott@chronis.pobox.com, 0@delerium.i.sourceforge.net, python-bugs-list@python.org Subject: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of data sent. Bug #111620, was updated on 2000-Aug-10 16:39 Here is a current snapshot of the bug. Project: Python Category: Library Status: Open Resolution: None Bug Group: None Priority: 5 Summary: lots of use of send() without verifying amount of data sent. Details: a quick grep of the standard python library (below) shows that there is lots of unchecked use of the send() function. Every unix system I've every used states that send() returns the number of bytes sent, which can be < length ( ). Using socket.send(s) without verifying that the return value is eq ual to the length of s is careless and can result in loss of data. I just submitted a patch for smtplib's use of send(), have patched a piece of Z ope the same way, and get the feeling that it's becoming standard to call send( ) without checking that the amount of data sent is the intended amount. While this is OK for a qu ick script, I don't feel it's OK for library code or anything that might be use d in production. scott For detailed info, follow this link: http://sourceforge.net/bugs/?func=detailbug&bug_id=111620&group_id=5470 _______________________________________________ Python-bugs-list maillist - Python-bugs-list@python.org http://www.python.org/mailman/listinfo/python-bugs-list ------- End of Forwarded Message From gmcm@hypernet.com Fri Aug 11 13:32:44 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Fri, 11 Aug 2000 08:32:44 -0400 Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of d In-Reply-To: <200008111255.HAA03735@cj20424-a.reston1.va.home.com> Message-ID: <1246125329-123433164@hypernet.com> [bug report] > Details: a quick grep of the standard python library (below) > shows that there is lots of unchecked use of the send() > function. [Guido] > I just noticed this. Is this true? Shouldn't we change send() > to raise an error instead of returning a small number? (The > number of bytes written can be an attribute of the exception.) No way! You'd break 90% of my sockets code! People who don't want to code proper sends / recvs can use that sissy makefile junk. - Gordon From thomas@xs4all.net Fri Aug 11 13:31:43 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 11 Aug 2000 14:31:43 +0200 Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of data sent. In-Reply-To: <200008111255.HAA03735@cj20424-a.reston1.va.home.com>; from guido@beopen.com on Fri, Aug 11, 2000 at 07:55:29AM -0500 References: <200008111255.HAA03735@cj20424-a.reston1.va.home.com> Message-ID: <20000811143143.G17171@xs4all.nl> On Fri, Aug 11, 2000 at 07:55:29AM -0500, Guido van Rossum wrote: > I just noticed this. Is this true? Shouldn't we change send() to > raise an error instead of returning a small number? (The number of > bytes written can be an attribute of the exception.) This would break a lot of code. (probably all that use send, with or without return-code checking.) I would propose a 'send_all' or some such instead, which would keep sending until either a real error occurs, or all data is sent (possibly with a timeout ?). And most uses of send could be replaced by send_all, both in the std. library and in user code. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From Vladimir.Marangozov@inrialpes.fr Fri Aug 11 13:39:36 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 11 Aug 2000 14:39:36 +0200 (CEST) Subject: [Python-Dev] Preventing recursion core dumps In-Reply-To: <200008111247.HAA03687@cj20424-a.reston1.va.home.com> from "Guido van Rossum" at Aug 11, 2000 07:47:09 AM Message-ID: <200008111239.OAA15818@python.inrialpes.fr> Guido van Rossum wrote: > > It seems better to tune the generic check than to special-case str, > repr, and getattr. Right. This would be a step forward, at least for recursive Python code (which is the most common complaint). Reducing the current value by half, i.e. setting MAX_RECURSION_DEPTH = 5000 works for me (Linux & AIX) Agreement on 5000? Doesn't solve the problem for C code (extensions) though... -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From Vladimir.Marangozov@inrialpes.fr Fri Aug 11 14:19:38 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 11 Aug 2000 15:19:38 +0200 (CEST) Subject: [Python-Dev] Preventing recursion core dumps In-Reply-To: <000401c00389$2fa577b0$060210ac@private> from "Barry Scott" at Aug 11, 2000 12:42:11 PM Message-ID: <200008111319.PAA16192@python.inrialpes.fr> Barry Scott wrote: > > Why not set a limit in the intepreter? Fixing this for every call in object.c > seems a lots of hard work and will always leave holes. Indeed. > > For embedding Python being able to control the recursion depth of the > intepreter is very useful. I would want to be able to set, from C, the > max call depth limit and the current call depth limit. Except exporting MAX_RECURSION_DEPTH as a variable (Py_MaxRecursionDepth) I don't see what you mean by current call depth limit. > I'd expect Python to set a min call depth limit. I don't understand this. Could you elaborate? Are you implying the introduction of a public function (ex. Py_SetRecursionDepth) that does some value checks? -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From paul@prescod.net Fri Aug 11 14:19:05 2000 From: paul@prescod.net (Paul Prescod) Date: Fri, 11 Aug 2000 08:19:05 -0500 Subject: [Python-Dev] Lockstep iteration - eureka! References: Your message of "Wed, 09 Aug 2000 02:37:07 MST." Message-ID: <3993FD49.C7E71108@prescod.net> Just van Rossum wrote: > > ... > > for indexing in : > ... Let me throw out another idea. What if sequences just had .items() methods? j=range(0,10) for index, element in j.items(): ... While we wait for the sequence "base class" we could provide helper functions that makes the implementation of both eager and lazy versions easier. -- Paul Prescod - Not encumbered by corporate consensus "I don't want you to describe to me -- not ever -- what you were doing to that poor boy to make him sound like that; but if you ever do it again, please cover his mouth with your hand," Grandmother said. -- John Irving, "A Prayer for Owen Meany" From guido@beopen.com Fri Aug 11 15:19:33 2000 From: guido@beopen.com (Guido van Rossum) Date: Fri, 11 Aug 2000 09:19:33 -0500 Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of data sent. In-Reply-To: Your message of "Fri, 11 Aug 2000 14:31:43 +0200." <20000811143143.G17171@xs4all.nl> References: <200008111255.HAA03735@cj20424-a.reston1.va.home.com> <20000811143143.G17171@xs4all.nl> Message-ID: <200008111419.JAA03948@cj20424-a.reston1.va.home.com> > > I just noticed this. Is this true? Shouldn't we change send() to > > raise an error instead of returning a small number? (The number of > > bytes written can be an attribute of the exception.) > > This would break a lot of code. (probably all that use send, with or without > return-code checking.) I would propose a 'send_all' or some such instead, > which would keep sending until either a real error occurs, or all data is > sent (possibly with a timeout ?). And most uses of send could be replaced by > send_all, both in the std. library and in user code. Really?!?! I just read the man page for send() (Red Hat linux 6.1) and it doesn't mention sending fewer than all bytes at all. In fact, while it says that the return value is the number of bytes sent, it at least *suggests* that it will return an error whenever not everything can be sent -- even in non-blocking mode. Under what circumstances can send() return a smaller number? --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From paul@prescod.net Fri Aug 11 14:25:27 2000 From: paul@prescod.net (Paul Prescod) Date: Fri, 11 Aug 2000 08:25:27 -0500 Subject: [Python-Dev] Winreg update Message-ID: <3993FEC7.4E38B4F1@prescod.net> I am in transit so I don't have time for a lot of back and forth email relating to winreg. It also seems that there are a lot of people (let's call them "back seat coders") who have vague ideas of what they want but don't want to spend a bunch of time in a long discussion about registry arcana. Therefore I am endevouring to make it as easy and fast to contribute to the discussion as possible. I'm doing this through a Python Module Proposal format. This can also serve as the basis of documentation. This is really easy so I want some real feedback this time. Distutils people, this means you! Mark! I would love to hear Bill Tutt, Greg Stein and anyone else who claims some knowledge of Windows! If you're one of the people who has asked for winreg in the core then you should respond. It isn't (IMO) sufficient to put in a hacky API to make your life easier. You need to give something to get something. You want windows registry support in the core -- fine, let's do it properly. Even people with a minimal understanding of the registry should be able to contribute: the registry isn't rocket surgery. I'll include a short primer in this email. All you need to do is read this email and comment on whether you agree with the overall principle and then give your opinion on fifteen possibly controversial issues. The "overall principle" is to steal shamelessly from Microsoft's new C#/VB/OLE/Active-X/CRL API instead of innovating for Python. That allows us to avoid starting the debate from scratch. It also eliminates the feature that Mark complained about (which was a Python-specific innovation). The fifteen issues are mostly extensions to the API to make it easier (convenience extensions) or more powerful (completeness extensions). Many of them are binary: "do this, don't do that." Others are choices: e.g. "Use tuples", "Use lists", "Use an instance". I will try to make sense of the various responses. Some issues will have strong consensus and I'll close those quickly. Others will require more (hopefully not much!) discussion. Windows Registry Primer: ======================== There are things called "keys". They aren't like Python keys so don't think of them that way. Keys have a list of subkeys indexed by name. Keys also have a list of "values". Values have names. Every value has a type. In some type-definition syntax: key is (name: string, subkeys: (string : key), values: (string : value )) value is ( name: string, type: enumeration, data: (depends on enumeration) ) That's the basic model. There are various helper facilities provided by the APIs, but really, the model is as above. ========================================================================= Python Module Proposal Title: Windows registry Version: $Revision: 1.0$ Owner: paul@prescod.net (Paul Prescod) Python-Version: 2.0 Status: Incomplete Overview It is convenient for Windows users to know that a Python module to access the registry is always available whenever Python is installed on Windows. This is especially useful for installation programs. There is a Windows registry module from the win32 extensions to Python. It is based directly on the original Microsoft APIs. This means that there are many backwards compatibility hacks, "reserved" parameters and other legacy features that are not interesting to most Python programmers. Microsoft is moving to a higher level API for languages other than C, as part of Microsoft's Common Runtime Library (CRL) initiative. This newer, higher level API serves as the basis for the module described herein. This higher level API would be implemented in Python and based upon the low-level API. They would not be in competition: a user would choose based on their preferences and needs. Module Exports These are taken directly from the Common Runtime Library: ClassesRoot The Windows Registry base key HKEY_CLASSES_ROOT. CurrentConfig The Windows Registry base key HKEY_CURRENT_CONFIG. CurrentUser The Windows Registry base key HKEY_CURRENT_USER. LocalMachine The Windows Registry base key HKEY_LOCAL_MACHINE. CurrentUser The Windows Registry base key HKEY_CURRENT_USER. DynData The Windows Registry base key HKEY_DYN_DATA. PerformanceData The Windows Registry base key HKEY_PERFORMANCE_DATA. Users The Windows Registry base key HKEY_USERS. RegistryKey Registry key class (important class in module) RegistryKey class Data Members These are taken directly from the Common Runtime Library: Name Retrieves the name of the key. [Issue: full path or just name within parent?] SubKeyCount Retrieves the count of subkeys. ValueCount Retrieves the count of values in the key. RegistryKey Methods These are taken directly from the Common Runtime Library: Close() Closes this key and flushes it to disk if the contents have been modified. CreateSubKey( subkeyname ) Creates a new subkey or opens an existing subkey. [Issue: SubKey_full_path]: Should it be possible to create a subkey deeply: >>> LocalMachine.CreateSubKey( r"foo\bar\baz" ) Presumably the result of this issue would also apply to every other method that takes a subkey parameter. It is not clear what the CRL API says yet (Mark?). If it says "yes" then we would follow it of course. If it says "no" then we could still consider the feature as an extension. [Yes] allow subkey parameters to be full paths [No] require them to be a single alphanumeric name, no slashes DeleteSubKey( subkeyname ) Deletes the specified subkey. To delete subkeys and all their children (recursively), use DeleteSubKeyTree. DeleteSubKeyTree( subkeyname ) Recursively deletes a subkey and any child subkeys. DeleteValue( valuename ) Deletes the specified value from this key. __cmp__( other ) Determines whether the specified key is the same key as the current key. GetSubKeyNames() Retrieves an array of strings containing all the subkey names. GetValue( valuename ) Retrieves the specified value. Registry types are converted according to the following table: REG_NONE: None REG_SZ: UnicodeType REG_MULTI_SZ: [UnicodeType, UnicodeType, ...] REG_DWORD: IntegerType REG_DWORD_LITTLE_ENDIAN: IntegerType REG_DWORD_BIG_ENDIAN: IntegerType REG_EXPAND_SZ: Same as REG_SZ REG_RESOURCE_LIST: Same as REG_BINARY REG_FULL_RESOURCE_DESCRIPTOR: Same as REG_BINARY REG_RESOURCE_REQUIREMENTS_LIST: Same as REG_BINARY REG_LINK: Same as REG_BINARY??? [Issue: more info needed!] REG_BINARY: StringType or array.array( 'c' ) [Issue: REG_BINARY Representation]: How should binary data be represented as Python data? [String] The win32 module uses "string". [Array] I propose that an array of bytes would be better. One benefit of "binary" is that allows SetValue to detect string data as REG_SZ and array.array('c') as REG_BINARY [Issue: Type getting method] Should there be a companion method called GetType that fetches the type of a registry value? Otherwise client code would not be able to distinguish between (e.g.) REG_SZ and REG_SZ_BINARY. [Yes] Add GetType( string ) [No] Do not add GetType GetValueNames() Retrieves a list of strings containing all the value names. OpenRemoteBaseKey( machinename, name ) Opens a new RegistryKey that represents the requested key on a foreign machine. OpenSubKey( subkeyname ) Retrieves a subkey. SetValue( keyname, value ) Sets the specified value Types are automatically mapped according to the following algorithm: None: REG_NONE String: REG_SZ UnicodeType: REG_SZ [UnicodeType, UnicodeType, ...]: REG_MULTI_SZ [StringType, StringType, ...]: REG_MULTI_SZ IntegerType: REG_DWORD array.array('c'): REG_BINARY [Issue: OptionalTypeParameter] Should there be an optional parameter that allows you to specify the type explicitly? Presume that the types are constants in the winreg modules (perhaps strings or integers). [Yes] Allow other types to be specified [No] People who want more control should use the underlying win32 module. Proposed Extensions The API above is a direct transliteration of the .NET API. It is somewhat underpowered in some senses and also is not entirely Pythonic. It is a good start as a basis for consensus, however, and these proposed extensions can be voted up or down individually. Two extensions are just the convenience functions (OpenRemoteKey and the top-level functions). Other extensions attempt to extend the API to support ALL features of the underlying API so that users never have to switch from one API to another to get a particular feature. Convenience Extension: OpenRemoteKey It is not clear to me why Microsoft restricts remote key opening to base keys. Why does it not allow a full path like this: >>> winreg.OpenRemoteKey( "machinename", r"HKEY_LOCAL_MACHINE\SOFTWARE\Python" ) [Issue: Add_OpenRemoteKey]: [Yes] add RemoteKey [No] do not add? [Issue: Remove_OpenRemoteBaseKey] [Remove] It's redundant! [Retain] For backwards compatibility Convenience Extension: Top-level Functions A huge number of registry-manipulating programs treat the registry namespace as "flat" and go directly to the interesting registry key. These top-level functions allow the Python user to skip all of the OO key object and get directly to what they want: key=OpenKey( keypath, machinename=None ) key=CreateKey( keypath, machinename=None ) DeleteKey( keypath, machinename=None ) val=GetValue( keypath, valname, machinename=None ) SetValue( keypath, valname, valdata, machinename=None ) [Yes] Add these functions [No] Do not add [Variant] I like the idea but would change the function signatures Completeness Extension: Type names If the type extensions are added to SetValue and GetValue then we need to decide how to represent types. It is fairly clear that they should be represented as constants in the module. The names of those constants could be the cryptic (but standard) Microsoft names or more descriptive, conventional names. Microsoft Names: REG_NONE REG_SZ REG_EXPAND_SZ REG_BINARY REG_DWORD REG_DWORD_LITTLE_ENDIAN REG_DWORD_BIG_ENDIAN REG_LINK REG_MULTI_SZ REG_RESOURCE_LIST REG_FULL_RESOURCE_DESCRIPTOR REG_RESOURCE_REQUIREMENTS_LIST Proposed Descriptive Names: NONE STRING EXPANDABLE_TEMPLATE_STRING BINARY_DATA INTEGER LITTLE_ENDIAN_INTEGER BIG_ENDIAN_INTEGER LINK STRING_LIST RESOURCE_LIST FULL_RESOURCE_DESCRIPTOR RESOURCE_REQUIREMENTS_LIST We could also allow both. One set would be aliases for the other. [Issue: TypeNames]: [MS Names]: Use the Microsoft names [Descriptive Names]: Use the more descriptive names [Both]: Use both Completeness Extension: Type representation No matter what the types are called, they must have values. The simplest thing would be to use the integers provided by the Microsoft header files. Unfortunately integers are not at all self-describing so getting from the integer value to something human readable requires some sort of switch statement or mapping. An alternative is to use strings and map them internally to the Microsoft integer constants. A third option is to use object instances. These instances would be useful for introspection and would have the following attributes: msname (e.g. REG_SZ) friendlyname (e.g. String) msinteger (e.g. 6 ) They would have only the following method: def __repr__( self ): "Return a useful representation of the type object" return " " % \ (self.msinteger, self.msname, self.friendlyname ) A final option is a tuple with the three attributes described above. [Issue: Type_Representation]: [Integers]: Use Microsoft integers [Strings]: Use string names [Instances]: Use object instances with three introspective attributes [Tuples]: Use 3-tuples Completeness Extension: Type Namespace Should the types be declared in the top level of the module (and thus show up in a "dir" or "from winreg import *") or should they live in their own dictionary, perhaps called "types" or "regtypes". They could also be attributes of some instance. [Issue: Type_Namespace]: [Module]: winreg.REG_SZ [Dictionary]: winreg.types["REG_SZ"] [Instance]: winreg.types["REG_SZ"] Completeness Extension: Saving/Loading Keys The underlying win32 registry API allows the loading and saving of keys to filenames. Therefore these could be implemented easily as methods: def save( self, filename ): "Save a key to a filename" _winreg.SaveKey( self.keyobj, filename ) def load( self, subkey, filename ): "Load a key from a filename" return _winreg.RegLoadKey( self.handle, subkey, filename ) >>> key.OpenSubKey("Python").save( "Python.reg" ) >>> key.load( "Python", "Python.reg" ) [Issue: Save_Load_Keys] [Yes] Support the saving and loading of keys [No] Do not add these methods Completeness Extension: Security Access Flags The underlying win32 registry API allows security flags to be applied to the OpenKey method. The flags are: "KEY_ALL_ACCESS" "KEY_CREATE_LINK" "KEY_CREATE_SUB_KEY" "KEY_ENUMERATE_SUB_KEYS" "KEY_EXECUTE" "KEY_NOTIFY" "KEY_QUERY_VALUE" "KEY_READ" "KEY_SET_VALUE" These are not documented in the underlying API but should be for this API. This documentation would be derived from the Microsoft documentation. They would be represented as integer or string constants in the Python API and used something like this: key=key.OpenKey( subkeyname, winreg.KEY_READ ) [Issue: Security_Access_Flags] [Yes] Allow the specification of security access flags. [No] Do not allow this specification. [Issue: Security_Access_Flags_Representation] [Integer] Use the Microsoft integers [String] Use string values [Tuples] Use (string, integer) tuples [Instances] Use instances with "name", "msinteger" attributes [Issue: Security_Access_Flags_Location] [Top-Level] winreg.KEY_READ [Dictionary] winreg.flags["KEY_READ"] [Instance] winreg.flags.KEY_READ Completeness Extension: Flush The underlying win32 registry API has a flush method for keys. The documentation is as follows: """Writes all the attributes of a key to the registry. It is not necessary to call RegFlushKey to change a key. Registry changes are flushed to disk by the registry using its lazy flusher. Registry changes are also flushed to disk at system shutdown. Unlike \function{CloseKey()}, the \function{FlushKey()} method returns only when all the data has been written to the registry. An application should only call \function{FlushKey()} if it requires absolute certainty that registry changes are on disk.""" If all completeness extensions are implemented, the author believes that this API will be as complete as the underlying API so programmers can choose which to use based on familiarity rather than feature-completeness. -- Paul Prescod - Not encumbered by corporate consensus "I don't want you to describe to me -- not ever -- what you were doing to that poor boy to make him sound like that; but if you ever do it again, please cover his mouth with your hand," Grandmother said. -- John Irving, "A Prayer for Owen Meany" From guido@beopen.com Fri Aug 11 15:28:09 2000 From: guido@beopen.com (Guido van Rossum) Date: Fri, 11 Aug 2000 09:28:09 -0500 Subject: [Python-Dev] Preventing recursion core dumps In-Reply-To: Your message of "Fri, 11 Aug 2000 14:39:36 +0200." <200008111239.OAA15818@python.inrialpes.fr> References: <200008111239.OAA15818@python.inrialpes.fr> Message-ID: <200008111428.JAA04464@cj20424-a.reston1.va.home.com> > > It seems better to tune the generic check than to special-case str, > > repr, and getattr. > > Right. This would be a step forward, at least for recursive Python code > (which is the most common complaint). Reducing the current value > by half, i.e. setting MAX_RECURSION_DEPTH = 5000 works for me (Linux & AIX) > > Agreement on 5000? No, the __getattr__ example still dumps core for me. With 4000 it is fine, but this indicates that this is totally the wrong approach: I can change the available stack size with ulimit -s and cause a core dump anyway. Or there could be a loger path through the C code where more C stack is used per recursion. We could set the maximum to 1000 and assume a "reasonable" stack size, but that doesn't make me feel comfortable either. It would be good if there was a way to sense the remaining available stack, even if it wasn't portable. Any Linux experts out there? > Doesn't solve the problem for C code (extensions) though... That wasn't what started this thread. Bugs in extensions are just that. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From gvwilson@nevex.com Fri Aug 11 14:39:38 2000 From: gvwilson@nevex.com (Greg Wilson) Date: Fri, 11 Aug 2000 09:39:38 -0400 (EDT) Subject: [Python-Dev] PEP 0211: Linear Algebra Operators Message-ID: This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. Send mail to mime@docserver.cac.washington.edu for more info. --168427786-36668808-966001178=:13482 Content-Type: TEXT/PLAIN; charset=US-ASCII Hi, everyone. Please find attached the latest version of PEP-0211, "Adding New Linear Algebra Operators to Python". As I don't have write access to the CVS repository, I'd be grateful if someone could check this in for me. Please send comments directly to me (gvwilson@nevex.com); I'll summarize, update the PEP, and re-post. Thanks, Greg --168427786-36668808-966001178=:13482 Content-Type: TEXT/PLAIN; charset=US-ASCII; name="pep-0211.txt" Content-Transfer-Encoding: BASE64 Content-ID: Content-Description: Content-Disposition: attachment; filename="pep-0211.txt" UEVQOiAyMTENClRpdGxlOiBBZGRpbmcgTmV3IExpbmVhciBBbGdlYnJhIE9w ZXJhdG9ycyB0byBQeXRob24NClZlcnNpb246ICRSZXZpc2lvbiQNCk93bmVy OiBndndpbHNvbkBuZXZleC5jb20gKEdyZWcgV2lsc29uKQ0KUHl0aG9uLVZl cnNpb246IDIuMQ0KQ3JlYXRlZDogMTUtSnVsLTIwMDANClN0YXR1czogRHJh ZnQNClBvc3QtSGlzdG9yeToNCg0KDQpJbnRyb2R1Y3Rpb24NCg0KICAgIFRo aXMgUEVQIGRlc2NyaWJlcyBhIHByb3Bvc2FsIHRvIGFkZCBsaW5lYXIgYWxn ZWJyYSBvcGVyYXRvcnMgdG8NCiAgICBQeXRob24gMi4wLiAgSXQgZGlzY3Vz c2VzIHdoeSBzdWNoIG9wZXJhdG9ycyBhcmUgZGVzaXJhYmxlLCBhbmQNCiAg ICBhbHRlcm5hdGl2ZXMgdGhhdCBoYXZlIGJlZW4gY29uc2lkZXJlZCBhbmQg ZGlzY2FyZGVkLiAgVGhpcyBQRVANCiAgICBzdW1tYXJpemVzIGRpc2N1c3Np b25zIGhlbGQgaW4gbWFpbGluZyBsaXN0IGZvcnVtcywgYW5kIHByb3ZpZGVz DQogICAgVVJMcyBmb3IgZnVydGhlciBpbmZvcm1hdGlvbiwgd2hlcmUgYXBw cm9wcmlhdGUuICBUaGUgQ1ZTIHJldmlzaW9uDQogICAgaGlzdG9yeSBvZiB0 aGlzIGZpbGUgY29udGFpbnMgdGhlIGRlZmluaXRpdmUgaGlzdG9yaWNhbCBy ZWNvcmQuDQoNCg0KUHJvcG9zYWwNCg0KICAgIEFkZCBhIHNpbmdsZSBuZXcg aW5maXggYmluYXJ5IG9wZXJhdG9yICdAJyAoImFjcm9zcyIpLCBhbmQNCiAg ICBjb3JyZXNwb25kaW5nIHNwZWNpYWwgbWV0aG9kcyAiX19hY3Jvc3NfXygp IiBhbmQgIl9fcmFjcm9zc19fKCkiLg0KICAgIFRoaXMgb3BlcmF0b3Igd2ls bCBwZXJmb3JtIG1hdGhlbWF0aWNhbCBtYXRyaXggbXVsdGlwbGljYXRpb24g b24NCiAgICBOdW1QeSBhcnJheXMsIGFuZCBnZW5lcmF0ZSBjcm9zcy1wcm9k dWN0cyB3aGVuIGFwcGxpZWQgdG8gYnVpbHQtaW4NCiAgICBzZXF1ZW5jZSB0 eXBlcy4gIE5vIGV4aXN0aW5nIG9wZXJhdG9yIGRlZmluaXRpb25zIHdpbGwg YmUgY2hhbmdlZC4NCg0KDQpCYWNrZ3JvdW5kDQoNCiAgICBDb21wdXRlcnMg d2VyZSBpbnZlbnRlZCB0byBkbyBhcml0aG1ldGljLCBhcyB3YXMgdGhlIGZp cnN0DQogICAgaGlnaC1sZXZlbCBwcm9ncmFtbWluZyBsYW5ndWFnZSwgRm9y dHJhbi4gIFdoaWxlIEZvcnRyYW4gd2FzIGENCiAgICBncmVhdCBhZHZhbmNl IG9uIGl0cyBtYWNoaW5lLWxldmVsIHByZWRlY2Vzc29ycywgdGhlcmUgd2Fz IHN0aWxsIGENCiAgICB2ZXJ5IGxhcmdlIGdhcCBiZXR3ZWVuIGl0cyBzeW50 YXggYW5kIHRoZSBub3RhdGlvbiB1c2VkIGJ5DQogICAgbWF0aGVtYXRpY2lh bnMuICBUaGUgbW9zdCBpbmZsdWVudGlhbCBlZmZvcnQgdG8gY2xvc2UgdGhp cyBnYXAgd2FzDQogICAgQVBMIFsxXToNCg0KICAgICAgICBUaGUgbGFuZ3Vh Z2UgW0FQTF0gd2FzIGludmVudGVkIGJ5IEtlbm5ldGggRS4gSXZlcnNvbiB3 aGlsZSBhdA0KICAgICAgICBIYXJ2YXJkIFVuaXZlcnNpdHkuIFRoZSBsYW5n dWFnZSwgb3JpZ2luYWxseSB0aXRsZWQgIkl2ZXJzb24NCiAgICAgICAgTm90 YXRpb24iLCB3YXMgZGVzaWduZWQgdG8gb3ZlcmNvbWUgdGhlIGluaGVyZW50 IGFtYmlndWl0aWVzDQogICAgICAgIGFuZCBwb2ludHMgb2YgY29uZnVzaW9u IGZvdW5kIHdoZW4gZGVhbGluZyB3aXRoIHN0YW5kYXJkDQogICAgICAgIG1h dGhlbWF0aWNhbCBub3RhdGlvbi4gSXQgd2FzIGxhdGVyIGRlc2NyaWJlZCBp biAxOTYyIGluIGENCiAgICAgICAgYm9vayBzaW1wbHkgdGl0bGVkICJBIFBy b2dyYW1taW5nIExhbmd1YWdlIiAoaGVuY2UgQVBMKS4NCiAgICAgICAgVG93 YXJkcyB0aGUgZW5kIG9mIHRoZSBzaXh0aWVzLCBsYXJnZWx5IHRocm91Z2gg dGhlIGVmZm9ydHMgb2YNCiAgICAgICAgSUJNLCB0aGUgY29tcHV0ZXIgY29t bXVuaXR5IGdhaW5lZCBpdHMgZmlyc3QgZXhwb3N1cmUgdG8NCiAgICAgICAg QVBMLiBJdmVyc29uIHJlY2VpdmVkIHRoZSBUdXJpbmcgQXdhcmQgaW4gMTk4 MCBmb3IgdGhpcyB3b3JrLg0KDQogICAgQVBMJ3Mgb3BlcmF0b3JzIHN1cHBv cnRlZCBib3RoIGZhbWlsaWFyIGFsZ2VicmFpYyBvcGVyYXRpb25zLCBzdWNo DQogICAgYXMgdmVjdG9yIGRvdCBwcm9kdWN0IGFuZCBtYXRyaXggbXVsdGlw bGljYXRpb24sIGFuZCBhIHdpZGUgcmFuZ2UNCiAgICBvZiBzdHJ1Y3R1cmFs IG9wZXJhdGlvbnMsIHN1Y2ggYXMgc3RpdGNoaW5nIHZlY3RvcnMgdG9nZXRo ZXIgdG8NCiAgICBjcmVhdGUgYXJyYXlzLiAgSXRzIG5vdGF0aW9uIHdhcyBl eGNlcHRpb25hbGx5IGNyeXB0aWM6IG1hbnkgb2YNCiAgICBpdHMgc3ltYm9s cyBkaWQgbm90IGV4aXN0IG9uIHN0YW5kYXJkIGtleWJvYXJkcywgYW5kIGV4 cHJlc3Npb25zDQogICAgaGFkIHRvIGJlIHJlYWQgcmlnaHQgdG8gbGVmdC4N Cg0KICAgIE1vc3Qgc3Vic2VxdWVudCB3b3JrIG9uIG51bWVyaWNhbCBsYW5n dWFnZXMsIHN1Y2ggYXMgRm9ydHJhbi05MCwNCiAgICBNQVRMQUIsIGFuZCBN YXRoZW1hdGljYSwgaGFzIHRyaWVkIHRvIHByb3ZpZGUgdGhlIHBvd2VyIG9m IEFQTA0KICAgIHdpdGhvdXQgdGhlIG9ic2N1cml0eS4gIFB5dGhvbidzIE51 bVB5IFsyXSBoYXMgbW9zdCBvZiB0aGUNCiAgICBmZWF0dXJlcyB0aGF0IHVz ZXJzIG9mIHN1Y2ggbGFuZ3VhZ2VzIGV4cGVjdCwgYnV0IHRoZXNlIGFyZQ0K ICAgIHByb3ZpZGVkIHRocm91Z2ggbmFtZWQgZnVuY3Rpb25zIGFuZCBtZXRo b2RzLCByYXRoZXIgdGhhbg0KICAgIG92ZXJsb2FkZWQgb3BlcmF0b3JzLiAg VGhpcyBtYWtlcyBOdW1QeSBjbHVtc2llciB0aGFuIGl0cw0KICAgIGNvbXBl dGl0b3JzLg0KDQogICAgT25lIHdheSB0byBtYWtlIE51bVB5IG1vcmUgY29t cGV0aXRpdmUgaXMgdG8gcHJvdmlkZSBncmVhdGVyDQogICAgc3ludGFjdGlj IHN1cHBvcnQgaW4gUHl0aG9uIGl0c2VsZiBmb3IgbGluZWFyIGFsZ2VicmEu ICBUaGlzDQogICAgcHJvcG9zYWwgdGhlcmVmb3JlIGV4YW1pbmVzIHRoZSBy ZXF1aXJlbWVudHMgdGhhdCBuZXcgbGluZWFyDQogICAgYWxnZWJyYSBvcGVy YXRvcnMgaW4gUHl0aG9uIG11c3Qgc2F0aXNmeSwgYW5kIHByb3Bvc2VzIGEg c3ludGF4DQogICAgYW5kIHNlbWFudGljcyBmb3IgdGhvc2Ugb3BlcmF0b3Jz Lg0KDQoNClJlcXVpcmVtZW50cw0KDQogICAgVGhlIG1vc3QgaW1wb3J0YW50 IHJlcXVpcmVtZW50IGlzIHRoYXQgdGhlcmUgYmUgbWluaW1hbCBpbXBhY3Qg b24NCiAgICB0aGUgZXhpc3RpbmcgZGVmaW5pdGlvbiBvZiBQeXRob24uICBU aGUgcHJvcG9zYWwgbXVzdCBub3QgYnJlYWsNCiAgICBleGlzdGluZyBwcm9n cmFtcywgZXhjZXB0IHBvc3NpYmx5IHRob3NlIHRoYXQgdXNlIE51bVB5Lg0K DQogICAgVGhlIHNlY29uZCBtb3N0IGltcG9ydGFudCByZXF1aXJlbWVudCBp cyB0byBiZSBhYmxlIHRvIGRvIGJvdGgNCiAgICBlbGVtZW50d2lzZSBhbmQg bWF0aGVtYXRpY2FsIG1hdHJpeCBtdWx0aXBsaWNhdGlvbiB1c2luZyBpbmZp eA0KICAgIG5vdGF0aW9uLiAgVGhlIG5pbmUgY2FzZXMgdGhhdCBtdXN0IGJl IGhhbmRsZWQgYXJlOg0KDQogICAgICAgIHw1IDZ8ICogICA5ICAgPSB8NDUg NTR8ICAgICAgTVM6IG1hdHJpeC1zY2FsYXIgbXVsdGlwbGljYXRpb24NCiAg ICAgICAgfDcgOHwgICAgICAgICAgIHw2MyA3MnwNCg0KICAgICAgICAgIDkg ICAqIHw1IDZ8ID0gfDQ1IDU0fCAgICAgIFNNOiBzY2FsYXItbWF0cml4IG11 bHRpcGxpY2F0aW9uDQogICAgICAgICAgICAgICAgfDcgOHwgICB8NjMgNzJ8 DQoNCiAgICAgICAgfDIgM3wgKiB8NCA1fCA9IHw4IDE1fCAgICAgICBWRTog dmVjdG9yIGVsZW1lbnR3aXNlIG11bHRpcGxpY2F0aW9uDQoNCg0KICAgICAg ICB8MiAzfCAqICB8NHwgID0gICAyMyAgICAgICAgIFZEOiB2ZWN0b3IgZG90 IHByb2R1Y3QNCiAgICAgICAgICAgICAgICAgfDV8DQoNCiAgICAgICAgIHwy fCAgKiB8NCA1fCA9IHwgOCAxMHwgICAgICBWTzogdmVjdG9yIG91dGVyIHBy b2R1Y3QNCiAgICAgICAgIHwzfCAgICAgICAgICAgIHwxMiAxNXwNCg0KICAg ICAgICB8MSAyfCAqIHw1IDZ8ID0gfCA1IDEyfCAgICAgIE1FOiBtYXRyaXgg ZWxlbWVudHdpc2UgbXVsdGlwbGljYXRpb24NCiAgICAgICAgfDMgNHwgICB8 NyA4fCAgIHwyMSAzMnwNCg0KICAgICAgICB8MSAyfCAqIHw1IDZ8ID0gfDE5 IDIyfCAgICAgIE1NOiBtYXRoZW1hdGljYWwgbWF0cml4IG11bHRpcGxpY2F0 aW9uDQogICAgICAgIHwzIDR8ICAgfDcgOHwgICB8NDMgNTB8DQoNCiAgICAg ICAgfDEgMnwgKiB8NSA2fCA9IHwxOSAyMnwgICAgICBWTTogdmVjdG9yLW1h dHJpeCBtdWx0aXBsaWNhdGlvbg0KICAgICAgICAgICAgICAgIHw3IDh8DQoN CiAgICAgICAgfDUgNnwgKiAgfDF8ICA9ICAgfDE3fCAgICAgICBNVjogbWF0 cml4LXZlY3RvciBtdWx0aXBsaWNhdGlvbg0KICAgICAgICB8NyA4fCAgICB8 MnwgICAgICB8MjN8DQoNCiAgICBOb3RlIHRoYXQgMS1kaW1lbnNpb25hbCB2 ZWN0b3JzIGFyZSB0cmVhdGVkIGFzIHJvd3MgaW4gVk0sIGFzDQogICAgY29s dW1ucyBpbiBNViwgYW5kIGFzIGJvdGggaW4gVkQgYW5kIFZPLiAgQm90aCBh cmUgc3BlY2lhbCBjYXNlcw0KICAgIG9mIDItZGltZW5zaW9uYWwgbWF0cmlj ZXMgKE54MSBhbmQgMXhOIHJlc3BlY3RpdmVseSkuICBJdCBtYXkNCiAgICB0 aGVyZWZvcmUgYmUgcmVhc29uYWJsZSB0byBkZWZpbmUgdGhlIG5ldyBvcGVy YXRvciBvbmx5IGZvcg0KICAgIDItZGltZW5zaW9uYWwgYXJyYXlzLCBhbmQg cHJvdmlkZSBhbiBlYXN5IChhbmQgZWZmaWNpZW50KSB3YXkgZm9yDQogICAg dXNlcnMgdG8gY29udmVydCAxLWRpbWVuc2lvbmFsIHN0cnVjdHVyZXMgdG8g Mi1kaW1lbnNpb25hbC4NCiAgICBCZWhhdmlvciBvZiBhIG5ldyBtdWx0aXBs aWNhdGlvbiBvcGVyYXRvciBmb3IgYnVpbHQtaW4gdHlwZXMgbWF5DQogICAg dGhlbjoNCg0KICAgIChhKSBiZSBhIHBhcnNpbmcgZXJyb3IgKHBvc3NpYmxl IG9ubHkgaWYgYSBjb25zdGFudCBpcyBvbmUgb2YgdGhlDQogICAgICAgIGFy Z3VtZW50cywgc2luY2UgbmFtZXMgYXJlIHVudHlwZWQgaW4gUHl0aG9uKTsN Cg0KICAgIChiKSBnZW5lcmF0ZSBhIHJ1bnRpbWUgZXJyb3I7IG9yDQoNCiAg ICAoYykgYmUgZGVyaXZlZCBieSBwbGF1c2libGUgZXh0ZW5zaW9uIGZyb20g aXRzIGJlaGF2aW9yIGluIHRoZQ0KICAgICAgICB0d28tZGltZW5zaW9uYWwg Y2FzZS4NCg0KICAgIFRoaXJkLCBzeW50YWN0aWMgc3VwcG9ydCBzaG91bGQg YmUgY29uc2lkZXJlZCBmb3IgdGhyZWUgb3RoZXINCiAgICBvcGVyYXRpb25z Og0KDQogICAgICAgICAgICAgICAgICAgICAgICAgVA0KICAgIChhKSB0cmFu c3Bvc2l0aW9uOiAgQSAgID0+IEFbaiwgaV0gZm9yIEFbaSwgal0NCg0KICAg ICAgICAgICAgICAgICAgICAgICAgIC0xDQogICAgKGIpIGludmVyc2U6ICAg ICAgICBBICAgPT4gQScgc3VjaCB0aGF0IEEnICogQSA9IEkgKHRoZSBpZGVu dGl0eSBtYXRyaXgpDQoNCiAgICAoYykgc29sdXRpb246ICAgICAgIEEvYiA9 PiB4ICBzdWNoIHRoYXQgQSAqIHggPSBiDQogICAgICAgICAgICAgICAgICAg ICAgICBBXGIgPT4geCAgc3VjaCB0aGF0IHggKiBBID0gYg0KDQogICAgV2l0 aCByZWdhcmQgdG8gKGMpLCBpdCBpcyB3b3J0aCBub3RpbmcgdGhhdCB0aGUg dHdvIHN5bnRheGVzIHVzZWQNCiAgICB3ZXJlIGludmVudGVkIGJ5IHByb2dy YW1tZXJzLCBub3QgbWF0aGVtYXRpY2lhbnMuICBNYXRoZW1hdGljaWFucw0K ICAgIGRvIG5vdCBoYXZlIGEgc3RhbmRhcmQsIHdpZGVseS11c2VkIG5vdGF0 aW9uIGZvciBtYXRyaXggc29sdXRpb24uDQoNCiAgICBJdCBpcyBhbHNvIHdv cnRoIG5vdGluZyB0aGF0IGRvemVucyBvZiBtYXRyaXggaW52ZXJzaW9uIGFu ZA0KICAgIHNvbHV0aW9uIGFsZ29yaXRobXMgYXJlIHdpZGVseSB1c2VkLiAg TUFUTEFCIGFuZCBpdHMga2luIGJpbmQNCiAgICB0aGVpciBpbnZlcnNpb24g YW5kL29yIHNvbHV0aW9uIG9wZXJhdG9ycyB0byBvbmUgd2hpY2ggaXMNCiAg ICByZWFzb25hYmx5IHJvYnVzdCBpbiBtb3N0IGNhc2VzLCBhbmQgcmVxdWly ZSB1c2VycyB0byBjYWxsDQogICAgZnVuY3Rpb25zIG9yIG1ldGhvZHMgdG8g YWNjZXNzIG90aGVycy4NCg0KICAgIEZvdXJ0aCwgY29uZnVzaW9uIGJldHdl ZW4gUHl0aG9uJ3Mgbm90YXRpb24gYW5kIHRob3NlIG9mIE1BVExBQg0KICAg IGFuZCBGb3J0cmFuLTkwIHNob3VsZCBiZSBhdm9pZGVkLiAgSW4gcGFydGlj dWxhciwgbWF0aGVtYXRpY2FsDQogICAgbWF0cml4IG11bHRpcGxpY2F0aW9u IChjYXNlIE1NKSBzaG91bGQgbm90IGJlIHJlcHJlc2VudGVkIGFzICcuKics DQogICAgc2luY2U6DQoNCiAgICAoYSkgTUFUTEFCIHVzZXMgcHJlZml4LScu JyBmb3JtcyB0byBtZWFuICdlbGVtZW50d2lzZScsIGFuZCByYXcNCiAgICAg ICAgZm9ybXMgdG8gbWVhbiAibWF0aGVtYXRpY2FsIiBbNF07IGFuZA0KDQog ICAgKGIpIGV2ZW4gaWYgdGhlIFB5dGhvbiBwYXJzZXIgY2FuIGJlIHRhdWdo dCBob3cgdG8gaGFuZGxlIGRvdHRlZA0KICAgICAgICBmb3JtcywgJzEuKkEn IHdpbGwgc3RpbGwgYmUgdmlzdWFsbHkgYW1iaWd1b3VzIFs0XS4NCg0KICAg IE9uZSBhbnRpLXJlcXVpcmVtZW50IGlzIHRoYXQgbmV3IG9wZXJhdG9ycyBh cmUgbm90IG5lZWRlZCBmb3INCiAgICBhZGRpdGlvbiwgc3VidHJhY3Rpb24s IGJpdHdpc2Ugb3BlcmF0aW9ucywgYW5kIHNvIG9uLCBzaW5jZQ0KICAgIG1h dGhlbWF0aWNpYW5zIGFscmVhZHkgdHJlYXQgdGhlbSBlbGVtZW50d2lzZS4N Cg0KDQpQcm9wb3NhbDoNCg0KICAgIFRoZSBtZWFuaW5ncyBvZiBhbGwgZXhp c3Rpbmcgb3BlcmF0b3JzIHdpbGwgYmUgdW5jaGFuZ2VkLiAgSW4NCiAgICBw YXJ0aWN1bGFyLCAnQSpCJyB3aWxsIGNvbnRpbnVlIHRvIGJlIGludGVycHJl dGVkIGVsZW1lbnR3aXNlLg0KICAgIFRoaXMgdGFrZXMgY2FyZSBvZiB0aGUg Y2FzZXMgTVMsIFNNLCBWRSwgYW5kIE1FLCBhbmQgZW5zdXJlcw0KICAgIG1p bmltYWwgaW1wYWN0IG9uIGV4aXN0aW5nIHByb2dyYW1zLg0KDQogICAgQSBu ZXcgb3BlcmF0b3IgJ0AnIChwcm9ub3VuY2VkICJhY3Jvc3MiKSB3aWxsIGJl IGFkZGVkIHRvIFB5dGhvbiwNCiAgICBhbG9uZyB3aXRoIHR3byBzcGVjaWFs IG1ldGhvZHMsICJfX2Fjcm9zc19fKCkiIGFuZA0KICAgICJfX3JhY3Jvc3Nf XygpIiwgd2l0aCB0aGUgdXN1YWwgc2VtYW50aWNzLg0KDQogICAgTnVtUHkg d2lsbCBvdmVybG9hZCAiQCIgdG8gcGVyZm9ybSBtYXRoZW1hdGljYWwgbXVs dGlwbGljYXRpb24gb2YNCiAgICBhcnJheXMgd2hlcmUgc2hhcGVzIHBlcm1p dCwgYW5kIHRvIHRocm93IGFuIGV4Y2VwdGlvbiBvdGhlcndpc2UuDQogICAg VGhlIG1hdHJpeCBjbGFzcydzIGltcGxlbWVudGF0aW9uIG9mICJAIiB3aWxs IHRyZWF0IGJ1aWx0LWluDQogICAgc2VxdWVuY2UgdHlwZXMgYXMgaWYgdGhl eSB3ZXJlIGNvbHVtbiB2ZWN0b3JzLiAgVGhpcyB0YWtlcyBjYXJlIG9mDQog ICAgdGhlIGNhc2VzIE1NIGFuZCBNVi4NCg0KICAgIEFuIGF0dHJpYnV0ZSAi VCIgd2lsbCBiZSBhZGRlZCB0byB0aGUgTnVtUHkgYXJyYXkgdHlwZSwgc3Vj aCB0aGF0DQogICAgIm0uVCIgaXM6DQoNCiAgICAoYSkgdGhlIHRyYW5zcG9z ZSBvZiAibSIgZm9yIGEgMi1kaW1lbnNpb25hbCBhcnJheQ0KDQogICAgKGIp IHRoZSAxeE4gbWF0cml4IHRyYW5zcG9zZSBvZiAibSIgaWYgIm0iIGlzIGEg MS1kaW1lbnNpb25hbA0KICAgICAgICBhcnJheTsgb3INCg0KICAgIChjKSBh IHJ1bnRpbWUgZXJyb3IgZm9yIGFuIGFycmF5IHdpdGggcmFuayA+PSAzLg0K DQogICAgVGhpcyBhdHRyaWJ1dGUgd2lsbCBhbGlhcyB0aGUgbWVtb3J5IG9m IHRoZSBiYXNlIG9iamVjdC4gIE51bVB5J3MNCiAgICAidHJhbnNwb3NlKCki IGZ1bmN0aW9uIHdpbGwgYmUgZXh0ZW5kZWQgdG8gdHVybiBidWlsdC1pbiBz ZXF1ZW5jZQ0KICAgIHR5cGVzIGludG8gcm93IHZlY3RvcnMuICBUaGlzIHRh a2VzIGNhcmUgb2YgdGhlIFZNLCBWRCwgYW5kIFZPDQogICAgY2FzZXMuICBX ZSBwcm9wb3NlIGFuIGF0dHJpYnV0ZSBiZWNhdXNlOg0KDQogICAgKGEpIHRo ZSByZXN1bHRpbmcgbm90YXRpb24gaXMgc2ltaWxhciB0byB0aGUgJ3N1cGVy c2NyaXB0IFQnIChhdA0KICAgICAgICBsZWFzdCwgYXMgc2ltaWxhciBhcyBB U0NJSSBhbGxvd3MpLCBhbmQNCg0KICAgIChiKSBpdCBzaWduYWxzIHRoYXQg dGhlIHRyYW5zcG9zaXRpb24gYWxpYXNlcyB0aGUgb3JpZ2luYWwgb2JqZWN0 Lg0KDQogICAgTm8gbmV3IG9wZXJhdG9ycyB3aWxsIGJlIGRlZmluZWQgdG8g bWVhbiAic29sdmUgYSBzZXQgb2YgbGluZWFyDQogICAgZXF1YXRpb25zIiwg b3IgImludmVydCBhIG1hdHJpeCIuICBJbnN0ZWFkLCBOdW1QeSB3aWxsIGRl ZmluZSBhDQogICAgdmFsdWUgImludiIsIHdoaWNoIHdpbGwgYmUgcmVjb2du aXplZCBieSB0aGUgZXhwb25lbnRpYXRpb24NCiAgICBvcGVyYXRvciwgc3Vj aCB0aGF0ICJBICoqIGludiIgaXMgdGhlIGludmVyc2Ugb2YgIkEiLiAgVGhp cyBpcw0KICAgIHNpbWlsYXIgaW4gc3Bpcml0IHRvIE51bVB5J3MgZXhpc3Rp bmcgIm5ld2F4aXMiIHZhbHVlLg0KDQogICAgKE9wdGlvbmFsKSBXaGVuIGFw cGxpZWQgdG8gc2VxdWVuY2VzLCB0aGUgb3BlcmF0b3Igd2lsbCByZXR1cm4g YQ0KICAgIGxpc3Qgb2YgdHVwbGVzIGNvbnRhaW5pbmcgdGhlIGNyb3NzLXBy b2R1Y3Qgb2YgdGhlaXIgZWxlbWVudHMgaW4NCiAgICBsZWZ0LXRvLXJpZ2h0 IG9yZGVyOg0KDQogICAgPj4+IFsxLCAyXSBAICgzLCA0KQ0KICAgIFsoMSwg MyksICgxLCA0KSwgKDIsIDMpLCAoMiwgNCldDQoNCiAgICA+Pj4gWzEsIDJd IEAgKDMsIDQpIEAgKDUsIDYpDQogICAgWygxLCAzLCA1KSwgKDEsIDMsIDYp LCANCiAgICAgKDEsIDQsIDUpLCAoMSwgNCwgNiksDQogICAgICgyLCAzLCA1 KSwgKDIsIDMsIDYpLA0KICAgICAoMiwgNCwgNSksICgyLCA0LCA2KV0NCg0K ICAgIFRoaXMgd2lsbCByZXF1aXJlIHRoZSBzYW1lIGtpbmQgb2Ygc3BlY2lh bCBzdXBwb3J0IGZyb20gdGhlIHBhcnNlcg0KICAgIGFzIGNoYWluZWQgY29t cGFyaXNvbnMgKHN1Y2ggYXMgImE8YjxjPD1kIikuICBIb3dldmVyLCBpdCB3 b3VsZA0KICAgIHBlcm1pdCB0aGUgZm9sbG93aW5nOg0KDQogICAgPj4+IGZv ciAoaSwgaikgaW4gWzEsIDJdIEAgWzMsIDRdOg0KICAgID4+PiAgICAgcHJp bnQgaSwgag0KICAgIDEgMw0KICAgIDEgNA0KICAgIDIgMw0KICAgIDIgNA0K DQogICAgYXMgYSBzaG9ydC1oYW5kIGZvciB0aGUgY29tbW9uIG5lc3RlZCBs b29wIGlkaW9tOg0KDQogICAgPj4+IGZvciBpIGluIFsxLCAyXToNCiAgICA+ Pj4gICAgZm9yIGogaW4gWzMsIDRdOg0KICAgID4+PiAgICAgICAgcHJpbnQg aSwgag0KDQogICAgUmVzcG9uc2UgdG8gdGhlICdsb2Nrc3RlcCBsb29wJyBx dWVzdGlvbm5haXJlIFs1XSBpbmRpY2F0ZWQgdGhhdA0KICAgIG5ld2NvbWVy cyB3b3VsZCBiZSBjb21mb3J0YWJsZSB3aXRoIHRoaXMgKHNvIGNvbWZvcnRh YmxlLCBpbiBmYWN0LA0KICAgIHRoYXQgbW9zdCBvZiB0aGVtIGludGVycHJl dGVkIG1vc3QgbXVsdGktbG9vcCAnemlwJyBzeW50YXhlcyBbNl0NCiAgICBh cyBpbXBsZW1lbnRpbmcgc2luZ2xlLXN0YWdlIG5lc3RpbmcpLg0KDQoNCkFs dGVybmF0aXZlczoNCg0KICAgIDAxLiBEb24ndCBhZGQgbmV3IG9wZXJhdG9y cyAtLS0gc3RpY2sgdG8gZnVuY3Rpb25zIGFuZCBtZXRob2RzLg0KDQogICAg UHl0aG9uIGlzIG5vdCBwcmltYXJpbHkgYSBudW1lcmljYWwgbGFuZ3VhZ2Uu ICBJdCBpcyBub3Qgd29ydGgNCiAgICBjb21wbGV4aWZ5aW5nIHRoZSBsYW5n dWFnZSBmb3IgdGhpcyBzcGVjaWFsIGNhc2UgLS0tIE51bVB5J3MNCiAgICBz dWNjZXNzIGlzIHByb29mIHRoYXQgdXNlcnMgY2FuIGFuZCB3aWxsIHVzZSBm dW5jdGlvbnMgYW5kIG1ldGhvZHMNCiAgICBmb3IgbGluZWFyIGFsZ2VicmEu DQoNCiAgICBPbiB0aGUgcG9zaXRpdmUgc2lkZSwgdGhpcyBtYWludGFpbnMg UHl0aG9uJ3Mgc2ltcGxpY2l0eS4gIEl0cw0KICAgIHdlYWtuZXNzIGlzIHRo YXQgc3VwcG9ydCBmb3IgcmVhbCBtYXRyaXggbXVsdGlwbGljYXRpb24gKGFu ZCwgdG8gYQ0KICAgIGxlc3NlciBleHRlbnQsIG90aGVyIGxpbmVhciBhbGdl YnJhIG9wZXJhdGlvbnMpIGlzIGZyZXF1ZW50bHkNCiAgICByZXF1ZXN0ZWQs IGFzIGZ1bmN0aW9uYWwgZm9ybXMgYXJlIGN1bWJlcnNvbWUgZm9yIGxlbmd0 aHkNCiAgICBmb3JtdWxhcywgYW5kIGRvIG5vdCByZXNwZWN0IHRoZSBvcGVy YXRvciBwcmVjZWRlbmNlIHJ1bGVzIG9mDQogICAgY29udmVudGlvbmFsIG1h dGhlbWF0aWNzLiAgSW4gYWRkaXRpb24sIHRoZSBtZXRob2QgZm9ybSBpcw0K ICAgIGFzeW1tZXRyaWMgaW4gaXRzIG9wZXJhbmRzLg0KDQogICAgMDIuIElu dHJvZHVjZSBwcmVmaXhlZCBmb3JtcyBvZiBleGlzdGluZyBvcGVyYXRvcnMs IHN1Y2ggYXMgIkAqIg0KICAgICAgICBvciAifioiLCBvciB1c2VkIGJveGVk IGZvcm1zLCBzdWNoIGFzICJbKl0iIG9yICIlKiUiLg0KDQogICAgVGhlcmUg YXJlIChhdCBsZWFzdCkgdGhyZWUgb2JqZWN0aW9ucyB0byB0aGlzLiAgRmly c3QsIGVpdGhlciBmb3JtDQogICAgc2VlbXMgdG8gaW1wbHkgdGhhdCBhbGwg b3BlcmF0b3JzIGV4aXN0IGluIGJvdGggZm9ybXMuICBUaGlzIGlzDQogICAg bW9yZSBuZXcgZW50aXRpZXMgdGhhbiB0aGUgcHJvYmxlbSBtZXJpdHMsIGFu ZCB3b3VsZCByZXF1aXJlIHRoZQ0KICAgIGFkZGl0aW9uIG9mIG1hbnkgbmV3 IG92ZXJsb2FkYWJsZSBtZXRob2RzLCBzdWNoIGFzIF9fYXRfbXVsX18uDQoN CiAgICBTZWNvbmQsIHdoaWxlIGl0IGlzIGNlcnRhaW5seSBwb3NzaWJsZSB0 byBpbnZlbnQgc2VtYW50aWNzIGZvcg0KICAgIHRoZXNlIG5ldyBvcGVyYXRv cnMgZm9yIGJ1aWx0LWluIHR5cGVzLCB0aGlzIHdvdWxkIGJlIGEgY2FzZSBv Zg0KICAgIHRoZSB0YWlsIHdhZ2dpbmcgdGhlIGRvZywgaS5lLiBvZiBsZXR0 aW5nIHRoZSBleGlzdGVuY2Ugb2YgYQ0KICAgIGZlYXR1cmUgImNyZWF0ZSIg YSBuZWVkIGZvciBpdC4NCg0KICAgIEZpbmFsbHksIHRoZSBib3hlZCBmb3Jt cyBtYWtlIGh1bWFuIHBhcnNpbmcgbW9yZSBjb21wbGV4LCBlLmcuOg0KDQog ICAgICAgIEFbKl0gPSBCICAgIHZzLiAgICBBWzpdID0gQg0KDQogICAgMDMu IChGcm9tIE1vc2hlIFphZGthIFs3XSwgYW5kIGFsc28gY29uc2lkZXJlZCBi eSBIdWFpeXUgWmhvdSBbOF0NCiAgICAgICAgaW4gaGlzIHByb3Bvc2FsIFs5 XSkgUmV0YWluIHRoZSBleGlzdGluZyBtZWFuaW5nIG9mIGFsbA0KICAgICAg ICBvcGVyYXRvcnMsIGJ1dCBjcmVhdGUgYSBiZWhhdmlvcmFsIGFjY2Vzc29y IGZvciBhcnJheXMsIHN1Y2gNCiAgICAgICAgdGhhdDoNCg0KICAgICAgICAg ICAgQSAqIEINCg0KICAgICAgICBpcyBlbGVtZW50d2lzZSBtdWx0aXBsaWNh dGlvbiAoTUUpLCBidXQ6DQoNCiAgICAgICAgICAgIEEubSgpICogQi5tKCkN Cg0KICAgICAgICBpcyBtYXRoZW1hdGljYWwgbXVsdGlwbGljYXRpb24gKE1N KS4gIFRoZSBtZXRob2QgIkEubSgpIiB3b3VsZA0KICAgICAgICByZXR1cm4g YW4gb2JqZWN0IHRoYXQgYWxpYXNlZCBBJ3MgbWVtb3J5IChmb3IgZWZmaWNp ZW5jeSksIGJ1dA0KICAgICAgICB3aGljaCBoYWQgYSBkaWZmZXJlbnQgaW1w bGVtZW50YXRpb24gb2YgX19tdWxfXygpLg0KDQogICAgVGhlIGFkdmFudGFn ZSBvZiB0aGlzIG1ldGhvZCBpcyB0aGF0IGl0IGhhcyBubyBlZmZlY3Qgb24g dGhlDQogICAgZXhpc3RpbmcgaW1wbGVtZW50YXRpb24gb2YgUHl0aG9uOiBj aGFuZ2VzIGFyZSBsb2NhbGl6ZWQgaW4gdGhlDQogICAgTnVtZXJpYyBtb2R1 bGUuICBUaGUgZGlzYWR2YW50YWdlcyBhcmU6DQoNCiAgICAoYSkgVGhlIHNl bWFudGljcyBvZiAiQS5tKCkgKiBCIiwgIkEgKyBCLm0oKSIsIGFuZCBzbyBv biB3b3VsZA0KICAgICAgICBoYXZlIHRvIGJlIGRlZmluZWQsIGFuZCB0aGVy ZSBpcyBubyAib2J2aW91cyIgY2hvaWNlIGZvciB0aGVtLg0KDQogICAgKGIp IEFsaWFzaW5nIG9iamVjdHMgdG8gdHJpZ2dlciBkaWZmZXJlbnQgb3BlcmF0 b3IgYmVoYXZpb3IgZmVlbHMNCiAgICAgICAgbGVzcyBQeXRob25pYyB0aGFu IGVpdGhlciBjYWxsaW5nIG1ldGhvZHMgKGFzIGluIHRoZSBleGlzdGluZw0K ICAgICAgICBOdW1lcmljIG1vZHVsZSkgb3IgdXNpbmcgYSBkaWZmZXJlbnQg b3BlcmF0b3IuICBUaGlzIFBFUCBpcw0KICAgICAgICBwcmltYXJpbHkgYWJv dXQgbG9vayBhbmQgZmVlbCwgYW5kIGFib3V0IG1ha2luZyBQeXRob24gbW9y ZQ0KICAgICAgICBhdHRyYWN0aXZlIHRvIHBlb3BsZSB3aG8gYXJlIG5vdCBh bHJlYWR5IHVzaW5nIGl0Lg0KDQogICAgMDQuIChGcm9tIGEgcHJvcG9zYWwg WzldIGJ5IEh1YWl5dSBaaG91IFs4XSkgSW50cm9kdWNlIGEgImRlbGF5ZWQN CiAgICAgICAgaW52ZXJzZSIgYXR0cmlidXRlLCBzaW1pbGFyIHRvIHRoZSAi dHJhbnNwb3NlIiBhdHRyaWJ1dGUNCiAgICAgICAgYWR2b2NhdGVkIGluIHRo ZSB0aGlyZCBwYXJ0IG9mIHRoaXMgcHJvcG9zYWwuICBUaGUgZXhwcmVzc2lv bg0KICAgICAgICAiYS5JIiB3b3VsZCBiZSBhIGRlbGF5ZWQgaGFuZGxlIG9u IHRoZSBpbnZlcnNlIG9mIHRoZSBtYXRyaXgNCiAgICAgICAgImEiLCB3aGlj aCB3b3VsZCBiZSBldmFsdWF0ZWQgaW4gY29udGV4dCBhcyByZXF1aXJlZC4g IEZvcg0KICAgICAgICBleGFtcGxlLCAiYS5JICogYiIgYW5kICJiICogYS5J IiB3b3VsZCBzb2x2ZSBzZXRzIG9mIGxpbmVhcg0KICAgICAgICBlcXVhdGlv bnMsIHdpdGhvdXQgYWN0dWFsbHkgY2FsY3VsYXRpbmcgdGhlIGludmVyc2Uu DQoNCiAgICBUaGUgbWFpbiBkcmF3YmFjayBvZiB0aGlzIHByb3Bvc2FsIGl0 IGlzIHJlbGlhbmNlIG9uIGxhenkNCiAgICBldmFsdWF0aW9uLCBhbmQgZXZl biBtb3JlIG9uICJzbWFydCIgbGF6eSBldmFsdWF0aW9uIChpLmUuIHRoZQ0K ICAgIG9wZXJhdGlvbiBwZXJmb3JtZWQgZGVwZW5kcyBvbiB0aGUgY29udGV4 dCBpbiB3aGljaCB0aGUgZXZhbHVhdGlvbg0KICAgIGlzIGRvbmUpLiAgVGhl IEJERkwgaGFzIHNvIGZhciByZXNpc3RlZCBpbnRyb2R1Y2luZyBMRSBpbnRv DQogICAgUHl0aG9uLg0KDQoNClJlbGF0ZWQgUHJvcG9zYWxzDQoNCiAgICAw MjAzIDogIEF1Z21lbnRlZCBBc3NpZ25tZW50cw0KDQogICAgICAgICAgICBJ ZiBuZXcgb3BlcmF0b3JzIGZvciBsaW5lYXIgYWxnZWJyYSBhcmUgaW50cm9k dWNlZCwgaXQgbWF5DQogICAgICAgICAgICBtYWtlIHNlbnNlIHRvIGludHJv ZHVjZSBhdWdtZW50ZWQgYXNzaWdubWVudCBmb3JtcyBmb3INCiAgICAgICAg ICAgIHRoZW0uDQoNCiAgICAwMjA3IDogIFJpY2ggQ29tcGFyaXNvbnMNCg0K ICAgICAgICAgICAgSXQgbWF5IGJlY29tZSBwb3NzaWJsZSB0byBvdmVybG9h ZCBjb21wYXJpc29uIG9wZXJhdG9ycw0KICAgICAgICAgICAgc3VjaCBhcyAn PCcgc28gdGhhdCBhbiBleHByZXNzaW9uIHN1Y2ggYXMgJ0EgPCBCJyByZXR1 cm5zDQogICAgICAgICAgICBhbiBhcnJheSwgcmF0aGVyIHRoYW4gYSBzY2Fs YXIgdmFsdWUuDQoNCiAgICAwMjA5IDogIEFkZGluZyBNdWx0aWRpbWVuc2lv bmFsIEFycmF5cw0KDQogICAgICAgICAgICBNdWx0aWRpbWVuc2lvbmFsIGFy cmF5cyBhcmUgY3VycmVudGx5IGFuIGV4dGVuc2lvbiB0bw0KICAgICAgICAg ICAgUHl0aG9uLCByYXRoZXIgdGhhbiBhIGJ1aWx0LWluIHR5cGUuDQoNCg0K QWNrbm93bGVkZ21lbnRzOg0KDQogICAgSSBhbSBncmF0ZWZ1bCB0byBIdWFp eXUgWmh1IFs4XSBmb3IgaW5pdGlhdGluZyB0aGlzIGRpc2N1c3Npb24sDQog ICAgYW5kIGZvciBzb21lIG9mIHRoZSBpZGVhcyBhbmQgdGVybWlub2xvZ3kg aW5jbHVkZWQgYmVsb3cuDQoNCg0KUmVmZXJlbmNlczoNCg0KICAgIFsxXSBo dHRwOi8vd3d3LmFjbS5vcmcvc2lnYXBsL3doeWFwbC5odG0NCiAgICBbMl0g aHR0cDovL251bXB5LnNvdXJjZWZvcmdlLm5ldA0KICAgIFszXSBQRVAtMDIw My50eHQgIkF1Z21lbnRlZCBBc3NpZ25tZW50cyIuDQogICAgWzRdIGh0dHA6 Ly9iZXZvLmNoZS53aXNjLmVkdS9vY3RhdmUvZG9jL29jdGF2ZV85Lmh0bWwj U0VDNjkNCiAgICBbNV0gaHR0cDovL3d3dy5weXRob24ub3JnL3BpcGVybWFp bC9weXRob24tZGV2LzIwMDAtSnVseS8wMTMxMzkuaHRtbA0KICAgIFs2XSBQ RVAtMDIwMS50eHQgIkxvY2tzdGVwIEl0ZXJhdGlvbiINCiAgICBbN10gTW9z aGUgWmFka2EgaXMgJ21vc2hlekBtYXRoLmh1amkuYWMuaWwnLg0KICAgIFs4 XSBIdWFpeXUgWmh1IGlzICdoemh1QHVzZXJzLnNvdXJjZWZvcmdlLm5ldCcN CiAgICBbOV0gaHR0cDovL3d3dy5weXRob24ub3JnL3BpcGVybWFpbC9weXRo b24tbGlzdC8yMDAwLUF1Z3VzdC8xMTI1MjkuaHRtbA0KDQoMDQpMb2NhbCBW YXJpYWJsZXM6DQptb2RlOiBpbmRlbnRlZC10ZXh0DQppbmRlbnQtdGFicy1t b2RlOiBuaWwNCkVuZDoNCg== --168427786-36668808-966001178=:13482-- From fredrik@pythonware.com Fri Aug 11 14:55:01 2000 From: fredrik@pythonware.com (Fredrik Lundh) Date: Fri, 11 Aug 2000 15:55:01 +0200 Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of data sent. References: <200008111255.HAA03735@cj20424-a.reston1.va.home.com> <20000811143143.G17171@xs4all.nl> <200008111419.JAA03948@cj20424-a.reston1.va.home.com> Message-ID: <016d01c0039b$bfb99a40$0900a8c0@SPIFF> guido wrote: > I just read the man page for send() (Red Hat linux 6.1) and it doesn't > mention sending fewer than all bytes at all. In fact, while it says > that the return value is the number of bytes sent, it at least > *suggests* that it will return an error whenever not everything can be > sent -- even in non-blocking mode. > > Under what circumstances can send() return a smaller number? never, it seems: The length of the message to be sent is specified by the length argument. If the message is too long to pass through the underlying protocol, send() fails and no data is transmitted. Successful completion of a call to send() does not guarantee delivery of the message. A return value of -1 indicates only locally-detected errors. If space is not available at the sending socket to hold the message to be transmitted and the socket file descriptor does not have O_NONBLOCK set, send() blocks until space is available. If space is not available at the sending socket to hold the message to be transmitted and the socket file descriptor does have O_NONBLOCK set, send() will fail. (from SUSv2) iow, it either blocks or fails. From fredrik@pythonware.com Fri Aug 11 15:01:17 2000 From: fredrik@pythonware.com (Fredrik Lundh) Date: Fri, 11 Aug 2000 16:01:17 +0200 Subject: [Python-Dev] Preventing recursion core dumps References: <000401c00389$2fa577b0$060210ac@private> Message-ID: <018a01c0039c$9f1949b0$0900a8c0@SPIFF> barry wrote: > For embedding Python being able to control the recursion depth of the intepreter > is very useful. I would want to be able to set, from C, the max call depth limit > and the current call depth limit. I'd expect Python to set a min call depth limit. +1 (on concept, at least). From thomas@xs4all.net Fri Aug 11 15:08:51 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 11 Aug 2000 16:08:51 +0200 Subject: [Python-Dev] Preventing recursion core dumps In-Reply-To: <200008111428.JAA04464@cj20424-a.reston1.va.home.com>; from guido@beopen.com on Fri, Aug 11, 2000 at 09:28:09AM -0500 References: <200008111239.OAA15818@python.inrialpes.fr> <200008111428.JAA04464@cj20424-a.reston1.va.home.com> Message-ID: <20000811160851.H17171@xs4all.nl> On Fri, Aug 11, 2000 at 09:28:09AM -0500, Guido van Rossum wrote: > It would be good if there was a way to sense the remaining available > stack, even if it wasn't portable. Any Linux experts out there? getrlimit and getrusage do what you want to, I think. getrusage() fills a struct rusage: struct rusage { struct timeval ru_utime; /* user time used */ struct timeval ru_stime; /* system time used */ long ru_maxrss; /* maximum resident set size */ long ru_ixrss; /* integral shared memory size */ long ru_idrss; /* integral unshared data size */ long ru_isrss; /* integral unshared stack size */ long ru_minflt; /* page reclaims */ long ru_majflt; /* page faults */ long ru_nswap; /* swaps */ long ru_inblock; /* block input operations */ long ru_oublock; /* block output operations */ long ru_msgsnd; /* messages sent */ long ru_msgrcv; /* messages received */ long ru_nsignals; /* signals received */ long ru_nvcsw; /* voluntary context switches */ long ru_nivcsw; /* involuntary context switches */ }; and you can get the actual stack limit with getrlimit(). The availability of getrusage/getrlimit is already checked by configure, and there's the resource module which wraps those functions and structs for Python code. Note that Linux isn't likely to be a problem, most linux distributions have liberal limits to start with (still the 'single-user OS' ;) BSDI, for instance, has very strict default limits -- the standard limits aren't even enough to start 'pine' on a few MB of mailbox. (But BSDI has rusage/rlimit, so we can 'autodetect' this.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From Moshe Zadka Fri Aug 11 15:13:13 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 11 Aug 2000 17:13:13 +0300 (IDT) Subject: [Python-Dev] Preventing recursion core dumps In-Reply-To: <200008111428.JAA04464@cj20424-a.reston1.va.home.com> Message-ID: On Fri, 11 Aug 2000, Guido van Rossum wrote: > It would be good if there was a way to sense the remaining available > stack, even if it wasn't portable. Any Linux experts out there? I'm far from an expert, but I might have an idea. The question is: must this works for embedded version of Python, or can I fool around with main()? Here's the approach: - In main(), get the address of some local variable. Call this min - Call getrlimit, and see the stack size. Call max = min+ ( There is no IGLU cabal. http://advogato.org/person/moshez From just@letterror.com Fri Aug 11 16:14:40 2000 From: just@letterror.com (Just van Rossum) Date: Fri, 11 Aug 2000 16:14:40 +0100 Subject: [Python-Dev] Preventing recursion core dumps Message-ID: > > Agreement on 5000? > > No, the __getattr__ example still dumps core for me. With 4000 it is > fine, but this indicates that this is totally the wrong approach: I > can change the available stack size with ulimit -s and cause a core > dump anyway. Or there could be a loger path through the C code where > more C stack is used per recursion. > > We could set the maximum to 1000 and assume a "reasonable" stack size, > but that doesn't make me feel comfortable either. > > It would be good if there was a way to sense the remaining available > stack, even if it wasn't portable. Any Linux experts out there? Gordon, how's that Stackless PEP coming along? Sorry, I couldn't resist ;-) Just From thomas@xs4all.net Fri Aug 11 15:21:09 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 11 Aug 2000 16:21:09 +0200 Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of data sent. In-Reply-To: <016d01c0039b$bfb99a40$0900a8c0@SPIFF>; from fredrik@pythonware.com on Fri, Aug 11, 2000 at 03:55:01PM +0200 References: <200008111255.HAA03735@cj20424-a.reston1.va.home.com> <20000811143143.G17171@xs4all.nl> <200008111419.JAA03948@cj20424-a.reston1.va.home.com> <016d01c0039b$bfb99a40$0900a8c0@SPIFF> Message-ID: <20000811162109.I17171@xs4all.nl> On Fri, Aug 11, 2000 at 03:55:01PM +0200, Fredrik Lundh wrote: > guido wrote: > > I just read the man page for send() (Red Hat linux 6.1) and it doesn't > > mention sending fewer than all bytes at all. In fact, while it says > > that the return value is the number of bytes sent, it at least > > *suggests* that it will return an error whenever not everything can be > > sent -- even in non-blocking mode. > > Under what circumstances can send() return a smaller number? > never, it seems: [snip manpage] Indeed. I didn't actually check the story, since Guido was apparently convinced by its validity. I was just operating under the assumption that send() did behave like write(). I won't blindly believe Guido anymore ! :) Someone set the patch to 'rejected' and tell the submittor that 'send' doesn't return the number of bytes written ;-P -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From Vladimir.Marangozov@inrialpes.fr Fri Aug 11 15:32:45 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 11 Aug 2000 16:32:45 +0200 (CEST) Subject: [Python-Dev] Preventing recursion core dumps In-Reply-To: from "Moshe Zadka" at Aug 11, 2000 05:13:13 PM Message-ID: <200008111432.QAA16648@python.inrialpes.fr> Moshe Zadka wrote: > > On Fri, 11 Aug 2000, Guido van Rossum wrote: > > > It would be good if there was a way to sense the remaining available > > stack, even if it wasn't portable. Any Linux experts out there? > > I'm far from an expert, but I might have an idea. The question is: must > this works for embedded version of Python, or can I fool around with > main()? Probably not main(), but Py_Initialize() for sure. > > Here's the approach: > > - In main(), get the address of some local variable. Call this > min > - Call getrlimit, and see the stack size. Call max = min+ ( - When checking for "too much recursion", take the address of a local > variable and compare it against max. If it's higher, stop. Sounds good. If getrlimit is not available, we can always fallback to some (yet to be computed) constant, i.e. the current state. [Just] > Gordon, how's that Stackless PEP coming along? > Sorry, I couldn't resist ;-) Ah, in this case, we'll get a memory error after filling the whole disk with frames -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From akuchlin@mems-exchange.org Fri Aug 11 15:33:35 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Fri, 11 Aug 2000 10:33:35 -0400 Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of data sent. In-Reply-To: <20000811162109.I17171@xs4all.nl>; from thomas@xs4all.net on Fri, Aug 11, 2000 at 04:21:09PM +0200 References: <200008111255.HAA03735@cj20424-a.reston1.va.home.com> <20000811143143.G17171@xs4all.nl> <200008111419.JAA03948@cj20424-a.reston1.va.home.com> <016d01c0039b$bfb99a40$0900a8c0@SPIFF> <20000811162109.I17171@xs4all.nl> Message-ID: <20000811103335.B20646@kronos.cnri.reston.va.us> On Fri, Aug 11, 2000 at 04:21:09PM +0200, Thomas Wouters wrote: >Someone set the patch to 'rejected' and tell the submittor that 'send' >doesn't return the number of bytes written ;-P What about reviving the idea of raising an exception, then? --amk From Moshe Zadka Fri Aug 11 15:40:10 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 11 Aug 2000 17:40:10 +0300 (IDT) Subject: [Python-Dev] Preventing recursion core dumps In-Reply-To: <200008111432.QAA16648@python.inrialpes.fr> Message-ID: On Fri, 11 Aug 2000, Vladimir Marangozov wrote: > Moshe Zadka wrote: > > > > On Fri, 11 Aug 2000, Guido van Rossum wrote: > > > > > It would be good if there was a way to sense the remaining available > > > stack, even if it wasn't portable. Any Linux experts out there? > > > > I'm far from an expert, but I might have an idea. The question is: must > > this works for embedded version of Python, or can I fool around with > > main()? > > Probably not main(), but Py_Initialize() for sure. Py_Initialize() isn't good enough -- I can put an upper bound on the difference between "min" and the top of the stack: I can't do so for the call to Py_Initialize(). Well, I probably can in some *really* ugly way. I'll have to think about it some more. > Sounds good. If getrlimit is not available, we can always fallback to > some (yet to be computed) constant, i.e. the current state. Well, since Guido asked for a non-portable Linuxish way, I think we can assume getrusage() is there. [Vladimir] > Ah, in this case, we'll get a memory error after filling the whole disk > with frames Which is great! Python promises to always throw an exception.... -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From thomas@xs4all.net Fri Aug 11 15:43:49 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 11 Aug 2000 16:43:49 +0200 Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of data sent. In-Reply-To: <20000811103335.B20646@kronos.cnri.reston.va.us>; from akuchlin@mems-exchange.org on Fri, Aug 11, 2000 at 10:33:35AM -0400 References: <200008111255.HAA03735@cj20424-a.reston1.va.home.com> <20000811143143.G17171@xs4all.nl> <200008111419.JAA03948@cj20424-a.reston1.va.home.com> <016d01c0039b$bfb99a40$0900a8c0@SPIFF> <20000811162109.I17171@xs4all.nl> <20000811103335.B20646@kronos.cnri.reston.va.us> Message-ID: <20000811164349.J17171@xs4all.nl> On Fri, Aug 11, 2000 at 10:33:35AM -0400, Andrew Kuchling wrote: > On Fri, Aug 11, 2000 at 04:21:09PM +0200, Thomas Wouters wrote: > >Someone set the patch to 'rejected' and tell the submittor that 'send' > >doesn't return the number of bytes written ;-P > What about reviving the idea of raising an exception, then? static PyObject * PySocketSock_send(PySocketSockObject *s, PyObject *args) { char *buf; int len, n, flags = 0; if (!PyArg_ParseTuple(args, "s#|i:send", &buf, &len, &flags)) return NULL; Py_BEGIN_ALLOW_THREADS n = send(s->sock_fd, buf, len, flags); Py_END_ALLOW_THREADS if (n < 0) return PySocket_Err(); return PyInt_FromLong((long)n); } (PySocket_Err() creates an error.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From guido@beopen.com Fri Aug 11 16:56:06 2000 From: guido@beopen.com (Guido van Rossum) Date: Fri, 11 Aug 2000 10:56:06 -0500 Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of data sent. In-Reply-To: Your message of "Fri, 11 Aug 2000 16:21:09 +0200." <20000811162109.I17171@xs4all.nl> References: <200008111255.HAA03735@cj20424-a.reston1.va.home.com> <20000811143143.G17171@xs4all.nl> <200008111419.JAA03948@cj20424-a.reston1.va.home.com> <016d01c0039b$bfb99a40$0900a8c0@SPIFF> <20000811162109.I17171@xs4all.nl> Message-ID: <200008111556.KAA05068@cj20424-a.reston1.va.home.com> > On Fri, Aug 11, 2000 at 03:55:01PM +0200, Fredrik Lundh wrote: > > guido wrote: > > > I just read the man page for send() (Red Hat linux 6.1) and it doesn't > > > mention sending fewer than all bytes at all. In fact, while it says > > > that the return value is the number of bytes sent, it at least > > > *suggests* that it will return an error whenever not everything can be > > > sent -- even in non-blocking mode. > > > > Under what circumstances can send() return a smaller number? > > > never, it seems: > > [snip manpage] > > Indeed. I didn't actually check the story, since Guido was apparently > convinced by its validity. I wasn't convinced! I wrote "is this true?" in my message!!! > I was just operating under the assumption that > send() did behave like write(). I won't blindly believe Guido anymore ! :) I bgelieve they do behave the same: in my mind, write() doesn't write fewer bytes than you tell it either! (Except maybe to a tty device when interrupted by a signal???) > Someone set the patch to 'rejected' and tell the submittor that 'send' > doesn't return the number of bytes written ;-P Done. Note that send() *does* return the number of bytes written. It's just always (supposed to be) the same as the length of the argument string. Since this is now established, should we change the send() method to raise an exception when it returns a smaller number? (The exception probably should be a subclass of socket.error and should carry the number of bytes written Could there be a signal interrupt issue here too? E.g. I send() a megabyte, which takes a while due to TCP buffer limits; before I'm done a signal handler interrupts the system call. Will send() now: (1) return a EINTR error (2) continue (3) return the number of bytes already written ??? --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From Vladimir.Marangozov@inrialpes.fr Fri Aug 11 16:58:45 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 11 Aug 2000 17:58:45 +0200 (CEST) Subject: [Python-Dev] Preventing recursion core dumps In-Reply-To: <200008111428.JAA04464@cj20424-a.reston1.va.home.com> from "Guido van Rossum" at Aug 11, 2000 09:28:09 AM Message-ID: <200008111558.RAA16953@python.inrialpes.fr> Guido van Rossum wrote: > > We could set the maximum to 1000 and assume a "reasonable" stack size, > but that doesn't make me feel comfortable either. Nor me, but it's more comfortable than a core dump, and is the only easy solution, solving most problems & probably breaking some code... After all, a max of 1024 seems to be a good suggestion. > > It would be good if there was a way to sense the remaining available > stack, even if it wasn't portable. Any Linux experts out there? On a second thought, I think this would be a bad idea, even if we manage to tweak the stack limits on most platforms. We would loose determinism = loose control -- no good. A depth-first algorithm may succeed on one machine, and fail on another. I strongly prefer to know that I'm limited to 1024 recursions ("reasonable" stack size assumptions included) and change my algorithm if it doesn't fly with my structure, than stumble subsequently on the fact that my algorithm works half the time. Changing this now *may* break such scripts, and there doesn't seem to be an immediate easy solution. But if I were to choose between breaking some scripts and preventing core dumps, well... -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From Moshe Zadka Fri Aug 11 17:12:21 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 11 Aug 2000 19:12:21 +0300 (IDT) Subject: [Python-Dev] Cookie.py Message-ID: This is a continuation of a previous server-side cookie support. There is a liberally licensed (old-Python license) framework called Webware, which includes Cookie.py, (apparently the same one by Timothy O'Malley). How about taking that Cookie.py? Webware can be found at http://webware.sourceforge.net/ -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From gmcm@hypernet.com Fri Aug 11 17:25:18 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Fri, 11 Aug 2000 12:25:18 -0400 Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of d In-Reply-To: <200008111419.JAA03948@cj20424-a.reston1.va.home.com> References: Your message of "Fri, 11 Aug 2000 14:31:43 +0200." <20000811143143.G17171@xs4all.nl> Message-ID: <1246111375-124272508@hypernet.com> [Guido] > I just read the man page for send() (Red Hat linux 6.1) and it > doesn't mention sending fewer than all bytes at all. In fact, > while it says that the return value is the number of bytes sent, > it at least *suggests* that it will return an error whenever not > everything can be sent -- even in non-blocking mode. It says (at least on RH 5.2): "If the message is too long to pass atomically through the underlying protocol...". Hey guys, TCP/IP is a stream protocol! For TCP/IP this is all completely misleading. Yes, it returns the number of bytes sent. For TCP/IP it is *not* an error to send less than the argument. It's only an error if the other end dies at the time of actual send. Python has been behaving properly all along. The bug report is correct. It's the usage of send in the std lib that is improper (though with a nearly infinitessimal chance of breaking, since it's almost all single threaded blocking usage of sockets). > Under what circumstances can send() return a smaller number? Just open a TCP/IP connection and send huge (64K or so) buffers. Current Python behavior is no different than C on Linux, HPUX and Windows. Look it up in Stevens if you don't believe me. Or try it. - Gordon From akuchlin@mems-exchange.org Fri Aug 11 17:26:08 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Fri, 11 Aug 2000 12:26:08 -0400 Subject: [Python-Dev] Cookie.py In-Reply-To: ; from moshez@math.huji.ac.il on Fri, Aug 11, 2000 at 07:12:21PM +0300 References: Message-ID: <20000811122608.F20646@kronos.cnri.reston.va.us> On Fri, Aug 11, 2000 at 07:12:21PM +0300, Moshe Zadka wrote: >This is a continuation of a previous server-side cookie support. >There is a liberally licensed (old-Python license) framework called >Webware, which includes Cookie.py, (apparently the same one by Timothy >O'Malley). How about taking that Cookie.py? O'Malley got in touch with me and let me know that the license has been changed to the 1.5.2 license with his departure from BBN. He hasn't sent me a URL where the current version can be downloaded, though. I don't know if WebWare has the most current version; it seems not, since O'Malley's was dated 06/21 and WebWare's was checked in on May 23. By the way, I'd suggest adding Cookie.py to a new 'web' package, and taking advantage of the move to break backward compatibility and remove the automatic usage of pickle (assuming it's still there). --amk From nascheme@enme.ucalgary.ca Fri Aug 11 17:37:01 2000 From: nascheme@enme.ucalgary.ca (Neil Schemenauer) Date: Fri, 11 Aug 2000 10:37:01 -0600 Subject: [Python-Dev] Preventing recursion core dumps In-Reply-To: <200008111558.RAA16953@python.inrialpes.fr>; from Vladimir Marangozov on Fri, Aug 11, 2000 at 05:58:45PM +0200 References: <200008111428.JAA04464@cj20424-a.reston1.va.home.com> <200008111558.RAA16953@python.inrialpes.fr> Message-ID: <20000811103701.A25386@keymaster.enme.ucalgary.ca> On Fri, Aug 11, 2000 at 05:58:45PM +0200, Vladimir Marangozov wrote: > On a second thought, I think this would be a bad idea, even if > we manage to tweak the stack limits on most platforms. We would > loose determinism = loose control -- no good. A depth-first algorithm > may succeed on one machine, and fail on another. So what? We don't limit the amount of memory you can allocate on all machines just because your program may run out of memory on some machine. It seems like the same thing to me. Neil From Moshe Zadka Fri Aug 11 17:40:31 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 11 Aug 2000 19:40:31 +0300 (IDT) Subject: [Python-Dev] Cookie.py In-Reply-To: <20000811122608.F20646@kronos.cnri.reston.va.us> Message-ID: On Fri, 11 Aug 2000, Andrew Kuchling wrote: > O'Malley got in touch with me and let me know that the license has > been changed to the 1.5.2 license with his departure from BBN. He > hasn't sent me a URL where the current version can be downloaded, > though. I don't know if WebWare has the most current version; it > seems not, since O'Malley's was dated 06/21 and WebWare's was checked > in on May 23. Well, as soon as you get a version, let me know: I've started working on documentation. > By the way, I'd suggest adding Cookie.py to a new 'web' package, and > taking advantage of the move to break backward compatibility and > remove the automatic usage of pickle (assuming it's still there). Well, depends on what you mean there: There are now three classes a) SimpleCookie -- never uses pickle b) SerilizeCookie -- always uses pickle c) SmartCookie -- uses pickle based on old heuristic. About web package: I'm +0. Fred has to think about how to document things in packages (we never had to until now). Well, who cares What is more important is working on documentation (which I'm doing), and on a regression test (for which the May 23 version is probably good enough). -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From thomas@xs4all.net Fri Aug 11 17:44:07 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 11 Aug 2000 18:44:07 +0200 Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of data sent. In-Reply-To: <200008111556.KAA05068@cj20424-a.reston1.va.home.com>; from guido@beopen.com on Fri, Aug 11, 2000 at 10:56:06AM -0500 References: <200008111255.HAA03735@cj20424-a.reston1.va.home.com> <20000811143143.G17171@xs4all.nl> <200008111419.JAA03948@cj20424-a.reston1.va.home.com> <016d01c0039b$bfb99a40$0900a8c0@SPIFF> <20000811162109.I17171@xs4all.nl> <200008111556.KAA05068@cj20424-a.reston1.va.home.com> Message-ID: <20000811184407.A14470@xs4all.nl> On Fri, Aug 11, 2000 at 10:56:06AM -0500, Guido van Rossum wrote: > > Indeed. I didn't actually check the story, since Guido was apparently > > convinced by its validity. > I wasn't convinced! I wrote "is this true?" in my message!!! I appologize... It's been a busy day for me, I guess I wasn't paying enough attention. I'll try to keep quiet when that happens, next time :P > > I was just operating under the assumption that > > send() did behave like write(). I won't blindly believe Guido anymore ! :) > I bgelieve they do behave the same: in my mind, write() doesn't write > fewer bytes than you tell it either! (Except maybe to a tty device > when interrupted by a signal???) Hm, I seem to recall write() could return after less than a full write, but I might be mistaken. I thought I was confusing send with write, but maybe I'm confusing both with some other function :-) I'm *sure* there is a function that behaves that way :P > Note that send() *does* return the number of bytes written. It's just > always (supposed to be) the same as the length of the argument string. > Since this is now established, should we change the send() method to > raise an exception when it returns a smaller number? (The exception > probably should be a subclass of socket.error and should carry the > number of bytes written Ahh, now it's starting to get clear to me. I'm not sure if it's worth it... It would require a different (non-POSIX) socket layer to return on 'incomplete' writes, and that is likely to break a number of other things, too. (Lets hope it does... a socket layer which has the same API but a different meaning would be very confusing !) > Could there be a signal interrupt issue here too? No, I don't think so. > E.g. I send() a megabyte, which takes a while due to TCP buffer limits; > before I'm done a signal handler interrupts the system call. Will send() > now: > (1) return a EINTR error Yes. From the manpage: If the message is too long to pass atomically through the underlying protocol, the error EMSGSIZE is returned, and the message is not transmitted. [..] ERRORS EINTR A signal occurred. [..] Because send() either completely succeeds or completely fails, I didn't see why you wanted an exception generated :) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From fdrake@beopen.com Fri Aug 11 17:45:13 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Fri, 11 Aug 2000 12:45:13 -0400 (EDT) Subject: [Python-Dev] Cookie.py In-Reply-To: References: <20000811122608.F20646@kronos.cnri.reston.va.us> Message-ID: <14740.11673.869664.837436@cj42289-a.reston1.va.home.com> Moshe Zadka writes: > About web package: I'm +0. Fred has to think about how to document > things in packages (we never had to until now). Well, who cares I'm not aware of any issues with documenting packages; the curses documentation seems to be coming along nicely, and that's a package. If you think I've missed something, we can (and should) deal with it in the Doc-SIG. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From akuchlin@mems-exchange.org Fri Aug 11 17:48:11 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Fri, 11 Aug 2000 12:48:11 -0400 Subject: [Python-Dev] Cookie.py In-Reply-To: ; from moshez@math.huji.ac.il on Fri, Aug 11, 2000 at 07:40:31PM +0300 References: <20000811122608.F20646@kronos.cnri.reston.va.us> Message-ID: <20000811124811.G20646@kronos.cnri.reston.va.us> On Fri, Aug 11, 2000 at 07:40:31PM +0300, Moshe Zadka wrote: >There are now three classes >a) SimpleCookie -- never uses pickle >b) SerilizeCookie -- always uses pickle >c) SmartCookie -- uses pickle based on old heuristic. Ah, good; never mind, then. >About web package: I'm +0. Fred has to think about how to document >things in packages (we never had to until now). Well, who cares Hmm... the curses.ascii module is already documented, so documenting packages shouldn't be a problem. --amk From esr@thyrsus.com Fri Aug 11 18:03:01 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Fri, 11 Aug 2000 13:03:01 -0400 Subject: [Python-Dev] Cookie.py In-Reply-To: <14740.11673.869664.837436@cj42289-a.reston1.va.home.com>; from fdrake@beopen.com on Fri, Aug 11, 2000 at 12:45:13PM -0400 References: <20000811122608.F20646@kronos.cnri.reston.va.us> <14740.11673.869664.837436@cj42289-a.reston1.va.home.com> Message-ID: <20000811130301.A7354@thyrsus.com> Fred L. Drake, Jr. : > I'm not aware of any issues with documenting packages; the curses > documentation seems to be coming along nicely, and that's a package. > If you think I've missed something, we can (and should) deal with it > in the Doc-SIG. The curses documentation is basically done. I've fleshed out the library reference and overhauled the HOWTO. I shipped the latter to amk yesterday because I couldn't beat CVS into checking out py-howtos for me. The items left on my to-do list are drafting PEP002 and doing something constructive about the Berkeley DB mess. I doubt I'll get to these things before LinuxWorld. Anybody else going? -- Eric S. Raymond It would be thought a hard government that should tax its people one tenth part. -- Benjamin Franklin From rushing@nightmare.com Fri Aug 11 17:59:07 2000 From: rushing@nightmare.com (Sam Rushing) Date: Fri, 11 Aug 2000 09:59:07 -0700 (PDT) Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of data sent. In-Reply-To: <284209072@toto.iv> Message-ID: <14740.12507.587044.121462@seattle.nightmare.com> Guido van Rossum writes: > Really?!?! > > I just read the man page for send() (Red Hat linux 6.1) and it doesn't > mention sending fewer than all bytes at all. In fact, while it says > that the return value is the number of bytes sent, it at least > *suggests* that it will return an error whenever not everything can be > sent -- even in non-blocking mode. > > Under what circumstances can send() return a smaller number? It's a feature of Linux... it will send() everything. Other unixen act in the classic fashion (it bit me on FreeBSD), and send only what fits right into the buffer that awaits. I think this could safely be added to the send method in socketmodule.c. Linux users wouldn't even notice. IMHO this is the kind of feature that people come to expect from programming in a HLL. Maybe disable the feature if it's a non-blocking socket? -Sam From billtut@microsoft.com Fri Aug 11 18:01:44 2000 From: billtut@microsoft.com (Bill Tutt) Date: Fri, 11 Aug 2000 10:01:44 -0700 Subject: [Python-Dev] Python keywords (was Lockstep iteration - eureka !) Message-ID: <58C671173DB6174A93E9ED88DCB0883D0A6121@red-msg-07.redmond.corp.microsoft.com> This is an alternative approach that we should certainly consider. We could use ANTLR (www.antlr.org) as our parser generator, and have it generate Java for JPython, and C++ for CPython. This would be a good chunk of work, and it's something I really don't have time to pursue. I don't even have time to pursue the idea about moving keyword recognition into the lexer. I'm just not sure if you want to bother introducing C++ into the Python codebase solely to only have one parser for CPython and JPython. Bill -----Original Message----- From: bwarsaw@beopen.com [mailto:bwarsaw@beopen.com] Sent: Thursday, August 10, 2000 8:01 PM To: Guido van Rossum Cc: Mark Hammond; python-dev@python.org Subject: Re: [Python-Dev] Python keywords (was Lockstep iteration - eureka!) >>>>> "GvR" == Guido van Rossum writes: GvR> Alas, I'm not sure how easy it will be. The parser generator GvR> will probably have to be changed to allow you to indicate not GvR> to do a resword lookup at certain points in the grammar. I GvR> don't know where to start. :-( Yet another reason why it would be nice to (eventually) merge the parsing technology in CPython and JPython. i-don't-wanna-work-i-jes-wanna-bang-on-my-drum-all-day-ly y'rs, -Barry _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://www.python.org/mailman/listinfo/python-dev From gmcm@hypernet.com Fri Aug 11 18:04:26 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Fri, 11 Aug 2000 13:04:26 -0400 Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of d In-Reply-To: <1246111375-124272508@hypernet.com> References: <200008111419.JAA03948@cj20424-a.reston1.va.home.com> Message-ID: <1246109027-124413737@hypernet.com> [I wrote, about send()] > Yes, it returns the number of bytes sent. For TCP/IP it is *not* > an error to send less than the argument. It's only an error if > the other end dies at the time of actual send. [and...] > Just open a TCP/IP connection and send huge (64K or so) > buffers. Current Python behavior is no different than C on > Linux, HPUX and Windows. And I just demonstrated it. Strangely enough, sending from Windows (where the dos say "send returns the total number of bytes sent, which can be less than the number indicated by len") it always sent the whole buffer, even when that was 1M on a non- blocking socket. (I select()'ed the socket first, to make sure it could send something). But from Linux, the largest buffer sent was 54,020 and typical was 27,740. No errors. - Gordon From thomas@xs4all.net Fri Aug 11 18:04:37 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 11 Aug 2000 19:04:37 +0200 Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of data sent. In-Reply-To: <14740.12507.587044.121462@seattle.nightmare.com>; from rushing@nightmare.com on Fri, Aug 11, 2000 at 09:59:07AM -0700 References: <284209072@toto.iv> <14740.12507.587044.121462@seattle.nightmare.com> Message-ID: <20000811190437.C17176@xs4all.nl> On Fri, Aug 11, 2000 at 09:59:07AM -0700, Sam Rushing wrote: > It's a feature of Linux... it will send() everything. Other unixen > act in the classic fashion (it bit me on FreeBSD), and send only what > fits right into the buffer that awaits. Ahhh, the downsides of working on the Most Perfect OS (writing this while our Technical Manager, a FreeBSD fan, is looking over my shoulder ;) Thanx for clearing that up. I was slowly going insane ;-P > I think this could safely be added to the send method in > socketmodule.c. Linux users wouldn't even notice. IMHO this is the > kind of feature that people come to expect from programming in a HLL. > Maybe disable the feature if it's a non-blocking socket? Hm, I'm not sure if that's the 'right' thing to do, though disabling it for non-blocking sockets is a nice idea. It shouldn't break anything, but it doesn't feel too 'right'. The safe option would be to add a function that resends as long as necessary, and point everyone to that function. But I'm not sure what the name should be -- send is just so obvious ;-) Perhaps you're right, perhaps we should consider this a job for the type of VHLL that Python is, and provide the opposite function separate instead: a non-resending send(), for those that really want it. But in the eyes of the Python programmer, socket.send() would just magically accept and send any message size you care to give it, so it shouldn't break things. I think ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From fdrake@beopen.com Fri Aug 11 18:16:43 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Fri, 11 Aug 2000 13:16:43 -0400 (EDT) Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of data sent. In-Reply-To: <20000811190437.C17176@xs4all.nl> References: <284209072@toto.iv> <14740.12507.587044.121462@seattle.nightmare.com> <20000811190437.C17176@xs4all.nl> Message-ID: <14740.13563.466035.477406@cj42289-a.reston1.va.home.com> Thomas Wouters writes: > Perhaps you're right, perhaps we should consider this a job for the type of > VHLL that Python is, and provide the opposite function separate instead: a > non-resending send(), for those that really want it. But in the eyes of the > Python programmer, socket.send() would just magically accept and send any > message size you care to give it, so it shouldn't break things. I think ;) This version receives my +1. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From gmcm@hypernet.com Fri Aug 11 18:38:01 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Fri, 11 Aug 2000 13:38:01 -0400 Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of d In-Reply-To: <20000811190437.C17176@xs4all.nl> References: <14740.12507.587044.121462@seattle.nightmare.com>; from rushing@nightmare.com on Fri, Aug 11, 2000 at 09:59:07AM -0700 Message-ID: <1246107013-124534915@hypernet.com> Thomas Wouters wrote: > On Fri, Aug 11, 2000 at 09:59:07AM -0700, Sam Rushing wrote: > > > It's a feature of Linux... it will send() everything. Other > > unixen act in the classic fashion (it bit me on FreeBSD), and > > send only what fits right into the buffer that awaits. ... > > I think this could safely be added to the send method in > > socketmodule.c. Linux users wouldn't even notice. IMHO this > > is the kind of feature that people come to expect from > > programming in a HLL. Maybe disable the feature if it's a > > non-blocking socket? > > Hm, I'm not sure if that's the 'right' thing to do, though > disabling it for non-blocking sockets is a nice idea. It's absolutely vital that it be disabled for non-blocking sockets. Otherwise you've just made it into a blocking socket. With that in place, I would be neutral on the change. I still feel that Python is already doing the right thing. The fact that everyone misunderstood the man page is not a good reason to change Python to match that misreading. > It > shouldn't break anything, but it doesn't feel too 'right'. The > safe option would be to add a function that resends as long as > necessary, and point everyone to that function. But I'm not sure > what the name should be -- send is just so obvious ;-) I've always thought that was why there was a makefile method. - Gordon From guido@beopen.com Fri Aug 11 23:05:32 2000 From: guido@beopen.com (Guido van Rossum) Date: Fri, 11 Aug 2000 17:05:32 -0500 Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of d In-Reply-To: Your message of "Fri, 11 Aug 2000 13:04:26 -0400." <1246109027-124413737@hypernet.com> References: <200008111419.JAA03948@cj20424-a.reston1.va.home.com> <1246109027-124413737@hypernet.com> Message-ID: <200008112205.RAA01218@cj20424-a.reston1.va.home.com> [Gordon] > [I wrote, about send()] > > Yes, it returns the number of bytes sent. For TCP/IP it is *not* > > an error to send less than the argument. It's only an error if > > the other end dies at the time of actual send. > > [and...] > > Just open a TCP/IP connection and send huge (64K or so) > > buffers. Current Python behavior is no different than C on > > Linux, HPUX and Windows. > > And I just demonstrated it. Strangely enough, sending from Windows > (where the dos say "send returns the total number of bytes sent, > which can be less than the number indicated by len") it always > sent the whole buffer, even when that was 1M on a non- > blocking socket. (I select()'ed the socket first, to make sure it > could send something). > > But from Linux, the largest buffer sent was 54,020 and typical > was 27,740. No errors. OK. So send() can do a partial write, but only on a stream connection. And most standard library code doesn't check for that condition, nor does (probably) much other code that used the standard library as an example. Worse, it seems that on some platforms send() *never* does a partial write (I couldn't reproduce it on Red Hat 6.1 Linux), so even stress testing may not reveal the lurking problem. Possible solutions: 1. Do nothing. Pro: least work. Con: subtle bugs remain. 2. Fix all code that's broken in the standard library, and try to encourage others to fix their code. Book authors need to be encouraged to add a warning. Pro: most thorough. Con: hard to fix every occurrence, especially in 3rd party code. 3. Fix the socket module to raise an exception when less than the number of bytes sent occurs. Pro: subtle bug exposed when it happens. Con: breaks code that did the right thing! 4. Fix the socket module to loop back on a partial send to send the remaining bytes. Pro: no more short writes. Con: what if the first few send() calls succeed and then an error is returned? Note: code that checks for partial writes will be redundant! I'm personally in favor of (4), despite the problem with errors after the first call. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From guido@beopen.com Fri Aug 11 23:14:23 2000 From: guido@beopen.com (Guido van Rossum) Date: Fri, 11 Aug 2000 17:14:23 -0500 Subject: [Python-Dev] missing mail Message-ID: <200008112214.RAA01257@cj20424-a.reston1.va.home.com> Just a note to you all. It seems I'm missing a lot of mail to python-dev. I noticed because I got a few mails cc'ed to me and never saw the copy sent via the list (which normally shows up within a minute). I looked in the archives and there were more messages that I hadn't seen at all (e.g. the entire Cookie thread). I don't know where the problem is (I *am* getting other mail to guido@python.org as well as to guido@beopen.com) and I have no time to research this right now. I'm going to be mostly off line this weekend and also all of next week. (I'll be able to read mail occasionally but I'll be too busy to keep track of everything.) So if you need me to reply, please cc me directly -- and please be patient! --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From huaiyu_zhu@yahoo.com Fri Aug 11 22:13:17 2000 From: huaiyu_zhu@yahoo.com (Huaiyu Zhu) Date: Fri, 11 Aug 2000 14:13:17 -0700 (PDT) Subject: [Python-Dev] Re: PEP 0211: Linear Algebra Operators In-Reply-To: Message-ID: As the PEP posted by Greg is substantially different from the one floating around in c.l.py, I'd like to post the latter here, which covers several weeks of discussions by dozens of discussants. I'd like to encourage Greg to post his version to python-list to seek comments. I'd be grateful to hear any comments. Python Extension Proposal: Adding new math operators Huaiyu Zhu 2000-08-11, draft 3 Introduction ------------ This PEP describes a proposal to add new math operators to Python, and summarises discussions in the news group comp.lang.python on this topic. Issues discussed here include: 1. Background. 2. Description of proposed operators and implementation issues. 3. Analysis of alternatives to new operators. 4. Analysis of alternative forms. 5. Compatibility issues 6. Description of wider extensions and other related ideas. A substantial portion of this PEP describes ideas that do not go into the proposed extension. They are presented because the extension is essentially syntactic sugar, so its adoption must be weighed against various possible alternatives. While many alternatives may be better in some aspects, the current proposal appears to be overall advantageous. Background ---------- Python provides five basic math operators, + - * / **. (Hereafter generically represented by "op"). They can be overloaded with new semantics for user-defined classes. However, for objects composed of homogeneous elements, such as arrays, vectors and matrices in numerical computation, there are two essentially distinct flavors of semantics. The objectwise operations treat these objects as points in multidimensional spaces. The elementwise operations treat them as collections of individual elements. These two flavors of operations are often intermixed in the same formulas, thereby requiring syntactical distinction. Many numerical computation languages provide two sets of math operators. For example, in Matlab, the ordinary op is used for objectwise operation while .op is used for elementwise operation. In R, op stands for elementwise operation while %op% stands for objectwise operation. In python, there are other methods of representation, some of which already used by available numerical packages, such as 1. function: mul(a,b) 2. method: a.mul(b) 3. casting: a.E*b In several aspects these are not as adequate as infix operators. More details will be shown later, but the key points are 1. Readability: Even for moderately complicated formulas, infix operators are much cleaner than alternatives. 2. Familiarity: Users are familiar with ordinary math operators. 3. Implementation: New infix operators will not unduly clutter python syntax. They will greatly ease the implementation of numerical packages. While it is possible to assign current math operators to one flavor of semantics, there is simply not enough infix operators to overload for the other flavor. It is also impossible to maintain visual symmetry between these two flavors if one of them does not contain symbols for ordinary math operators. Proposed extension ------------------ 1. New operators ~+ ~- ~* ~/ ~** ~+= ~-= ~*= ~/= ~**= are added to core Python. They parallel the existing operators + - * / ** and the (soon to be added) += -= *= /= **= operators. 2. Operator ~op retains the syntactical properties of operator op, including precedence. 3. Operator ~op retains the semantical properties of operator op on built-in number types. They raise syntax error on other types. 4. These operators are overloadable in classes with names that prepend "alt" to names of ordinary math operators. For example, __altadd__ and __raltadd__ work for ~+ just as __add__ and __radd__ work for +. 5. As with standard math operators, the __r*__() methods are invoked when the left operand does not provide the appropriate method. The symbol ~ is already used in Python as the unary "bitwise not" operator. Currently it is not allowed for binary operators. To allow it as part of binary operators, the tokanizer would treat ~+ as one token. This means that currently valid expression ~+1 would be tokenized as ~+ 1 instead of ~ + 1. The compiler would then treat ~+ as composite of ~ +. The proposed implementation is to patch several files relating to the parser and compiler to duplicate the functionality of existing math operators as necessary. All new semantics are to be implemented in the application that overloads them, but they are recommended to be conceptually similar to existing math operators. It is not specified which version of operators stands for elementwise or objectwise operations, leaving the decision to applications. A prototype implementation already exists. Alternatives to adding new operators ------------------------------------ Some of the leading alternatives, using the multiplication as an example. 1. Use function mul(a,b). Advantage: - No need for new operators. Disadvantage: - Prefix forms are cumbersome for composite formulas. - Unfamiliar to the intended users. - Too verbose for the intended users. - Unable to use natural precedence rules. 2. Use method call a.mul(b) Advantage: - No need for new operators. Disadvantage: - Asymmetric for both operands. - Unfamiliar to the intended users. - Too verbose for the intended users. - Unable to use natural precedence rules. 3. Use "shadow classes". For matrix class define a shadow array class accessible through a method .E, so that for matrices a and b, a.E*b would be a matrix object that is elementwise_mul(a,b). Likewise define a shadow matrix class for arrays accessible through a method .M so that for arrays a and b, a.M*b would be an array that is matrixwise_mul(a,b). Advantage: - No need for new operators. - Benefits of infix operators with correct precedence rules. - Clean formulas in applications. Disadvantage: - Hard to maintain in current Python because ordinary numbers cannot have user defined class methods. (a.E*b will fail if a is a pure number.) - Difficult to implement, as this will interfere with existing method calls, like .T for transpose, etc. - Runtime overhead of object creation and method lookup. - The shadowing class cannot replace a true class, because it does not return its own type. So there need to be a M class with shadow E class, and an E class with shadow M class. - Unnatural to mathematicians. 4. Implement matrixwise and elementwise classes with easy casting to the other class. So matrixwise operations for arrays would be like a.M*b.M and elementwise operations for matrices would be like a.E*b.E. For error detection a.E*b.M would raise exceptions. Advantage: - No need for new operators. - Similar to infix notation with correct precedence rules. Disadvantage: - Similar difficulty due to lack of user-methods for pure numbers. - Runtime overhead of object creation and method lookup. - More cluttered formulas - Switching of flavor of objects to facilitate operators becomes persistent. This introduces long range context dependencies in application code that would be extremely hard to maintain. 5. Using mini parser to parse formulas written in arbitrary extension placed in quoted strings. Advantage: - Pure Python, without new operators Disadvantage: - The actual syntax is within the quoted string, which does not resolve the problem itself. - Introducing zones of special syntax. - Demanding on the mini-parser. Among these alternatives, the first and second are used in current applications to some extent, but found inadequate. The third is the most favorite for applications, but it will incur huge implementation complexity. The fourth would make applications codes very contex-sensitive and hard to maintain. These two alternatives also share significant implementational difficulties due to current type/class split. The fifth appears to create more problems than it would solve. Alternative forms of infix operators ------------------------------------ Two major forms and several minor variants of new infix operators were discussed: 1. Bracketed form (op) [op] {op} :op: ~op~ %op% 2. Meta character form .op @op ~op Alternatively the meta character is put after the operator. 3. Less consistent variations of these themes. These are considered unfavorably. For completeness some are listed here - Use @/ and /@ for left and right division - Use [*] and (*) for outer and inner products 4. Use __call__ to simulate multiplication. a(b) or (a)(b) Criteria for choosing among the representations include: - No syntactical ambiguities with existing operators. - Higher readability in actual formulas. This makes the bracketed forms unfavorable. See examples below. - Visually similar to existing math operators. - Syntactically simple, without blocking possible future extensions. With these criteria the overall winner in bracket form appear to be {op}. A clear winner in the meta character form is ~op. Comparing these it appears that ~op is the favorite among them all. Some analysis are as follows: - The .op form is ambiguous: 1.+a would be different from 1 .+a. - The bracket type operators are most favorable when standing alone, but not in formulas, as they interfere with visual parsing of parenthesis for precedence and function argument. This is so for (op) and [op], and somewhat less so for {op} and . - The form has the potential to be confused with < > and =. - The @op is not favored because @ is visually heavy (dense, more like a letter): a@+b is more readily read as a@ + b than a @+ b. - For choosing meta-characters: Most of existing ASCII symbols have already been used. The only three unused are @ $ ?. Semantics of new operators -------------------------- There are convincing arguments for using either set of operators as objectwise or elementwise. Some of them are listed here: 1. op for element, ~op for object - Consistent with current multiarray interface of Numeric package - Consistent with some other languages - Perception that elementwise operations are more natural - Perception that elementwise operations are used more frequently 2. op for object, ~op for element - Consistent with current linear algebra interface of MatPy package - Consistent with some other languages - Perception that objectwise operations are more natural - Perception that objectwise operations are used more frequently - Consistent with the current behavior of operators on lists - Allow ~ to be a general elementwise meta-character in future extensions. It is generally agreed upon that - there is no absolute reason to favor one or the other - it is easy to cast from one representation to another in a sizable chunk of code, so the other flavor of operators is always minority - there are other semantic differences that favor existence of array-oriented and matrix-oriented packages, even if their operators are unified. - whatever the decision is taken, codes using existing interfaces should not be broken for a very long time. Therefore not much is lost, and much flexibility retained, if the semantic flavors of these two sets of operators are not dictated by the core language. The application packages are responsible for making the most suitable choice. This is already the case for NumPy and MatPy which use opposite semantics. Adding new operators will not break this. See also observation after subsection 2 in the Examples below. The issue of numerical precision was raised, but if the semantics is left to the applications, the actual precisions should also go there. Examples -------- Following are examples of the actual formulas that will appear using various operators or other representations described above. 1. The matrix inversion formula: - Using op for object and ~op for element: b = a.I - a.I * u / (c.I + v/a*u) * v / a b = a.I - a.I * u * (c.I + v*a.I*u).I * v * a.I - Using op for element and ~op for object: b = a.I @- a.I @* u @/ (c.I @+ v@/a@*u) @* v @/ a b = a.I ~- a.I ~* u ~/ (c.I ~+ v~/a~*u) ~* v ~/ a b = a.I (-) a.I (*) u (/) (c.I (+) v(/)a(*)u) (*) v (/) a b = a.I [-] a.I [*] u [/] (c.I [+] v[/]a[*]u) [*] v [/] a b = a.I <-> a.I <*> u (c.I <+> va<*>u) <*> v a b = a.I {-} a.I {*} u {/} (c.I {+} v{/}a{*}u) {*} v {/} a Observation: For linear algebra using op for object is preferable. Observation: The ~op type operators look better than (op) type in complicated formulas. - using named operators b = a.I @sub a.I @mul u @div (c.I @add v @div a @mul u) @mul v @div a b = a.I ~sub a.I ~mul u ~div (c.I ~add v ~div a ~mul u) ~mul v ~div a Observation: Named operators are not suitable for math formulas. 2. Plotting a 3d graph - Using op for object and ~op for element: z = sin(x~**2 ~+ y~**2); plot(x,y,z) - Using op for element and ~op for object: z = sin(x**2 + y**2); plot(x,y,z) Observation: Elementwise operations with broadcasting allows much more efficient implementation than Matlab. Observation: Swapping the semantics of op and ~op (by casting the objects) is often advantageous, as the ~op operators would only appear in chunks of code where the other flavor dominate. 3. Using + and - with automatic broadcasting a = b - c; d = a.T*a Observation: This would silently produce hard-to-trace bugs if one of b or c is row vector while the other is column vector. Miscellaneous issues: --------------------- 1. Need for the ~+ ~- operators. The objectwise + - are important because they provide important sanity checks as per linear algebra. The elementwise + - are important because they allow broadcasting that are very efficient in applications. 2. Left division (solve). For matrix, a*x is not necessarily equal to x*a. The solution of a*x==b, denoted x=solve(a,b), is therefore different from the solution of x*a==b, denoted x=div(b,a). There are discussions about finding a new symbol for solve. [Background: Matlab use b/a for div(b,a) and a\b for solve(a,b).] It is recognized that Python provides a better solution without requiring a new symbol: the inverse method .I can be made to be delayed so that a.I*b and b*a.I are equivalent to Matlab's a\b and b/a. The implementation is quite simple and the resulting application code clean. 3. Power operator. Python's use of a**b as pow(a,b) has two perceived disadvantages: - Most mathematicians are more familiar with a^b for this purpose. - It results in long augmented assignment operator ~**=. However, this issue is distinct from the main issue here. 4. Additional multiplication operators. Several forms of multiplications are used in (multi-)linear algebra. Most can be seen as variations of multiplication in linear algebra sense (such as Kronecker product). But two forms appear to be more fundamental: outer product and inner product. However, their specification includes indices, which can be either - associated with the operator, or - associated with the objects. The latter (the Einstein notation) is used extensively on paper, and is also the easier one to implement. By implementing a tensor-with-indices class, a general form of multiplication would cover both outer and inner products, and specialize to linear algebra multiplication as well. The index rule can be defined as class methods, like, a = b.i(1,2,-1,-2) * c.i(4,-2,3,-1) # a_ijkl = b_ijmn c_lnkm Therefore one objectwise multiplication is sufficient. 5. Bitwise operators. Currently Python assigns six operators to bitwise operations: and (&), or (|), xor (^), complement (~), left shift (<<) and right shift (>>), with their own precedence levels. This has some barings on the new math operators in several ways: - The proposed new math operators use the symbol ~ that is "bitwise not" operator. This poses no compatibility problem but somewhat complicates implementation. - The symbol ^ might be better used for pow than bitwise xor. But this depends on the future of bitwise operators. It does not immediately impact on the proposed math operator. - The symbol | was suggested to be used for matrix solve. But the new solution of using delayed .I is better in several ways. - The bitwise operators assign special syntactical and semantical structures to operations, which could be more consistently regarded as elementwise lattice operators. (see below) Most of their usage could be replaced by a bitwise module with named functions. Removing ~ as a single operator could also allow notations that link them to logical operators (see below). However, this issue is separate from the current proposed extension. 6. Lattice operators. It was suggested that similar operators be combined with bitwise operators to represent lattice operations. For example, ~| and ~& could represent "lattice or" and "lattice and". But these can already be achieved by overloading existing logical or bitwise operators. On the other hand, these operations might be more deserving for infix operators than the built-in bitwise operators do (see below). 7. Alternative to special operator names used in definition, def "+"(a, b) in place of def __add__(a, b) This appears to require greater syntactical change, and would only be useful when arbitrary additional operators are allowed. 8. There was a suggestion to provide a copy operator :=, but this can already be done by a=b.copy. Impact on possible future extensions: ------------------------------------- More general extensions could lead from the current proposal. Although they would be distinct proposals, they might have syntactical or semantical implications on each other. It is prudent to ensure that the current extension do not restrict any future possibilities. 1. Named operators. The news group discussion made it generally clear that infix operators is a scarce resource in Python, not only in numerical computation, but in other fields as well. Several proposals and ideas were put forward that would allow infix operators be introduced in ways similar to named functions. The idea of named infix operators is essentially this: Choose a meta character, say @, so that for any identifier "opname", the combination "@opname" would be a binary infix operator, and a @opname b == opname(a,b) Other representations mentioned include .name ~name~ :name: (.name) %name% and similar variations. The pure bracket based operators cannot be used this way. This requires a change in the parser to recognize @opname, and parse it into the same structure as a function call. The precedence of all these operators would have to be fixed at one level, so the implementation would be different from additional math operators which keep the precedence of existing math operators. The current proposed extension do not limit possible future extensions of such form in any way. 2. More general symbolic operators. One additional form of future extension is to use meta character and operator symbols (symbols that cannot be used in syntactical structures other than operators). Suppose @ is the meta character. Then a + b, a @+ b, a @@+ b, a @+- b would all be operators with a hierarchy of precedence, defined by def "+"(a, b) def "@+"(a, b) def "@@+"(a, b) def "@+-"(a, b) One advantage compared with named operators is greater flexibility for precedences based on either the meta character or the ordinary operator symbols. This also allows operator composition. The disadvantage is that they are more like "line noise". In any case the current proposal does not impact its future possibility. These kinds of future extensions may not be necessary when Unicode becomes generally available. 3. Object/element dichotomy for other types of objects. The distinction between objectwise and elementwise operations are meaningful in other contexts as well, where an object can be conceptually regarded as a collection of homogeneous elements. Several examples are listed here: - List arithmetics [1, 2] + [3, 4] # [1, 2, 3, 4] [1, 2] ~+ [3, 4] # [4, 6] ['a', 'b'] * 2 # ['a', 'b', 'a', 'b'] 'ab' * 2 # 'abab' ['a', 'b'] ~* 2 # ['aa', 'bb'] [1, 2] ~* 2 # [2, 4] It is also consistent to Cartesian product [1,2]*[3,4] # [(1,3),(1,4),(2,3),(2,4)] - Tuple generation [1, 2, 3], [4, 5, 6] # ([1,2, 3], [4, 5, 6]) [1, 2, 3]~,[4, 5, 6] # [(1,4), (2, 5), (3,6)] This has the same effect as the proposed zip function. - Bitwise operation (regarding integer as collection of bits, and removing the dissimilarity between bitwise and logical operators) 5 and 6 # 6 5 or 6 # 5 5 ~and 6 # 4 5 ~or 6 # 7 - Elementwise format operator (with broadcasting) a = [1,2,3,4,5] print ["%5d "] ~% a # print ("%5s "*len(a)) % tuple(a) a = [[1,2],[3,4]] print ["%5d "] ~~% a - Using ~ as generic elementwise meta-character to replace map ~f(a, b) # map(f, a, b) ~~f(a, b) # map(lambda *x:map(f, *x), a, b) More generally, def ~f(*x): return map(f, *x) def ~~f(*x): return map(~f, *x) ... - Rich comparison [1, 2, 3, 4] ~< [4, 3, 2, 1] # [1, 1, 0, 0] There are probably many other similar situations. This general approach seems well suited for most of them, in place of several separated proposals for each of them (parallel and cross iteration, list comprehension, rich comparison, and some others). Of course, the sementics of "elementwise" depends on applications. For example an element of matrix is two levels down from list of list point of view. In any case, the current proposal will not negatively impact on future possibilities of this nature. Note that this section discusses compatibility of the proposed extension with possible future extensions. The desirability or compatibility of these other extensions themselves are specifically not considered here. -- Huaiyu Zhu hzhu@users.sourceforge.net Matrix for Python Project http://MatPy.sourceforge.net From trentm@ActiveState.com Fri Aug 11 22:30:31 2000 From: trentm@ActiveState.com (Trent Mick) Date: Fri, 11 Aug 2000 14:30:31 -0700 Subject: [Python-Dev] *.dsp and *.dsw are treated by CVS as binary. Why? Message-ID: <20000811143031.A13790@ActiveState.com> These files (PCbuild/*.dsw PCbuild/*.dsp) are just normal text files. Why, then, do we treat them as binary files. Would it not be preferable to have those files be handled like a normal text files, i.e. check it out on Unix and it uses Unix line terminators, check it out on Windows and it uses DOS line terminators. This way you are using the native line terminator format and text processing tools you use on them a less likely to screw them up. (Anyone see my motivation?). Does anybody see any problems treating them as text files? And, if not, who knows how to get rid of the '-kb' sticky tag on those files. Thanks, Trent -- Trent Mick TrentM@ActiveState.com From gmcm@hypernet.com Fri Aug 11 22:34:54 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Fri, 11 Aug 2000 17:34:54 -0400 Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of d In-Reply-To: <200008112205.RAA01218@cj20424-a.reston1.va.home.com> References: Your message of "Fri, 11 Aug 2000 13:04:26 -0400." <1246109027-124413737@hypernet.com> Message-ID: <1246092799-125389828@hypernet.com> [Guido] > OK. So send() can do a partial write, but only on a stream > connection. And most standard library code doesn't check for > that condition, nor does (probably) much other code that used the > standard library as an example. Worse, it seems that on some > platforms send() *never* does a partial write (I couldn't > reproduce it on Red Hat 6.1 Linux), so even stress testing may > not reveal the lurking problem. I'm quite sure you can force it with a non-blocking socket (on RH 5.2 64K blocks did it - but that's across a 10baseT ethernet connection). > Possible solutions: > > 1. Do nothing. Pro: least work. Con: subtle bugs remain. Yes, but they're application-level bugs, even if they're in the std lib. They're not socket-support level bugs. > 2. Fix all code that's broken in the standard library, and try to > encourage others to fix their code. Book authors need to be > encouraged to add a warning. Pro: most thorough. Con: hard to > fix every occurrence, especially in 3rd party code. As far as I can tell, Linux and Windows can't fail with the std lib code (it's all blocking sockets). Sam says BSDI could fail, and as I recall HPUX could too. > 3. Fix the socket module to raise an exception when less than the > number of bytes sent occurs. Pro: subtle bug exposed when it > happens. Con: breaks code that did the right thing! > > 4. Fix the socket module to loop back on a partial send to send > the remaining bytes. Pro: no more short writes. Con: what if > the first few send() calls succeed and then an error is returned? > Note: code that checks for partial writes will be redundant! If you can exempt non-blocking sockets, either 3 or 4 (preferably 4) is OK. But if you can't exempt non-blocking sockets, I'll scream bloody murder. It would mean you could not write high performance socket code in Python (which it currently is very good for). For one thing, you'd break Medusa. > I'm personally in favor of (4), despite the problem with errors > after the first call. The sockets HOWTO already documents the problem. Too bad I didn't write it before that std lib code got written . I still prefer leaving it alone and telling people to use makefile if they can't deal with it. I'll vote +0 on 4 if and only if it exempts nonblocking sockets. - Gordon From nowonder@nowonder.de Sat Aug 12 00:48:20 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Fri, 11 Aug 2000 23:48:20 +0000 Subject: [Python-Dev] Python keywords (was Lockstep iteration - eureka!) References: <58C671173DB6174A93E9ED88DCB0883D0A6121@red-msg-07.redmond.corp.microsoft.com> Message-ID: <399490C4.F234D68A@nowonder.de> Bill Tutt wrote: > > This is an alternative approach that we should certainly consider. We could > use ANTLR (www.antlr.org) as our parser generator, and have it generate Java > for JPython, and C++ for CPython. This would be a good chunk of work, and > it's something I really don't have time to pursue. I don't even have time to > pursue the idea about moving keyword recognition into the lexer. ANTLR is a great tool. Unfortunately - although trying hard to change it this morning in order to suppress keyword lookup in certain places - I don't know anything about the interface between Python and its parser. Is there some documentation on that (or can some divine deity guide me with a few hints where to look in Parser/*)? > I'm just not sure if you want to bother introducing C++ into the Python > codebase solely to only have one parser for CPython and JPython. Which compilers/platforms would this affect? VC++/Windows won't be a problem, I guess; gcc mostly comes with g++, but not always as a default. Probably more problematic. don't-know-about-VMS-and-stuff-ly y'rs Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From guido@beopen.com Fri Aug 11 23:56:23 2000 From: guido@beopen.com (Guido van Rossum) Date: Fri, 11 Aug 2000 17:56:23 -0500 Subject: [Python-Dev] *.dsp and *.dsw are treated by CVS as binary. Why? In-Reply-To: Your message of "Fri, 11 Aug 2000 14:30:31 MST." <20000811143031.A13790@ActiveState.com> References: <20000811143031.A13790@ActiveState.com> Message-ID: <200008112256.RAA01675@cj20424-a.reston1.va.home.com> > These files (PCbuild/*.dsw PCbuild/*.dsp) are just normal text files. Why, > then, do we treat them as binary files. DevStudio doesn't (or at least 5.x didn't) like it if not all lines used CRLF terminators. > Would it not be preferable to have those files be handled like a normal text > files, i.e. check it out on Unix and it uses Unix line terminators, check it > out on Windows and it uses DOS line terminators. I think I made them binary during the period when I was mounting the Unix source directory on a Windows machine. I don't do that any more and I don't know anyone who does, so I think it's okay to change. > This way you are using the native line terminator format and text processing > tools you use on them a less likely to screw them up. (Anyone see my > motivation?). > > Does anybody see any problems treating them as text files? And, if not, who > knows how to get rid of the '-kb' sticky tag on those files. cvs admin -kkv file ... --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From guido@beopen.com Sat Aug 12 00:00:29 2000 From: guido@beopen.com (Guido van Rossum) Date: Fri, 11 Aug 2000 18:00:29 -0500 Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of d In-Reply-To: Your message of "Fri, 11 Aug 2000 17:34:54 -0400." <1246092799-125389828@hypernet.com> References: Your message of "Fri, 11 Aug 2000 13:04:26 -0400." <1246109027-124413737@hypernet.com> <1246092799-125389828@hypernet.com> Message-ID: <200008112300.SAA01726@cj20424-a.reston1.va.home.com> > > 4. Fix the socket module to loop back on a partial send to send > > the remaining bytes. Pro: no more short writes. Con: what if > > the first few send() calls succeed and then an error is returned? > > Note: code that checks for partial writes will be redundant! > > If you can exempt non-blocking sockets, either 3 or 4 > (preferably 4) is OK. But if you can't exempt non-blocking > sockets, I'll scream bloody murder. It would mean you could > not write high performance socket code in Python (which it > currently is very good for). For one thing, you'd break Medusa. Of course. Don't worry. > > I'm personally in favor of (4), despite the problem with errors > > after the first call. > > The sockets HOWTO already documents the problem. Too > bad I didn't write it before that std lib code got written . > > I still prefer leaving it alone and telling people to use makefile if > they can't deal with it. I'll vote +0 on 4 if and only if it exempts > nonblocking sockets. Understood. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From fdrake@beopen.com Fri Aug 11 23:21:18 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Fri, 11 Aug 2000 18:21:18 -0400 (EDT) Subject: [Python-Dev] Python keywords (was Lockstep iteration - eureka!) In-Reply-To: <399490C4.F234D68A@nowonder.de> References: <58C671173DB6174A93E9ED88DCB0883D0A6121@red-msg-07.redmond.corp.microsoft.com> <399490C4.F234D68A@nowonder.de> Message-ID: <14740.31838.336790.710005@cj42289-a.reston1.va.home.com> Peter Schneider-Kamp writes: > parser. Is there some documentation on that (or can some divine deity > guide me with a few hints where to look in Parser/*)? Not that I'm aware of! Feel free to write up any overviews you think appropriate, and it can become part of the standard documentation or be a README in the Parser/ directory. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From tim_one@email.msn.com Sat Aug 12 01:59:22 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 11 Aug 2000 20:59:22 -0400 Subject: [Python-Dev] *.dsp and *.dsw are treated by CVS as binary. Why? In-Reply-To: <20000811143031.A13790@ActiveState.com> Message-ID: [Trent Mick] > These files (PCbuild/*.dsw PCbuild/*.dsp) are just normal text files. > Why, then, do we treat them as binary files. > > Would it not be preferable to have those files be handled like > a normal text files, i.e. check it out on Unix and it uses Unix > line terminators, check it out on Windows and it uses DOS line > terminators. > > This way you are using the native line terminator format and text > processing tools you use on them a less likely to screw them up. > (Anyone see my motivation?). Not really. They're not human-editable! Leave 'em alone. Keeping them in binary mode is a clue to people that they aren't *supposed* to go mucking with them via text processing tools. > Does anybody see any problems treating them as text files? And, > if not, who knows how to get rid of the '-kb' sticky tag on those > files. Well, whatever you did didn't work. I'm dead in the water on Windows now -- VC6 refuses to open the new & improved .dsw and .dsp files. I *imagine* it's because they've got Unix line-ends now, but haven't yet checked. Can you fix it or back it out? From skip@mojam.com (Skip Montanaro) Sat Aug 12 02:07:31 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Fri, 11 Aug 2000 20:07:31 -0500 (CDT) Subject: [Python-Dev] list comprehensions Message-ID: <14740.41811.590487.13187@beluga.mojam.com> I believe the latest update to the list comprehensions patch by Ping resolved the last concert the BDFL(*) had. As the owner of the patch is it my responsibility to check it in or do I need to assign it to Guido for final dispensation. Skip (*) Took me a week or so to learn what BDFL meant. :-) I tried a lot of "somewhat inaccurate" expansions before seeing it expanded in a message from Barry... From esr@thyrsus.com Sat Aug 12 03:50:17 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Fri, 11 Aug 2000 22:50:17 -0400 Subject: [Python-Dev] Stop the presses! Message-ID: <20000811225016.A18449@thyrsus.com> The bad news: I've found another reproducible core-dump bug in Python-2.0 under Linux. Actually I found it in 1.5.2 while making some changes to CML2, and just verified that the CVS snapshot of Python 2.0 bombs identically. The bad news II: it really seems to be in the Python core, not one of the extensions like Tkinter. My curses and Tk front ends both segfault in the same place, the guard of an innocuous-looking if statement. The good news: the patch to go from code-that-runs to code-that-bombs is pretty small and clean. I suspect anybody who really knows the ceval internals will be able to use it to nail this bug fairly quickly. Damn, seems like I found the core dump in Pickle just yesterday. This is getting to be a habit I don't enjoy much :-(. I'm putting together a demonstration package now. Stay tuned; I'll ship it tonight. -- Eric S. Raymond "One of the ordinary modes, by which tyrants accomplish their purposes without resistance, is, by disarming the people, and making it an offense to keep arms." -- Constitutional scholar and Supreme Court Justice Joseph Story, 1840 From ping@lfw.org Sat Aug 12 03:56:50 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Fri, 11 Aug 2000 19:56:50 -0700 (PDT) Subject: [Python-Dev] Stop the presses! In-Reply-To: <20000811225016.A18449@thyrsus.com> Message-ID: On Fri, 11 Aug 2000, Eric S. Raymond wrote: > The good news: the patch to go from code-that-runs to code-that-bombs > is pretty small and clean. I suspect anybody who really knows the ceval > internals will be able to use it to nail this bug fairly quickly. [...] > I'm putting together a demonstration package now. Stay tuned; I'll > ship it tonight. Oooh, i can't wait. How exciting! Post it, post it! :) -- ?!ng "This code is better than any code that doesn't work has any right to be." -- Roger Gregory, on Xanadu From fdrake@beopen.com Sat Aug 12 04:30:23 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Fri, 11 Aug 2000 23:30:23 -0400 (EDT) Subject: [Python-Dev] list comprehensions In-Reply-To: <14740.41811.590487.13187@beluga.mojam.com> References: <14740.41811.590487.13187@beluga.mojam.com> Message-ID: <14740.50383.386575.806754@cj42289-a.reston1.va.home.com> Skip Montanaro writes: > I believe the latest update to the list comprehensions patch by > Ping resolved the last concert the BDFL(*) had. As the owner of > the patch is it my responsibility to check it in or do I need to > assign it to Guido for final dispensation. Given the last comment added to the patch, check it in and close the patch. Then finish the PEP so we don't have to explain it over and over and ... ;-) -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From esr@thyrsus.com Sat Aug 12 04:56:33 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Fri, 11 Aug 2000 23:56:33 -0400 Subject: [Python-Dev] Directions for reproducing the coredump Message-ID: <20000811235632.A19358@thyrsus.com> Here are the directions to reproduce the core dump. 1. Download and unpack CML2 version 0.7.6 from . Change directory into it. 2. Do `cmlcompile.py kernel-rules.cml' to generate a pickled rulebase. 3. Run `make xconfig'. Ignore the error message about the arch defconfig. 4. Set NETDEVICES on the main menu to 'y'. 5. Select the "Network Device Configuration" menu that appears below. 6. Set PPP to 'y'. 7. Select the "PPP options" menu that appears below it. 8. Set PPP_BSDCOMP to 'y'. 9. Observe and dismiss the pop-up window. Quit the configurator using the "File" menu on the menu bar. 10. Now apply the attached patch. 11. Repeat steps 2-10. 12. Observe the core dump. If you look near cmlsystem.py:770, you'll see that the patch inserted two print statements that bracket the apparent point of the core dump. 13. To verify that this core dump is neither a Tkinter nor an ncurses problem, run `make menuconfig'. 14. Repeat steps 2-8. To set symbols in the curses interface, use the arrow keys to select each one and type "y". To select a menu, use the arrow keys and type a space or Enter when the selection bar is over the entry. 15. Observe the core dump at the same spot. This bug bites both a stock 1.5.2 and today's CVS snapshoot of 2.0. --- cml.py 2000/08/12 03:21:40 1.97 +++ cml.py 2000/08/12 03:25:45 @@ -111,6 +111,21 @@ res = res + self.dump() return res[:-1] + "}" +class Requirement: + "A requirement, together with a message to be shown if it's violated." + def __init__(self, wff, message=None): + self.predicate = wff + self.message = message + + def str(self): + return display_expression(self.predicate) + + def __repr__(self): + if self.message: + return self.message + else: + return str(self) + # This describes an entire configuration. class CMLRulebase: --- cmlcompile.py 2000/08/10 16:22:39 1.131 +++ cmlcompile.py 2000/08/12 03:24:31 @@ -12,7 +12,7 @@ _keywords = ('symbols', 'menus', 'start', 'unless', 'suppress', 'dependent', 'menu', 'choices', 'derive', 'default', - 'range', 'require', 'prohibit', 'private', 'debug', + 'range', 'require', 'prohibit', 'explanation', 'private', 'debug', 'helpfile', 'prefix', 'banner', 'icon', 'condition', 'trits', 'on', 'warndepend') @@ -432,7 +432,14 @@ expr = parse_expr(input) if leader.type == "prohibit": expr = ('==', expr, cml.n.value) - requirements.append(expr) + msg = None + #next = input.lex_token() + #if next.type != 'explanation': + # input.push_token(next) + # continue + #else: + # msg = input.demand("word") + requirements.append(cml.Requirement(expr, msg)) bool_tests.append((expr, input.infile, input.lineno)) elif leader.type == "default": symbol = input.demand("word") @@ -746,7 +753,7 @@ entry.visibility = resolve(entry.visibility) if entry.default: entry.default = resolve(entry.default) - requirements = map(resolve, requirements) + requirements = map(lambda x: cml.Requirement(resolve(x.predicate), x.message), requirements) if bad_symbols: sys.stderr.write("cmlcompile: %d symbols could not be resolved:\n"%(len(bad_symbols),)) sys.stderr.write(`bad_symbols.keys()` + "\n") @@ -868,7 +875,7 @@ # rule file are not consistent, it's not likely the user will make # a consistent one. for wff in requirements: - if not cml.evaluate(wff, debug): + if not cml.evaluate(wff.predicate, debug): print "cmlcompile: constraint violation:", wff errors = 1 --- cmlsystem.py 2000/07/25 04:24:53 1.98 +++ cmlsystem.py 2000/08/12 03:29:21 @@ -28,6 +28,7 @@ "INCAUTOGEN":"/*\n * Automatically generated, don't edit\n */\n", "INCDERIVED":"/*\n * Derived symbols\n */\n", "ISNOTSET":"# %s is not set\n", + "NOTRITS":"Trit values are not currently allowed.", "RADIOINVIS":" Query of choices menu %s elided, button pressed", "READING":"Reading configuration from %s", "REDUNDANT":" Redundant assignment forced by %s", @@ -100,10 +101,10 @@ "Assign constraints to their associated symbols." for entry in self.dictionary.values(): entry.constraints = [] - for wff in self.constraints: - for symbol in cml.flatten_expr(wff): - if not wff in symbol.constraints: - symbol.constraints.append(wff) + for requirement in self.constraints: + for symbol in cml.flatten_expr(requirement.predicate): + if not requirement.predicate in symbol.constraints: + symbol.constraints.append(requirement) if self.debug: cc = dc = tc = 0 for symbol in self.dictionary.values(): @@ -436,8 +437,8 @@ if symbol.constraints: self.set_symbol(symbol, value) for constraint in symbol.constraints: - if not cml.evaluate(constraint, self.debug): - self.debug_emit(1, self.lang["CONSTRAINT"] % (value, symbol.name, constraint)) + if not cml.evaluate(constraint.predicate, self.debug): + self.debug_emit(1, self.lang["CONSTRAINT"] % (value, symbol.name, str(constraint))) self.rollback() return 0 self.rollback() @@ -544,7 +545,7 @@ # be unfrozen. Simplify constraints to remove newly frozen variables. # Then rerun optimize_constraint_access. if freeze: - self.constraints = map(lambda wff, self=self: self.simplify(wff), self.constraints) + self.constraints = map(lambda requirement, self=self: cml.Requirement(self.simplify(requirement.predicate), requirement.message), self.constraints) self.optimize_constraint_access() for entry in self.dictionary.values(): if self.bindcheck(entry, self.newbindings) and entry.menu and entry.menu.type=="choices": @@ -559,7 +560,7 @@ violations = [] # First, check the explicit constraints. for constraint in self.constraints: - if not cml.evaluate(constraint, self.debug): + if not cml.evaluate(constraint.predicate, self.debug): violations.append(constraint); self.debug_emit(1, self.lang["FAILREQ"] % (constraint,)) # If trits are disabled, any variable having a trit value is wrong. @@ -570,7 +571,7 @@ mvalued = ('and', ('!=', entry,cml.m), mvalued) if mvalued != cml.y: mvalued = self.simplify(mvalued) - violations.append(('implies', ('==', self.trit_tie, cml.n), mvalued)) + violations.append(cml.Requirement(('implies', ('==', self.trit_tie, cml.n), mvalued), self.lang["NOTRITS"])) return violations def set_symbol(self, symbol, value, source=None): @@ -631,10 +632,10 @@ dups = {} relevant = [] for csym in touched: - for wff in csym.constraints: - if not dups.has_key(wff): - relevant.append(wff) - dups[wff] = 1 + for requirement in csym.constraints: + if not dups.has_key(requirement.predicate): + relevant.append(requirement.predicate) + dups[requirement.predicate] = 1 # Now loop through the constraints, simplifying out assigned # variables and trying to freeze more variables each time. # The outer loop guarantees that as long as the constraints @@ -765,7 +766,9 @@ else: self.set_symbol(left, cml.n.value, source) return 1 + print "Just before the core-dump point" if right_mutable and left == cml.n.value: + print "Just after the core-dump point" if rightnonnull == cml.n.value: self.debug_emit(1, self.lang["REDUNDANT"] % (wff,)) return 0 End of diffs, -- Eric S. Raymond A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects. -- Robert A. Heinlein, "Time Enough for Love" From nhodgson@bigpond.net.au Sat Aug 12 05:16:54 2000 From: nhodgson@bigpond.net.au (Neil Hodgson) Date: Sat, 12 Aug 2000 14:16:54 +1000 Subject: [Python-Dev] Winreg update References: <3993FEC7.4E38B4F1@prescod.net> Message-ID: <045901c00414$27a67010$8119fea9@neil> > .... It also seems that there are a lot of people (let's > call them "back seat coders") who have vague ideas of what they want but > don't want to spend a bunch of time in a long discussion about registry > arcana. Therefore I am endevouring to make it as easy and fast to > contribute to the discussion as possible. I think a lot of the registry using people are unwilling to spend too much energy on this because, while it looks useless, its not really going to be a problem so long as the low level module is available. > If you're one of the people who has asked for winreg in the core then > you should respond. It isn't (IMO) sufficient to put in a hacky API to > make your life easier. You need to give something to get something. You > want windows registry support in the core -- fine, let's do it properly. Hacky API only please. The registry is just not important enough to have this much attention or work. > All you need to do is read this email and comment on whether you agree > with the overall principle and then give your opinion on fifteen > possibly controversial issues. The "overall principle" is to steal > shamelessly from Microsoft's new C#/VB/OLE/Active-X/CRL API instead of > innovating for Python. That allows us to avoid starting the debate from > scratch. It also eliminates the feature that Mark complained about > (which was a Python-specific innovation). The Microsoft.Win32.Registry* API appears to be a hacky legacy API to me. Its there for compatibility during the transition to the System.Configuration API. Read the blurb for ConfigManager to understand the features of System.Configuration. Its all based on XML files. What a surprise. Neil From ping@lfw.org Sat Aug 12 05:52:54 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Fri, 11 Aug 2000 21:52:54 -0700 (PDT) Subject: [Python-Dev] Directions for reproducing the coredump In-Reply-To: <20000811235632.A19358@thyrsus.com> Message-ID: On Fri, 11 Aug 2000, Eric S. Raymond wrote: > Here are the directions to reproduce the core dump. I have successfully reproduced the core dump. -- ?!ng "This code is better than any code that doesn't work has any right to be." -- Roger Gregory, on Xanadu From ping@lfw.org Sat Aug 12 05:57:02 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Fri, 11 Aug 2000 21:57:02 -0700 (PDT) Subject: [Python-Dev] Directions for reproducing the coredump In-Reply-To: Message-ID: On Fri, 11 Aug 2000, Ka-Ping Yee wrote: > I have successfully reproduced the core dump. I'm investigating. Top of the stack looks like: #0 0x40061e39 in __pthread_lock (lock=0x0, self=0x40067f20) at spinlock.c:41 #1 0x4005f8aa in __pthread_mutex_lock (mutex=0xbfe0277c) at mutex.c:92 #2 0x400618cb in __flockfile (stream=0xbfe02794) at lockfile.c:32 #3 0x400d2955 in _IO_vfprintf (s=0xbfe02794, format=0x80d1500 "'%.50s' instance has no attribute '%.400s'", ap=0xbfe02a54) at vfprintf.c:1041 #4 0x400e00b3 in _IO_vsprintf (string=0xbfe02850 "Ø/à ¿", format=0x80d1500 "'%.50s' instance has no attribute '%.400s'", args=0xbfe02a54) at iovsprintf.c:47 #5 0x80602c5 in PyErr_Format (exception=0x819783c, format=0x80d1500 "'%.50s' instance has no attribute '%.400s'") at errors.c:377 #6 0x806eac4 in instance_getattr1 (inst=0x84ecdd4, name=0x81960a8) at classobject.c:594 #7 0x806eb97 in instance_getattr (inst=0x84ecdd4, name=0x81960a8) at classobject.c:639 #8 0x807b445 in PyObject_GetAttrString (v=0x84ecdd4, name=0x80d306b "__str__") at object.c:595 #9 0x807adf8 in PyObject_Str (v=0x84ecdd4) at object.c:291 #10 0x8097d1e in builtin_str (self=0x0, args=0x85adc3c) at bltinmodule.c:2034 #11 0x805a490 in call_builtin (func=0x81917e0, arg=0x85adc3c, kw=0x0) at ceval.c:2369 -- ?!ng "This code is better than any code that doesn't work has any right to be." -- Roger Gregory, on Xanadu From tim_one@email.msn.com Sat Aug 12 07:29:42 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 12 Aug 2000 02:29:42 -0400 Subject: [Python-Dev] list comprehensions In-Reply-To: <14740.41811.590487.13187@beluga.mojam.com> Message-ID: [Skip Montanaro] > I believe the latest update to the list comprehensions patch by Ping > resolved the last concert the BDFL(*) had. As the owner of the > patch is it my responsibility to check it in or do I need to assign > it to Guido for final dispensation. As the owner of the listcomp PEP, I both admonish you to wait until the PEP is complete, and secretly encourage you to check it in anyway (unlike most PEPs, this one is pre-approved no matter what I write <0.5 wink> -- better to get the code out there now! if anything changes due to the PEP, should be easy to twiddle). acting-responsibly-despite-appearances-ly y'rs - tim From tim_one@email.msn.com Sat Aug 12 08:32:17 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 12 Aug 2000 03:32:17 -0400 Subject: [Python-Dev] Directions for reproducing the coredump In-Reply-To: <20000811235632.A19358@thyrsus.com> Message-ID: [Eric S. Raymond, with a lot of code that dies in a lot of pain] Eric, as I'm running on a Windows laptop right now, there's not much I can do to try to run this code. However, something struck me in your patch "by eyeball", and here's a self-contained program that crashes under Windows: # This is esr's new class. class Requirement: "A requirement, together with a message to be shown if it's violated." def __init__(self, wff, message=None): self.predicate = wff self.message = message def str(self): return display_expression(self.predicate) def __repr__(self): if self.message: return self.message else: return str(self) # This is my driver. r = Requirement("trust me, I'm a wff!") print r Could that be related to your problem? I think you really wanted to name "str" as "__str__" in this class (or if not, comment in extreme detail why you want to confuse the hell out of the reader ). As is, my print r attempts to get look up r.__str__, which isn't found, so Python falls back to using r.__repr__. That *is* found, but r.message is None, so Requirement.__repr__ executes return str(self) And then we go thru the whole "no __str__" -> "try __repr__" -> "message is None" -> "return str(self)" business again, and end up with unbounded recursion. The top of the stacktrace Ping posted *does* show that the code is failing to find a "__str__" attr, so that's consistent with the scenario described here. If this is the problem, note that ways to detect such kinds of unbounded recursion have been discussed here within the last week. You're a clever enough fellow that I have to suspect you concocted this test case as a way to support the more extreme of those proposals without just saying "+1" . From ping@lfw.org Sat Aug 12 09:09:53 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Sat, 12 Aug 2000 01:09:53 -0700 (PDT) Subject: [Python-Dev] Directions for reproducing the coredump In-Reply-To: Message-ID: On Fri, 11 Aug 2000, Ka-Ping Yee wrote: > On Fri, 11 Aug 2000, Ka-Ping Yee wrote: > > I have successfully reproduced the core dump. > > I'm investigating. Top of the stack looks like: This chunk of stack repeats lots and lots of times. The problem is due to infinite recursion in your __repr__ routine: class Requirement: "A requirement, together with a message to be shown if it's violated." def __init__(self, wff, message=None): self.predicate = wff self.message = message def str(self): return display_expression(self.predicate) def __repr__(self): if self.message: return self.message else: return str(self) Notice that Requirement.__repr__ calls str(self), which triggers Requirement.__repr__ again because there is no __str__ method. If i change "def str(self)" to "def __str__(self)", the problem goes away and everything works properly. With a reasonable stack depth limit in place, this would produce a run-time error rather than a segmentation fault. -- ?!ng "This code is better than any code that doesn't work has any right to be." -- Roger Gregory, on Xanadu From ping@lfw.org Sat Aug 12 09:22:40 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Sat, 12 Aug 2000 01:22:40 -0700 (PDT) Subject: [Python-Dev] Directions for reproducing the coredump In-Reply-To: Message-ID: On Sat, 12 Aug 2000, Tim Peters wrote: > Could that be related to your problem? I think you really wanted to name > "str" as "__str__" in this class Oops. I guess i should have just read the code before going through the whole download procedure. Uh, yeah. What he said. :) That wise man with the moustache over there. One thing i ran into as a result of trying to run it under the debugger, though: turning on cursesmodule was slightly nontrivial. There's no cursesmodule.c; it's _cursesmodule.c instead; but Modules/Setup says "#curses cursesmodule.c". Taking out the "#" wasn't sufficient; i had to edit and insert the underscores by hand to get curses to work. -- ?!ng From Fredrik Lundh" <20000811143143.G17171@xs4all.nl> <200008111419.JAA03948@cj20424-a.reston1.va.home.com> <016d01c0039b$bfb99a40$0900a8c0@SPIFF> <20000811162109.I17171@xs4all.nl> <200008111556.KAA05068@cj20424-a.reston1.va.home.com> Message-ID: <00d301c0043d$7eb0b540$f2a6b5d4@hagrid> guido wrote: > > Indeed. I didn't actually check the story, since Guido was = apparently > > convinced by its validity. >=20 > I wasn't convinced! I wrote "is this true?" in my message!!! >=20 > > I was just operating under the assumption that > > send() did behave like write(). I won't blindly believe Guido = anymore ! :) >=20 > I bgelieve they do behave the same: in my mind, write() doesn't write > fewer bytes than you tell it either! (Except maybe to a tty device > when interrupted by a signal???) SUSv2 again: If a write() requests that more bytes be written than there is room for (for example, the ulimit or the physical end of a medium), only as many bytes as there is room for will be written. For example, suppose there is space for 20 bytes more in a file before reaching a limit. A write of 512 bytes will return 20. The next write of a non-zero number of bytes will give a failure return (except as noted below) and the implementation will generate a SIGXFSZ signal for the thread.=20 If write() is interrupted by a signal before it writes any data, it will return -1 with errno set to [EINTR].=20 If write() is interrupted by a signal after it successfully writes some data, it will return the number of bytes written.=20 sockets are an exception: If fildes refers to a socket, write() is equivalent to send() with no flags set. fwiw, if "send" may send less than the full buffer in blocking mode on some platforms (despite what the specification implies), it's quite interesting that *nobody* has ever noticed before... From Fredrik Lundh" <200008112256.RAA01675@cj20424-a.reston1.va.home.com> Message-ID: <00d601c0043d$a2e66c20$f2a6b5d4@hagrid> guido wrote: > I think I made them binary during the period when I was mounting the > Unix source directory on a Windows machine. I don't do that any more > and I don't know anyone who does we do. trent wrote: > > Does anybody see any problems treating them as text files? developer studio 5.0 does: "This makefile was not generated by Developer Studio" "Continuing will create a new Developer Studio project to wrap this makefile. You will be prompted to save after the new project has been created". "Do you want to continue" (click yes) "The options file (.opt) for this workspace specified a project configuration "... - Win32 Alpha Release" that no longer exists. The configuration will be set to "... - Win32 Debug" (click OK) (click build) "MAKE : fatal error U1052: file '....mak' not found" From thomas@xs4all.net Sat Aug 12 10:21:19 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sat, 12 Aug 2000 11:21:19 +0200 Subject: [Python-Dev] Directions for reproducing the coredump In-Reply-To: ; from ping@lfw.org on Sat, Aug 12, 2000 at 01:22:40AM -0700 References: Message-ID: <20000812112119.C14470@xs4all.nl> On Sat, Aug 12, 2000 at 01:22:40AM -0700, Ka-Ping Yee wrote: > One thing i ran into as a result of trying to run it under the > debugger, though: turning on cursesmodule was slightly nontrivial. > There's no cursesmodule.c; it's _cursesmodule.c instead; but > Modules/Setup says "#curses cursesmodule.c". Taking out the "#" > wasn't sufficient; i had to edit and insert the underscores by hand > to get curses to work. You should update your Setup file, then ;) Compare it with Setup.in and see what else changed since the last time you configured Python. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From martin@loewis.home.cs.tu-berlin.de Sat Aug 12 10:29:25 2000 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Sat, 12 Aug 2000 11:29:25 +0200 Subject: [Python-Dev] Processing CVS diffs Message-ID: <200008120929.LAA01434@loewis.home.cs.tu-berlin.de> While looking at the comments for Patch #100654, I noticed a complaint about the patch being a CVS diff, which is not easily processed by patch. There is a simple solution to that: process the patch with the script below. It will change the patch in-place, and it works well for me even though it is written in the Evil Language :-) Martin #! /usr/bin/perl -wi # Propagate the full pathname from the Index: line in CVS output into # the diff itself so that patch will use it. # Thrown together by Jason Merrill while (<>) { if (/^Index: (.*)/) { $full = $1; print; for (1..7) { $_ = <>; s/ [^\t]+\t/ $full\t/; print; } } else { print; } } From mal@lemburg.com Sat Aug 12 10:48:25 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Sat, 12 Aug 2000 11:48:25 +0200 Subject: [Python-Dev] Python Announcements ??? Message-ID: <39951D69.45D01703@lemburg.com> Could someone at BeOpen please check what happened to the python-announce mailing list ?! Messages to that list don't seem to show up anywhere and I've been getting strange reports from the mail manager software in the past when I've tried to post there. Also, what happened to the idea of hooking that list onto the c.l.p.a newsgroup. I don't remember the details of how this is done (had to do something with adding some approved header), but this would be very helpful. The Python community currently has no proper way of announcing new projects, software or gigs. A post to c.l.p which has grown to be a >4K posts/month list does not have the same momentum as pushing it through c.l.p.a had in the past. Thanks, -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From just@letterror.com Sat Aug 12 12:51:31 2000 From: just@letterror.com (Just van Rossum) Date: Sat, 12 Aug 2000 12:51:31 +0100 Subject: [Python-Dev] Preventing recursion core dumps Message-ID: (Sorry for the late reply, that's what you get when you don't Cc me...) Vladimir Marangozov wrote: > [Just] > > Gordon, how's that Stackless PEP coming along? > > Sorry, I couldn't resist ;-) > > Ah, in this case, we'll get a memory error after filling the whole disk > with frames No matter how much we wink to each other, that was a cheap shot; especially since it isn't true: Stackless has a MAX_RECURSION_DEPTH value. Someone who has studied Stackless "in detail" (your words ;-) should know that. Admittedly, that value is set way too high in the last stackless release (123456 ;-), but that doesn't change the principle that Stackless could solve the problem discussed in this thread in a reliable and portable manner. Of course there's be work to do: - MAX_RECURSION_DEPTH should be changeable at runtime - __str__ (and a bunch of others) isn't yet stackless - ... But the hardest task seems to be to get rid of the hostility and prejudices against Stackless :-( Just From esr@thyrsus.com Sat Aug 12 12:22:55 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Sat, 12 Aug 2000 07:22:55 -0400 Subject: [Python-Dev] Directions for reproducing the coredump In-Reply-To: ; from tim_one@email.msn.com on Sat, Aug 12, 2000 at 03:32:17AM -0400 References: <20000811235632.A19358@thyrsus.com> Message-ID: <20000812072255.C20109@thyrsus.com> Tim Peters : > If this is the problem, note that ways to detect such kinds of unbounded > recursion have been discussed here within the last week. You're a clever > enough fellow that I have to suspect you concocted this test case as a way > to support the more extreme of those proposals without just saying "+1" > . I may be that clever, but I ain't that devious. I'll try the suggested fix. Very likely you're right, though the location of the core dump is peculiar if this is the case. It's inside bound_from_constraint(), whereas in your scenario I'd expect it to be in the Requirement method code. -- Eric S. Raymond The day will come when the mystical generation of Jesus by the Supreme Being as his father, in the womb of a virgin, will be classed with the fable of the generation of Minerva in the brain of Jupiter. -- Thomas Jefferson, 1823 From esr@thyrsus.com Sat Aug 12 12:34:19 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Sat, 12 Aug 2000 07:34:19 -0400 Subject: [Python-Dev] Directions for reproducing the coredump In-Reply-To: ; from ping@lfw.org on Sat, Aug 12, 2000 at 01:22:40AM -0700 References: Message-ID: <20000812073419.D20109@thyrsus.com> Ka-Ping Yee : > One thing i ran into as a result of trying to run it under the > debugger, though: turning on cursesmodule was slightly nontrivial. > There's no cursesmodule.c; it's _cursesmodule.c instead; but > Modules/Setup says "#curses cursesmodule.c". Taking out the "#" > wasn't sufficient; i had to edit and insert the underscores by hand > to get curses to work. Your Setup is out of date. But this reminds me. There's way too much hand-hacking in the Setup mechanism. It wouldn't be hard to enhance the Setup format to support #if/#endif so that config.c generation could take advantage of configure tests. That way, Setup could have constructs in it like this: #if defined(CURSES) #if defined(linux) _curses _cursesmodule.c -lncurses #else _curses _cursesmodule.c -lcurses -ltermcap #endif #endif I'm willing to do and test this. -- Eric S. Raymond The right of the citizens to keep and bear arms has justly been considered as the palladium of the liberties of a republic; since it offers a strong moral check against usurpation and arbitrary power of rulers; and will generally, even if these are successful in the first instance, enable the people to resist and triumph over them." -- Supreme Court Justice Joseph Story of the John Marshall Court From esr@snark.thyrsus.com Sat Aug 12 12:44:54 2000 From: esr@snark.thyrsus.com (Eric S. Raymond) Date: Sat, 12 Aug 2000 07:44:54 -0400 Subject: [Python-Dev] Core dump is dead, long live the core dump Message-ID: <200008121144.HAA20230@snark.thyrsus.com> Tim's diagnosis of fatal recursion was apparently correct; apologies, all. This still leaves the question of why the core dump happened so far from the actual scene of the crime. -- Eric S. Raymond In every country and in every age, the priest has been hostile to liberty. He is always in alliance with the despot, abetting his abuses in return for protection to his own. -- Thomas Jefferson, 1814 From mal@lemburg.com Sat Aug 12 12:36:14 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Sat, 12 Aug 2000 13:36:14 +0200 Subject: [Python-Dev] Directions for reproducing the coredump References: <20000812073419.D20109@thyrsus.com> Message-ID: <399536AE.309D456C@lemburg.com> "Eric S. Raymond" wrote: > > Ka-Ping Yee : > > One thing i ran into as a result of trying to run it under the > > debugger, though: turning on cursesmodule was slightly nontrivial. > > There's no cursesmodule.c; it's _cursesmodule.c instead; but > > Modules/Setup says "#curses cursesmodule.c". Taking out the "#" > > wasn't sufficient; i had to edit and insert the underscores by hand > > to get curses to work. > > Your Setup is out of date. > > But this reminds me. There's way too much hand-hacking in the Setup > mechanism. It wouldn't be hard to enhance the Setup format to support > #if/#endif so that config.c generation could take advantage of > configure tests. That way, Setup could have constructs in it like > this: > > #if defined(CURSES) > #if defined(linux) > _curses _cursesmodule.c -lncurses > #else > _curses _cursesmodule.c -lcurses -ltermcap > #endif > #endif > > I'm willing to do and test this. This would be a *cool* thing to have :-) Definitely +1 from me if it's done in a portable way. (Not sure how you would get this to run without the C preprocessor though -- and Python's Makefile doesn't provide any information on how to call it in a platform independent way. It's probably cpp on most platforms, but you never know...) -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From esr@thyrsus.com Sat Aug 12 12:50:57 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Sat, 12 Aug 2000 07:50:57 -0400 Subject: [Python-Dev] Directions for reproducing the coredump In-Reply-To: <399536AE.309D456C@lemburg.com>; from mal@lemburg.com on Sat, Aug 12, 2000 at 01:36:14PM +0200 References: <20000812073419.D20109@thyrsus.com> <399536AE.309D456C@lemburg.com> Message-ID: <20000812075056.A20245@thyrsus.com> M.-A. Lemburg : > > But this reminds me. There's way too much hand-hacking in the Setup > > mechanism. It wouldn't be hard to enhance the Setup format to support > > #if/#endif so that config.c generation could take advantage of > > configure tests. That way, Setup could have constructs in it like > > this: > > > > #if defined(CURSES) > > #if defined(linux) > > _curses _cursesmodule.c -lncurses > > #else > > _curses _cursesmodule.c -lcurses -ltermcap > > #endif > > #endif > > > > I'm willing to do and test this. > > This would be a *cool* thing to have :-) > > Definitely +1 from me if it's done in a portable way. > > (Not sure how you would get this to run without the C preprocessor > though -- and Python's Makefile doesn't provide any information > on how to call it in a platform independent way. It's probably > cpp on most platforms, but you never know...) Ah. The Makefile may not provide this information -- but I believe configure can be made to! -- Eric S. Raymond Ideology, politics and journalism, which luxuriate in failure, are impotent in the face of hope and joy. -- P. J. O'Rourke From thomas@xs4all.net Sat Aug 12 12:53:46 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sat, 12 Aug 2000 13:53:46 +0200 Subject: [Python-Dev] Directions for reproducing the coredump In-Reply-To: <20000812073419.D20109@thyrsus.com>; from esr@thyrsus.com on Sat, Aug 12, 2000 at 07:34:19AM -0400 References: <20000812073419.D20109@thyrsus.com> Message-ID: <20000812135346.D14470@xs4all.nl> On Sat, Aug 12, 2000 at 07:34:19AM -0400, Eric S. Raymond wrote: > But this reminds me. There's way too much hand-hacking in the Setup > mechanism. It wouldn't be hard to enhance the Setup format to support > #if/#endif so that config.c generation could take advantage of > configure tests. That way, Setup could have constructs in it like > this: > #if defined(CURSES) > #if defined(linux) > _curses _cursesmodule.c -lncurses > #else > _curses _cursesmodule.c -lcurses -ltermcap > #endif > #endif Why go through that trouble ? There already is a 'Setup.config' file, which is used to pass Setup info for the thread and gc modules. It can easily be extended to include information on all other locatable modules, leaving 'Setup' or 'Setup.local' for people who have their modules in strange places. What would be a cool idea as well would be a configuration tool. Not as complex as the linux kernel config tool, but something to help people select the modules they want. Though it might not be necessary if configure finds out what modules can be safely built. I'm willing to write some autoconf tests to locate modules as well, if this is deemed a good idea. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Sat Aug 12 12:54:31 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sat, 12 Aug 2000 13:54:31 +0200 Subject: [Python-Dev] Core dump is dead, long live the core dump In-Reply-To: <200008121144.HAA20230@snark.thyrsus.com>; from esr@snark.thyrsus.com on Sat, Aug 12, 2000 at 07:44:54AM -0400 References: <200008121144.HAA20230@snark.thyrsus.com> Message-ID: <20000812135431.E14470@xs4all.nl> On Sat, Aug 12, 2000 at 07:44:54AM -0400, Eric S. Raymond wrote: > Tim's diagnosis of fatal recursion was apparently correct; apologies, > all. This still leaves the question of why the core dump happened so > far from the actual scene of the crime. Blame it on your stack :-) It could have been that the appropriate error was generated, and that the stack overflowed *again* during the processing of that error :-) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From gmcm@hypernet.com Sat Aug 12 14:16:47 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Sat, 12 Aug 2000 09:16:47 -0400 Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of d In-Reply-To: <00d301c0043d$7eb0b540$f2a6b5d4@hagrid> Message-ID: <1246036275-128789882@hypernet.com> Fredrik wrote: > fwiw, if "send" may send less than the full buffer in blocking > mode on some platforms (despite what the specification implies), > it's quite interesting that *nobody* has ever noticed before... I noticed, but I expected it, so had no reason to comment. The Linux man pages are the only specification of send that I've seen that don't make a big deal out it. And clearly I'm not the only one, otherwise there would never have been a bug report (he didn't experience it, he just noticed sends without checks). - Gordon From guido@beopen.com Sat Aug 12 15:48:11 2000 From: guido@beopen.com (Guido van Rossum) Date: Sat, 12 Aug 2000 09:48:11 -0500 Subject: [Python-Dev] Re: PEP 0211: Linear Algebra Operators In-Reply-To: Your message of "Fri, 11 Aug 2000 14:13:17 MST." References: Message-ID: <200008121448.JAA03545@cj20424-a.reston1.va.home.com> > As the PEP posted by Greg is substantially different from the one floating > around in c.l.py, I'd like to post the latter here, which covers several > weeks of discussions by dozens of discussants. I'd like to encourage Greg > to post his version to python-list to seek comments. A procedural suggestion: let's have *two* PEPs, one for Huaiyu's proposal, one for Greg's. Each PEP should in its introduction briefly mention the other as an alternative. I don't generally recommend that alternative proposals develop separate PEPs, but in this case the potential impact on Python is so large that I think it's the only way to proceed that doesn't give one group an unfair advantage over the other. I haven't had the time to read either proposal yet, so I can't comment on their (in)compatibility, but I would surmise that at most one can be accepted -- with the emphasis on *at most* (possibly neither is ready for prime time), and with the understanding that each proposal may be able to borrow ideas or code from the other anyway. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From Vladimir.Marangozov@inrialpes.fr Sat Aug 12 15:21:50 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Sat, 12 Aug 2000 16:21:50 +0200 (CEST) Subject: [Python-Dev] Preventing recursion core dumps In-Reply-To: <20000811103701.A25386@keymaster.enme.ucalgary.ca> from "Neil Schemenauer" at Aug 11, 2000 10:37:01 AM Message-ID: <200008121421.QAA20095@python.inrialpes.fr> Neil Schemenauer wrote: > > On Fri, Aug 11, 2000 at 05:58:45PM +0200, Vladimir Marangozov wrote: > > On a second thought, I think this would be a bad idea, even if > > we manage to tweak the stack limits on most platforms. We would > > loose determinism = loose control -- no good. A depth-first algorithm > > may succeed on one machine, and fail on another. > > So what? Well, the point is that people like deterministic behavior and tend to really dislike unpredictable systems, especially when the lack of determinism is due to platform heterogeneity. > We don't limit the amount of memory you can allocate on all > machines just because your program may run out of memory on some > machine. We don't because we can't do it portably. But if we could, this would have been a very useful setting -- there has been demand for Python on embedded systems where memory size is a constraint. And note that after the malloc cleanup, we *can* do this with a specialized Python malloc (control how much memory is allocated from Python). -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From Fredrik Lundh" Message-ID: <001301c00469$cb380fe0$f2a6b5d4@hagrid> gordon wrote: > Fredrik wrote: >=20 > > fwiw, if "send" may send less than the full buffer in blocking > > mode on some platforms (despite what the specification implies), > > it's quite interesting that *nobody* has ever noticed before... >=20 > I noticed, but I expected it, so had no reason to comment. The=20 > Linux man pages are the only specification of send that I've=20 > seen that don't make a big deal out it. And clearly I'm not the=20 > only one, otherwise there would never have been a bug report=20 > (he didn't experience it, he just noticed sends without checks). I meant "I wonder why my script fails" rather than "that piece of code looks funky". ::: fwiw, I still haven't found a single reference (SUSv2 spec, man- pages, Stevens, the original BSD papers) that says that a blocking socket may do anything but sending all the data, or fail. if that's true, I'm not sure we really need to "fix" anything here... From Vladimir.Marangozov@inrialpes.fr Sat Aug 12 15:46:40 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Sat, 12 Aug 2000 16:46:40 +0200 (CEST) Subject: [Python-Dev] Preventing recursion core dumps In-Reply-To: from "Just van Rossum" at Aug 12, 2000 12:51:31 PM Message-ID: <200008121446.QAA20112@python.inrialpes.fr> Just van Rossum wrote: > > (Sorry for the late reply, that's what you get when you don't Cc me...) > > Vladimir Marangozov wrote: > > [Just] > > > Gordon, how's that Stackless PEP coming along? > > > Sorry, I couldn't resist ;-) > > > > Ah, in this case, we'll get a memory error after filling the whole disk > > with frames > > No matter how much we wink to each other, that was a cheap shot; I can't say that yours was more expensive . > especially since it isn't true: Stackless has a MAX_RECURSION_DEPTH value. > Someone who has studied Stackless "in detail" (your words ;-) should know > that. As I said - it has been years ago. Where's that PEP draft? Please stop dreaming about hostility . I am all for Stackless, but the implementation wasn't mature enough at the time when I looked at it. Now I hear it has evolved and does not allow graph cycles. Okay, good -- tell me more in a PEP and submit a patch. > > Admittedly, that value is set way too high in the last stackless release > (123456 ;-), but that doesn't change the principle that Stackless could > solve the problem discussed in this thread in a reliable and portable > manner. Indeed, if it didn't reduce the stack dependency in a portable way, it couldn't have carried the label "Stackless" for years. BTW, I'm more interested in the stackless aspect than the call/cc aspect of the code. > > Of course there's be work to do: > - MAX_RECURSION_DEPTH should be changeable at runtime > - __str__ (and a bunch of others) isn't yet stackless > - ... Tell me more in the PEP. > > But the hardest task seems to be to get rid of the hostility and prejudices > against Stackless :-( Dream on . -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From skip@mojam.com (Skip Montanaro) Sat Aug 12 18:56:23 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Sat, 12 Aug 2000 12:56:23 -0500 (CDT) Subject: [Python-Dev] Include/graminit.h, Python/graminit.c obsolete? Message-ID: <14741.36807.101870.221890@beluga.mojam.com> With Thomas's patch to the top-level Makefile that makes Grammar a more first-class directory, are the generated graminit.h and graminit.c files needed any longer? Skip From guido@beopen.com Sat Aug 12 20:12:23 2000 From: guido@beopen.com (Guido van Rossum) Date: Sat, 12 Aug 2000 14:12:23 -0500 Subject: [Python-Dev] Include/graminit.h, Python/graminit.c obsolete? In-Reply-To: Your message of "Sat, 12 Aug 2000 12:56:23 EST." <14741.36807.101870.221890@beluga.mojam.com> References: <14741.36807.101870.221890@beluga.mojam.com> Message-ID: <200008121912.OAA00807@cj20424-a.reston1.va.home.com> > With Thomas's patch to the top-level Makefile that makes Grammar a more > first-class directory, are the generated graminit.h and graminit.c files > needed any longer? I still like to keep them around. Most people don't hack the grammar. --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/) From trentm@ActiveState.com Sat Aug 12 19:39:00 2000 From: trentm@ActiveState.com (Trent Mick) Date: Sat, 12 Aug 2000 11:39:00 -0700 Subject: [Python-Dev] *.dsp and *.dsw are treated by CVS as binary. Why? In-Reply-To: <00d601c0043d$a2e66c20$f2a6b5d4@hagrid>; from effbot@telia.com on Sat, Aug 12, 2000 at 11:13:45AM +0200 References: <20000811143031.A13790@ActiveState.com> <200008112256.RAA01675@cj20424-a.reston1.va.home.com> <00d601c0043d$a2e66c20$f2a6b5d4@hagrid> Message-ID: <20000812113900.D3528@ActiveState.com> On Sat, Aug 12, 2000 at 11:13:45AM +0200, Fredrik Lundh wrote: > guido wrote: > > I think I made them binary during the period when I was mounting the > > Unix source directory on a Windows machine. I don't do that any more > > and I don't know anyone who does > > we do. > > trent wrote: > > > Does anybody see any problems treating them as text files? > > developer studio 5.0 does: > > "This makefile was not generated by Developer Studio" > > "Continuing will create a new Developer Studio project to > wrap this makefile. You will be prompted to save after the > new project has been created". > > "Do you want to continue" > > (click yes) > > "The options file (.opt) for this workspace specified a project > configuration "... - Win32 Alpha Release" that no longer exists. > The configuration will be set to "... - Win32 Debug" > > (click OK) > > (click build) > > "MAKE : fatal error U1052: file '....mak' not found" > > I admit that I have not tried a clean checkout and used DevStudio 5 (I will try at home alter today). Howver, I *do* think that the problem here is that you grabbed in the short iterim before patch: http://www.python.org/pipermail/python-checkins/2000-August/007072.html I hope I hope. If it is broken for MSVC 5 when I tried in a little bit I will back out. Trent -- Trent Mick TrentM@ActiveState.com From trentm@ActiveState.com Sat Aug 12 19:47:19 2000 From: trentm@ActiveState.com (Trent Mick) Date: Sat, 12 Aug 2000 11:47:19 -0700 Subject: [Python-Dev] *.dsp and *.dsw are treated by CVS as binary. Why? In-Reply-To: ; from tim_one@email.msn.com on Fri, Aug 11, 2000 at 08:59:22PM -0400 References: <20000811143031.A13790@ActiveState.com> Message-ID: <20000812114719.E3528@ActiveState.com> On Fri, Aug 11, 2000 at 08:59:22PM -0400, Tim Peters wrote: > Not really. They're not human-editable! Leave 'em alone. Keeping them in > binary mode is a clue to people that they aren't *supposed* to go mucking > with them via text processing tools. I think that putting them in binary mode is a misleading clue that people should not muck with them. The *are* text files. Editable or not the are not binary. I shouldn't go mucking with 'configure' either, because it is a generated file, but we shouldn't call it binary. Yes, I agree, people should not muck with .dsp files. I am not suggesting that we do. The "text-processing" I was referring to are my attempts to keep a local repository of Python in our local SCM tool (Perforce) in sync with Python-CVS. When I suck in Python-CVS on linux and them shove it in Perforce: - the .dsp's land on my linux box with DOS terminators - I check everything into Perforce - I check Python out of Perforce on a Windows box and the .dsp's are all terminated with \r\n\n. This is because the .dsp were not marked as binary in Perforce because I logically didn't think that they *should* be marked as binary. Having them marked as binary is just misleading I think. Anyway, as Guido said, this is not worth arguing over too much and it should have been fixed for you about an hour after I broke it (sorry). If it is still broken for you then I will back out. Trent -- Trent Mick TrentM@ActiveState.com From nascheme@enme.ucalgary.ca Sat Aug 12 19:58:20 2000 From: nascheme@enme.ucalgary.ca (Neil Schemenauer) Date: Sat, 12 Aug 2000 12:58:20 -0600 Subject: [Python-Dev] Lib/symbol.py needs update after listcomp Message-ID: <20000812125820.A567@keymaster.enme.ucalgary.ca> Someone needs to run: ./python Lib/symbol.py and check in the changes. Neil From akuchlin@mems-exchange.org Sat Aug 12 20:09:44 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Sat, 12 Aug 2000 15:09:44 -0400 Subject: [Python-Dev] Lib/symbol.py needs update after listcomp In-Reply-To: <20000812125820.A567@keymaster.enme.ucalgary.ca>; from nascheme@enme.ucalgary.ca on Sat, Aug 12, 2000 at 12:58:20PM -0600 References: <20000812125820.A567@keymaster.enme.ucalgary.ca> Message-ID: <20000812150944.A9653@kronos.cnri.reston.va.us> On Sat, Aug 12, 2000 at 12:58:20PM -0600, Neil Schemenauer wrote: >Someone needs to run: > ./python Lib/symbol.py >and check in the changes. Done. --amk From tim_one@email.msn.com Sat Aug 12 20:10:30 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 12 Aug 2000 15:10:30 -0400 Subject: [Python-Dev] *.dsp and *.dsw are treated by CVS as binary. Why? In-Reply-To: <20000812113900.D3528@ActiveState.com> Message-ID: Note that an update isn't enough to get you going again on Windows, and neither is (the moral equivalent of) "rm *" in PCbuild followed by an update. But "rm -rf PCbuild" followed by an update was enough for me (I'm working via phone modem -- a fresh full checkout is too time-consuming for me). From Vladimir.Marangozov@inrialpes.fr Sat Aug 12 20:16:17 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Sat, 12 Aug 2000 21:16:17 +0200 (CEST) Subject: [Python-Dev] minimal stackless Message-ID: <200008121916.VAA20873@python.inrialpes.fr> I'd like to clarify my position about the mythical Stackless issue. I would be okay to evaluate a minimal stackless implementation of the current VM, and eventually consider it for inclusion if it doesn't slow down the interpreter (and if it does, I don't know yet how much would be tolerable). However, I would be willing to do this only if such implementation is distilled from the call/cc stuff completely. That is, a minimal stackless implementation which gives us an equivalent VM as we have it today with the C stack. This is what I'd like to see first in the stackless PEP too. No mixtures with continuations & co. The call/cc issue is "application domain" for me -- it relies on top of the minimal stackless and would come only as an exported interface to the control flow management of the VM. Therefore, it must be completely optional (both in terms of lazy decision on whether it should be included someday). So, if such distilled, minimal stackless implementation hits the SourceForge shelves by the next week, I, at least, will give it a try and will report impressions. By that time, it would also be nice to see a clear summary of the frame management ideas in the 1st draft of the PEP. If the proponents of Stackless are ready for the challenge, give it a go (this seems to be a required first step in the right direction anyway). I can't offer any immediate help though, given the list of Python-related tasks I'd like to finish (as always, done in my spare minutes) and I'll be almost, if not completely, unavailable the last week of August. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From trentm@ActiveState.com Sat Aug 12 20:22:58 2000 From: trentm@ActiveState.com (Trent Mick) Date: Sat, 12 Aug 2000 12:22:58 -0700 Subject: [Python-Dev] *.dsp and *.dsw are treated by CVS as binary. Why? In-Reply-To: ; from tim_one@email.msn.com on Sat, Aug 12, 2000 at 03:10:30PM -0400 References: <20000812113900.D3528@ActiveState.com> Message-ID: <20000812122258.A4684@ActiveState.com> On Sat, Aug 12, 2000 at 03:10:30PM -0400, Tim Peters wrote: > Note that an update isn't enough to get you going again on Windows, and > neither is (the moral equivalent of) "rm *" in PCbuild followed by an > update. But "rm -rf PCbuild" followed by an update was enough for me (I'm > working via phone modem -- a fresh full checkout is too time-consuming for > me). Oh right. The '-kb' is sticky to you checked out version. I forgot about that. Thanks, Tim. Trent -- Trent Mick TrentM@ActiveState.com From esr@thyrsus.com Sat Aug 12 20:37:42 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Sat, 12 Aug 2000 15:37:42 -0400 Subject: [Python-Dev] minimal stackless In-Reply-To: <200008121916.VAA20873@python.inrialpes.fr>; from Vladimir.Marangozov@inrialpes.fr on Sat, Aug 12, 2000 at 09:16:17PM +0200 References: <200008121916.VAA20873@python.inrialpes.fr> Message-ID: <20000812153742.A25529@thyrsus.com> Vladimir Marangozov : > That is, a minimal stackless implementation which gives us an equivalent > VM as we have it today with the C stack. This is what I'd like to see > first in the stackless PEP too. No mixtures with continuations & co. > > The call/cc issue is "application domain" for me -- it relies on top of > the minimal stackless and would come only as an exported interface to the > control flow management of the VM. Therefore, it must be completely > optional (both in terms of lazy decision on whether it should be included > someday). I'm certainly among the call/cc fans, and I guess I'm weakly in the "Stackless proponent" camp, and I agree. These issues should be separated. If minimal stackless mods to ceval can solve (for example) the stack overflow problem I just got bitten by, we ought to integrate them for 2.0 and then give any new features a separate and thorough debate. I too will be happy to test a minimal-stackless patch. Come on, Christian, the ball's in your court. This is your best chance to get stackless accepted. -- Eric S. Raymond When only cops have guns, it's called a "police state". -- Claire Wolfe, "101 Things To Do Until The Revolution" From Fredrik Lundh" Message-ID: <002b01c00495$32df3120$f2a6b5d4@hagrid> skip wrote: > With Thomas's patch to the top-level Makefile that makes Grammar a more > first-class directory, are the generated graminit.h and graminit.c files > needed any longer? yes please -- thomas' patch only generates those files on unix boxes. as long as we support other platforms too, the files should be in the repository, and new versions should be checked in whenever the grammar is changed. From tim_one@email.msn.com Sat Aug 12 20:39:03 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 12 Aug 2000 15:39:03 -0400 Subject: [Python-Dev] *.dsp and *.dsw are treated by CVS as binary. Why? In-Reply-To: <20000812114719.E3528@ActiveState.com> Message-ID: [Trent Mick] > I think that putting them in binary mode is a misleading clue that > people should not muck with them. The *are* text files. But you don't know that. They're internal Microsoft files in an undocumented, proprietary format. You'll find nothing in MS docs guaranteeing they're text files, but will find the direst warnings against attempting to edit them. MS routinely changes *scads* of things about DevStudio-internal files across releases. For all the rest, you created your own problems by insisting on telling Perforce they're text files, despite that they're clearly marked binary under CVS. I'm unstuck now, but Fredrik will likely still have new problems cross-mounting file systems between Windows and Linux (see his msg). Since nothing here *was* broken (except for your private and self-created problems under Perforce), "fixing" it was simply a bad idea. We're on a very tight schedule, and the CVS tree isn't a playground. From thomas@xs4all.net Sat Aug 12 20:45:24 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sat, 12 Aug 2000 21:45:24 +0200 Subject: [Python-Dev] 're' crashes ? Message-ID: <20000812214523.H14470@xs4all.nl> I'm not trying to sound like Eric (though I don't mind if I do ;) but my Python crashes. Or rather, test_re fails with a coredump, since this afternoon or so. I'm fairly certain it was working fine yesterday, and it's an almost-vanilla CVS tree (I was about to check-in the fixes to Tools/compiler, and tried to use the compiler on the std lib and the test suite, when I noticed the coredump.) The coredump says this: #0 eval_code2 (co=0x824ba50, globals=0x82239b4, locals=0x0, args=0x827e18c, argcount=2, kws=0x827e194, kwcount=0, defs=0x82211c0, defcount=1, owner=0x0) at ceval.c:1474 1474 Py_DECREF(w); Which is part of the FOR_LOOP opcode: 1461 case FOR_LOOP: 1462 /* for v in s: ... 1463 On entry: stack contains s, i. 1464 On exit: stack contains s, i+1, s[i]; 1465 but if loop exhausted: 1466 s, i are popped, and we jump */ 1467 w = POP(); /* Loop index */ 1468 v = POP(); /* Sequence object */ 1469 u = loop_subscript(v, w); 1470 if (u != NULL) { 1471 PUSH(v); 1472 x = PyInt_FromLong(PyInt_AsLong(w)+1); 1473 PUSH(x); 1474 Py_DECREF(w); 1475 PUSH(u); 1476 if (x != NULL) continue; 1477 } 1478 else { 1479 Py_DECREF(v); 1480 Py_DECREF(w); 1481 /* A NULL can mean "s exhausted" 1482 but also an error: */ 1483 if (PyErr_Occurred()) 1484 why = WHY_EXCEPTION; I *think* this isn't caused by this code, but rather by a refcounting bug somewhere. 'w' should be an int, and it's used on line 1472, and doesn't cause an error there (unless passing a NULL pointer to PyInt_AsLong() isn't an error ?) But it's NULL at line 1474. Is there an easy way to track an error like this ? Otherwise I'll play around a bit using breakpoints and such in gdb. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From tim_one@email.msn.com Sat Aug 12 21:03:20 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 12 Aug 2000 16:03:20 -0400 Subject: [Python-Dev] Feature freeze! Message-ID: The 2.0 release manager (Jeremy) is on vacation. In his absence, here's a reminder from the 2.0 release schedule: Aug. 14: All 2.0 PEPs finished / feature freeze See the rest at: http://python.sourceforge.net/peps/pep-0200.html Note that that's Monday! Any new "new feature" patches submitted after Sunday will be mindlessly assigned Postponed status. New "new feature" patches submitted after this instant but before Monday will almost certainly be assigned Postponed status too -- just not *entirely* mindlessly . "Sunday" and "Monday" are defined by wherever Guido happens to be. "This instant" is defined by me, and probably refers to some time in the past from your POV; it's negotiable. From akuchlin@mems-exchange.org Sat Aug 12 21:06:28 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Sat, 12 Aug 2000 16:06:28 -0400 Subject: [Python-Dev] Location of compiler code Message-ID: I noticed that Jeremy checked in his compiler code; however, it lives in Tools/compiler/compiler. Any reason it isn't in Lib/compiler? --amk From tim_one@email.msn.com Sat Aug 12 21:11:50 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 12 Aug 2000 16:11:50 -0400 Subject: [Python-Dev] Location of compiler code In-Reply-To: Message-ID: [Andrew Kuchling] > I noticed that Jeremy checked in his compiler code; however, it lives > in Tools/compiler/compiler. Any reason it isn't in Lib/compiler? Suggest waiting for Jeremy to return from vacation (22 Aug). From Vladimir.Marangozov@inrialpes.fr Sat Aug 12 22:08:44 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Sat, 12 Aug 2000 23:08:44 +0200 (CEST) Subject: [Python-Dev] Feature freeze! In-Reply-To: from "Tim Peters" at Aug 12, 2000 04:03:20 PM Message-ID: <200008122108.XAA21412@python.inrialpes.fr> Tim Peters wrote: > > The 2.0 release manager (Jeremy) is on vacation. In his absence, here's a > reminder from the 2.0 release schedule: > > Aug. 14: All 2.0 PEPs finished / feature freeze > > See the rest at: > > http://python.sourceforge.net/peps/pep-0200.html > > Note that that's Monday! Any new "new feature" patches submitted after > Sunday will be mindlessly assigned Postponed status. New "new feature" > patches submitted after this instant but before Monday will almost certainly > be assigned Postponed status too -- just not *entirely* mindlessly . > "Sunday" and "Monday" are defined by wherever Guido happens to be. "This > instant" is defined by me, and probably refers to some time in the past from > your POV; it's negotiable. This reminder comes JIT! Then please make coincide the above dates/instants with the status of the open patches and take a stance on them: assign them to people, postpone, whatever. I deliberately postponed my object malloc patch. PS: this is also JIT as per the stackless discussion -- I mentioned "consider for inclusion" which was interpreted as "inclusion for 2.0" . God knows that I tried to be very careful when writing my position statement... OTOH, there's still a valid deadline for 2.0! PPS: is the pep-0200.html referenced above up to date? For instance, I see it mentions SET_LINENO pointing to old references, while a newer postponed patch is at SourceForge. A "last modified " stamp would be nice. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From trentm@ActiveState.com Sat Aug 12 22:51:55 2000 From: trentm@ActiveState.com (Trent Mick) Date: Sat, 12 Aug 2000 14:51:55 -0700 Subject: [Python-Dev] can this overflow (list insertion)? Message-ID: <20000812145155.A7629@ActiveState.com> from Objects/listobject.c: static int ins1(PyListObject *self, int where, PyObject *v) { int i; PyObject **items; if (v == NULL) { PyErr_BadInternalCall(); return -1; } items = self->ob_item; NRESIZE(items, PyObject *, self->ob_size+1); if (items == NULL) { PyErr_NoMemory(); return -1; } if (where < 0) where = 0; if (where > self->ob_size) where = self->ob_size; for (i = self->ob_size; --i >= where; ) items[i+1] = items[i]; Py_INCREF(v); items[where] = v; self->ob_item = items; self->ob_size++; <-------------- can this overflow? return 0; } In the case of sizeof(int) < sizeof(void*), can this overflow. I have a small patch to text self->ob_size against INT_MAX and I was going to submit it but I am not so sure that overflow is not checked by some other mechanism for list insert. Is it or was this relying on sizeof(ob_size) == sizeof(void*), hence a list being able to hold as many items as there is addressable memory? scared-to-patch-ly yours, Trent proposed patch: *** python/dist/src/Objects/listobject.c Fri Aug 11 16:25:08 2000 --- Python/dist/src/Objects/listobject.c Fri Aug 11 16:25:36 2000 *************** *** 149,155 **** Py_INCREF(v); items[where] = v; self->ob_item = items; ! self->ob_size++; return 0; } --- 149,159 ---- Py_INCREF(v); items[where] = v; self->ob_item = items; ! if (self->ob_size++ == INT_MAX) { ! PyErr_SetString(PyExc_OverflowError, ! "cannot add more objects to list"); ! return -1; ! } return 0; } -- Trent Mick TrentM@ActiveState.com From thomas@xs4all.net Sat Aug 12 22:52:47 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sat, 12 Aug 2000 23:52:47 +0200 Subject: [Python-Dev] Feature freeze! In-Reply-To: <200008122108.XAA21412@python.inrialpes.fr>; from Vladimir.Marangozov@inrialpes.fr on Sat, Aug 12, 2000 at 11:08:44PM +0200 References: <200008122108.XAA21412@python.inrialpes.fr> Message-ID: <20000812235247.I14470@xs4all.nl> On Sat, Aug 12, 2000 at 11:08:44PM +0200, Vladimir Marangozov wrote: > PPS: is the pep-0200.html referenced above up to date? For instance, > I see it mentions SET_LINENO pointing to old references, while a newer > postponed patch is at SourceForge. I asked similar questions about PEP 200, in particular on which new features were considered for 2.0 and what their status is (PEP 200 doesn't mention augmented assignment, which as far as I know has been on Guido's "2.0" list since 2.0 and 1.6 became different releases.) I apparently caught Jeremy just before he left for his holiday, and directed me towards Guido regarding those questions, and Guido has apparently been too busy (or he missed that email as well as some python-dev email.) All my PEPs are in, though, unless I should write a PEP on 'import as', which I really think should go in 2.0. I'd be suprised if 'import as' needs a PEP, since the worst vote on 'import as' was Eric's '+0', and there seems little concern wrt. syntax or implementation. It's more of a fix for overlooked syntax than it is a new feature<0.6 wink>. I just assumed the PyLabs team (or at least 4/5th of it) were too busy with getting 1.6 done and finished to be concerned with non-pressing 2.0 issues, and didn't want to press them on these issues until 1.6 is truely finished. Pity 1.6-beta-cycle and 2.0-feature-freeze overlap :P -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From nascheme@enme.ucalgary.ca Sat Aug 12 23:03:57 2000 From: nascheme@enme.ucalgary.ca (Neil Schemenauer) Date: Sat, 12 Aug 2000 18:03:57 -0400 Subject: [Python-Dev] parsers and compilers for 2.0 Message-ID: <20000812180357.A18816@acs.ucalgary.ca> With all the recent proposed and accepted language changes, we should be a careful to keep everything up to date. The parser module, Jeremy's compiler, and I suspect JPython have been left behind by the recent changes. In the past we have been blessed by a very stable core language. Times change. :) I'm trying to keep Jeremy's compiler up to date. Modifying the parser module to understand list comprehensions seems to be none trivial however. Does anyone else have the time and expertise to make these changes? The compiler/transformer.py module will also have to be modified to understand the new parse tree nodes. That part should be somewhat easier. On a related note, I think the SyntaxError messages generated by the compile() function and the parser module could be improved. This is annoying: >>> compile("$x", "myfile.py", "eval") Traceback (most recent call last): File " ", line 1, in ? File " ", line 1 $x ^ SyntaxError: invalid syntax Is there any reason why the error message does not say "myfile.py" instead of " "? If not I can probably put together a patch to fix it. As far as I can tell, the parser ParserError exceptions are almost useless. At least a line number could be given. I'm not sure how much work that is to fix though. Neil From nascheme@enme.ucalgary.ca Sat Aug 12 23:06:07 2000 From: nascheme@enme.ucalgary.ca (Neil Schemenauer) Date: Sat, 12 Aug 2000 18:06:07 -0400 Subject: [Python-Dev] compiler package in Lib? Message-ID: <20000812180607.A18938@acs.ucalgary.ca> Shouldn't the compiler package go in Lib instead of Tools? The AST used the by compiler should be very useful to things like lint checkers, optimizers, and "refactoring" tools. Neil From Vladimir.Marangozov@inrialpes.fr Sat Aug 12 23:24:39 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Sun, 13 Aug 2000 00:24:39 +0200 (CEST) Subject: [Python-Dev] can this overflow (list insertion)? In-Reply-To: <20000812145155.A7629@ActiveState.com> from "Trent Mick" at Aug 12, 2000 02:51:55 PM Message-ID: <200008122224.AAA21816@python.inrialpes.fr> Trent Mick wrote: > > [listobject.c/ins1()] > ... > self->ob_item = items; > self->ob_size++; <-------------- can this overflow? > return 0; > } > > > In the case of sizeof(int) < sizeof(void*), can this overflow. I have a small > patch to text self->ob_size against INT_MAX and I was going to submit it but > I am not so sure that overflow is not checked by some other mechanism for > list insert. +0. It could overflow but if it does, this is a bad sign about using a list for such huge amount of data. And this is the second time in a week that I see an attempt to introduce a bogus counter due to post-increments embedded in an if statement! > Is it or was this relying on sizeof(ob_size) == sizeof(void*), > hence a list being able to hold as many items as there is addressable memory? > > scared-to-patch-ly yours, > Trent And you're right > > > proposed patch: > > *** python/dist/src/Objects/listobject.c Fri Aug 11 16:25:08 2000 > --- Python/dist/src/Objects/listobject.c Fri Aug 11 16:25:36 2000 > *************** > *** 149,155 **** > Py_INCREF(v); > items[where] = v; > self->ob_item = items; > ! self->ob_size++; > return 0; > } > > --- 149,159 ---- > Py_INCREF(v); > items[where] = v; > self->ob_item = items; > ! if (self->ob_size++ == INT_MAX) { > ! PyErr_SetString(PyExc_OverflowError, > ! "cannot add more objects to list"); > ! return -1; > ! } > return 0; > } > > > > > -- > Trent Mick > TrentM@ActiveState.com > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://www.python.org/mailman/listinfo/python-dev > -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From esr@thyrsus.com Sat Aug 12 23:31:32 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Sat, 12 Aug 2000 18:31:32 -0400 Subject: [Python-Dev] parsers and compilers for 2.0 In-Reply-To: <20000812180357.A18816@acs.ucalgary.ca>; from nascheme@enme.ucalgary.ca on Sat, Aug 12, 2000 at 06:03:57PM -0400 References: <20000812180357.A18816@acs.ucalgary.ca> Message-ID: <20000812183131.A26660@thyrsus.com> Neil Schemenauer : > I'm trying to keep Jeremy's compiler up to date. Modifying the > parser module to understand list comprehensions seems to be none > trivial however. Last I checked, list comprehensions hadn't been accepted. I think there's at least one more debate waiting there... -- Eric S. Raymond If a thousand men were not to pay their tax-bills this year, that would ... [be] the definition of a peaceable revolution, if any such is possible. -- Henry David Thoreau From trentm@ActiveState.com Sat Aug 12 23:33:12 2000 From: trentm@ActiveState.com (Trent Mick) Date: Sat, 12 Aug 2000 15:33:12 -0700 Subject: [Python-Dev] can this overflow (list insertion)? In-Reply-To: <200008122224.AAA21816@python.inrialpes.fr>; from Vladimir.Marangozov@inrialpes.fr on Sun, Aug 13, 2000 at 12:24:39AM +0200 References: <20000812145155.A7629@ActiveState.com> <200008122224.AAA21816@python.inrialpes.fr> Message-ID: <20000812153312.B7629@ActiveState.com> On Sun, Aug 13, 2000 at 12:24:39AM +0200, Vladimir Marangozov wrote: > +0. > > It could overflow but if it does, this is a bad sign about using a list > for such huge amount of data. Point taken. > > And this is the second time in a week that I see an attempt to introduce > a bogus counter due to post-increments embedded in an if statement! > If I read you correctly then I think that you are mistaking my intention. Do you mean that I am doing the comparison *before* the increment takes place here: > > ! if (self->ob_size++ == INT_MAX) { > > ! PyErr_SetString(PyExc_OverflowError, > > ! "cannot add more objects to list"); > > ! return -1; > > ! } That is my intention. You can increment up to INT_MAX but not over..... ... heh heh actually my code *is* wrong. But for a slightly different reason. I trash the value of self->ob_size on overflow. You are right, I made a mistake trying to be cute with autoincrement in an 'if' statement. I should do the check and *then* increment if okay. Thanks, Trent -- Trent Mick TrentM@ActiveState.com From thomas@xs4all.net Sat Aug 12 23:34:43 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sun, 13 Aug 2000 00:34:43 +0200 Subject: [Python-Dev] parsers and compilers for 2.0 In-Reply-To: <20000812183131.A26660@thyrsus.com>; from esr@thyrsus.com on Sat, Aug 12, 2000 at 06:31:32PM -0400 References: <20000812180357.A18816@acs.ucalgary.ca> <20000812183131.A26660@thyrsus.com> Message-ID: <20000813003443.J14470@xs4all.nl> On Sat, Aug 12, 2000 at 06:31:32PM -0400, Eric S. Raymond wrote: > Neil Schemenauer : > > I'm trying to keep Jeremy's compiler up to date. Modifying the > > parser module to understand list comprehensions seems to be none > > trivial however. > Last I checked, list comprehensions hadn't been accepted. I think > there's at least one more debate waiting there... Check again, they're already checked in. The implementation may change later, but the syntax has been decided (by Guido): [(x, y) for y in something for x in somewhere if y in x] The parentheses around the leftmost expression are mandatory. It's currently implemented something like this: L = [] __x__ = [].append for y in something: for x in somewhere: if y in x: __x__((x, y)) del __x__ (where 'x' is a number, chosen to *probably* not conflict with any other local vrbls or other (nested) list comprehensions, and the result of the expression is L, which isn't actually stored anywhere during evaluation.) See the patches list archive and the SF patch info about the patch (#100654) for more information on how and why. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From esr@thyrsus.com Sun Aug 13 00:01:54 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Sat, 12 Aug 2000 19:01:54 -0400 Subject: [Python-Dev] parsers and compilers for 2.0 In-Reply-To: <20000813003443.J14470@xs4all.nl>; from thomas@xs4all.net on Sun, Aug 13, 2000 at 12:34:43AM +0200 References: <20000812180357.A18816@acs.ucalgary.ca> <20000812183131.A26660@thyrsus.com> <20000813003443.J14470@xs4all.nl> Message-ID: <20000812190154.B26719@thyrsus.com> Thomas Wouters : > > Last I checked, list comprehensions hadn't been accepted. I think > > there's at least one more debate waiting there... > > Check again, they're already checked in. The implementation may change > later, but the syntax has been decided (by Guido): > > [(x, y) for y in something for x in somewhere if y in x] Damn. That's unfortunate. With all due respect to the BDFL, I've come to believe that having special syntax for this (rather than constructor functions a la zip()) is a bad mistake. I predict it's going to come back to bite us hard in the future. -- Eric S. Raymond I cannot undertake to lay my finger on that article of the Constitution which grant[s] a right to Congress of expending, on objects of benevolence, the money of their constituents. -- James Madison, 1794 From bckfnn@worldonline.dk Sun Aug 13 00:29:14 2000 From: bckfnn@worldonline.dk (Finn Bock) Date: Sat, 12 Aug 2000 23:29:14 GMT Subject: [Python-Dev] parsers and compilers for 2.0 In-Reply-To: <20000812180357.A18816@acs.ucalgary.ca> References: <20000812180357.A18816@acs.ucalgary.ca> Message-ID: <3995dd8b.34665776@smtp.worldonline.dk> [Neil Schemenauer] >With all the recent proposed and accepted language changes, we >should be a careful to keep everything up to date. The parser >module, Jeremy's compiler, and I suspect JPython have been left >behind by the recent changes. WRT JPython, the list comprehensions have not yet been added. Then again, the feature was only recently checked in. You raise a good point however. There are many compilers/parsers that have to be updated before we can claim that a feature is fully implemented. [Thomas Wouters] >[(x, y) for y in something for x in somewhere if y in x] > >The parentheses around the leftmost expression are mandatory. It's currently >implemented something like this: > >L = [] >__x__ = [].append >for y in something: > for x in somewhere: > if y in x: > __x__((x, y)) >del __x__ Thank you for the fine example. At least I now think that know what the feature is about. regards, finn From tim_one@email.msn.com Sun Aug 13 00:37:14 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 12 Aug 2000 19:37:14 -0400 Subject: [Python-Dev] can this overflow (list insertion)? In-Reply-To: <20000812145155.A7629@ActiveState.com> Message-ID: [Trent Mick] > from Objects/listobject.c: > > static int > ins1(PyListObject *self, int where, PyObject *v) > { > ... > self->ob_size++; <-------------- can this overflow? > return 0; > } > ... > Is it or was this relying on sizeof(ob_size) == sizeof(void*), > hence a list being able to hold as many items as there is > addressable memory? I think it's more relying on the product of two other assumptions: (a) sizeof(int) >= 4, and (b) nobody is going to make a list with 2 billion elements in Python. But you're right, sooner or later that's going to bite us. > proposed patch: > > *** python/dist/src/Objects/listobject.c Fri Aug 11 16:25:08 2000 > --- Python/dist/src/Objects/listobject.c Fri Aug 11 16:25:36 2000 > *************** > *** 149,155 **** > Py_INCREF(v); > items[where] = v; > self->ob_item = items; > ! self->ob_size++; > return 0; > } > > --- 149,159 ---- > Py_INCREF(v); > items[where] = v; > self->ob_item = items; > ! if (self->ob_size++ == INT_MAX) { > ! PyErr_SetString(PyExc_OverflowError, > ! "cannot add more objects to list"); > ! return -1; > ! } > return 0; > } +1 on catching it, -1 on this technique. You noted later that this will make trash out of ob_size if it triggers, but the list has already grown and been shifted by this time too, so it's left in an insane state (to the user, the last element will appear to vanish). Suggest checking at the *start* of the routine instead: if (self->ob_size == INT_MAX) { PyErr_SetString(PyExc_OverflowError, "cannot add more objects to list"); return -1; } Then the list isn't harmed. From tim_one@email.msn.com Sun Aug 13 00:57:29 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 12 Aug 2000 19:57:29 -0400 Subject: [Python-Dev] Feature freeze! In-Reply-To: <200008122108.XAA21412@python.inrialpes.fr> Message-ID: [Vladimir Marangozov] > This reminder comes JIT! > > Then please make coincide the above dates/instants with the status of > the open patches and take a stance on them: assign them to people, > postpone, whatever. > > I deliberately postponed my object malloc patch. I don't know why. It's been there quite a while, and had non-trivial support for inclusion in 2.0. A chance to consider the backlog of patches as a stable whole is why the two weeks between "feature freeze" and 2.0b1 exists! > PS: this is also JIT as per the stackless discussion -- I mentioned > "consider for inclusion" which was interpreted as "inclusion for 2.0" > . God knows that I tried to be very careful when writing my > position statement... OTOH, there's still a valid deadline for 2.0! I doubt any variant of Stackless has a real shot for 2.0 at this point, although if a patch shows up before Sunday ends I won't Postpone it without reason (like, say, Guido tells me to). > PPS: is the pep-0200.html referenced above up to date? For instance, > I see it mentions SET_LINENO pointing to old references, while a newer > postponed patch is at SourceForge. > > A "last modified " stamp would be nice. I agree, but yaaaawn . CVS says it was last modified before Jeremy went on vacation. It's not up to date. The designated release manager in Jeremy's absence apparently didn't touch it. I can't gripe about that, though, because he's my boss . He sent me email today saying "tag, now you're it!" (Guido will be gone all next week). My plate is already full, though, and I won't get around to updating it today. Yes, this is no way to run a release, and so I don't have any confidence that the release dates in pep200 will be met. Still, I was arguing for feature freeze two months ago, and so as long as "I'm it" I'm not inclined to slip the schedule on *that* part. I bet it will be at least 3 weeks before 2.0b1 hits the streets, though. in-any-case-feature-freeze-is-on-the-critical-path-so-the-sooner- the-better-ly y'rs - tim From tim_one@email.msn.com Sun Aug 13 01:11:30 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 12 Aug 2000 20:11:30 -0400 Subject: [Python-Dev] Feature freeze! In-Reply-To: <20000812235247.I14470@xs4all.nl> Message-ID: [Thomas Wouters] > I asked similar questions about PEP 200, in particular on which > new features were considered for 2.0 and what their status is > (PEP 200 doesn't mention augmented assignment, which as far as I > know has been on Guido's "2.0" list since 2.0 and 1.6 became > different releases.) Yes, augmented assignment is golden for 2.0. > I apparently caught Jeremy just before he left for his holiday, > and directed me towards Guido regarding those questions, and > Guido has apparently been too busy (or he missed that email as > well as some python-dev email.) Indeed, we're never going to let Guido be Release Manager again . > All my PEPs are in, though, unless I should write a PEP on 'import as', > which I really think should go in 2.0. I'd be suprised if 'import > as' needs a PEP, since the worst vote on 'import as' was Eric's '+0', > and there seems little concern wrt. syntax or implementation. It's > more of a fix for overlooked syntax than it is a new feature<0.6 wink>. Why does everyone flee from the idea of writing a PEP? Think of it as a chance to let future generations know what a cool idea you had. I agree this change is too simple and too widely loved to *need* a PEP, but if you write one anyway you can add it to your resume under your list of peer-reviewed publications . > I just assumed the PyLabs team (or at least 4/5th of it) were too > busy with getting 1.6 done and finished to be concerned with non- > pressing 2.0 issues, and didn't want to press them on these issues > until 1.6 is truely finished. Actually, Fred Drake has done almost everything in the 1.6 branch by himself, while Guido has done all the installer and web-page work for that. The rest of our time has been eaten away by largely invisible cruft, from haggling over the license to haggling over where to put python.org next. Lots of haggling! You guys get to do the *fun* parts (btw, it's occurred to me often that I did more work on Python proper when I had a speech recognition job!). > Pity 1.6-beta-cycle and 2.0-feature-freeze overlap :P Ya, except it's too late to stop 1.6 now . but-not-too-late-to-stop-2.0-ly y'rs - tim From MarkH@ActiveState.com Sun Aug 13 02:02:36 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Sun, 13 Aug 2000 11:02:36 +1000 Subject: [Python-Dev] RE: list comprehensions (was parsers and compilers for 2.0) In-Reply-To: <20000812190154.B26719@thyrsus.com> Message-ID: ESR, responding to [(x, y) for y in something for x in somewhere if y in x] for list comprehension syntax: > Damn. That's unfortunate. With all due respect to the BDFL, I've come > to believe that having special syntax for this (rather than constructor > functions a la zip()) is a bad mistake. I predict it's going to come > back to bite us hard in the future. FWIW, these are my thoughts exactly (for this particular issue, anyway). Wont-bother-voting-cos-nobody-is-counting ly, Mark. From trentm@ActiveState.com Sun Aug 13 02:25:18 2000 From: trentm@ActiveState.com (Trent Mick) Date: Sat, 12 Aug 2000 18:25:18 -0700 Subject: [Python-Dev] parsers and compilers for 2.0 In-Reply-To: <3995dd8b.34665776@smtp.worldonline.dk>; from bckfnn@worldonline.dk on Sat, Aug 12, 2000 at 11:29:14PM +0000 References: <20000812180357.A18816@acs.ucalgary.ca> <3995dd8b.34665776@smtp.worldonline.dk> Message-ID: <20000812182518.B10528@ActiveState.com> On Sat, Aug 12, 2000 at 11:29:14PM +0000, Finn Bock wrote: > [Thomas Wouters] > > >[(x, y) for y in something for x in somewhere if y in x] > > > >The parentheses around the leftmost expression are mandatory. It's currently > >implemented something like this: > > > >L = [] > >__x__ = [].append > >for y in something: > > for x in somewhere: > > if y in x: > > __x__((x, y)) > >del __x__ > > Thank you for the fine example. At least I now think that know what the > feature is about. > Maybe that example should get in the docs for list comprehensions. Trent -- Trent Mick TrentM@ActiveState.com From trentm@ActiveState.com Sun Aug 13 02:30:02 2000 From: trentm@ActiveState.com (Trent Mick) Date: Sat, 12 Aug 2000 18:30:02 -0700 Subject: [Python-Dev] Feature freeze! In-Reply-To: ; from tim_one@email.msn.com on Sat, Aug 12, 2000 at 08:11:30PM -0400 References: <20000812235247.I14470@xs4all.nl> Message-ID: <20000812183002.C10528@ActiveState.com> On Sat, Aug 12, 2000 at 08:11:30PM -0400, Tim Peters wrote: > You guys get to do the *fun* parts Go give Win64 a whirl for a while. Trent -- Trent Mick TrentM@ActiveState.com From tim_one@email.msn.com Sun Aug 13 02:33:43 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 12 Aug 2000 21:33:43 -0400 Subject: [Python-Dev] RE: list comprehensions (was parsers and compilers for 2.0) In-Reply-To: Message-ID: [ESR, responding to [(x, y) for y in something for x in somewhere if y in x] for list comprehension syntax: ] > Damn. That's unfortunate. With all due respect to the BDFL, I've come > to believe that having special syntax for this (rather than constructor > functions a la zip()) is a bad mistake. I predict it's going to come > back to bite us hard in the future. [Mark Hammond] > FWIW, these are my thoughts exactly (for this particular issue, > anyway). > > Wont-bother-voting-cos-nobody-is-counting ly, Counting, no; listening, yes; persuaded, no. List comprehensions are one of the best-loved features of Haskell (really!), and Greg/Skip/Ping's patch implements as an exact a parallel to Haskell's syntax and semantics as is possible in Python. Predictions of doom thus need to make a plausible case for why a rousing success in Haskell is going to be a disaster in Python. The only basis I can see for such a claim (and I have to make one up myself because nobody else has ) is that Haskell is lazy, while Python is eager. I can't get from there to "disaster", though, or even "plausible regret". Beyond that, Guido dislikes the way Lisp spells most things, so it's this or nothing. I'm certain I'll use it, and with joy. Do an update and try it. C:\Code\python\dist\src\PCbuild>python Python 2.0b1 (#0, Aug 12 2000, 14:57:27) [MSC 32 bit (Intel)] on win32 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam Copyright 1995-2000 Corporation for National Research Initiatives (CNRI) >>> [x**2 for x in range(10)] [0, 1, 4, 9, 16, 25, 36, 49, 64, 81] >>> [x**2 for x in range(10) if x & 1] [1, 9, 25, 49, 81] >>> [x**2 if 3] [81] >>> Now even as a fan, I'll admit that last line sucks . bug-in-the-grammar-ly y'rs - tim From thomas@xs4all.net Sun Aug 13 08:53:57 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Sun, 13 Aug 2000 09:53:57 +0200 Subject: [Python-Dev] RE: list comprehensions (was parsers and compilers for 2.0) In-Reply-To: ; from tim_one@email.msn.com on Sat, Aug 12, 2000 at 09:33:43PM -0400 References: Message-ID: <20000813095357.K14470@xs4all.nl> On Sat, Aug 12, 2000 at 09:33:43PM -0400, Tim Peters wrote: [ ESR and Mark griping about list comprehensions syntax, which I can relate to, so I'll bother to try and exlain what bothers *me* wrt list comprehensions. Needn't be the same as what bothers them, though ] > List comprehensions are one of the best-loved features of Haskell > (really!), and Greg/Skip/Ping's patch implements as an exact a parallel to > Haskell's syntax and semantics as is possible in Python. I don't see "it's cool in language X" as a particular good reason to include a feature... We don't add special syntax for regular expressions, support for continuations or direct access to hardware because of that, do we ? > Predictions of doom thus need to make a plausible case for why a rousing > success in Haskell is going to be a disaster in Python. The only basis I > can see for such a claim (and I have to make one up myself because nobody > else has ) is that Haskell is lazy, while Python is eager. I can't > get from there to "disaster", though, or even "plausible regret". My main beef with the syntax is that it is, in my eyes, unpythonic. It has an alien, forced feel to it, much more so than the 'evil' map/filter/reduce. It doesn't 'fit' into Python the way most of the other features do; it's simply syntactic sugar for a specific kind of for-loop. It doesn't add any extra functionality, and for that large a syntactic change, I guess that scares me. Those doubts were why I was glad you were going to write the PEP. I was looking forward to you explaining why I had those doubts and giving sound arguments against them :-) > Beyond that, Guido dislikes the way Lisp spells most things, so it's this or > nothing. I'm certain I'll use it, and with joy. Do an update and try it. Oh, I've tried it. It's not included in the 'heavily patched Python 2.0b1' I have running on a couple of machines to impress my colleagues, (which includes the obmalloc patch, augmented assignment, range literals, import as, indexing-for, and extended-slicing-on-lists) but that's mostly because I was expecting, like ESR, a huge debate on its syntax. Lets say that most my doubts arose after playing with it for a while. I fear people will start using it in odd construct, and in odd ways, expecting other aspects of for-loops to be included in list comprehensions (break, else, continue, etc.) And then there's the way it's hard to parse because of the lack of punctuation in it: [((a,b)*c, (spam(d)%34)^e) for a in [(x, y) for x in L for y in S] for b in [b for b in B if mean(b)] for b,c in C for a,d in D for e in [Egg(a, b, c, d, e) for e in E]] I hope anyone writing something like that (notice the shadowing of some of the outer vrbls in the inner loops) will either add some newlines and indentation by themselves, or will be hunted down and shot (or at least winged) by the PSU. I'm not arguing to remove list comprehensions. I think they are cool features that can replace map/filter, I just don't think they're that much better than the use of map/filter. Write-that-PEP-Tim-it-will-look-good-on-your-resume-ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From esr@thyrsus.com Sun Aug 13 09:13:40 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Sun, 13 Aug 2000 04:13:40 -0400 Subject: [Python-Dev] RE: list comprehensions (was parsers and compilers for 2.0) In-Reply-To: <20000813095357.K14470@xs4all.nl>; from thomas@xs4all.net on Sun, Aug 13, 2000 at 09:53:57AM +0200 References: <20000813095357.K14470@xs4all.nl> Message-ID: <20000813041340.B27949@thyrsus.com> Thomas Wouters : > My main beef with the syntax is that it is, in my eyes, unpythonic. It has > an alien, forced feel to it, much more so than the 'evil' map/filter/reduce. > It doesn't 'fit' into Python the way most of the other features do; it's > simply syntactic sugar for a specific kind of for-loop. It doesn't add any > extra functionality, and for that large a syntactic change, I guess that > scares me. I agree 100% with all of this. -- Eric S. Raymond "This country, with its institutions, belongs to the people who inhabit it. Whenever they shall grow weary of the existing government, they can exercise their constitutional right of amending it or their revolutionary right to dismember it or overthrow it." -- Abraham Lincoln, 4 April 1861 From Moshe Zadka Sun Aug 13 09:15:15 2000 From: Moshe Zadka (Moshe Zadka) Date: Sun, 13 Aug 2000 11:15:15 +0300 (IDT) Subject: [Python-Dev] *.dsp and *.dsw are treated by CVS as binary. Why? In-Reply-To: Message-ID: On Sat, 12 Aug 2000, Tim Peters wrote: > [Trent Mick] > > I think that putting them in binary mode is a misleading clue that > > people should not muck with them. The *are* text files. > > But you don't know that. They're internal Microsoft files in an > undocumented, proprietary format. You'll find nothing in MS docs > guaranteeing they're text files, but will find the direst warnings against > attempting to edit them. MS routinely changes *scads* of things about > DevStudio-internal files across releases. Hey, I parsed those beasts, and edited them by hand. of-course-my-co-workers-hated-me-for-that-ly y'rs, Z. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From Vladimir.Marangozov@inrialpes.fr Sun Aug 13 10:16:50 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Sun, 13 Aug 2000 11:16:50 +0200 (CEST) Subject: [Python-Dev] can this overflow (list insertion)? In-Reply-To: from "Tim Peters" at Aug 12, 2000 07:37:14 PM Message-ID: <200008130916.LAA29139@python.inrialpes.fr> Tim Peters wrote: > > I think it's more relying on the product of two other assumptions: (a) > sizeof(int) >= 4, and (b) nobody is going to make a list with 2 billion > elements in Python. But you're right, sooner or later that's going to bite > us. +1 on your patch, but frankly, if we reach a situation to be bitten by this overflow, chances are that we've already dumped core or will be very soon -- billions of objects = soon to be overflowing ob_refcnt integer counters. Py_None looks like a fine candidate for this. Now I'm sure you're going to suggest again making the ob_refcnt a long, as you did before . > Suggest checking at the *start* of the routine instead: > > if (self->ob_size == INT_MAX) { > PyErr_SetString(PyExc_OverflowError, > "cannot add more objects to list"); > return -1; > } > > Then the list isn't harmed. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From Vladimir.Marangozov@inrialpes.fr Sun Aug 13 10:32:25 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Sun, 13 Aug 2000 11:32:25 +0200 (CEST) Subject: [Python-Dev] Feature freeze! In-Reply-To: from "Tim Peters" at Aug 12, 2000 07:57:29 PM Message-ID: <200008130932.LAA29181@python.inrialpes.fr> Tim Peters wrote: > > [Vladimir Marangozov] > > This reminder comes JIT! > > > > Then please make coincide the above dates/instants with the status of > > the open patches and take a stance on them: assign them to people, > > postpone, whatever. > > > > I deliberately postponed my object malloc patch. > > I don't know why. It's been there quite a while, and had non-trivial > support for inclusion in 2.0. A chance to consider the backlog of patches > as a stable whole is why the two weeks between "feature freeze" and 2.0b1 > exists! Because the log message reads that I'm late with the stat interface which shows what the situation is with and without it. If I want to finish that part, I'll need to block my Sunday afternoon. Given that now it's 11am, I have an hour to think what to do about it -- resurrect or leave postponed. > I doubt any variant of Stackless has a real shot for 2.0 at this point, > although if a patch shows up before Sunday ends I won't Postpone it without > reason (like, say, Guido tells me to). I'm doubtful too, but if there's a clean & solid minimal implementation which removes the stack dependency -- I'll have a look. > > > PPS: is the pep-0200.html referenced above up to date? For instance, > > I see it mentions SET_LINENO pointing to old references, while a newer > > postponed patch is at SourceForge. > > > > A "last modified " stamp would be nice. > > I agree, but yaaaawn . CVS says it was last modified before Jeremy > went on vacation. It's not up to date. The designated release manager in > Jeremy's absence apparently didn't touch it. I can't gripe about that, > though, because he's my boss . He sent me email today saying "tag, > now you're it!" (Guido will be gone all next week). My plate is already > full, though, and I won't get around to updating it today. Okay - just wanted to make this point clear, since your reminder reads "see the details there". -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From trentm@ActiveState.com Sun Aug 13 19:04:49 2000 From: trentm@ActiveState.com (Trent Mick) Date: Sun, 13 Aug 2000 11:04:49 -0700 Subject: [Python-Dev] can this overflow (list insertion)? In-Reply-To: <200008130916.LAA29139@python.inrialpes.fr>; from Vladimir.Marangozov@inrialpes.fr on Sun, Aug 13, 2000 at 11:16:50AM +0200 References: <200008130916.LAA29139@python.inrialpes.fr> Message-ID: <20000813110449.A23269@ActiveState.com> On Sun, Aug 13, 2000 at 11:16:50AM +0200, Vladimir Marangozov wrote: > Tim Peters wrote: > > > > I think it's more relying on the product of two other assumptions: (a) > > sizeof(int) >= 4, and (b) nobody is going to make a list with 2 billion > > elements in Python. But you're right, sooner or later that's going to bite > > us. > > +1 on your patch, but frankly, if we reach a situation to be bitten I'll check it in later today. Trent -- Trent Mick TrentM@ActiveState.com From trentm@ActiveState.com Mon Aug 14 00:08:43 2000 From: trentm@ActiveState.com (Trent Mick) Date: Sun, 13 Aug 2000 16:08:43 -0700 Subject: [Python-Dev] you may have some PCbuild hiccups Message-ID: <20000813160843.A27104@ActiveState.com> Hello all, Recently I spearheaded a number of screw ups in the PCbuild/ directory. PCbuild/*.dsp and *.dsw went from binary to text to binary again. These are sticky CVS attributes on files in your checked out Python tree. If you care about the PCbuild/ content (i.e. you build Python on Windows) then you may need to completely delete the PCbuild directory and re-get it from CVS. You can tell if you *need* to by doing a 'cvs status *.dsw *.dsp'. If any of those files *don't* have the "Sticky Option: -kb", they should. If they all do and MSDEV loads the project files okay, then you are fine. NOTE: You have to delete the *whole* PCbuild\ directory, not just its contents. The PCbuild\CVS control directory is part of what you have to re-get. Sorry, Trent -- Trent Mick TrentM@ActiveState.com From tim_one@email.msn.com Mon Aug 14 01:08:45 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 13 Aug 2000 20:08:45 -0400 Subject: [Python-Dev] RE: list comprehensions (was parsers and compilers for 2.0) In-Reply-To: <20000813095357.K14470@xs4all.nl> Message-ID: [Tim] >> List comprehensions are one of the best-loved features of Haskell >> (really!), and Greg/Skip/Ping's patch implements as an exact a >> parallel to Haskell's syntax and semantics as is possible in Python. [Thomas Wouters] > I don't see "it's cool in language X" as a particular good reason > to include a feature... We don't add special syntax for regular > expressions, support for continuations or direct access to hardware > because of that, do we ? As Guido (overly!) modestly says, the only language idea he ever invented was an "else" clause on loops. He decided listcomps "were Pythonic" before knowing anything about Haskell (or SETL, from which Haskell took the idea). Given that he *already* liked them, the value in looking at Haskell is for its actual experience with them. It would be pretty stupid *not* to look at experience with other languages that already have it! And that's whether you're pro or con. So it's not "cool in language X" that drives it at all, it's "cool in language X" that serves to confirm or refute the prior judgment that "it's Pythonic, period". When, e.g., Eric predicts it will bite us hard someday, I can point to Haskell and legitimately ask "why here and not there?". There was once a great push for adding some notion of "protected" class members to Python. Guido was initially opposed, but tempted to waffle because proponents kept pointing to C++. Luckily, Stroustrup had something to say about this in his "Design and Evolution of C++" book, including that he thought adding "protected" was a mistake, driven by relentless "good arguments" that opposed his own initial intuition. So in that case, *really* looking at C++ may have saved Guido from making the same mistake. As another example, few arguments are made more frequently than that Python should add embedded assignments in conditionals. *Lots* of other languages have that -- but they mostly serve to tell us it's a bug factory in practice! The languages that avoid the bugs point to ways to get the effect safely (although none yet Pythonically enough for Guido). So this is a fact: language design is very little about wholesale invention, and that's especially true of Python. It's a mystically difficult blending of borrowed ideas, and it's rare as readable Perl that an idea will get borrowed if it didn't even work well in its native home. listcomps work great where they came from, and that plus "hey, Guido likes 'em!" makes it 99% a done deal. > My main beef with the syntax is that it is, in my eyes, unpythonic. > It has an alien, forced feel to it, much more so than the 'evil' > map/filter/reduce. It doesn't 'fit' into Python the way most of > the other features do; Guido feels exactly the opposite: the business about "alien, forced feel, not fitting" is exactly what he's said about map/filter/reduce/lambda on many occasions. listcomps strike him (me too, for that matter) as much more Pythonic than those. > it's simply syntactic sugar for a specific kind of for-loop. It > doesn't add any extra functionality, All overwhelmingly true of augmented assignments, and range literals, and three-argument getattr, and list.pop, etc etc etc too. Python has lots of syntactic sugar -- making life pleasant is not a bad thing. > and for that large a syntactic change, I guess that scares me. The only syntactic change is to add a new form of list constructor. It's isolated and self-contained, and so "small" in that sense. > Those doubts were why I was glad you were going to write the PEP. I > was looking forward to you explaining why I had those doubts and > giving sound arguments against them :-) There is no convincing argument to made either way on whether "it's Pythonic", which I think is your primary worry. People *never* reach consensus on whether a given feature X "is Pythonic". That's why it's always Guido's job. You've been here long enough to see that -1 and +1 are about evenly balanced, except on (in recent memory) "import x as y" -- which I conviently neglected to mention had been dismissed as unPythonic by Guido just a couple weeks ago . > ... > but that's mostly because I was expecting, like ESR, a huge debate > on its syntax. Won't happen, because it already did. This was debated to death long ago, and more than once, and Guido likes what he likes now. Greg Wilson made the only new point on listcomps I've seen since two weeks after they were first proposed by Greg Ewing (i.e., that the ";" notation *really* sucked). > Lets say that most my doubts arose after playing with it for > a while. I fear people will start using it in odd construct, and > in odd ways, Unlike, say, filter, map and reduce ? > expecting other aspects of for-loops to be included > in list comprehensions (break, else, continue, etc.) Those ideas were rejected long ago too (and that Haskell and SETL also rejected them independently shows that, whether we can explain it or not, they're simply bad ideas). > And then there's the way it's hard to parse because of the > lack of punctuation in it: > > [((a,b)*c, (spam(d)%34)^e) for a in [(x, y) for x in L for y in > S] for b in [b for b in B if mean(b)] for b,c in C for a,d in D > for e in [Egg(a, b, c, d, e) for e in E]] That isn't a serious argument, to my eyes. Write that as a Lisp one-liner and see what it looks like then -- nuts is nuts, and a "scare example" could just as easily be concocted out of insane nesting of subscripts and/or function calls and/or parenthesized arithmetic. Idiotic nesting is a possibility for any construct that nests! BTW, you're missing the possibility to nest listcomps in "the expression" position too, a la >>> [[1 for i in range(n)] for n in range(10)] [[], [1], [1, 1], [1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 1, 1, 1, 1]] >>> I know you missed that possibility above because, despite your claim of being hard to parse, it's dead easy to spot where your listcomps begin: "[" is easy for the eye to find. > I hope anyone writing something like that (notice the shadowing of > some of the outer vrbls in the inner loops) You can find the same in nested lambdas littering map/reduce/etc today. > will either add some newlines and indentation by themselves, or > will be hunted down and shot (or at least winged) by the PSU. Nope! We just shun them. Natural selection will rid the Earth of them without violence . > I'm not arguing to remove list comprehensions. I think they are cool > features that can replace map/filter, I just don't think they're that > much better than the use of map/filter. Haskell programmers have map/filter too, and Haskell programmers routinely favor using listcomps. This says something about what people who have both prefer. I predict that once you're used to them, you'll find them much more expressive: "[" tells you immediately you're getting a list, then the next thing you see is what the list is built out of, and then there's a bunch of lower-level detail. It's great. > Write-that-PEP-Tim-it-will-look-good-on-your-resume-ly y'rs, except-i'm-too-old-to-need-a-resume-anymore -ly y'rs - tim From tim_one@email.msn.com Mon Aug 14 02:31:20 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 13 Aug 2000 21:31:20 -0400 Subject: [Python-Dev] you may have some PCbuild hiccups In-Reply-To: <20000813160843.A27104@ActiveState.com> Message-ID: [Trent Mick] > Recently I spearheaded a number of screw ups in the PCbuild/ > directory. Let's say the intent was laudable but the execution a bit off the mark . [binary -> text -> binary again] > ... > NOTE: You have to delete the *whole* PCbuild\ directory, not just > its contents. The PCbuild\CVS control directory is part of what you > have to re-get. Actually, I don't think you have to bother this time -- just do a regular update. The files *were* marked as text this time around, but there is no "sticky bit" saying so in the local config, so a plain update replaces them now. OK, I made most of that up. But a plain update did work fine for me ... From nowonder@nowonder.de Mon Aug 14 05:27:07 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Mon, 14 Aug 2000 04:27:07 +0000 Subject: [*].items() (was: Re: [Python-Dev] Lockstep iteration - eureka!) References: Your message of "Wed, 09 Aug 2000 02:37:07 MST." <3993FD49.C7E71108@prescod.net> Message-ID: <3997751B.9BB1D9FA@nowonder.de> Paul Prescod wrote: > > Just van Rossum wrote: > > > > for indexing in : > > Let me throw out another idea. What if sequences just had .items() > methods? > > j=range(0,10) > > for index, element in j.items(): I like the idea and so I've uploaded a patch for this to SF: https://sourceforge.net/patch/?func=detailpatch&patch_id=101178&group_id=5470 For ease of reading: This patch adds a .items() method to the list object. .items() returns a list with of tuples. E.g.: for index, value in ["a", "b", "c"].items(): print index, ":", value will print: 0: a 1: b 2: c I think this is an easy way to achieve looping over index AND elements in parallel. Semantically the following two expressions should be equivalent: for index, value in zip(range(len(mylist)), mylist): for index, value in mylist.items(): In opposition to patch #110138 I would call this: "Adding syntactic sugar without adding syntax (or sugar ):" this-doesn't-deserve-new-syntax-ly y'rs Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From greg@cosc.canterbury.ac.nz Mon Aug 14 05:01:35 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Mon, 14 Aug 2000 16:01:35 +1200 (NZST) Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of data sent. In-Reply-To: <20000811184407.A14470@xs4all.nl> Message-ID: <200008140401.QAA14955@s454.cosc.canterbury.ac.nz> > ERRORS > > EINTR A signal occurred. Different unices seem to have manpages which differ considerably in these areas. The Solaris manpage says: EINTR The operation was interrupted by delivery of a signal before any data could be buffered to be sent. which suggests that you won't get EINTR if some data *has* been sent before the signal arrives. It seems to me the only thing that could possibly happen in this case is to return with fewer bytes than requested, whether the socket is non-blocking or not. So it seems that, in the presence of signals, neither write() nor send() can be relied upon to either completely succeed or completely fail. Perhaps the reason this hasn't caused anyone a problem is that the combination of blocking sockets and signals that you want to handle and then carry on after are fairly rare. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Mon Aug 14 04:51:45 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Mon, 14 Aug 2000 15:51:45 +1200 (NZST) Subject: [Python-Dev] Preventing recursion core dumps In-Reply-To: <20000811103701.A25386@keymaster.enme.ucalgary.ca> Message-ID: <200008140351.PAA14951@s454.cosc.canterbury.ac.nz> > We don't limit the amount of memory you can allocate on all > machines just because your program may run out of memory on some > machine. Legend has it that Steve Jobs tried to do something like that with the original 128K Macintosh. He was against making the machine expandable in any way, so that any program which ran one Mac would run on all Macs. Didn't stay that way for very long... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Mon Aug 14 05:17:30 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Mon, 14 Aug 2000 16:17:30 +1200 (NZST) Subject: [Python-Dev] RE: list comprehensions (was parsers and compilers for 2.0) In-Reply-To: <20000813095357.K14470@xs4all.nl> Message-ID: <200008140417.QAA14959@s454.cosc.canterbury.ac.nz> Two reasons why list comprehensions fit better in Python than the equivalent map/filter/lambda constructs: 1) Scoping. The expressions in the LC have direct access to the enclosing scope, which is not true of lambdas in Python. 2) Efficiency. An LC with if-clauses which weed out many potential list elements can be much more efficient than the equivalent filter operation, which must build the whole list first and then remove unwanted items. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Mon Aug 14 05:24:43 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Mon, 14 Aug 2000 16:24:43 +1200 (NZST) Subject: [Python-Dev] noreply@sourceforge.net: [Python-bugs-list] [Bug #111620] lots of use of send() without verifyi ng amount of d In-Reply-To: <001301c00469$cb380fe0$f2a6b5d4@hagrid> Message-ID: <200008140424.QAA14962@s454.cosc.canterbury.ac.nz> Fredrik Lundh : > fwiw, I still haven't found a single reference (SUSv2 spec, man- > pages, Stevens, the original BSD papers) that says that a blocking > socket may do anything but sending all the data, or fail. The Solaris manpage sort of seems to indirectly suggest that it might conceivabley be possible: EMSGSIZE The socket requires that message be sent atomi- cally, and the message was too long. Which suggests that some types of socket may not require the message to be sent atomically. (TCP/IP, for example.) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From thomas@xs4all.net Mon Aug 14 06:38:55 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 14 Aug 2000 07:38:55 +0200 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/api api.tex,1.76,1.77 In-Reply-To: <200008140250.TAA31549@slayer.i.sourceforge.net>; from fdrake@users.sourceforge.net on Sun, Aug 13, 2000 at 07:50:23PM -0700 References: <200008140250.TAA31549@slayer.i.sourceforge.net> Message-ID: <20000814073854.O14470@xs4all.nl> On Sun, Aug 13, 2000 at 07:50:23PM -0700, Fred L. Drake wrote: > In the section on the "Very High Level Layer", address concerns brought up > by Edward K. Ream about FILE* values and > incompatible C libraries in dynamically linked extensions. It is not clear > (to me) how realistic the issue is, but it is better documented than not. > + Note also that several of these functions take \ctype{FILE*} > + parameters. On particular issue which needs to be handled carefully > + is that the \ctype{FILE} structure for different C libraries can be > + different and incompatible. Under Windows (at least), it is possible > + for dynamically linked extensions to actually use different libraries, > + so care should be taken that \ctype{FILE*} parameters are only passed > + to these functions if it is certain that they were created by the same > + library that the Python runtime is using. I saw a Jitterbug 'suggestion' bugthread, where Guido ended up liking the idea of wrapping fopen() and fclose() in the Python library, so that you got the right FILE structures when linking with another libc/compiler. Whatever happened to that idea ? Or does it just await implementation ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Mon Aug 14 06:57:13 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 14 Aug 2000 07:57:13 +0200 Subject: [Python-Dev] RE: list comprehensions (was parsers and compilers for 2.0) In-Reply-To: ; from tim_one@email.msn.com on Sun, Aug 13, 2000 at 08:08:45PM -0400 References: <20000813095357.K14470@xs4all.nl> Message-ID: <20000814075713.P14470@xs4all.nl> Well, Tim, thanx for that mini-PEP, if I can call your recap of years of discussion that ;-) It did clear up my mind, though I have a few comments to make. This is the last I have to say about it, though, I didn't intend to drag you into a new long discussion ;) On Sun, Aug 13, 2000 at 08:08:45PM -0400, Tim Peters wrote: > Guido feels exactly the opposite: the business about "alien, forced feel, > not fitting" is exactly what he's said about map/filter/reduce/lambda on > many occasions. Note that I didn't mention lambda, and did so purposely ;) Yes, listcomps are much better than lambda. And I'll grant the special case of 'None' as the function is unpythonic, in map/filter/reduce. Other than that, they're just functions, which I hope aren't too unpythonic > > [((a,b)*c, (spam(d)%34)^e) for a in [(x, y) for x in L for y in > > S] for b in [b for b in B if mean(b)] for b,c in C for a,d in D > > for e in [Egg(a, b, c, d, e) for e in E]] > That isn't a serious argument, to my eyes. Well, it's at the core of my doubts :) 'for' and 'if' start out of thin air. I don't think any other python statement or expression can be repeated and glued together without any kind of separator, except string literals (which I can see the point of, but scared me a little none the less.) I don't know enough lisp to write this expression in that, but I assume you could still match the parentheses to find out how they are grouped. > I know you missed that possibility above because, despite your claim of > being hard to parse, it's dead easy to spot where your listcomps begin: "[" > is easy for the eye to find. That's the start of a listcomp, but not of a specific listcomp-for or -if. > > I hope anyone writing something like that (notice the shadowing of > > some of the outer vrbls in the inner loops) > You can find the same in nested lambdas littering map/reduce/etc today. Yes, and wasn't the point to remove those ? Like I said, I'm not arguing against listcomprehensions, I'm just saying I'm sorry we didn't get yet another debate on syntax ;) Having said that, I'll step back and let Eric's predicted doom fall over Python; hopefully we are wrong and you all are right :-) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From jack@oratrix.nl Mon Aug 14 10:44:39 2000 From: jack@oratrix.nl (Jack Jansen) Date: Mon, 14 Aug 2000 11:44:39 +0200 Subject: [Python-Dev] Preventing recursion core dumps In-Reply-To: Message by Guido van Rossum , Fri, 11 Aug 2000 09:28:09 -0500 , <200008111428.JAA04464@cj20424-a.reston1.va.home.com> Message-ID: <20000814094440.0BC7F303181@snelboot.oratrix.nl> Isn't the solution to this problem to just implement PyOS_CheckStack() for unix? I assume you can implement it fairly cheaply by having the first call compute a stack warning address and subsequent calls simply checking that the stack hasn't extended below the limit yet. It might also be needed to add a few more PyOS_CheckStack() calls here and there, but I think most of them are in place. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From mal@lemburg.com Mon Aug 14 12:27:27 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 14 Aug 2000 13:27:27 +0200 Subject: [Python-Dev] Doc-strings for class attributes ?! Message-ID: <3997D79F.CC4A5A0E@lemburg.com> I've been doing a lot of auto-doc style documenation lately and have wondered how to include documentation for class attributes in a nice and usable way. Right now, we already have doc-strings for modules, classes, functions and methods. Yet there is no way to assign doc-strings to arbitrary class attributes. I figured that it would be nice to have the doc-strings for attributes use the same notation as for the other objects, e.g. class C " My class C " a = 1 " This is the attribute a of class C, used for ..." b = 0 " Setting b to 1 causes..." The idea is to create an implicit second attribute for every instance of documented attribute with a special name, e.g. for attribute b: __doc__b__ = " Setting b to 1 causes..." That way doc-strings would be able to use class inheritance just like the attributes themselves. The extra attributes can be created by the compiler. In -OO mode, these attributes would not be created. What do you think about this idea ? -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From bwarsaw@beopen.com Mon Aug 14 15:13:21 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Mon, 14 Aug 2000 10:13:21 -0400 (EDT) Subject: [Python-Dev] 2nd thought: fully qualified host names References: <3992DF9E.BF5A080C@nowonder.de> <200008101614.LAA28785@cj20424-a.reston1.va.home.com> <20000810174026.D17171@xs4all.nl> <39933AD8.B8EF5D59@nowonder.de> <20000811005013.F17171@xs4all.nl> Message-ID: <14743.65153.264194.444209@anthem.concentric.net> >>>>> "TW" == Thomas Wouters writes: TW> Fine, the patch addresses that. When the hostname passed to TW> smtplib is "" (which is the default), it should be turned into TW> a FQDN. I agree. However, if someone passed in a name, we do TW> not know if they even *want* the name turned into a FQDN. In TW> the face of ambiguity, refuse the temptation to guess. Just to weigh in after the fact, I agree with Thomas. All this stuff is primarily there to generate something sane for the default empty string argument. If the library client passes in their own name, smtplib.py should use that as given. -Barry From fdrake@beopen.com Mon Aug 14 15:46:17 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Mon, 14 Aug 2000 10:46:17 -0400 (EDT) Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Lib/test test_ntpath.py,1.2,1.3 In-Reply-To: <200008140621.XAA12890@slayer.i.sourceforge.net> References: <200008140621.XAA12890@slayer.i.sourceforge.net> Message-ID: <14744.1593.850598.411098@cj42289-a.reston1.va.home.com> Mark Hammond writes: > Test for fix to bug #110673: os.abspatth() now always returns > os.getcwd() on Windows, if an empty path is specified. It > previously did not if an empty path was delegated to > win32api.GetFullPathName()) ... > + tester('ntpath.abspath("")', os.getcwd()) This doesn't work. The test should pass on non-Windows platforms as well; on Linux I get this: cj42289-a(.../python/linux-beowolf); ./python ../Lib/test/test_ntpath.py error! evaluated: ntpath.abspath("") should be: /home/fdrake/projects/python/linux-beowolf returned: \home\fdrake\projects\python\linux-beowolf\ 1 errors. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From skip@mojam.com (Skip Montanaro) Mon Aug 14 15:56:39 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Mon, 14 Aug 2000 09:56:39 -0500 (CDT) Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0001.txt,1.4,1.5 In-Reply-To: <200008141448.HAA18067@slayer.i.sourceforge.net> References: <200008141448.HAA18067@slayer.i.sourceforge.net> Message-ID: <14744.2215.11395.695253@beluga.mojam.com> Barry> There are now three basic types of PEPs: informational, standards Barry> track, and technical. Looking more like RFCs all the time... ;-) Skip From jim@interet.com Mon Aug 14 16:25:59 2000 From: jim@interet.com (James C. Ahlstrom) Date: Mon, 14 Aug 2000 11:25:59 -0400 Subject: [Python-Dev] Python keywords (was Lockstep iteration - eureka!) References: <58C671173DB6174A93E9ED88DCB0883D0A6121@red-msg-07.redmond.corp.microsoft.com> Message-ID: <39980F87.85641FD2@interet.com> Bill Tutt wrote: > > This is an alternative approach that we should certainly consider. We could > use ANTLR (www.antlr.org) as our parser generator, and have it generate Java What about using Bison/Yacc? I have been playing with a lint tool for Python, and have been using it. JimA From trentm@ActiveState.com Mon Aug 14 16:41:28 2000 From: trentm@ActiveState.com (Trent Mick) Date: Mon, 14 Aug 2000 08:41:28 -0700 Subject: Python lint tool (was: Re: [Python-Dev] Python keywords (was Lockstep iteration - eureka!)) In-Reply-To: <39980F87.85641FD2@interet.com>; from jim@interet.com on Mon, Aug 14, 2000 at 11:25:59AM -0400 References: <58C671173DB6174A93E9ED88DCB0883D0A6121@red-msg-07.redmond.corp.microsoft.com> <39980F87.85641FD2@interet.com> Message-ID: <20000814084128.A7537@ActiveState.com> On Mon, Aug 14, 2000 at 11:25:59AM -0400, James C. Ahlstrom wrote: > What about using Bison/Yacc? I have been playing with a > lint tool for Python, and have been using it. > Oh yeah? What does the linter check? I would be interested in seeing that. Trent -- Trent Mick TrentM@ActiveState.com From bwarsaw@beopen.com Mon Aug 14 16:46:50 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Mon, 14 Aug 2000 11:46:50 -0400 (EDT) Subject: [Python-Dev] xxx.get_fqdn() for the standard lib References: <3992DF9E.BF5A080C@nowonder.de> <200008101614.LAA28785@cj20424-a.reston1.va.home.com> <20000810174026.D17171@xs4all.nl> <3993D570.7578FE71@nowonder.de> Message-ID: <14744.5229.470633.973850@anthem.concentric.net> >>>>> "PS" == Peter Schneider-Kamp writes: PS> After sleeping over it, I noticed that at least PS> BaseHTTPServer and ftplib also use a similar PS> algorithm to get a fully qualified domain name. PS> Together with smtplib there are four occurences PS> of the algorithm (2 in BaseHTTPServer). I think PS> it would be good not to have four, but one PS> implementation. PS> First I thought it could be socket.get_fqdn(), PS> but it seems a bit troublesome to write it in C. PS> Should this go somewhere? If yes, where should PS> it go? PS> I'll happily prepare a patch as soon as I know PS> where to put it. I wonder if we should move socket to _socket and write a Python wrapper which would basically import * from _socket and add make_fqdn(). -Barry From thomas@xs4all.net Mon Aug 14 16:48:37 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Mon, 14 Aug 2000 17:48:37 +0200 Subject: [Python-Dev] xxx.get_fqdn() for the standard lib In-Reply-To: <14744.5229.470633.973850@anthem.concentric.net>; from bwarsaw@beopen.com on Mon, Aug 14, 2000 at 11:46:50AM -0400 References: <3992DF9E.BF5A080C@nowonder.de> <200008101614.LAA28785@cj20424-a.reston1.va.home.com> <20000810174026.D17171@xs4all.nl> <3993D570.7578FE71@nowonder.de> <14744.5229.470633.973850@anthem.concentric.net> Message-ID: <20000814174837.S14470@xs4all.nl> On Mon, Aug 14, 2000 at 11:46:50AM -0400, Barry A. Warsaw wrote: > >>>>> "PS" == Peter Schneider-Kamp writes: > PS> After sleeping over it, I noticed that at least > PS> BaseHTTPServer and ftplib also use a similar > PS> algorithm to get a fully qualified domain name. > > PS> Together with smtplib there are four occurences > PS> of the algorithm (2 in BaseHTTPServer). I think > PS> it would be good not to have four, but one > PS> implementation. > > PS> First I thought it could be socket.get_fqdn(), > PS> but it seems a bit troublesome to write it in C. > > PS> Should this go somewhere? If yes, where should > PS> it go? > > PS> I'll happily prepare a patch as soon as I know > PS> where to put it. > > I wonder if we should move socket to _socket and write a Python > wrapper which would basically import * from _socket and add > make_fqdn(). +1 on that idea, especially since BeOS and Windows (I think ?) already have that constructions. If we are going to place this make_fqdn() function anywhere, it should be the socket module or a 'dns' module. (And I mean a real DNS module, not the low-level wrapper around raw DNS packets that Guido wrote ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From bwarsaw@beopen.com Mon Aug 14 16:56:15 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Mon, 14 Aug 2000 11:56:15 -0400 (EDT) Subject: [Python-Dev] Lockstep iteration - eureka! References: <3993FD49.C7E71108@prescod.net> Message-ID: <14744.5791.895030.893545@anthem.concentric.net> >>>>> "PP" == Paul Prescod writes: PP> Let me throw out another idea. What if sequences just had PP> .items() methods? Funny, I remember talking with Guido about this on a lunch trip several years ago. Tim will probably chime in that /he/ proposed it in the Python 0.9.3 time frame. :) -Barry From fdrake@beopen.com Mon Aug 14 16:59:53 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Mon, 14 Aug 2000 11:59:53 -0400 (EDT) Subject: [Python-Dev] xxx.get_fqdn() for the standard lib In-Reply-To: <14744.5229.470633.973850@anthem.concentric.net> References: <3992DF9E.BF5A080C@nowonder.de> <200008101614.LAA28785@cj20424-a.reston1.va.home.com> <20000810174026.D17171@xs4all.nl> <3993D570.7578FE71@nowonder.de> <14744.5229.470633.973850@anthem.concentric.net> Message-ID: <14744.6009.66009.888078@cj42289-a.reston1.va.home.com> Barry A. Warsaw writes: > I wonder if we should move socket to _socket and write a Python > wrapper which would basically import * from _socket and add > make_fqdn(). I think we could either do this or use PyRun_String() from initsocket(). -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From bwarsaw@beopen.com Mon Aug 14 17:09:11 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Mon, 14 Aug 2000 12:09:11 -0400 (EDT) Subject: [Python-Dev] Cookie.py References: <20000811122608.F20646@kronos.cnri.reston.va.us> Message-ID: <14744.6567.225562.458943@anthem.concentric.net> >>>>> "MZ" == Moshe Zadka writes: | a) SimpleCookie -- never uses pickle | b) SerilizeCookie -- always uses pickle | c) SmartCookie -- uses pickle based on old heuristic. Very cool. The autopicklification really bugged me too (literally) in Mailman. -Barry From bwarsaw@beopen.com Mon Aug 14 17:12:45 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Mon, 14 Aug 2000 12:12:45 -0400 (EDT) Subject: [Python-Dev] Python keywords (was Lockstep iteration - eureka !) References: <58C671173DB6174A93E9ED88DCB0883D0A6121@red-msg-07.redmond.corp.microsoft.com> Message-ID: <14744.6781.535265.161119@anthem.concentric.net> >>>>> "BT" == Bill Tutt writes: BT> This is an alternative approach that we should certainly BT> consider. We could use ANTLR (www.antlr.org) as our parser BT> generator, and have it generate Java for JPython, and C++ for BT> CPython. This would be a good chunk of work, and it's BT> something I really don't have time to pursue. I don't even BT> have time to pursue the idea about moving keyword recognition BT> into the lexer. BT> I'm just not sure if you want to bother introducing C++ into BT> the Python codebase solely to only have one parser for CPython BT> and JPython. We've talked about exactly those issues internally a while back, but never came to a conclusion (IIRC) about the C++ issue for CPython. -Barry From jim@interet.com Mon Aug 14 17:29:08 2000 From: jim@interet.com (James C. Ahlstrom) Date: Mon, 14 Aug 2000 12:29:08 -0400 Subject: Python lint tool (was: Re: [Python-Dev] Python keywords (was Lockstep iteration - eureka!)) References: <58C671173DB6174A93E9ED88DCB0883D0A6121@red-msg-07.redmond.corp.microsoft.com> <39980F87.85641FD2@interet.com> <20000814084128.A7537@ActiveState.com> Message-ID: <39981E54.D50BD0B4@interet.com> Trent Mick wrote: > > On Mon, Aug 14, 2000 at 11:25:59AM -0400, James C. Ahlstrom wrote: > > What about using Bison/Yacc? I have been playing with a > > lint tool for Python, and have been using it. > > > Oh yeah? What does the linter check? I would be interested in seeing that. Actually I have better luck parsing Python than linting it. My initial naive approach using C-language wisdom such as checking for line numbers where variables are set/used failed. I now feel that a Python lint tool must either use complete data flow analysis (hard) or must actually interpret the code as Python does (hard). All I can really do so far is get and check function signatures. I can supply more details if you want, but remember it doesn't work yet, and I may not have time to complete it. I learned a lot though. To parse Python I first use Parser/tokenizer.c to return tokens, then a Yacc grammar file. This parses all of Lib/*.py in less than two seconds on a modest machine. The tokens returned by tokenizer.c must be massaged a bit to be suitable for Yacc, but nothing major. All the Yacc actions are calls to Python methods, so the real work is written in Python. Yacc just represents the grammar. The problem I have with the current grammar is the large number of confusing shifts required. The grammar can't specify operator precedence, so it uses shift/reduce conflicts instead. Yacc eliminates this problem. JimA From tim_one@email.msn.com Mon Aug 14 17:42:14 2000 From: tim_one@email.msn.com (Tim Peters) Date: Mon, 14 Aug 2000 12:42:14 -0400 Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: <14744.5791.895030.893545@anthem.concentric.net> Message-ID: [Paul Prescod] > Let me throw out another idea. What if sequences just had > .items() methods? [Barry A. Warsaw] > Funny, I remember talking with Guido about this on a lunch trip > several years ago. Tim will probably chime in that /he/ proposed it > in the Python 0.9.3 time frame. :) Not me, although *someone* proposed it at least that early, perhaps at 0.9.1 already. IIRC, that was the very first time Guido used the term "hypergeneralization" in a cluck-cluck kind of public way. That is, sequences and mappings are different concepts in Python, and intentionally so. Don't know how he feels now. But if you add seq.items(), you had better add seq.keys() too, and seq.values() as a synonym for seq[:]. I guess the perceived advantage of adding seq.items() is that it supplies yet another incredibly slow and convoluted way to get at the for-loop index? "Ah, that's the ticket! Let's allocate gazillabytes of storage and compute all the indexes into a massive data structure up front, and then we can use the loop index that's already sitting there for free anyway to index into that and get back a redundant copy of itself!" . not-a-good-sign-when-common-sense-is-offended-ly y'rs - tim From bwarsaw@beopen.com Mon Aug 14 17:48:59 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Mon, 14 Aug 2000 12:48:59 -0400 (EDT) Subject: [Python-Dev] Python Announcements ??? References: <39951D69.45D01703@lemburg.com> Message-ID: <14744.8955.35531.757406@anthem.concentric.net> >>>>> "M" == M writes: M> Could someone at BeOpen please check what happened to the M> python-announce mailing list ?! This is on my task list, but I was on vacation last week and have been swamped with various other things. My plan is to feed the announcements to a Mailman list, where approval can happen using the same moderator interface. But I need to make a few changes to Mailman to support this. -Barry From mal@lemburg.com Mon Aug 14 17:54:05 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 14 Aug 2000 18:54:05 +0200 Subject: [Python-Dev] Python Announcements ??? References: <39951D69.45D01703@lemburg.com> <14744.8955.35531.757406@anthem.concentric.net> Message-ID: <3998242D.A61010FB@lemburg.com> "Barry A. Warsaw" wrote: > > >>>>> "M" == M writes: > > M> Could someone at BeOpen please check what happened to the > M> python-announce mailing list ?! > > This is on my task list, but I was on vacation last week and have been > swamped with various other things. My plan is to feed the > announcements to a Mailman list, where approval can happen using the > same moderator interface. But I need to make a few changes to Mailman > to support this. Great :-) BTW, doesn't SourceForge have some News channel for Python as well (I have seen these for other projects) ? Would be cool to channel the announcements there as well. Thanks, -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From ping@lfw.org Mon Aug 14 19:58:11 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Mon, 14 Aug 2000 13:58:11 -0500 (CDT) Subject: Python lint tool (was: Re: [Python-Dev] Python keywords (was Lockstep iteration - eureka!)) In-Reply-To: <39981E54.D50BD0B4@interet.com> Message-ID: Trent Mick wrote: Oh yeah? What does the linter check? I would be interested in seeing that. James C. Ahlstrom wrote: > Actually I have better luck parsing Python than linting it. [...] > All I can really do so far is get and check function signatures. Python is hard to lint-check because types and objects are so dynamic. Last time i remember visiting this issue, Tim Peters came up with a lint program that was based on warning you if you used a particular spelling of an identifier only once (thus likely to indicate a typing mistake). I enhanced this a bit to follow imports and the result is at http://www.lfw.org/python/ (look for "pylint"). The rule is pretty simplistic, but i've tweaked it a bit and it has actually worked pretty well for me. Anyway, feel free to give it a whirl. -- ?!ng From bwarsaw@beopen.com Mon Aug 14 20:12:04 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Mon, 14 Aug 2000 15:12:04 -0400 (EDT) Subject: [Python-Dev] xxx.get_fqdn() for the standard lib References: <3992DF9E.BF5A080C@nowonder.de> <200008101614.LAA28785@cj20424-a.reston1.va.home.com> <20000810174026.D17171@xs4all.nl> <3993D570.7578FE71@nowonder.de> <14744.5229.470633.973850@anthem.concentric.net> <14744.6009.66009.888078@cj42289-a.reston1.va.home.com> Message-ID: <14744.17540.586064.729048@anthem.concentric.net> >>>>> "Fred" == Fred L Drake, Jr writes: | I think we could either do this or use PyRun_String() from | initsocket(). Ug. -1 on using PyRun_String(). Doing the socket->_socket shuffle is better for the long term. -Barry From nowonder@nowonder.de Mon Aug 14 22:12:03 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Mon, 14 Aug 2000 21:12:03 +0000 Subject: [Python-Dev] Lockstep iteration - eureka! References: Message-ID: <399860A3.4E9A340E@nowonder.de> Tim Peters wrote: > > But if you add seq.items(), you had better add seq.keys() too, and > seq.values() as a synonym for seq[:]. I guess the perceived advantage of > adding seq.items() is that it supplies yet another incredibly slow and > convoluted way to get at the for-loop index? "Ah, that's the ticket! Let's > allocate gazillabytes of storage and compute all the indexes into a massive > data structure up front, and then we can use the loop index that's already > sitting there for free anyway to index into that and get back a redundant > copy of itself!" . That's a -1, right? <0.1 wink> Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From fdrake@beopen.com Mon Aug 14 20:13:29 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Mon, 14 Aug 2000 15:13:29 -0400 (EDT) Subject: [Python-Dev] xxx.get_fqdn() for the standard lib In-Reply-To: <14744.17540.586064.729048@anthem.concentric.net> References: <3992DF9E.BF5A080C@nowonder.de> <200008101614.LAA28785@cj20424-a.reston1.va.home.com> <20000810174026.D17171@xs4all.nl> <3993D570.7578FE71@nowonder.de> <14744.5229.470633.973850@anthem.concentric.net> <14744.6009.66009.888078@cj42289-a.reston1.va.home.com> <14744.17540.586064.729048@anthem.concentric.net> Message-ID: <14744.17625.935969.667720@cj42289-a.reston1.va.home.com> Barry A. Warsaw writes: > Ug. -1 on using PyRun_String(). Doing the socket->_socket shuffle is > better for the long term. I'm inclined to agree, simply because it allows at least a slight simplification in socketmodule.c since the conditional naming of the module init function can be removed. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From bwarsaw@beopen.com Mon Aug 14 20:24:10 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Mon, 14 Aug 2000 15:24:10 -0400 (EDT) Subject: [Python-Dev] Lockstep iteration - eureka! References: <14744.5791.895030.893545@anthem.concentric.net> Message-ID: <14744.18266.840173.466719@anthem.concentric.net> >>>>> "TP" == Tim Peters writes: TP> But if you add seq.items(), you had better add seq.keys() too, TP> and seq.values() as a synonym for seq[:]. I guess the TP> perceived advantage of adding seq.items() is that it supplies TP> yet another incredibly slow and convoluted way to get at the TP> for-loop index? "Ah, that's the ticket! Let's allocate TP> gazillabytes of storage and compute all the indexes into a TP> massive data structure up front, and then we can use the loop TP> index that's already sitting there for free anyway to index TP> into that and get back a redundant copy of itself!" . Or create a generator. -Barry From bwarsaw@beopen.com Mon Aug 14 20:25:07 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Mon, 14 Aug 2000 15:25:07 -0400 (EDT) Subject: [Python-Dev] Python Announcements ??? References: <39951D69.45D01703@lemburg.com> <14744.8955.35531.757406@anthem.concentric.net> <3998242D.A61010FB@lemburg.com> Message-ID: <14744.18323.499501.115700@anthem.concentric.net> >>>>> "M" == M writes: M> BTW, doesn't SourceForge have some News channel for Python M> as well (I have seen these for other projects) ? Would be M> cool to channel the announcements there as well. Yes, but it's a bit clunky. -Barry From esr@thyrsus.com Mon Aug 14 23:57:18 2000 From: esr@thyrsus.com (esr@thyrsus.com) Date: Mon, 14 Aug 2000 18:57:18 -0400 Subject: [Python-Dev] RE: list comprehensions (was parsers and compilers for 2.0) In-Reply-To: <200008140417.QAA14959@s454.cosc.canterbury.ac.nz> References: <20000813095357.K14470@xs4all.nl> <200008140417.QAA14959@s454.cosc.canterbury.ac.nz> Message-ID: <20000814185718.A2509@thyrsus.com> Greg Ewing : > Two reasons why list comprehensions fit better in Python > than the equivalent map/filter/lambda constructs: > > 1) Scoping. The expressions in the LC have direct access to the > enclosing scope, which is not true of lambdas in Python. This is a bug in lambdas, not a feature of syntax. > 2) Efficiency. An LC with if-clauses which weed out many potential > list elements can be much more efficient than the equivalent > filter operation, which must build the whole list first and > then remove unwanted items. A better argument. To refute it, I'd need to open a big can of worms labeled "lazy evaluation". -- Eric S. Raymond Freedom, morality, and the human dignity of the individual consists precisely in this; that he does good not because he is forced to do so, but because he freely conceives it, wants it, and loves it. -- Mikhail Bakunin From esr@thyrsus.com Mon Aug 14 23:59:08 2000 From: esr@thyrsus.com (esr@thyrsus.com) Date: Mon, 14 Aug 2000 18:59:08 -0400 Subject: [Python-Dev] RE: list comprehensions (was parsers and compilers for 2.0) In-Reply-To: <20000814075713.P14470@xs4all.nl> References: <20000813095357.K14470@xs4all.nl> <20000814075713.P14470@xs4all.nl> Message-ID: <20000814185908.B2509@thyrsus.com> Thomas Wouters : > Like I said, I'm not arguing against listcomprehensions, I'm just saying I'm > sorry we didn't get yet another debate on syntax ;) Having said that, I'll > step back and let Eric's predicted doom fall over Python; hopefully we are > wrong and you all are right :-) Now, now. I'm not predicting the doom of Python as a whole, just that listcomp syntax will turn out to have been a bad, limiting mistake. -- Eric S. Raymond It is proper to take alarm at the first experiment on our liberties. We hold this prudent jealousy to be the first duty of citizens and one of the noblest characteristics of the late Revolution. The freemen of America did not wait till usurped power had strengthened itself by exercise and entangled the question in precedents. They saw all the consequences in the principle, and they avoided the consequences by denying the principle. We revere this lesson too much ... to forget it -- James Madison. From MarkH@ActiveState.com Tue Aug 15 01:46:56 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Tue, 15 Aug 2000 10:46:56 +1000 Subject: [Python-Dev] WindowsError repr Message-ID: I have just checked in a fix for: [ Bug #110670 ] Win32 os.listdir raises confusing errors http://sourceforge.net/bugs/?group_id=5470&func=detailbug&bug_id=110670 In a nutshell: >>> os.listdir('/cow') ... OSError: [Errno 3] No such process: '/cow' >>> The solution here was to use the new WindowsError object that was defined back in February (http://www.python.org/pipermail/python-dev/2000-February/008803.html) As this is a sub-class of OSError, nothing will break. However, the _look_ of the error does change. After my fix, it now looks like: >>> os.listdir('/cow') ... WindowsError: [Errno 3] The system cannot find the path specified: '/cow' >>> AGAIN - I stress - catching "OSError" or "os.error" _will_ continue to work, as WindowsError derives from OSError. It just worries me that people will start explicitly catching "WindowsError", regardless of whatever documentation we might write on the subject. Does anyone see this as a problem? Should a WindowsError masquerade as "OSError", or maybe just look a little more like it - eg, "OSError (windows)" ?? Thoughts, Mark. From tim_one@email.msn.com Tue Aug 15 02:01:55 2000 From: tim_one@email.msn.com (Tim Peters) Date: Mon, 14 Aug 2000 21:01:55 -0400 Subject: [Python-Dev] WindowsError repr In-Reply-To: Message-ID: [Mark Hammond] > ... > However, the _look_ of the error does change. After my fix, it now looks > like: > > >>> os.listdir('/cow') > ... > WindowsError: [Errno 3] The system cannot find the path specified: '/cow' > >>> Thank you! > AGAIN - I stress - catching "OSError" or "os.error" _will_ continue to > work, as WindowsError derives from OSError. It just worries me > that people will start explicitly catching "WindowsError", regardless > of whatever documentation we might write on the subject. > > Does anyone see this as a problem? Should a WindowsError masquerade as > "OSError", or maybe just look a little more like it - eg, "OSError > (windows)" ?? I can assure you that nobody running on a Unix(tm) derivative is going to catch WindowsError as such on purpose, so the question is how stupid are Windows users? I say leave it alone and let them tell us . From greg@cosc.canterbury.ac.nz Tue Aug 15 02:08:00 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 15 Aug 2000 13:08:00 +1200 (NZST) Subject: [Python-Dev] RE: list comprehensions (was parsers and compilers for 2.0) In-Reply-To: <20000814075713.P14470@xs4all.nl> Message-ID: <200008150108.NAA15067@s454.cosc.canterbury.ac.nz> > > [((a,b)*c, (spam(d)%34)^e) for a in [(x, y) for x in L for y in > > S] for b in [b for b in B if mean(b)] for b,c in C for a,d in D > > for e in [Egg(a, b, c, d, e) for e in E]] Note that shadowing of the local variables like that in an LC is NOT allowed, because, like the variables in a normal for loop, they're all at the same scope level. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim_one@email.msn.com Tue Aug 15 05:43:44 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 15 Aug 2000 00:43:44 -0400 Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: <399860A3.4E9A340E@nowonder.de> Message-ID: [Tim] > But if you add seq.items(), you had better add seq.keys() too, and > seq.values() as a synonym for seq[:]. I guess the perceived > advantage of adding seq.items() is that it supplies yet another > incredibly slow and convoluted way to get at the for-loop index? > "Ah, that's the ticket! Let's allocate gazillabytes of storage and > compute all the indexes into a massive data structure up front, and > then we can use the loop index that's already sitting there for > free anyway to index into that and get back a redundant copy of > itself!" . [Peter Schneider-Kamp]] > That's a -1, right? <0.1 wink> -0 if you also add .keys() and .values() (if you're going to hypergeneralize, don't go partway nuts -- then it's both more general than it should be yet still not as general as people will expect). -1 if it's *just* seq.items(). +1 on an "indexing" clause (the BDFL liked that enough to implement it a few years ago, but it didn't go in then because he found some random putz who had used "indexing" as a vrbl name; but if doesn't need to be a keyword, even that lame (ask Just ) objection goes away). sqrt(-1) on Barry's generator tease, because that's an imaginary proposal at this stage of the game. From Fredrik Lundh" Message-ID: <003d01c0067a$4aa6dc40$f2a6b5d4@hagrid> mark wrote: > AGAIN - I stress - catching "OSError" or "os.error" _will_ continue to > work, as WindowsError derives from OSError. It just worries me that people > will start explicitly catching "WindowsError", regardless of whatever > documentation we might write on the subject. > > Does anyone see this as a problem? I've seen bigger problems -- but I think it's a problem. any reason you cannot just use a plain OSError? is the extra "this is not a generic OSError" information bit actually used by anyone? From Fredrik Lundh" Message-ID: <006d01c00680$1c4469c0$f2a6b5d4@hagrid> tim wrote: > ! test_popen2 Win32 X X 26-Jul-2000 > [believe this was fix by /F] > ! [still fails 15-Aug-2000 for me, on Win98 - tim > ! test test_popen2 crashed -- exceptions.WindowsError : > ! [Errno 2] The system cannot find the file specified > ! ] do you have w9xpopen.exe in place? (iirc, mark just added the build files) From tim_one@email.msn.com Tue Aug 15 07:30:40 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 15 Aug 2000 02:30:40 -0400 Subject: [Python-Dev] [PEP 200] Help! Message-ID: I took a stab at updating PEP200 (the 2.0 release plan), but if you know more about any of it that should be recorded or changed, please just do so! There's no reason to funnel updates thru me. Jeremy may feel differently when he gets back, but in the meantime this is just more time-consuming stuff I hadn't planned on needing to do. Windows geeks: what's going on with test_winreg2 and test_popen2? Those tests have been failing forever (at least on Win98 for me), and the grace period has more than expired. Fredrik, if you're still waiting for me to do something with popen2 (rings a vague bell), please remind me -- I've forgotten what it was! thrashingly y'rs - tim From tim_one@email.msn.com Tue Aug 15 07:43:06 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 15 Aug 2000 02:43:06 -0400 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/nondist/peps pep-0200.txt,1.7,1.8 In-Reply-To: <006d01c00680$1c4469c0$f2a6b5d4@hagrid> Message-ID: > tim wrote: > > ! test_popen2 Win32 X X 26-Jul-2000 > > [believe this was fix by /F] > > ! [still fails 15-Aug-2000 for me, on Win98 - tim > > ! test test_popen2 crashed -- exceptions.WindowsError : > > ! [Errno 2] The system cannot find the file specified > > ! ] [/F] > do you have w9xpopen.exe in place? > > (iirc, mark just added the build files) Ah, thanks! This is coming back to me now ... kinda ... will pursue. From tim_one@email.msn.com Tue Aug 15 08:07:49 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 15 Aug 2000 03:07:49 -0400 Subject: [Python-Dev] test_popen2 on Windows In-Reply-To: Message-ID: [/F] > do you have w9xpopen.exe in place? > > (iirc, mark just added the build files) Heh -- yes, and I wasn't building them. Now test_popen2 fails for a different reason: def _test(): teststr = "abc\n" print "testing popen2..." r, w = popen2('cat') ... Ain't no "cat" on Win98! The test is specific to Unix derivatives. Other than that, popen2 is working for me now. Mumble. From MarkH@ActiveState.com Tue Aug 15 09:08:33 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Tue, 15 Aug 2000 18:08:33 +1000 Subject: [Python-Dev] test_popen2 on Windows In-Reply-To: Message-ID: > Ain't no "cat" on Win98! The test is specific to Unix > derivatives. Other than that, popen2 is working for me heh - I noticed that yesterday, then lumped it in the too hard basket. What I really wanted was for test_popen2 to use python itself for the sub-process. This way, commands like 'python -c "import sys;sys.exit(2)"' could test the handle close semantics, for example. I gave up when I realized I would probably need to create temp files with the mini-programs. I was quite confident that if I attempted this, I would surely break the test suite on a few platforms. I wasn't brave enough to risk those testicles of wrath at this stage in the game Mark. From thomas@xs4all.net Tue Aug 15 12:15:42 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Tue, 15 Aug 2000 13:15:42 +0200 Subject: [Python-Dev] New PEP for import-as Message-ID: <20000815131542.B14470@xs4all.nl> --QKdGvSO+nmPlgiQ/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline I wrote a quick PEP describing the 'import as' proposal I posted a patch for last week. Mostly since I was bored in the train to work (too many kids running around to play Diablo II or any other game -- I hate it when those brats go 'oh cool' and keep hanging around looking over my shoulder ;-) but also a bit because Tim keeps insisting it should be easy to write a PEP. Perhaps lowering the standard by providing a few *small* PEPs helps with that ;) Just's 'indexing-for' PEP would be a good one, too, in that case. Anyway, the proto-PEP is attached. It's in draft status as far as I'm concerned, but the PEP isn't really necessary if the feature is accepted by acclamation. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! --QKdGvSO+nmPlgiQ/ Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="pep-02XX.txt" PEP: 2XX Title: Import As Version: $Revision: 1.0 $ Owner: thomas@xs4all.net (Thomas Wouters) Python-Version: 2.0 Status: Draft Introduction This PEP describes the `import as' proposal for Python 2.0. This PEP tracks the status and ownership of this feature. It contains a description of the feature and outlines changes necessary to support the feature. The CVS revision history of this file contains the definitive historical record. Rationale This PEP proposes a small extention of current Python syntax regarding the `import' and `from import' statements. These statements load in a module, and either bind that module to a local name, or binds objects from that module to a local name. However, it is sometimes desirable to bind those objects to a different name, for instance to avoid name clashes. Currently, a round-about way has to be used to achieve this: import os real_os = os del os And similar for the `from ... import' statement: from os import fdopen, exit, stat os_fdopen = fdopen os_stat = stat del fdopen, stat The proposed syntax change would add an optional `as' clause to both these statements, as follows: import os as real_os from os import fdopen as os_fdopen, exit, stat as os_stat The `as' name is not intended to be a keyword, and some trickery has to be used to convince the CPython parser it isn't one. For more advanced parsers/tokenizers, however, this should not be a problem. Implementation details A proposed implementation of this new clause can be found in the SourceForge patch manager[XX]. The patch uses a NAME field in the grammar rather than a bare string, to avoid the keyword issue. It also introduces a new bytecode, IMPORT_FROM_AS, which loads an object from a module and pushes it onto the stack, so it can be stored by a normal STORE_NAME opcode. The special case of `from module import *' remains a special case, in that it cannot accomodate an `as' clause. Also, the current implementation does not use IMPORT_FROM_AS for the old form of from-import, even though it would make sense to do so. The reason for this is that the current IMPORT_FROM bytecode loads objects directly from a module into the local namespace, in one bytecode operation, and also handles the special case of `*'. As a result of moving to the IMPORT_FROM_AS bytecode, two things would happen: - Two bytecode operations would have to be performed, per symbol, rather than one. - The names imported through `from-import' would be susceptible to the `global' keyword, which they currently are not. This means that `from-import' outside of the `*' special case behaves more like the normal `import' statement, which already follows the `global' keyword. It also means, however, that the `*' special case is even more special, compared to the ordinary form of `from-import' However, for consistency and for simplicity of implementation, it is probably best to split off the special case entirely, making a separate bytecode `IMPORT_ALL' that handles the special case of `*', and handle all other forms of `from-import' the way the proposed `IMPORT_FROM_AS' bytecode does. This dilemma does not apply to the normal `import' statement, because this is alread split into two opcodes, a `LOAD_MODULE' and a `STORE_NAME' opcode. Supporting the `import as' syntax is a slight change to the compiler only. Copyright This document has been placed in the Public Domain. References [1] http://sourceforge.net/patch/?func=detailpatch&patch_id=101135&group_id=5470 Local Variables: mode: indented-text indent-tabs-mode: nil End: --QKdGvSO+nmPlgiQ/-- From nowonder@nowonder.de Tue Aug 15 16:32:50 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Tue, 15 Aug 2000 15:32:50 +0000 Subject: [Python-Dev] IDLE development - Call for participation Message-ID: <399962A2.D53A048F@nowonder.de> To (hopefully) speed up the devlopment of IDLE a temporary fork has been created as a seperate project at SourceForge: http://idlefork.sourceforge.net http://sourceforge.net/projects/idlefork The CVS version represents the enhanced IDLE version sed by David Scherer in his VPython. Besides other improvements this version executes threads in a seperate process. The spanish inquisition invites everybody interested in IDLE (and not keen to participate in any witch trials) to contribute to the project. Any kind of contribution (discussion of new features, bug reports, patches) will be appreciated. If we can get the new IDLE version stable and Python's benevolent dictator for life blesses our lines of code, the improved IDLE may go back into Python's source tree proper. at-least-it'll-be-part-of-Py3K- -ly y'rs Peter P.S.: You do not have to be a member of the Flying Circus. P.P.S.: There is no Spanish inquisition <0.5 wink>! -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From thomas@xs4all.net Tue Aug 15 16:56:46 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Tue, 15 Aug 2000 17:56:46 +0200 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Python pythonrun.c,2.105,2.106 In-Reply-To: <200008151549.IAA25722@slayer.i.sourceforge.net>; from fdrake@users.sourceforge.net on Tue, Aug 15, 2000 at 08:49:06AM -0700 References: <200008151549.IAA25722@slayer.i.sourceforge.net> Message-ID: <20000815175646.A376@xs4all.nl> On Tue, Aug 15, 2000 at 08:49:06AM -0700, Fred L. Drake wrote: > + #include "osdefs.h" /* SEP */ This comment is kind of cryptic... I know of only one SEP, and that's in "a SEP field", a construct we use quite often at work ;-) Does this comment mean the same ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From fdrake@beopen.com Tue Aug 15 17:09:34 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Tue, 15 Aug 2000 12:09:34 -0400 (EDT) Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Python pythonrun.c,2.105,2.106 In-Reply-To: <20000815175646.A376@xs4all.nl> References: <200008151549.IAA25722@slayer.i.sourceforge.net> <20000815175646.A376@xs4all.nl> Message-ID: <14745.27454.815489.456310@cj42289-a.reston1.va.home.com> Thomas Wouters writes: > On Tue, Aug 15, 2000 at 08:49:06AM -0700, Fred L. Drake wrote: > > > + #include "osdefs.h" /* SEP */ > > This comment is kind of cryptic... I know of only one SEP, and that's in "a > SEP field", a construct we use quite often at work ;-) Does this comment > mean the same ? Very cryptic indeed! It meant I was including osdefs.h to get the SEP #define from there, but then I didn't need it in the final version of the code, so the #include can be removed. I'll remove those pronto! Thanks for pointing out my sloppiness! -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From trentm@ActiveState.com Tue Aug 15 18:47:23 2000 From: trentm@ActiveState.com (Trent Mick) Date: Tue, 15 Aug 2000 10:47:23 -0700 Subject: [Python-Dev] segfault in sre on 64-bit plats Message-ID: <20000815104723.A27306@ActiveState.com> Fredrik, The sre module currently segfaults on one of the tests suite tests on both Win64 and 64-bit linux: [trentm@nickel src]$ ./python -c "import sre; sre.match('(x)*', 50000*'x')" > srefail.out Segmentation fault (core dumped) I know that I can't expect you to debug this completely, as you don't have to hardware, but I was hoping you might be able to shed some light on the subject for me. This test on Win32 and Linux32 hits the recursion limit check of 10000 in SRE_MATCH(). However, on Linux64 the segfault occurs at a recursion depth of 7500. I don't want to just willy-nilly drop the recursion limit down to make the problem go away. Do you have any idea why the segfault may be occuring on 64-bit platforms? Mark (Favas), have you been having any problems with sre on your 64-bit plats? In the example above I turned VERBOSE on in _sre.c. WOuld the trace help you? Here is the last of it (the whole thing is 2MB so I am not sending it all): copy 0:1 to 15026 (2) |0x600000000020b90c|0x6000000000200d72|ENTER 7517 |0x600000000020b90e|0x6000000000200d72|MARK 0 |0x600000000020b912|0x6000000000200d72|LITERAL 120 |0x600000000020b916|0x6000000000200d73|MARK 1 |0x600000000020b91a|0x6000000000200d73|MAX_UNTIL 7515 copy 0:1 to 15028 (2) |0x600000000020b90c|0x6000000000200d73|ENTER 7518 |0x600000000020b90e|0x6000000000200d73|MARK 0 |0x600000000020b912|0x6000000000200d73|LITERAL 120 |0x600000000020b916|0x6000000000200d74|MARK 1 |0x600000000020b91a|0x6000000000200d74|MAX_UNTIL 7516 copy 0:1 to 15030 (2) |0x600000000020b90c|0x6000000000200d74|ENTER 7519 |0x600000000020b90e|0x6000000000200d74|MARK 0 |0x600000000020b912|0x6000000000200d74|LITERAL 120 |0x600000000020b916|0x6000000000200d75|MARK 1 |0x600000000020b91a|0x6000000000200d75|MAX_UNTIL 7517 copy 0:1 to 15032 (2) |0x600000000020b90c|0x6000000000200d75|ENTER 7520 |0x600000000020b90e|0x6000000000200d75|MARK 0 |0x600000000020b912|0x6000000000200d75|LITERAL 120 |0x600000000020b916|0x6000000000200d76|MARK 1 |0x600000000020b91a|0x6000000000200d76|MAX_UNTIL 7518 copy 0:1 to 15034 (2) |0x600000000020b90c|0x6000000000200d76|ENTER 7521 |0x600000000020b90e|0x6000000000200d76|MARK 0 |0x600000000020b912|0x6000000000200d76|LITERAL 120 |0x600000000020b916|0x6000000000200d77|MARK 1 |0x600000000020b91a|0x6000000000200d77|MAX_UNTIL 7519 copy 0:1 to 15036 (2) |0x600000000020b90c|0x600 Thanks, Trent -- Trent Mick TrentM@ActiveState.com From thomas@xs4all.net Tue Aug 15 19:24:14 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Tue, 15 Aug 2000 20:24:14 +0200 Subject: [Python-Dev] Re: [Patches] [Patch #101175] Fix slight bug in the Ref manual docs on listcomprehensions In-Reply-To: <200008151746.KAA06454@bush.i.sourceforge.net>; from noreply@sourceforge.net on Tue, Aug 15, 2000 at 10:46:39AM -0700 References: <200008151746.KAA06454@bush.i.sourceforge.net> Message-ID: <20000815202414.B376@xs4all.nl> On Tue, Aug 15, 2000 at 10:46:39AM -0700, noreply@sourceforge.net wrote: [ About my slight fix to ref5.tex, on list comprehensions syntax ] > Comment by tim_one: > Reassigned to Fred, because it's a simple doc change. Fred, accept this > and check it in. Note that the grammar has a bug, though, so this > will need to be changed again (and so will the implementation). That is, > [x if 6] should not be a legal expression but the grammar allows it today. A comment by someone (?!ng ?) who forgot to login, at the original list-comprehensions patch suggests that Skip forgot to include the documentation patch to listcomps he provided. Ping, Skip, can you sort this out and check in the rest of that documentation (which supposedly includes a tutorial section as well) ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Tue Aug 15 19:27:38 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Tue, 15 Aug 2000 20:27:38 +0200 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/ref ref5.tex,1.32,1.33 In-Reply-To: <200008151754.KAA19233@slayer.i.sourceforge.net>; from fdrake@users.sourceforge.net on Tue, Aug 15, 2000 at 10:54:51AM -0700 References: <200008151754.KAA19233@slayer.i.sourceforge.net> Message-ID: <20000815202737.C376@xs4all.nl> On Tue, Aug 15, 2000 at 10:54:51AM -0700, Fred L. Drake wrote: > Index: ref5.tex > diff -C2 -r1.32 -r1.33 > *** ref5.tex 2000/08/12 18:09:50 1.32 > --- ref5.tex 2000/08/15 17:54:49 1.33 > *************** > *** 153,157 **** > > \begin{verbatim} > ! list_display: "[" [expression_list [list_iter]] "]" > list_iter: list_for | list_if > list_for: "for" expression_list "in" testlist [list_iter] > --- 153,158 ---- > > \begin{verbatim} > ! list_display: "[" [listmaker] "]" > ! listmaker: expression_list ( list_iter | ( "," expression)* [","] ) Uhm, this is wrong, and I don't think it was what I submitted either (though, if I did, I apologize :) The first element of listmaker is an expression, not an expression_list. I'll change that, unless Ping and Skip wake up and fix it in a much better way instead. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From fdrake@beopen.com Tue Aug 15 19:32:07 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Tue, 15 Aug 2000 14:32:07 -0400 (EDT) Subject: [Python-Dev] Re: [Patches] [Patch #101175] Fix slight bug in the Ref manual docs on listcomprehensions In-Reply-To: <20000815202414.B376@xs4all.nl> References: <200008151746.KAA06454@bush.i.sourceforge.net> <20000815202414.B376@xs4all.nl> Message-ID: <14745.36007.423378.87635@cj42289-a.reston1.va.home.com> Thomas Wouters writes: > A comment by someone (?!ng ?) who forgot to login, at the original > list-comprehensions patch suggests that Skip forgot to include the > documentation patch to listcomps he provided. Ping, Skip, can you sort this > out and check in the rest of that documentation (which supposedly includes a > tutorial section as well) ? I've not been tracking the list comprehensions discussion, but there is a (minimal) entry in the tutorial. It could use some fleshing out. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From fdrake@beopen.com Tue Aug 15 19:34:43 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Tue, 15 Aug 2000 14:34:43 -0400 (EDT) Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/ref ref5.tex,1.32,1.33 In-Reply-To: <20000815202737.C376@xs4all.nl> References: <200008151754.KAA19233@slayer.i.sourceforge.net> <20000815202737.C376@xs4all.nl> Message-ID: <14745.36163.362268.388275@cj42289-a.reston1.va.home.com> Thomas Wouters writes: > Uhm, this is wrong, and I don't think it was what I submitted either > (though, if I did, I apologize :) The first element of listmaker is an > expression, not an expression_list. I'll change that, unless Ping and Skip > wake up and fix it in a much better way instead. You're right; that's what I get for applying it manually (trying to avoid all the machinery of saving/patching from SF...). Fixed in a sec! -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From Fredrik Lundh" Message-ID: <005401c006ec$a95a74a0$f2a6b5d4@hagrid> trent wrote: > This test on Win32 and Linux32 hits the recursion limit check of 10000 in > SRE_MATCH(). However, on Linux64 the segfault occurs at a recursion depth of > 7500. I don't want to just willy-nilly drop the recursion limit down to make > the problem go away. SRE is overflowing the stack, of course :-( ::: I spent a little time surfing around on the MSDN site, and came up with the following little PyOS_CheckStack implementation for Visual C (and compatibles): #include int __inline PyOS_CheckStack() { __try { /* alloca throws a stack overflow exception if there's less than 2k left on the stack */ alloca(2000); return 0; } __except (1) { /* just ignore the error */ } return 1; } a quick look at the resulting assembler code indicates that this should be pretty efficient (some exception-related stuff, and a call to an internal stack probe function), but I haven't added it to the interpreter (and I don't have time to dig deeper into this before the weekend). maybe someone else has a little time to spare? it shouldn't be that hard to figure out 1) where to put this, 2) what ifdef's to use around it, and 3) what "2000" should be changed to... (and don't forget to set USE_STACKCHECK) From Fredrik Lundh" <005401c006ec$a95a74a0$f2a6b5d4@hagrid> Message-ID: <008601c006ed$8100c120$f2a6b5d4@hagrid> I wrote: > } __except (1) { should probably be: } __except (EXCEPTION_EXECUTE_HANDLER) { From Fredrik Lundh" I wrote: > } __except (EXCEPTION_EXECUTE_HANDLER) { which is defined in "excpt.h"... From tim_one@email.msn.com Tue Aug 15 20:19:23 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 15 Aug 2000 15:19:23 -0400 Subject: [Python-Dev] Call for reviewer! Message-ID: There are 5 open & related patches to getopt.py: 101106 thru 101110 inclusive. Who wants to review these? Fair warning in advance that Guido usually hates adding stuff to getopt, and the patch comment I examined the entire 1.6b1 tarball for incompatibilities, and found only 2 in 90+ modules using getopt.py. probably means it's dead on arrival (2+% is infinitely more than 0% <0.1 wink>). On that basis alone, my natural inclination is to reject them for lack of backward compatibility. So let's get some votes and see whether there's sufficient support to overcome that. From trentm@ActiveState.com Tue Aug 15 20:53:46 2000 From: trentm@ActiveState.com (Trent Mick) Date: Tue, 15 Aug 2000 12:53:46 -0700 Subject: [Python-Dev] Call for reviewer! In-Reply-To: ; from tim_one@email.msn.com on Tue, Aug 15, 2000 at 03:19:23PM -0400 References: Message-ID: <20000815125346.I30086@ActiveState.com> On Tue, Aug 15, 2000 at 03:19:23PM -0400, Tim Peters wrote: > There are 5 open & related patches to getopt.py: 101106 thru 101110 > inclusive. Who wants to review these? Fair warning in advance that Guido > usually hates adding stuff to getopt, and the patch comment > > I examined the entire 1.6b1 tarball for incompatibilities, > and found only 2 in 90+ modules using getopt.py. > > probably means it's dead on arrival (2+% is infinitely more than 0% <0.1 > wink>). > > On that basis alone, my natural inclination is to reject them for lack of > backward compatibility. So let's get some votes and see whether there's > sufficient support to overcome that. > -0 (too timid to use -1) getopt is a nice simple, quick, useful module. Rather than extending it I would rather see a separate getopt-like module for those who need some more heavy duty option processing. One that supports windows '/' switch markers. One where each option is maybe a class instance with methods that do the processing and record state for that option and with attributes for help strings and the number of arguments accepted and argument validification methods. One that supports abstraction of options to capabilities (e.g. two compiler interfaces, same capability, different option to specify it, share option processing). One that support different algorithms for parsing the command line (some current apps like to run through and grab *all* the options, some like to stop option processing at the first non-option). Call it 'supergetopt' and whoever cam 'import supergetopt as getopt'. Keep getopt the way it is. Mind you, I haven't looked at the proposed patches so my opinion might be unfair. Trent -- Trent Mick TrentM@ActiveState.com From akuchlin@mems-exchange.org Tue Aug 15 21:01:56 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Tue, 15 Aug 2000 16:01:56 -0400 Subject: [Python-Dev] Call for reviewer! In-Reply-To: <20000815125346.I30086@ActiveState.com>; from trentm@ActiveState.com on Tue, Aug 15, 2000 at 12:53:46PM -0700 References: <20000815125346.I30086@ActiveState.com> Message-ID: <20000815160156.D16506@kronos.cnri.reston.va.us> On Tue, Aug 15, 2000 at 12:53:46PM -0700, Trent Mick wrote: >Call it 'supergetopt' and whoever cam 'import supergetopt as getopt'. Note that there's Lib/distutils/fancy_getopt.py. The docstring reads: Wrapper around the standard getopt module that provides the following additional features: * short and long options are tied together * options have help strings, so fancy_getopt could potentially create a complete usage summary * options set attributes of a passed-in object --amk From bwarsaw@beopen.com Tue Aug 15 21:30:59 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Tue, 15 Aug 2000 16:30:59 -0400 (EDT) Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/lib libsocket.tex,1.46,1.47 References: <200008151930.MAA10234@slayer.i.sourceforge.net> Message-ID: <14745.43139.834290.323136@anthem.concentric.net> >>>>> "TW" == Thomas Wouters writes: TW> Apply SF patch #101151, by Peter S-K, which fixes smtplib's TW> passing of the 'helo' and 'ehlo' message, and exports the TW> 'make_fqdn' function. This function should be moved to TW> socket.py, if that module ever gets a Python wrapper. Should I work on this for 2.0? Specifically 1) moving socketmodule to _socket and writing a socket.py wrapper; 2) exporting make_fqdn() in socket.py instead of smtplib. It makes no sense for make_fqdn to live in smtplib. I'd be willing to do this. -Barry From gstein@lyra.org Tue Aug 15 21:42:02 2000 From: gstein@lyra.org (Greg Stein) Date: Tue, 15 Aug 2000 13:42:02 -0700 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/lib libsocket.tex,1.46,1.47 In-Reply-To: <14745.43139.834290.323136@anthem.concentric.net>; from bwarsaw@beopen.com on Tue, Aug 15, 2000 at 04:30:59PM -0400 References: <200008151930.MAA10234@slayer.i.sourceforge.net> <14745.43139.834290.323136@anthem.concentric.net> Message-ID: <20000815134202.K19525@lyra.org> On Tue, Aug 15, 2000 at 04:30:59PM -0400, Barry A. Warsaw wrote: > > >>>>> "TW" == Thomas Wouters writes: > > TW> Apply SF patch #101151, by Peter S-K, which fixes smtplib's > TW> passing of the 'helo' and 'ehlo' message, and exports the > TW> 'make_fqdn' function. This function should be moved to > TW> socket.py, if that module ever gets a Python wrapper. > > Should I work on this for 2.0? Specifically 1) moving socketmodule to > _socket and writing a socket.py wrapper; 2) exporting make_fqdn() in > socket.py instead of smtplib. > > It makes no sense for make_fqdn to live in smtplib. > > I'd be willing to do this. Note that Windows already has a socket.py module (under plat-win or somesuch). You will want to integrate that with any new socket.py that you implement. Also note that Windows does some funny stuff in socketmodule.c to export itself as _socket. (the *.dsp files already build it as _socket.dll) +1 Cheers, -g -- Greg Stein, http://www.lyra.org/ From bwarsaw@beopen.com Tue Aug 15 21:46:15 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Tue, 15 Aug 2000 16:46:15 -0400 (EDT) Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/lib libsocket.tex,1.46,1.47 References: <200008151930.MAA10234@slayer.i.sourceforge.net> <14745.43139.834290.323136@anthem.concentric.net> <20000815134202.K19525@lyra.org> Message-ID: <14745.44055.15573.283903@anthem.concentric.net> >>>>> "GS" == Greg Stein writes: GS> Note that Windows already has a socket.py module (under GS> plat-win or somesuch). You will want to integrate that with GS> any new socket.py that you implement. GS> Also note that Windows does some funny stuff in socketmodule.c GS> to export itself as _socket. (the *.dsp files already build it GS> as _socket.dll) GS> +1 Should we have separate plat-*/socket.py files or does it make more sense to try to integrate them into one shared socket.py? From quick glance it certainly looks like there's Windows specific stuff in plat-win/socket.py (big surprise, huh?) -Barry From nowonder@nowonder.de Tue Aug 15 23:47:24 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Tue, 15 Aug 2000 22:47:24 +0000 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/lib libsocket.tex,1.46,1.47 References: <200008151930.MAA10234@slayer.i.sourceforge.net> <14745.43139.834290.323136@anthem.concentric.net> Message-ID: <3999C87C.24A0DF82@nowonder.de> "Barry A. Warsaw" wrote: > > Should I work on this for 2.0? Specifically 1) moving socketmodule to > _socket and writing a socket.py wrapper; 2) exporting make_fqdn() in > socket.py instead of smtplib. +1 on you doing that. I'd volunteer, but I am afraid ... Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From thomas@xs4all.net Tue Aug 15 22:04:11 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Tue, 15 Aug 2000 23:04:11 +0200 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/lib libsocket.tex,1.46,1.47 In-Reply-To: <14745.44055.15573.283903@anthem.concentric.net>; from bwarsaw@beopen.com on Tue, Aug 15, 2000 at 04:46:15PM -0400 References: <200008151930.MAA10234@slayer.i.sourceforge.net> <14745.43139.834290.323136@anthem.concentric.net> <20000815134202.K19525@lyra.org> <14745.44055.15573.283903@anthem.concentric.net> Message-ID: <20000815230411.D376@xs4all.nl> On Tue, Aug 15, 2000 at 04:46:15PM -0400, Barry A. Warsaw wrote: > GS> Note that Windows already has a socket.py module (under > GS> plat-win or somesuch). You will want to integrate that with > GS> any new socket.py that you implement. BeOS also has its own socket.py wrapper, to provide some functions BeOS itself is missing (dup, makefile, fromfd, ...) I'm not sure if that's still necessary, though, perhaps BeOS decided to implement those functions in a later version ? > Should we have separate plat-*/socket.py files or does it make more > sense to try to integrate them into one shared socket.py? From quick > glance it certainly looks like there's Windows specific stuff in > plat-win/socket.py (big surprise, huh?) And don't forget the BeOS stuff ;P This is the biggest reason I didn't do it myself: it takes some effort and a lot of grokking to fix this up properly, without spreading socket.py out in every plat-dir. Perhaps it needs to be split up like the os module ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From gmcm@hypernet.com Tue Aug 15 23:25:33 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Tue, 15 Aug 2000 18:25:33 -0400 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/lib libsocket.tex,1.46,1.47 In-Reply-To: <20000815230411.D376@xs4all.nl> References: <14745.44055.15573.283903@anthem.concentric.net>; from bwarsaw@beopen.com on Tue, Aug 15, 2000 at 04:46:15PM -0400 Message-ID: <1245744161-146360088@hypernet.com> Thomas Wouters wrote: > On Tue, Aug 15, 2000 at 04:46:15PM -0400, Barry A. Warsaw wrote: > > > GS> Note that Windows already has a socket.py module (under > > GS> plat-win or somesuch). You will want to integrate that > > with GS> any new socket.py that you implement. > > BeOS also has its own socket.py wrapper, to provide some > functions BeOS itself is missing (dup, makefile, fromfd, ...) I'm > not sure if that's still necessary, though, perhaps BeOS decided > to implement those functions in a later version ? Sounds very close to what Windows left out. As for *nixen, there are some differences between BSD and SysV sockets, but they're really, really arcane. - Gordon From fdrake@beopen.com Wed Aug 16 00:06:15 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Tue, 15 Aug 2000 19:06:15 -0400 (EDT) Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/lib libsocket.tex,1.46,1.47 In-Reply-To: <14745.43139.834290.323136@anthem.concentric.net> References: <200008151930.MAA10234@slayer.i.sourceforge.net> <14745.43139.834290.323136@anthem.concentric.net> Message-ID: <14745.52455.487734.450253@cj42289-a.reston1.va.home.com> Barry A. Warsaw writes: > Should I work on this for 2.0? Specifically 1) moving socketmodule to > _socket and writing a socket.py wrapper; 2) exporting make_fqdn() in > socket.py instead of smtplib. > > It makes no sense for make_fqdn to live in smtplib. I've started, but am momentarily interupted. Watch for it late tonight. ;) -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From bwarsaw@beopen.com Wed Aug 16 00:19:42 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Tue, 15 Aug 2000 19:19:42 -0400 (EDT) Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Doc/lib libsocket.tex,1.46,1.47 References: <200008151930.MAA10234@slayer.i.sourceforge.net> <14745.43139.834290.323136@anthem.concentric.net> <14745.52455.487734.450253@cj42289-a.reston1.va.home.com> Message-ID: <14745.53262.605601.806635@anthem.concentric.net> >>>>> "Fred" == Fred L Drake, Jr writes: Fred> I've started, but am momentarily interupted. Watch for it Fred> late tonight. ;) Okay fine. I'll hold off on socket module then, and will take a look at whatever you come up with. -Barry From gward@mems-exchange.org Wed Aug 16 00:57:51 2000 From: gward@mems-exchange.org (Greg Ward) Date: Tue, 15 Aug 2000 19:57:51 -0400 Subject: [Python-Dev] Winreg update In-Reply-To: <3993FEC7.4E38B4F1@prescod.net>; from paul@prescod.net on Fri, Aug 11, 2000 at 08:25:27AM -0500 References: <3993FEC7.4E38B4F1@prescod.net> Message-ID: <20000815195751.A16100@ludwig.cnri.reston.va.us> On 11 August 2000, Paul Prescod said: > This is really easy so I want > some real feedback this time. Distutils people, this means you! Mark! I > would love to hear Bill Tutt, Greg Stein and anyone else who claims some > knowledge of Windows! All I know is that the Distutils only use the registry for one thing: finding the MSVC binaries (in distutils/msvccompiler.py). The registry access is coded in such a way that we can use either the win32api/win32con modules ("old way") or _winreg ("new way", but still the low-level interface). I'm all in favour of high-level interfaces, and I'm also in favour of speaking the local tongue -- when in Windows, follow the Windows API (at least for features that are totally Windows-specific, like the registry). But I know nothing about all this stuff, and as far as I know the registry access in distutils/msvccompiler.py works just fine as is. Greg From tim_one@email.msn.com Wed Aug 16 01:28:10 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 15 Aug 2000 20:28:10 -0400 Subject: [Python-Dev] Release Satii Message-ID: 1.6: Is done, but being held back (by us -- two can play at this game ) pending resolution of license issues. Since 2.0 will be a derivative work of 1.6, the license that goes out with 1.6 affects us forever after. Can't say more about that because I don't know more; and Guido is out of town this week. 2.0: Full steam ahead! Just finished going thru every patch on SourceForge. What's Open at this instant is *it* for new 2.0 features. More accurately, they're the only new features that will still be *considered* for 2.0 (not everything in Open now will necessarily be accepted). The only new patches that won't be instantly Postponed from now until 2.0 final ships are bugfixes. Some oddities: + 8 patches remain unassigned. 7 of those are part of a single getopt crusade (well, two getopt crusades, since as always happens when people go extending getopt, they can't agree about what to do), and if nobody speaks in their favor they'll probably get gently rejected. The eighth is a CGI patch from Ping that looks benign to me but is incomplete (missing doc changes). + /F's Py_ErrFormat patch got moved back from Rejected to Open so we can find all the putative 2.0 patches in one SF view (i.e., Open). I've said before that I have no faith in the 2.0 release schedule. Here's your chance to make a fool of me -- and in public too ! nothing-would-make-me-happier-ly y'rs - tim From greg@cosc.canterbury.ac.nz Wed Aug 16 01:57:18 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Wed, 16 Aug 2000 12:57:18 +1200 (NZST) Subject: [Python-Dev] Re: [Patches] [Patch #101175] Fix slight bug in the Ref manual docs on listcomprehensions In-Reply-To: <20000815202414.B376@xs4all.nl> Message-ID: <200008160057.MAA15191@s454.cosc.canterbury.ac.nz> Thomas Wouters : > Comment by tim_one: > [x if 6] should not be a legal expression but the grammar allows it today. Why shouldn't it be legal? The meaning is quite clear (either a one-element list or an empty list). It's something of a degenerate case, but I don't think degenerate cases should be excluded simply because they're degenerate. Excluding it will make both the implementation and documentation more complicated, with no benefit that I can see. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim_one@email.msn.com Wed Aug 16 02:26:36 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 15 Aug 2000 21:26:36 -0400 Subject: [Python-Dev] Re: [Patches] [Patch #101175] Fix slight bug in the Ref manual docs on listcomprehensions In-Reply-To: <200008160057.MAA15191@s454.cosc.canterbury.ac.nz> Message-ID: [Tim] > [x if 6] should not be a legal expression but the grammar > allows it today. [Greg Ewing] > Why shouldn't it be legal? Because Guido hates it. It's almost certainly an error on the part of the user; really the same reason that zip() without arguments raises an exception. > ... > Excluding it will make both the implementation and documentation > more complicated, Of course, but marginally so. "The first clause must be an iterator"; end of doc changes. > with no benefit that I can see. Catching likely errors is a benefit for the user. I realize that Haskell does allow it -- although that would be a surprise to most Haskell users . From dgoodger@bigfoot.com Wed Aug 16 03:36:02 2000 From: dgoodger@bigfoot.com (David Goodger) Date: Tue, 15 Aug 2000 22:36:02 -0400 Subject: [Python-Dev] Re: Call for reviewer! In-Reply-To: Message-ID: I thought the "backwards compatibility" issue might be a sticking point ;> And I *can* see why. So, if I were to rework the patch to remove the incompatibility, would it fly or still be shot down? Here's the change, in a nutshell: Added a function getoptdict(), which returns the same data as getopt(), but instead of a list of [(option, optarg)], it returns a dictionary of {option:optarg}, enabling random/direct access. getoptdict() turns this: if __name__ == '__main__': import getopt opts, args = getopt.getopt(sys.argv[1:], 'a:b') if len(args) <> 2: raise getopt.error, 'Exactly two arguments required.' options = {'a': [], 'b': 0} # default option values for opt, optarg in opts: if opt == '-a': options['a'].append(optarg) elif opt == '-b': options['b'] = 1 main(args, options) into this: if __name__ == '__main__': import getopt opts, args = getopt.getoptdict(sys.argv[1:], 'a:b', repeatedopts=APPEND) if len(args) <> 2: raise getopt.error, 'Exactly two arguments required.' options = {'a': opts.get('-a', [])} options['b'] = opts.has_key('-b') main(args, options) (Notice how the defaults get subsumed into the option processing, which goes from 6 lines to 2 for this short example. A much higher-level interface, IMHO.) BUT WAIT, THERE'S MORE! As part of the deal, you get a free test_getopt.py regression test module! Act now; vote +1! (Actually, you'll get that no matter what you vote. I'll remove the getoptdict-specific stuff and resubmit it if this patch is rejected.) The incompatibility was introduced because the current getopt() returns an empty string as the optarg (second element of the tuple) for an argumentless option. I changed it to return None. Otherwise, it's impossible to differentiate between an argumentless option '-a' and an empty string argument '-a ""'. But I could rework it to remove the incompatibility. Again: If the patch were to become 100% backwards-compatible, with just the addition of getoptdict(), would it still be rejected, or does it have a chance? Eagerly awaiting your judgement... -- David Goodger dgoodger@bigfoot.com Open-source projects: - The Go Tools Project: http://gotools.sourceforge.net (more to come!) From akuchlin@mems-exchange.org Wed Aug 16 04:13:08 2000 From: akuchlin@mems-exchange.org (A.M. Kuchling) Date: Tue, 15 Aug 2000 23:13:08 -0400 Subject: [Python-Dev] Fate of Include/my*.h Message-ID: <20000815231308.A1157@207-172-36-205.s205.tnt6.ann.va.dialup.rcn.com> The now-redundant Include/my*.h files in Include should either be deleted, or at least replaced with empty files containing only a "This file is obsolete" comment. I don't think they were ever part of the public API (Python.h always included them), so deleting them shouldn't break anything. --amk From greg@cosc.canterbury.ac.nz Wed Aug 16 04:04:33 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Wed, 16 Aug 2000 15:04:33 +1200 (NZST) Subject: [Python-Dev] Re: [Patches] [Patch #101175] Fix slight bug in the Ref manual docs on listcomprehensions In-Reply-To: Message-ID: <200008160304.PAA15215@s454.cosc.canterbury.ac.nz> Tim Peters: > Because Guido hates it. It's almost certainly an error on the part > of the user Guido doesn't like it, therefore it must be an error. Great piece of logic there. > Catching likely errors is a benefit for the user. What evidence is there that this particular "likely error" is going to be prevalent enough to justify outlawing a potentially useful construct? Where are the hoardes of Haskell user falling into this trap and clamouring for it to be disallowed? > really the same reason that zip() without arguments raises an > exception. No, I don't think it's the same reason. It's not clear what zip() without arguments should return. There's no such difficulty in this case. For the most part, Python is free of artificial restrictions, and I like it that way. Imposing a restriction of this sort seems un-Pythonic. This is the second gratuitous change that's been made to my LC syntax without any apparent debate. While I acknowledge the right of the BDFL to do this, I'm starting to feel a bit left out... > I realize that Haskell does allow it -- although that would be a > surprise to most Haskell users Which suggests that they don't trip over this feature very often, otherwise they'd soon find out about it! Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From gstein@lyra.org Wed Aug 16 04:20:13 2000 From: gstein@lyra.org (Greg Stein) Date: Tue, 15 Aug 2000 20:20:13 -0700 Subject: [Python-Dev] Fate of Include/my*.h In-Reply-To: <20000815231308.A1157@207-172-36-205.s205.tnt6.ann.va.dialup.rcn.com>; from amk@s205.tnt6.ann.va.dialup.rcn.com on Tue, Aug 15, 2000 at 11:13:08PM -0400 References: <20000815231308.A1157@207-172-36-205.s205.tnt6.ann.va.dialup.rcn.com> Message-ID: <20000815202013.H17689@lyra.org> On Tue, Aug 15, 2000 at 11:13:08PM -0400, A.M. Kuchling wrote: > The now-redundant Include/my*.h files in Include should either be > deleted, or at least replaced with empty files containing only a "This > file is obsolete" comment. I don't think they were ever part of the > public API (Python.h always included them), so deleting them shouldn't > break anything. +1 on deleting them. -- Greg Stein, http://www.lyra.org/ From tim_one@email.msn.com Wed Aug 16 04:23:44 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 15 Aug 2000 23:23:44 -0400 Subject: [Python-Dev] Nasty new bug in test_longexp Message-ID: Fred, I vaguely recall you touched something here recently, so you're top o' the list. Smells like an uninitialized variable. 1 of 4: test_longexp fails in release build: C:\PCbuild>python ..\lib\test\regrtest.py test_longexp test_longexp test test_longexp failed -- Writing: '\012', expected: ' ' 1 test failed: test_longexp 2 of 4: but passes in verbose mode, despite that the output doesn't appear to match what's expected (missing " (line 1)"): C:\PCbuild>python ..\lib\test\regrtest.py -v test_longexp test_longexp test_longexp Caught SyntaxError for long expression: expression too long 1 test OK. 3 of 4: but passes in debug build: C:\PCbuild>python_d ..\lib\test\regrtest.py test_longexp Adding parser accelerators ... Done. test_longexp 1 test OK. [3962 refs] 4 of 4: and verbose debug output does appear to match what's expected: C:\PCbuild>python_d ..\lib\test\regrtest.py -v test_longexp Adding parser accelerators ... Done. test_longexp test_longexp Caught SyntaxError for long expression: expression too long (line 1) 1 test OK. [3956 refs] C:\PCbuild> From tim_one@email.msn.com Wed Aug 16 04:24:44 2000 From: tim_one@email.msn.com (Tim Peters) Date: Tue, 15 Aug 2000 23:24:44 -0400 Subject: [Python-Dev] Fate of Include/my*.h In-Reply-To: <20000815231308.A1157@207-172-36-205.s205.tnt6.ann.va.dialup.rcn.com> Message-ID: [A.M. Kuchling] > The now-redundant Include/my*.h files in Include should either be > deleted, or at least replaced with empty files containing only a "This > file is obsolete" comment. I don't think they were ever part of the > public API (Python.h always included them), so deleting them shouldn't > break anything. +1 From tim_one@email.msn.com Wed Aug 16 05:13:00 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 16 Aug 2000 00:13:00 -0400 Subject: [Python-Dev] Re: [Patches] [Patch #101175] Fix slight bug in the Ref manual docs on listcomprehensions In-Reply-To: <200008160304.PAA15215@s454.cosc.canterbury.ac.nz> Message-ID: [Tim] >> Because Guido hates it. It's almost certainly an error on the part >> of the user [Greg Ewing] > Guido doesn't like it, therefore it must be an error. Great > piece of logic there. Perhaps I should have used a colon: Guido hates it *because* it's almost certainly an error. I expect the meaning was plain enough without that, though. >> Catching likely errors is a benefit for the user. > What evidence is there that this particular "likely error" is Nobody said it was likely. Scare quotes don't work unless you quote something that was actually said . Likeliness has nothing to do with whether Python calls something an error anyway, here or anywhere else. > going to be prevalent enough to justify outlawing a potentially > useful construct? Making a list that's either empty or a singleton is useful? Fine, here you go: (boolean and [x] or []) We don't need listcomps for that. listcomps are a concrete implementation of mathematical set-builder notation, and without iterators to supply a universe of elements to build *on*, it may make *accidental* sense thanks to this particular implementation -- but about as much *logical* sense as map(None, seq1, seq2, ...) makes now. SETL is the original computer language home for comprehensions (both set and list), and got this part right (IMO; Guido just hates it for his own incrutable reasons ). > Where are the hoardes of Haskell user falling into this trap and > clamouring for it to be disallowed? I'd look over on comp.lang.haskell -- provided anyone is still hanging out there. >> really the same reason that zip() without arguments raises an >> exception. > No, I don't think it's the same reason. It's not clear what > zip() without arguments should return. There's no such difficulty > in this case. A zip with no arguments has no universe to zip over; a listcomp without iterators has no universe to build on. I personally don't want syntax that's both a floor wax and a dessert topping. The *intent* here is to supply a flexible and highly expressive way to build lists out of other sequences; no other sequences, use something else. > For the most part, Python is free of artificial restrictions, and I > like it that way. Imposing a restriction of this sort seems > un-Pythonic. > > This is the second gratuitous change that's been made to my > LC syntax without any apparent debate. The syntax hasn't been changed yet -- this *is* the debate. I won't say any more about it, let's hear what others think. As to being upset over changes to your syntax, I offered you ownership of the PEP the instant it was dumped on me (26-Jul), but didn't hear back. Perhaps you didn't get the email. BTW, what was the other gratuitous change? Requiring parens around tuple targets? That was discussed here too, but the debate was brief as consensus seemed clearly to favor requiring them. That, plus Guido suggested it at a PythonLabs mtg, and agreement was unanimous on that point. Or are you talking about some other change (I can't recall any other one)? > While I acknowledge the right of the BDFL to do this, I'm starting > to feel a bit left out... Well, Jeez, Greg -- Skip took over the patch, Ping made changes to it after, I got stuck with the PEP and the Python-Dev rah-rah stuff, and you just sit back and snipe. That's fine, you're entitled, but if you choose not to do the work anymore, you took yourself out of the loop. >> I realize that Haskell does allow it -- although that would be a >> surprise to most Haskell users > Which suggests that they don't trip over this feature very > often, otherwise they'd soon find out about it! While also suggesting it's useless to allow it. From paul@prescod.net Wed Aug 16 05:30:06 2000 From: paul@prescod.net (Paul Prescod) Date: Wed, 16 Aug 2000 00:30:06 -0400 Subject: [Python-Dev] Lockstep iteration - eureka! References: Message-ID: <399A18CE.6CFFCAB9@prescod.net> Tim Peters wrote: > > ... > > But if you add seq.items(), you had better add seq.keys() too, and > seq.values() as a synonym for seq[:]. I guess the perceived advantage of > adding seq.items() is that it supplies yet another incredibly slow and > convoluted way to get at the for-loop index? "Ah, that's the ticket! Let's > allocate gazillabytes of storage and compute all the indexes into a massive > data structure up front, and then we can use the loop index that's already > sitting there for free anyway to index into that and get back a redundant > copy of itself!" . > > not-a-good-sign-when-common-sense-is-offended-ly y'rs - tim .items(), .keys(), .values() and range() all offended my common sense when I started using Python in the first place. I got over it. I really don't see this "indexing" issue to be common enough either for special syntax OR to worry alot about efficiency. Nobody is forcing anyone to use .items(). If you want a more efficient way to do it, it's available (just not as syntactically beautifu -- same as range/xrangel). That isn't the case for dictionary .items(), .keys() and .values(). Also, if .keys() returns a range object then theoretically the interpreter could recognize that it is looping over a range and optimize it at runtime. That's an alternate approach to optimizing range literals through new byte-codes. I don't have time to think about what that would entail right now.... :( -- Paul Prescod - Not encumbered by corporate consensus Simplicity does not precede complexity, but follows it. - http://www.cs.yale.edu/homes/perlis-alan/quotes.html From fdrake@beopen.com Wed Aug 16 05:51:34 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Wed, 16 Aug 2000 00:51:34 -0400 (EDT) Subject: [Python-Dev] socket module changes Message-ID: <14746.7638.870650.747281@cj42289-a.reston1.va.home.com> This is a brief description of what I plan to check in to make the changes we've discussed regarding the socket module. I'll make the checkins tomorrow morning, allowing you all night to scream if you think that'll help. ;) Windows and BeOS both use a wrapper module, but these are essentially identical; the Windows variant has evolved a bit more, but that evolution is useful for BeOS as well, aside from the errorTab table (which gives messages for Windows-specific error numbers). I will be moving the sharable portions to a new module, _dupless_socket, which the new socket module will import on Windows and BeOS. (That name indicates why they use a wrapper in the first place!) The errorTab definition will be moved to the new socket module and will only be defined on Windows. The exist wrappers, plat-beos/socket.py and plat-win/socket.py, will be removed. socketmodule.c will only build as _socket, allowing much simplification of the conditional compilation at the top of the initialization function. The socket module will include the make_fqdn() implementation, adjusted to make local references to the socket module facilities it requires and to use string methods instead of using the string module. It is documented. The docstring in _socket will be moved to socket.py. If the screaming doesn't wake me, I'll check this in in the morning. The regression test isn't complaining! ;) -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From tim_one@email.msn.com Wed Aug 16 06:12:21 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 16 Aug 2000 01:12:21 -0400 Subject: [Python-Dev] Lockstep iteration - eureka! In-Reply-To: <399A18CE.6CFFCAB9@prescod.net> Message-ID: [Paul Prescod] > ... > I really don't see this "indexing" issue to be common enough A simple grep (well, findstr under Windows) finds over 300 instances of for ... in range(len(... in the .py files on my laptop. I don't recall exactly what the percentages were when I looked over a very large base of Python code several years ago, but I believe it was about 1 in 7 for loops. > for special syntax OR to worry alot about efficiency. 1 in 7 is plenty. range(len(seq)) is a puzzler to newbies, too -- it's *such* an indirect way of saying what they say directly in other languages. > Nobody is forcing anyone to use .items(). Agreed, but since seq.items() doesn't exist now . > If you want a more efficient way to do it, it's available (just not as > syntactically beautiful -- same as range/xrangel). Which way would that be? I don't know of one, "efficient" either in the sense of runtime speed or of directness of expression. xrange is at least a storage-efficient way, and isn't it grand that we index the xrange object with the very integer we're (usually) trying to get it to return ? The "loop index" isn't an accident of the way Python happens to implement "for" today, it's the very basis of Python's thing.__getitem__(i)/IndexError iteration protocol. Exposing it is natural, because *it* is natural. > ... > Also, if .keys() returns a range object then theoretically the > interpreter could recognize that it is looping over a range and optimize > it at runtime. Sorry, but seq.keys() just makes me squirm. It's a little step down the Lispish path of making everything look the same. I don't want to see float.write() either . although-that-would-surely-be-more-orthogonal-ly y'rs - tim From thomas@xs4all.net Wed Aug 16 06:34:29 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 16 Aug 2000 07:34:29 +0200 Subject: [Python-Dev] Re: [Patches] [Patch #101175] Fix slight bug in the Ref manual docs on listcomprehensions In-Reply-To: ; from tim_one@email.msn.com on Wed, Aug 16, 2000 at 12:13:00AM -0400 References: <200008160304.PAA15215@s454.cosc.canterbury.ac.nz> Message-ID: <20000816073429.E376@xs4all.nl> On Wed, Aug 16, 2000 at 12:13:00AM -0400, Tim Peters wrote: > > This is the second gratuitous change that's been made to my > > LC syntax without any apparent debate. > > The syntax hasn't been changed yet -- this *is* the debate. I won't say any > more about it, let's hear what others think. It'd be nice to hear *what* the exact syntax issue is. At first I thought you meant forcing parentheses around all forms of iterator expressions, but apparently you mean requiring at least a single 'for' statement in a listcomp ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From tim_one@email.msn.com Wed Aug 16 06:36:24 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 16 Aug 2000 01:36:24 -0400 Subject: [Python-Dev] Re: [Patches] [Patch #101175] Fix slight bug in the Ref manual docs on listcomprehensions In-Reply-To: <200008160304.PAA15215@s454.cosc.canterbury.ac.nz> Message-ID: Clarification: [Tim] >>> Catching likely errors is a benefit for the user. [Greg Ewing] >> What evidence is there that this particular "likely error" is .. [Tim] > Nobody said it was likely. ... Ha! I did! But not in Greg's sense. It was originally in the sense of "if we see it, it's almost certainly an error on the part of the user", not that "it's likely we'll see this". This is in the same sense that Python considers x = float(i,,) or x = for i [1,2,3] to be likely errors -- you don't see 'em often, but they're most likely errors on the part of the user when you do. back-to-the-more-mundane-confusions-ly y'rs - tim From greg@cosc.canterbury.ac.nz Wed Aug 16 07:02:23 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Wed, 16 Aug 2000 18:02:23 +1200 (NZST) Subject: [Python-Dev] Re: [Patches] [Patch #101175] Fix slight bug in the Ref manual docs on listcomprehensions In-Reply-To: Message-ID: <200008160602.SAA15239@s454.cosc.canterbury.ac.nz> > Guido hates it *because* it's almost certainly an error. Yes, I know what you meant. I was just trying to point out that, as far as I can see, it's only Guido's *opinion* that it's almost certainly an error. Let n1 be the number of times that [x if y] appears in some program and the programmer actually meant to write something else. Let n2 be the number of times [x if y] appears and the programmer really meant it. Now, I agree that n1+n2 will probably be a very small number. But from that alone it doesn't follow that a given instance of [x if y] is probably an error. That is only true if n1 is much greater than n2, and in the absence of any experience, there's no reason to believe that. > A zip with no arguments has no universe to zip over; a listcomp without > iterators has no universe to build on... The *intent* here is to > supply a flexible and highly expressive way to build lists out of other > sequences; no other sequences, use something else. That's a reasonable argument. It might even convince me if I think about it some more. I'll think about it some more. > if you choose not to do the work anymore, you took yourself out of the > loop. You're absolutely right. I'll shut up now. (By the way, I think your mail must have gone astray, Tim -- I don't recall ever being offered ownership of a PEP, whatever that might entail.) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim_one@email.msn.com Wed Aug 16 07:18:30 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 16 Aug 2000 02:18:30 -0400 Subject: [Python-Dev] Re: [Patches] [Patch #101175] Fix slight bug in the Ref manual docs on listcomprehensions In-Reply-To: <20000816073429.E376@xs4all.nl> Message-ID: [Thomas Wouters] > It'd be nice to hear *what* the exact syntax issue is. At first I thought > you meant forcing parentheses around all forms of iterator > expressions, No, that's an old one, and was requiring parens around a target expression iff it's a tuple. So [x, y for x in s for y in t] # BAD [(x, y) for x in s for y in t] # good [(((x, y))) for x in s for y in t] # good, though silly [x+y for x in s for y in t] # good [(x+y) for x in s for y in t] # good [x , for x in s] # BAD [(x ,) for x in s] # good That much is already implemented in the patch currently on SourceForge. > but apparently you mean requiring at least a single 'for' statement > in a listcomp ? No too , but closer: it's that the leftmost ("first") clause must be a "for". So, yes, at least one for, but also that an "if" can't precede *all* the "for"s: [x for x in s if x & 1] # good [x if x & 1 for x in s] # BAD [x for x in s] # good [x if y & 1] # BAD Since the leftmost clause can't refer to any bindings established "to its right", an "if" as the leftmost clause can't act to filter the elements generated by the iterators, and so Guido (me too) feels it's almost certainly an error on the user's part if Python sees an "if" in the leftmost position. The current patch allows all of these, though. In (mathematical) set-builder notation, you certainly see things like odds = {x | mod(x, 2) = 1} That is, with "just a condition". But in such cases the universe over which x ranges is implicit from context (else the expression is not well-defined!), and can always be made explicit; e.g., perhaps the above is in a text where it's assumed everything is a natural number, and then it can be made explicit via odds = {x in Natural | mod(x, 2) = 1| In the concrete implementation afforded by listcomps, there is no notion of an implicit universal set, so (as in SETL too, where this all came from originally) explicit naming of the universe is required. The way listcomps are implemented *can* make [x if y] "mean something" (namely one of [x] or [], depending on y's value), but that has nothing to do with its set-builder heritage. Looks to me like the user is just confused! To Guido too. Hence the desire not to allow this form at all. From ping@lfw.org Wed Aug 16 07:23:57 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Tue, 15 Aug 2000 23:23:57 -0700 (PDT) Subject: [Python-Dev] Re: [Patches] [Patch #101175] Fix slight bug in the Ref manual docs on listcomprehensions In-Reply-To: <200008160057.MAA15191@s454.cosc.canterbury.ac.nz> Message-ID: On Wed, 16 Aug 2000, Greg Ewing wrote: > > [x if 6] should not be a legal expression but the grammar allows it today. > > Why shouldn't it be legal? [...] > Excluding it will make both the implementation and documentation > more complicated, with no benefit that I can see. I don't have a strong opinion on this either way, but i can state pretty confidently that the change would be tiny and simple: just replace "list_iter" in the listmaker production with "list_for", and you are done. -- ?!ng "I'm not trying not to answer the question; i'm just not answering it." -- Lenore Snell From tim_one@email.msn.com Wed Aug 16 07:59:06 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 16 Aug 2000 02:59:06 -0400 Subject: [Python-Dev] Re: [Patches] [Patch #101175] Fix slight bug in the Ref manual docs on listcomprehensions In-Reply-To: <200008160602.SAA15239@s454.cosc.canterbury.ac.nz> Message-ID: [Tim] >> Guido hates it *because* it's almost certainly an error. [Greg Ewing] > Yes, I know what you meant. I was just trying to point out > that, as far as I can see, it's only Guido's *opinion* that > it's almost certainly an error. Well, it's mine too, but I always yield to him on stuff like that anyway; and I guess I *have* to now, because he's my boss . > Let n1 be the number of times that [x if y] appears in some > program and the programmer actually meant to write something > else. Let n2 be the number of times [x if y] appears and > the programmer really meant it. > > Now, I agree that n1+n2 will probably be a very small number. > But from that alone it doesn't follow that a given instance > of [x if y] is probably an error. That is only true if > n1 is much greater than n2, and in the absence of any > experience, there's no reason to believe that. I argued that one all I'm going to -- I think there is. >> ... The *intent* here is to supply a flexible and highly expressive > way to build lists out of other sequences; no other sequences, use > something else. > That's a reasonable argument. It might even convince me if > I think about it some more. I'll think about it some more. Please do, because ... >> if you choose not to do the work anymore, you took yourself out >> of the loop. > You're absolutely right. I'll shut up now. Please don't! This patch is not without opposition, and while consensus is rarely reached on Python-Dev, I think that's partly because "the BDFL ploy" is overused to avoid the pain of principled compromise. If this ends in a stalement among the strongest proponents, it may not be such a good idea after all. > (By the way, I think your mail must have gone astray, Tim -- > I don't recall ever being offered ownership of a PEP, whatever > that might entail.) All explained at http://python.sourceforge.net/peps/ Although in this particular case, I haven't done anything with the PEP except argue in favor of what I haven't yet written! Somebody else filled in the skeletal text that's there now. If you still want it, it's yours; I'll attach the email in question. ok-that's-16-hours-of-python-today-in-just-a-few-more-i'll- have-to-take-a-pee -ly y'rs - tim -----Original Message----- From: Tim Peters [mailto:tim_one@email.msn.com] Sent: Wednesday, July 26, 2000 1:25 AM To: Greg Ewing Subject: RE: [Python-Dev] PEP202 Greg, nice to see you on Python-Dev! I became the PEP202 shepherd because nobody else volunteered, and I want to see the patch get into 2.0. That's all there was to it, though: if you'd like to be its shepherd, happy to yield to you. You've done the most to make this happen! Hmm -- but maybe that also means you don't *want* to do more. That's OK too. From bwarsaw@beopen.com Wed Aug 16 14:21:59 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Wed, 16 Aug 2000 09:21:59 -0400 (EDT) Subject: [Python-Dev] Re: Call for reviewer! References: Message-ID: <14746.38263.433927.239480@anthem.concentric.net> I used to think getopt needed a lot of changes, but I'm not so sure anymore. getopt's current API works fine for me and I use it in all my scripts. However, >>>>> "DG" == David Goodger writes: DG> The incompatibility was introduced because the current DG> getopt() returns an empty string as the optarg (second element DG> of the tuple) for an argumentless option. I changed it to DG> return None. Otherwise, it's impossible to differentiate DG> between an argumentless option '-a' and an empty string DG> argument '-a ""'. But I could rework it to remove the DG> incompatibility. I don't think that's necessary. In my own use, if I /know/ -a doesn't have an argument (because I didn't specify as "a:"), then I never check the optarg. And it's bad form for a flag to take an optional argument; it either does or it doesn't and you know that in advance. -Barry From bwarsaw@beopen.com Wed Aug 16 14:23:45 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Wed, 16 Aug 2000 09:23:45 -0400 (EDT) Subject: [Python-Dev] Fate of Include/my*.h References: <20000815231308.A1157@207-172-36-205.s205.tnt6.ann.va.dialup.rcn.com> Message-ID: <14746.38369.116212.875999@anthem.concentric.net> >>>>> "AMK" == A M Kuchling writes: AMK> The now-redundant Include/my*.h files in Include should AMK> either be deleted +1 -Barry From fdrake@beopen.com Wed Aug 16 15:26:29 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Wed, 16 Aug 2000 10:26:29 -0400 (EDT) Subject: [Python-Dev] socket module changes Message-ID: <14746.42133.355087.417895@cj42289-a.reston1.va.home.com> The changes to the socket module are now complete. Note two changes to yesterdays plan: - there is no _dupless_socket; I just merged that into socket.py - make_fqdn() got renamed to getfqdn() for consistency with the rest of the module. I also remembered to update smptlib. ;) I'll be away from email during the day; Windows & BeOS users, please test this! -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From bwarsaw@beopen.com Wed Aug 16 15:46:26 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Wed, 16 Aug 2000 10:46:26 -0400 (EDT) Subject: [Python-Dev] socket module changes References: <14746.42133.355087.417895@cj42289-a.reston1.va.home.com> Message-ID: <14746.43330.134066.238781@anthem.concentric.net> >> - there is no _dupless_socket; I just merged that into socket.py - Thanks, that's the one thing I was going to complain about. :) -Barry From bwarsaw@beopen.com Wed Aug 16 16:11:57 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Wed, 16 Aug 2000 11:11:57 -0400 (EDT) Subject: [Python-Dev] PEP 214, extended print statement Message-ID: <14746.44861.78992.343012@anthem.concentric.net> After channeling and encouragement by Tim Peters, I've updated PEP 214, the extended print statement. Text is included below, but is also available at http://python.sourceforge.net/peps/pep-0214.html SourceForge patch #100970 contains the patch to apply against the current CVS tree if you want to play with it http://sourceforge.net/patch/download.php?id=100970 -Barry -------------------- snip snip -------------------- PEP: 214 Title: Extended Print Statement Version: $Revision: 1.3 $ Author: bwarsaw@beopen.com (Barry A. Warsaw) Python-Version: 2.0 Status: Draft Created: 24-Jul-2000 Post-History: 16-Aug-2000 Introduction This PEP describes a syntax to extend the standard `print' statement so that it can be used to print to any file-like object, instead of the default sys.stdout. This PEP tracks the status and ownership of this feature. It contains a description of the feature and outlines changes necessary to support the feature. This PEP summarizes discussions held in mailing list forums, and provides URLs for further information, where appropriate. The CVS revision history of this file contains the definitive historical record. Proposal This proposal introduces a syntax extension to the print statement, which allows the programmer to optionally specify the output file target. An example usage is as follows: print >> mylogfile, 'this message goes to my log file' Formally, the syntax of the extended print statement is print_stmt: ... | '>>' test [ (',' test)+ [','] ] ) where the ellipsis indicates the original print_stmt syntax unchanged. In the extended form, the expression just after >> must yield an object with a write() method (i.e. a file-like object). Thus these two statements are equivalent: print 'hello world' print >> sys.stdout, 'hello world' As are these two statements: print print >> sys.stdout These two statements are syntax errors: print , print >> sys.stdout, Justification `print' is a Python keyword and introduces the print statement as described in section 6.6 of the language reference manual[1]. The print statement has a number of features: - it auto-converts the items to strings - it inserts spaces between items automatically - it appends a newline unless the statement ends in a comma The formatting that the print statement performs is limited; for more control over the output, a combination of sys.stdout.write(), and string interpolation can be used. The print statement by definition outputs to sys.stdout. More specifically, sys.stdout must be a file-like object with a write() method, but it can be rebound to redirect output to files other than specifically standard output. A typical idiom is sys.stdout = mylogfile try: print 'this message goes to my log file' finally: sys.stdout = sys.__stdout__ The problem with this approach is that the binding is global, and so affects every statement inside the try: clause. For example, if we added a call to a function that actually did want to print to stdout, this output too would get redirected to the logfile. This approach is also very inconvenient for interleaving prints to various output streams, and complicates coding in the face of legitimate try/except or try/finally clauses. Reference Implementation A reference implementation, in the form of a patch against the Python 2.0 source tree, is available on SourceForge's patch manager[2]. This approach adds two new opcodes, PRINT_ITEM_TO and PRINT_NEWLINE_TO, which simply pop the file like object off the top of the stack and use it instead of sys.stdout as the output stream. Alternative Approaches An alternative to this syntax change has been proposed (originally by Moshe Zadka) which requires no syntax changes to Python. A writeln() function could be provided (possibly as a builtin), that would act much like extended print, with a few additional features. def writeln(*args, **kws): import sys file = sys.stdout sep = ' ' end = '\n' if kws.has_key('file'): file = kws['file'] del kws['file'] if kws.has_key('nl'): if not kws['nl']: end = ' ' del kws['nl'] if kws.has_key('sep'): sep = kws['sep'] del kws['sep'] if kws: raise TypeError('unexpected keywords') file.write(sep.join(map(str, args)) + end) writeln() takes a three optional keyword arguments. In the context of this proposal, the relevant argument is `file' which can be set to a file-like object with a write() method. Thus print >> mylogfile, 'this goes to my log file' would be written as writeln('this goes to my log file', file=mylogfile) writeln() has the additional functionality that the keyword argument `nl' is a flag specifying whether to append a newline or not, and an argument `sep' which specifies the separator to output in between each item. References [1] http://www.python.org/doc/current/ref/print.html [2] http://sourceforge.net/patch/download.php?id=100970 From gvwilson@nevex.com Wed Aug 16 16:49:06 2000 From: gvwilson@nevex.com (Greg Wilson) Date: Wed, 16 Aug 2000 11:49:06 -0400 (EDT) Subject: [Python-Dev] PEP 214, extended print statement Message-ID: > Barry Warsaw wrote: > [extended print PEP] +1 --- it'll come in handy when teaching newbies on Windows and Unix simultaneously. Greg From skip@mojam.com (Skip Montanaro) Wed Aug 16 17:33:30 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 16 Aug 2000 11:33:30 -0500 (CDT) Subject: [Python-Dev] Re: [Patches] [Patch #101175] Fix slight bug in the Ref manual docs on listcomprehensions In-Reply-To: <20000815202414.B376@xs4all.nl> References: <200008151746.KAA06454@bush.i.sourceforge.net> <20000815202414.B376@xs4all.nl> Message-ID: <14746.49754.697401.684106@beluga.mojam.com> Thomas> A comment by someone (?!ng ?) who forgot to login, at the Thomas> original list-comprehensions patch suggests that Skip forgot to Thomas> include the documentation patch to listcomps he provided. Ping, Thomas> Skip, can you sort this out and check in the rest of that Thomas> documentation (which supposedly includes a tutorial section as Thomas> well) ? Ping & I have already taken care of this off-list. His examples should be checked in shortly, if not already. Skip From skip@mojam.com (Skip Montanaro) Wed Aug 16 17:43:44 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 16 Aug 2000 11:43:44 -0500 (CDT) Subject: [Python-Dev] Re: [Patches] [Patch #101175] Fix slight bug in the Ref manual docs on listcomprehensions In-Reply-To: References: <200008160304.PAA15215@s454.cosc.canterbury.ac.nz> Message-ID: <14746.50368.982239.813435@beluga.mojam.com> Tim> Well, Jeez, Greg -- Skip took over the patch, Ping made changes to Tim> it after, I got stuck with the PEP and the Python-Dev rah-rah Tim> stuff, and you just sit back and snipe. That's fine, you're Tim> entitled, but if you choose not to do the work anymore, you took Tim> yourself out of the loop. Tim, I think that's a bit unfair to Greg. Ages ago Greg offered up a prototype implementation of list comprehensions based upon a small amount of discussion on c.l.py. I took over the patch earlier because I wanted to see it added to Python (originally 1.7, which is now 2.0). I knew it would languish or die if someone on python-dev didn't shepherd it. I was just get the thing out there for discussion, and I knew that Greg wasn't on python-dev to do it himself, which is where most of the discussion about list comprehensions has taken place. When I've remembered to, I've tried to at least CC him on threads I've started so he could participate. My apologies to Greg for not being more consistent in that regard. I don't think we can fault him for not having been privy to all the discussion. Skip From gward@mems-exchange.org Wed Aug 16 18:34:02 2000 From: gward@mems-exchange.org (Greg Ward) Date: Wed, 16 Aug 2000 13:34:02 -0400 Subject: [Python-Dev] Python 1.6 & Distutils 0.9.1: success Message-ID: <20000816133401.F16672@ludwig.cnri.reston.va.us> [oops, screwed up the cc: python-dev when I sent this to Fred. let's try again, shall we?] Hi Fred -- I went ahead and tried out the current cnri-16-start branch on Solaris 2.6. (I figured you guys are all using Linux by now, so you might want to hear back how it works on Solaris.) In short: no problem! It built, tested, and installed just fine. Oops, just noticed that my configure.in fix from late May didn't make the cut: revision 1.124 date: 2000/05/26 12:22:54; author: gward; state: Exp; lines: +6 -2 When building on Solaris and the compiler is GCC, use '$(CC) -G' to create shared extensions rather than 'ld -G'. This ensures that shared extensions link against libgcc.a, in case there are any functions in the GCC runtime not already in the Python core. Oh well. This means that Robin Dunn's bsddb extension won't work with Python 1.6 under Solaris. So then I tried Distutils 0.9.1 with the new build: again, it worked just fine. I was able to build and install the Distutils proper, and then NumPy. And I made a NumPy source dist. Looks like it works just fine, although this is hardly a rigorous test (sigh). I'd say go ahead and release Distutils 0.9.1 with Python 1.6... Greg -- Greg Ward - software developer gward@mems-exchange.org MEMS Exchange / CNRI voice: +1-703-262-5376 Reston, Virginia, USA fax: +1-703-262-5367 From thomas@xs4all.net Wed Aug 16 21:55:53 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Wed, 16 Aug 2000 22:55:53 +0200 Subject: [Python-Dev] Pending patches for 2.0 Message-ID: <20000816225552.H376@xs4all.nl> I have a small problem with the number of pending patches that I wrote, and haven't fully finished yet: I'm going to be away on vacation from about September 1st until about October 1st or so ;P I'll try to finish them as much as possible before then (they mostly need just documentation anyway) but if Guido decides to go for a different approach for one or more of them (like allowing floats and/or longs in range literals) someone will have to take them over to finish them in time for 2.0. I'm not sure when I'll be leaving my internet connection behind, where we'll be going or when I'll be back, but I won't be able to do too much rewriting in the next two weeks either -- work is killing me. (Which is one of the reasons I'm going to try to be as far away from work as possible, on September 2nd ;) However, if a couple of patches are rejected/postponed and others don't require substantial changes, and if those decisions are made before, say, August 30th, I think I can move them into the CVS tree before leaving and just shove the responsibility for them on the entire dev team ;) This isn't a push to get them accepted ! Just a warning that if they aren't accepted before then, someone will have to take over the breastfeeding ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From fdrake@beopen.com Wed Aug 16 22:07:35 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Wed, 16 Aug 2000 17:07:35 -0400 (EDT) Subject: [Python-Dev] Pending patches for 2.0 In-Reply-To: <20000816225552.H376@xs4all.nl> References: <20000816225552.H376@xs4all.nl> Message-ID: <14747.663.260950.537440@cj42289-a.reston1.va.home.com> Thomas Wouters writes: > much as possible before then (they mostly need just documentation anyway) ^^^^^^^^^^^^^^^^^^ Don't underestimate that requirement! > This isn't a push to get them accepted ! Just a warning that if they aren't > accepted before then, someone will have to take over the breastfeeding ;) I don't think I want to know too much about your development tools! ;-) -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From fdrake@beopen.com Wed Aug 16 22:24:19 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Wed, 16 Aug 2000 17:24:19 -0400 (EDT) Subject: [Python-Dev] Python 1.6 & Distutils 0.9.1: success In-Reply-To: <20000816133401.F16672@ludwig.cnri.reston.va.us> References: <20000816133401.F16672@ludwig.cnri.reston.va.us> Message-ID: <14747.1667.252426.489530@cj42289-a.reston1.va.home.com> Greg Ward writes: > I went ahead and tried out the current cnri-16-start branch on Solaris > 2.6. (I figured you guys are all using Linux by now, so you might want > to hear back how it works on Solaris.) Great! I've just updated 1.6 to include the Distutils-0_9_1 tagged version of the distutils package and the documentation. I'm rebuilding our release candidates now. > In short: no problem! It built, tested, and installed just fine. Great! Thanks! > Oops, just noticed that my configure.in fix from late May didn't make > the cut: ... > Oh well. This means that Robin Dunn's bsddb extension won't work with > Python 1.6 under Solaris. That's unfortunate. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From thomas@xs4all.net Wed Aug 16 23:22:05 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 17 Aug 2000 00:22:05 +0200 Subject: [Python-Dev] Pending patches for 2.0 In-Reply-To: <14747.663.260950.537440@cj42289-a.reston1.va.home.com>; from fdrake@beopen.com on Wed, Aug 16, 2000 at 05:07:35PM -0400 References: <20000816225552.H376@xs4all.nl> <14747.663.260950.537440@cj42289-a.reston1.va.home.com> Message-ID: <20000817002205.I376@xs4all.nl> On Wed, Aug 16, 2000 at 05:07:35PM -0400, Fred L. Drake, Jr. wrote: > Thomas Wouters writes: > > much as possible before then (they mostly need just documentation anyway) > ^^^^^^^^^^^^^^^^^^ > Don't underestimate that requirement! I'm not, especially since the things that need documentation (if they are in principle acceptable to Guido) are range literals (tutorials and existing code examples), 'import as' (ref, tut), augmented assignment (ref, tut, lib, api, ext, existing examples), the getslice->getitem change (tut, lib, all other references to getslice/extended slices and existing example code) and possibly the 'indexing for' patch (ref, tut, a large selection of existing example code.) Oh, and I forgot, some patches would benefit from more library changes, too, like augmented assignment and getslice-to-getitem. That can always be done after the patches are in, by other people (if they can't, the patch shouldn't go in in the first place!) I guess I'll be doing one large, intimate pass over all documentation, do everything at once, and later split it up. I also think I'm going to post them seperately, to allow for easier proofreading. I also think I'm in need of some sleep, and will think about this more tomorrow, after I get LaTeX2HTML working on my laptop, so I can at least review my own changes ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From trentm@ActiveState.com Thu Aug 17 00:55:42 2000 From: trentm@ActiveState.com (Trent Mick) Date: Wed, 16 Aug 2000 16:55:42 -0700 Subject: [Python-Dev] autoconf question: howto add to CFLAGS and LDFLAGS? Message-ID: <20000816165542.D29260@ActiveState.com> Hello autoconf-masters, I am currently trying to port Python to Monterey (64-bit AIX) and I need to add a couple of Monterey specific options to CFLAGS and LDFLAGS (or to whatever appropriate variables for all 'cc' and 'ld' invocations) but it is not obvious *at all* how to do that in configure.in. Can anybody helpme on that? ANother issue that I am having. This is how the python executable is linked on Linux with gcc: gcc -Xlinker -export-dynamic python.o ../libpython2.0.a -lpthread -ldl -lutil -lm -o python It, of course, works fine, but shouldn't the proper (read "portable") invocation to include the python2.0 library be gcc -Xlinker -export-dynamic python.o -L.. -lpython2.0 -lpthread -ldl -lutil -lm -o python That invocation form (i.e. with the '-L.. -lpython2.0') works on Linux, and is *required* on Monterey. Does this problem not show up with other Unix compilers. My hunch is that simply listing library (*.a) arguments on the gcc command line is a GNU gcc/ld shortcut to the more portable usage of -L and -l. Any opinions. I would either like to change the form to the latter or I'll have to special case the invocation for Monterey. ANy opinions on which is worse. Thanks, Trent -- Trent Mick TrentM@ActiveState.com From trentm@ActiveState.com Thu Aug 17 01:24:25 2000 From: trentm@ActiveState.com (Trent Mick) Date: Wed, 16 Aug 2000 17:24:25 -0700 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements Message-ID: <20000816172425.A32338@ActiveState.com> I am porting Python to Monterey (64-bit AIX) and have a small (hopefully) question about POSIX threads. I have Monterey building and passing the threads test suite using Python/thread_pthread.h with just one issue: -------------- snipped from current thread_pthread.h --------------- long PyThread_get_thread_ident(void) { volatile pthread_t threadid; if (!initialized) PyThread_init_thread(); /* Jump through some hoops for Alpha OSF/1 */ threadid = pthread_self(); return (long) *(long *) &threadid; } ------------------------------------------------------------------- Does the POSIX threads spec specify a C type or minimum size for pthread_t? Or can someone point me to the appropriate resource to look this up. On Linux (mine at least): /usr/include/bits/pthreadtypes.h:120:typedef unsigned long int pthread_t; On Monterey: typedef unsigned int pthread_t; That is fine, they are both 32-bits, however Monterey is an LP64 platform (sizeof(long)==8, sizeof(int)=4), which brings up the question: WHAT IS UP WITH THAT return STATEMENT? return (long) *(long *) &threadid; My *guess* is that this is an attempt to just cast 'threadid' (a pthread_t) to a long and go through hoops to avoid compiler warnings. I dont' know what else it could be. Is that what the "Alpha OSF/1" comment is about? Anybody have an Alpha OSF/1 hanging around. The problem is that when sizeof(pthread_t) != sizeof(long) this line is just broken. Could this be changed to return threadid; safely? Thanks, Trent -- Trent Mick TrentM@ActiveState.com From greg@cosc.canterbury.ac.nz Thu Aug 17 01:33:40 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 17 Aug 2000 12:33:40 +1200 (NZST) Subject: [Python-Dev] Re: [Patches] [Patch #101175] Fix slight bug in the Ref manual docs on listcomprehensions In-Reply-To: Message-ID: <200008170033.MAA15351@s454.cosc.canterbury.ac.nz> > If this ends in a stalement among the strongest proponents, it may not > be such a good idea after all. Don't worry, I really don't have any strong objection to either of these changes. They're only cosmetic, after all. It's still a good idea. Just one comment: even if the first clause *is* a 'for', there's no guarantee that the rest of the clauses have to have anything to do with what it produces. E.g. [x for y in [1] if z] The [x if y] case is only one of an infinite number of possible abuses. Do you still think it's worth taking special steps to catch that particular one? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Thu Aug 17 02:17:53 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 17 Aug 2000 13:17:53 +1200 (NZST) Subject: [Python-Dev] PEP 214, extended print statement In-Reply-To: Message-ID: <200008170117.NAA15360@s454.cosc.canterbury.ac.nz> Looks reasonably good. Not entirely sure I like the look of >> though -- a bit too reminiscent of C++. How about print to myfile, x, y, z with 'to' as a non-reserved keyword. Or even print to myfile: x, y, z but that might be a bit too radical! Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From m.favas@per.dem.csiro.au Thu Aug 17 02:17:42 2000 From: m.favas@per.dem.csiro.au (Mark Favas) Date: Thu, 17 Aug 2000 09:17:42 +0800 Subject: [Python-Dev] [Fwd: segfault in sre on 64-bit plats] Message-ID: <399B3D36.6921271@per.dem.csiro.au> This is a multi-part message in MIME format. --------------53A0211FF1B8BFA142994B9A Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit --------------53A0211FF1B8BFA142994B9A Content-Type: message/rfc822 Content-Disposition: inline Message-ID: <399B3CAA.C8815E61@per.dem.csiro.au> Date: Thu, 17 Aug 2000 09:15:22 +0800 From: Mark Favas Organization: CSIRO Exploration & Mining X-Mailer: Mozilla 4.73 [en] (X11; U; OSF1 V4.0 alpha) X-Accept-Language: en MIME-Version: 1.0 To: Trent Mick Subject: Re: segfault in sre on 64-bit plats References: <20000815104723.A27306@ActiveState.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Trent Mick wrote: > > Fredrik, > > The sre module currently segfaults on one of the tests suite tests on both > Win64 and 64-bit linux: > > [trentm@nickel src]$ ./python -c "import sre; sre.match('(x)*', 50000*'x')" > srefail.out > Segmentation fault (core dumped) > > I know that I can't expect you to debug this completely, as you don't have to > hardware, but I was hoping you might be able to shed some light on the > subject for me. > > This test on Win32 and Linux32 hits the recursion limit check of 10000 in > SRE_MATCH(). However, on Linux64 the segfault occurs at a recursion depth of > 7500. I don't want to just willy-nilly drop the recursion limit down to make > the problem go away. > > Do you have any idea why the segfault may be occuring on 64-bit platforms? > > Mark (Favas), have you been having any problems with sre on your 64-bit plats? > Sorry for the delay - yes, I had these segfaults due to exceeding the stack size on Tru64 Unix (which, by default, is 2048 kbytes) before Fredrick introduced the recusrion limit of 10000 in _sre.c. You'd expect a 64-bit OS to use a bit more bytes of the stack when handling recursive calls, but your 7500 down from 10000 sounds a bit much - unless the stack size limit you're using on Linux64 is smaller than that for Linux32 - what are they? I certainly agree that it'd be better to solve this in a way other than selecting a sufficiently low recursion limit (I won't mention stackless here... .) - getrlimit(2), getrusage(2), or other??? -- Mark --------------53A0211FF1B8BFA142994B9A-- From greg@cosc.canterbury.ac.nz Thu Aug 17 02:26:59 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 17 Aug 2000 13:26:59 +1200 (NZST) Subject: [Python-Dev] autoconf question: howto add to CFLAGS and LDFLAGS? In-Reply-To: <20000816165542.D29260@ActiveState.com> Message-ID: <200008170126.NAA15363@s454.cosc.canterbury.ac.nz> > My hunch is that simply listing library (*.a) arguments on the gcc > command line is a GNU gcc/ld shortcut to the more portable usage of -L > and -l. I've never encountered a Unix that wouldn't let you explicity give .a files to cc or ld. It's certainly not a GNU invention. Sounds like Monterey is the odd one out here. ("Broken" is another word that comes to mind.) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From Vladimir.Marangozov@inrialpes.fr Thu Aug 17 02:41:48 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Thu, 17 Aug 2000 03:41:48 +0200 (CEST) Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: <20000816172425.A32338@ActiveState.com> from "Trent Mick" at Aug 16, 2000 05:24:25 PM Message-ID: <200008170141.DAA17229@python.inrialpes.fr> Trent Mick wrote: > > I am porting Python to Monterey (64-bit AIX) and have a small (hopefully) > question about POSIX threads. I have Monterey building and passing the > threads test suite using Python/thread_pthread.h with just one issue: > > -------------- snipped from current thread_pthread.h --------------- > long > PyThread_get_thread_ident(void) > { > volatile pthread_t threadid; > if (!initialized) > PyThread_init_thread(); > /* Jump through some hoops for Alpha OSF/1 */ > threadid = pthread_self(); > return (long) *(long *) &threadid; > } > ------------------------------------------------------------------- > > ... > > WHAT IS UP WITH THAT return STATEMENT? > return (long) *(long *) &threadid; I don't know and I had the same question at the time when there was some obscure bug on my AIX combo at this location. I remember that I had played with the debugger and the only workaround at the time which solved the mystery was to add the 'volatile' qualifier. So if you're asking yourself what that 'volatile' is for, you have one question less... > > My *guess* is that this is an attempt to just cast 'threadid' (a pthread_t) > to a long and go through hoops to avoid compiler warnings. I dont' know what > else it could be. Is that what the "Alpha OSF/1" comment is about? Anybody > have an Alpha OSF/1 hanging around. The problem is that when > sizeof(pthread_t) != sizeof(long) this line is just broken. > > Could this be changed to > return threadid; > safely? I have the same question. If Guido can't answer this straight, we need to dig the CVS logs. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From Vladimir.Marangozov@inrialpes.fr Thu Aug 17 02:43:33 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Thu, 17 Aug 2000 03:43:33 +0200 (CEST) Subject: [Python-Dev] autoconf question: howto add to CFLAGS and LDFLAGS? In-Reply-To: <20000816165542.D29260@ActiveState.com> from "Trent Mick" at Aug 16, 2000 04:55:42 PM Message-ID: <200008170143.DAA17238@python.inrialpes.fr> Trent Mick wrote: > > Hello autoconf-masters, > > I am currently trying to port Python to Monterey (64-bit AIX) and I need to > add a couple of Monterey specific options to CFLAGS and LDFLAGS (or to > whatever appropriate variables for all 'cc' and 'ld' invocations) but it is > not obvious *at all* how to do that in configure.in. Can anybody helpme on > that? How can we help? What do want to do, exactly? -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From fdrake@beopen.com Thu Aug 17 02:40:32 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Wed, 16 Aug 2000 21:40:32 -0400 (EDT) Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: <200008170141.DAA17229@python.inrialpes.fr> References: <20000816172425.A32338@ActiveState.com> <200008170141.DAA17229@python.inrialpes.fr> Message-ID: <14747.17040.968927.914435@cj42289-a.reston1.va.home.com> Vladimir Marangozov writes: > I have the same question. If Guido can't answer this straight, we need > to dig the CVS logs. Guido is out of town right now, and doesn't have his usual email tools with him, so he may not respond this week. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From Vladimir.Marangozov@inrialpes.fr Thu Aug 17 03:12:18 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Thu, 17 Aug 2000 04:12:18 +0200 (CEST) Subject: [Python-Dev] shm + Win32 + docs (was: Adding library modules to the core) In-Reply-To: <20000808114655.C29686@thyrsus.com> from "Eric S. Raymond" at Aug 08, 2000 11:46:55 AM Message-ID: <200008170212.EAA17523@python.inrialpes.fr> Eric S. Raymond wrote: > > Vladimir, I suggest that the most useful thing you could do to advance > the process at this point would be to document shm in core-library style. Eric, I'm presently suffering from chronic lack of time (as you probably do too) so if you could write the docs for me and take all associated credits for them, please do so (shouldn't be that hard, after all -- the web page and the comments are self-explanatory :-). I'm willing to "unblock" you on this, but I can hardly make the time for it -- it's low-priority on my dynamic task schedule. :( I'd also love to assemble the win32 bits on the matter (what's in win32event for the semaphore interface + my Windows book) to add shm and sem for Windows and rewrite the interface, but I have no idea on when this could happen. I will disappear from the face of the World sometime soon and it's unclear on when I'll be able to reappear (nor how soon I'll disappear, btw) So, be aware of that. I hope to be back again before 2.1 so if we can wrap up a Unix + win32 shm, that would be much appreciated! > > At the moment, core Python has nothing (with the weak and nonportable > exception of open(..., O_EXCL)) that can do semaphores properly. Thus > shm would address a real gap in the language. Indeed. This is currently being discussed on the french Python list, where Richard Gruet (rgruet@ina.fr) posted the following code for inter-process locks: glock.py I don't have the time to look at it in detail, just relaying here for food and meditation :-) -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 ------------------------------[ glock.py ]---------------------------- #!/usr/bin/env python #---------------------------------------------------------------------------- # glock.py: Global mutex # # Prerequisites: # - Python 1.5.2 or newer (www.python.org) # - On windows: win32 extensions installed # (http://www.python.org/windows/win32all/win32all.exe) # - OS: Unix, Windows. # # History: # -22 Jan 2000 (R.Gruet): creation # # Limitations: # TODO: #---------------------------------------------------------------------------- ''' This module defines the class GlobalLock that implements a global (inter-process) mutex that works on Windows and Unix, using file-locking on Unix (I also tried this approach on Windows but got some tricky problems so I ended using Win32 Mutex).. See class GlobalLock for more details. ''' __version__ = 1,0,2 __author__ = ('Richard Gruet', 'rgruet@ina.fr') # Imports: import sys, string, os # System-dependent imports for locking implementation: _windows = (sys.platform == 'win32') if _windows: try: import win32event, win32api, pywintypes except ImportError: sys.stderr.write('The win32 extensions need to be installed!') else: # assume Unix try: import fcntl except ImportError: sys.stderr.write("On what kind of OS am I ? (Mac?) I should be on " "Unix but can't import fcntl.\n") raise import threading # Exceptions : # ---------- class GlobalLockError(Exception): ''' Error raised by the glock module. ''' pass class NotOwner(GlobalLockError): ''' Attempt to release somebody else's lock. ''' pass # Constants # ---------: true=-1 false=0 #---------------------------------------------------------------------------- class GlobalLock: #---------------------------------------------------------------------------- ''' A global mutex. *Specification: ------------- The lock must act as a global mutex, ie block between different candidate processus, but ALSO between different candidate threads of the same process. It must NOT block in case of recursive lock request issued by the SAME thread. Extraneous unlocks should be ideally harmless. *Implementation: -------------- In Python there is no portable global lock AFAIK. There is only a LOCAL/ in-process Lock mechanism (threading.RLock), so we have to implement our own solution. Unix: use fcntl.flock(). Recursive calls OK. Different process OK. But <> threads, same process don't block so we have to use an extra threading.RLock to fix that point. Win: We use WIN32 mutex from Python Win32 extensions. Can't use std module msvcrt.locking(), because global lock is OK, but blocks also for 2 calls from the same thread! ''' def __init__(self, fpath, lockInitially=false): ''' Creates (or opens) a global lock. @param fpath Path of the file used as lock target. This is also the global id of the lock. The file will be created if non existent. @param lockInitially if true locks initially. ''' if _windows: self.name = string.replace(fpath, '\\', '_') self.mutex = win32event.CreateMutex(None, lockInitially, self.name) else: # Unix self.name = fpath self.flock = open(fpath, 'w') self.fdlock = self.flock.fileno() self.threadLock = threading.RLock() if lockInitially: self.acquire() def __del__(self): #print '__del__ called' ## try: self.release() except: pass if _windows: win32api.CloseHandle(self.mutex) else: try: self.flock.close() except: pass def __repr__(self): return ' ' % self.name def acquire(self): ''' Locks. Suspends caller until done. On windows an IOError is raised after ~10 sec if the lock can't be acquired. @exception GlobalLockError if lock can't be acquired (timeout) ''' if _windows: r = win32event.WaitForSingleObject(self.mutex, win32event.INFINITE) if r == win32event.WAIT_FAILED: raise GlobalLockError("Can't acquire mutex.") else: # Acquire 1st the global (inter-process) lock: try: fcntl.flock(self.fdlock, fcntl.LOCK_EX) # blocking except IOError: #(errno 13: perm. denied, # 36: Resource deadlock avoided) raise GlobalLockError('Cannot acquire lock on "file" %s\n' % self.name) #print 'got file lock.' ## # Then acquire the local (inter-thread) lock: self.threadLock.acquire() #print 'got thread lock.' ## def release(self): ''' Unlocks. (caller must own the lock!) @return The lock count. @exception IOError if file lock can't be released @exception NotOwner Attempt to release somebody else's lock. ''' if _windows: try: win32event.ReleaseMutex(self.mutex) except pywintypes.error, e: errCode, fctName, errMsg = e.args if errCode == 288: raise NotOwner("Attempt to release somebody else's lock") else: raise GlobalLockError('%s: err#%d: %s' % (fctName, errCode, errMsg)) else: # Acquire 1st the local (inter-thread) lock: try: self.threadLock.release() except AssertionError: raise NotOwner("Attempt to release somebody else's lock") # Then release the global (inter-process) lock: try: fcntl.flock(self.fdlock, fcntl.LOCK_UN) except IOError: # (errno 13: permission denied) raise GlobalLockError('Unlock of file "%s" failed\n' % self.name) #---------------------------------------------------------------------------- # M A I N #---------------------------------------------------------------------------- def main(): # unfortunately can't test inter-process lock here! lockName = 'myFirstLock' l = GlobalLock(lockName) if not _windows: assert os.path.exists(lockName) l.acquire() l.acquire() # rentrant lock, must not block l.release() l.release() if _windows: try: l.release() except NotOwner: pass else: raise Exception('should have raised a NotOwner exception') # Check that <> threads of same process do block: import threading, time thread = threading.Thread(target=threadMain, args=(l,)) print 'main: locking...', l.acquire() print ' done.' thread.start() time.sleep(3) print '\nmain: unlocking...', l.release() print ' done.' time.sleep(0.1) del l # to close file print 'tests OK.' def threadMain(lock): print 'thread started(%s).' % lock print 'thread: locking (should stay blocked for ~ 3 sec)...', lock.acquire() print 'thread: locking done.' print 'thread: unlocking...', lock.release() print ' done.' print 'thread ended.' if __name__ == "__main__": main() From bwarsaw@beopen.com Thu Aug 17 04:17:23 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Wed, 16 Aug 2000 23:17:23 -0400 (EDT) Subject: [Python-Dev] PEP 214, extended print statement References: <200008170117.NAA15360@s454.cosc.canterbury.ac.nz> Message-ID: <14747.22851.266303.28877@anthem.concentric.net> >>>>> "GE" == Greg Ewing writes: GE> Looks reasonably good. Not entirely sure I like the look GE> of >> though -- a bit too reminiscent of C++. GE> How about GE> print to myfile, x, y, z Not bad at all. Seems quite Pythonic to me. GE> with 'to' as a non-reserved keyword. Or even GE> print to myfile: x, y, z GE> but that might be a bit too radical! Definitely so. -Barry From bwarsaw@beopen.com Thu Aug 17 04:19:25 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Wed, 16 Aug 2000 23:19:25 -0400 (EDT) Subject: [Python-Dev] autoconf question: howto add to CFLAGS and LDFLAGS? References: <20000816165542.D29260@ActiveState.com> <200008170126.NAA15363@s454.cosc.canterbury.ac.nz> Message-ID: <14747.22973.502494.739270@anthem.concentric.net> >>>>> "GE" == Greg Ewing writes: >> My hunch is that simply listing library (*.a) arguments on the >> gcc command line is a GNU gcc/ld shortcut to the more portable >> usage of -L and -l. GE> I've never encountered a Unix that wouldn't let you explicity GE> give .a files to cc or ld. It's certainly not a GNU invention. That certainly jives with my experience. All the other non-gcc C compilers I've used (admittedly only on *nix) have always accepted explicit .a files on the command line. -Barry From MarkH@ActiveState.com Thu Aug 17 04:32:25 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Thu, 17 Aug 2000 13:32:25 +1000 Subject: [Python-Dev] os.path.commonprefix breakage Message-ID: Hi, I believe that Skip recently made a patch to os.path.commonprefix to only return the portion of the common prefix that corresponds to a directory. I have just dicovered some code breakage from this change. On 1.5.2, the behaviour was: >>> os.path.commonprefix(["../foo/bar", "../foo/spam"]) '../foo/' While since the change we have: '../foo' Note that the trailing slash has been dropped. The code this broke did similar to: prefix = os.path.commonprefix(files) for file in files: tail_portion = file[len(prefix):] In 1.6, the "tail_portion" result looks like an absolute path "/bar" and "/spam", respectively. The intent was obviously to get absolute path names back ("bar" and "spam") The code that broke is not mine, so you can safely be horrified at how broken it is :-) The point, however, is that code like this does exist out there. I'm obviously going to change the code that broke, and don't have time to look into the posixpath.py code - but is this level of possible breakage acceptable? Thanks, Mark. From tim_one@email.msn.com Thu Aug 17 04:34:12 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 16 Aug 2000 23:34:12 -0400 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: <20000816172425.A32338@ActiveState.com> Message-ID: [Trent Mick] > I am porting Python to Monterey (64-bit AIX) and have a small > (hopefully) question about POSIX threads. POSIX threads. "small question". HAHAHAHAHAHA. Thanks, that felt good . > I have Monterey building and passing the threads test suite using > Python/thread_pthread.h with just one issue: > > > -------------- snipped from current thread_pthread.h --------------- > long > PyThread_get_thread_ident(void) > { > volatile pthread_t threadid; > if (!initialized) > PyThread_init_thread(); > /* Jump through some hoops for Alpha OSF/1 */ > threadid = pthread_self(); > return (long) *(long *) &threadid; > } > ------------------------------------------------------------------- > > Does the POSIX threads spec specify a C type or minimum size for > pthread_t? Which POSIX threads spec? There are so very many (it went thru many incompatible changes). But, to answer your question, I don't know but doubt it. In practice, some implementations return pointers into kernel space, others pointers into user space, others small integer indices into kernel- or user-space arrays of structs. So I think it's *safe* to assume it will always fit in an integral type large enough to hold a pointer, but not guaranteed. Plain "long" certainly isn't safe in theory. > Or can someone point me to the appropriate resource to look > this up. On Linux (mine at least): > /usr/include/bits/pthreadtypes.h:120:typedef unsigned long int > pthread_t; And this is a 32- or 64-bit Linux? > On Monterey: > typedef unsigned int pthread_t; > > That is fine, they are both 32-bits, however Monterey is an LP64 platform > (sizeof(long)==8, sizeof(int)=4), which brings up the question: > > WHAT IS UP WITH THAT return STATEMENT? > return (long) *(long *) &threadid; Heh heh. Thanks for the excuse! I contributed the pthreads implementation originally, and that eyesore sure as hell wasn't in it when I passed it on. That's easy for me to be sure of, because that entire function was added by somebody after me . I've been meaning to track down where that crap line came from for *years*, but never had a good reason before. So, here's the scoop: + The function was added in revision 2.3, more than 6 years ago. At that time, the return had a direct cast to long. + The "Alpha OSF/1" horror was the sole change made to get revision 2.5. Back in those days, the "patches list" was Guido's mailbox, and *all* CVS commits were done by him. So he checked in everything everyone could convince them they needed, and sometimes without knowing exactly why. So I strongly doubt he'll even remember this change, and am certain it's not his code. > My *guess* is that this is an attempt to just cast 'threadid' (a > pthread_t) to a long and go through hoops to avoid compiler warnings. I > dont' know what else it could be. Me neither. > Is that what the "Alpha OSF/1" comment is about? That comment was introduced by the commit that added the convoluted casting, so yes, that's what the comment is talking about. > Anybody have an Alpha OSF/1 hanging around. The problem is that when > sizeof(pthread_t) != sizeof(long) this line is just broken. > > Could this be changed to > return threadid; > safely? Well, that would return it to exactly the state it was in at revision 2.3, except with the cast to long left implicit. Apparently that "didn't work"! Something else is broken here, too, and has been forever: the thread docs claim that thread.get_ident() returns "a nonzero integer". But across all the thread implementations, there's nothing that guarantees that! It's a goof, based on the first thread implementation in which it just happened to be true for that platform. So thread.get_ident() is plain braindead: if Python wants to return a unique non-zero long across platforms, the current code doesn't guarantee any of that. So one of two things can be done: 1. Bite the bullet and do it correctly. For example, maintain a static dict mapping the native pthread_self() return value to Python ints, and return the latter as Python's thread.get_ident() value. Much better would to implement a x-platform thread-local storage abstraction, and use that to hold a Python-int ident value. 2. Continue in the tradition already established , and #ifdef the snot out of it for Monterey. In favor of #2, the code is already so hosed that making it hosier won't be a significant relative increase in its inherent hosiness. spoken-like-a-true-hoser-ly y'rs - tim From tim_one@email.msn.com Thu Aug 17 04:47:04 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 16 Aug 2000 23:47:04 -0400 Subject: [Python-Dev] PEP 214, extended print statement In-Reply-To: <14747.22851.266303.28877@anthem.concentric.net> Message-ID: [Greg Ewing] > Looks reasonably good. Not entirely sure I like the look > of >> though -- a bit too reminiscent of C++. > > How about > > print to myfile, x, y, z [Barry Warsaw] > Not bad at all. Seems quite Pythonic to me. Me too! +1 on changing ">>" to "to" here. Then we can introduce x = print from myfile, 3 as a synonym for x = myfile.read(3) too . People should know that Guido doesn't seem to like the idea of letting print specify the output target at all. "Why not?" "Because people say print is pretty useless anyway, for example, when they want to write to something other than stdout." "But that's the whole point of this change! To make print more useful!" "Well, but then ...". After years of channeling, you get a feel for when to change the subject and bring it up again later as if it were brand new . half-of-channeling-is-devious-persuasion-ly y'rs - tim From skip@mojam.com (Skip Montanaro) Thu Aug 17 05:04:54 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 16 Aug 2000 23:04:54 -0500 (CDT) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: References: Message-ID: <14747.25702.435148.549678@beluga.mojam.com> >>>>> "Mark" == Mark Hammond writes: Mark> I believe that Skip recently made a patch to os.path.commonprefix Mark> to only return the portion of the common prefix that corresponds Mark> to a directory. Mark> I have just dicovered some code breakage from this change. On Mark> 1.5.2, the behaviour was: >>>> os.path.commonprefix(["../foo/bar", "../foo/spam"]) Mark> '../foo/' Mark> While since the change we have: Mark> '../foo' I'm sure it can be argued that the slash should be there. The previous behavior was clearly broken, however, because it was advancing character-by-character instead of directory-by-directory. Consequently, calling os.path.commonprefix(["/home/swen", "/home/swenson"]) would yield the most likely invalid path "/home/sw" as the common prefix. It would be easy enough to append the appropriate path separator to the the result before returning. I have no problem with that. Others with more knowledge of path semantics should chime in. Also, should the behavior be consistent across platforms or should it do what is correct for each platform on which it's implemented (dospath, ntpath, macpath)? Skip From tim_one@email.msn.com Thu Aug 17 05:05:12 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 17 Aug 2000 00:05:12 -0400 Subject: [Python-Dev] os.path.commonprefix breakage In-Reply-To: Message-ID: I agree this is Bad Damage, and should be fixed before 2.0b1 goes out. Can you enter a bug report? > -----Original Message----- > From: python-dev-admin@python.org [mailto:python-dev-admin@python.org]On > Behalf Of Mark Hammond > Sent: Wednesday, August 16, 2000 11:32 PM > To: python-dev@python.org > Subject: [Python-Dev] os.path.commonprefix breakage > > > Hi, > I believe that Skip recently made a patch to > os.path.commonprefix to only > return the portion of the common prefix that corresponds to a directory. > > I have just dicovered some code breakage from this change. On 1.5.2, the > behaviour was: > > >>> os.path.commonprefix(["../foo/bar", "../foo/spam"]) > '../foo/' > > While since the change we have: > '../foo' > > Note that the trailing slash has been dropped. > > The code this broke did similar to: > > prefix = os.path.commonprefix(files) > for file in files: > tail_portion = file[len(prefix):] > > In 1.6, the "tail_portion" result looks like an absolute path "/bar" and > "/spam", respectively. The intent was obviously to get absolute > path names > back ("bar" and "spam") > > The code that broke is not mine, so you can safely be horrified at how > broken it is :-) The point, however, is that code like this does > exist out > there. > > I'm obviously going to change the code that broke, and don't have time to > look into the posixpath.py code - but is this level of possible breakage > acceptable? > > Thanks, > > Mark. From greg@cosc.canterbury.ac.nz Thu Aug 17 05:11:51 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 17 Aug 2000 16:11:51 +1200 (NZST) Subject: [Python-Dev] PEP 214, extended print statement In-Reply-To: Message-ID: <200008170411.QAA15381@s454.cosc.canterbury.ac.nz> tim_one: > +1 on changing ">>" to "to" here. Your +1 might be a bit hasty. I've just realised that a non-reserved word in that position would be ambiguous, as can be seen by considering print to(myfile), x, y, z > Then we can introduce > > x = print from myfile, 3 Actually, for the sake of symmetry, I was going to suggest input from myfile, x, y ,z except that the word 'input' is already taken. Bummer. But wait a moment, we could have from myfile input x, y, z Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From fdrake@beopen.com Thu Aug 17 05:11:44 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Thu, 17 Aug 2000 00:11:44 -0400 (EDT) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <14747.25702.435148.549678@beluga.mojam.com> References: <14747.25702.435148.549678@beluga.mojam.com> Message-ID: <14747.26112.609255.338170@cj42289-a.reston1.va.home.com> Skip Montanaro writes: > I'm sure it can be argued that the slash should be there. The previous > behavior was clearly broken, however, because it was advancing > character-by-character instead of directory-by-directory. Consequently, > calling > > os.path.commonprefix(["/home/swen", "/home/swenson"]) > > would yield the most likely invalid path "/home/sw" as the common prefix. You have a typo in there... ;) > It would be easy enough to append the appropriate path separator to the the > result before returning. I have no problem with that. Others with more > knowledge of path semantics should chime in. Also, should the behavior be I'd guess that the path separator should only be appended if it's part of the passed-in strings; that would make it a legitimate part of the prefix. If it isn't present for all of them, it shouldn't be part of the result: >>> os.path.commonprefix(["foo", "foo/bar"]) 'foo' -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From skip@mojam.com (Skip Montanaro) Thu Aug 17 05:23:37 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 16 Aug 2000 23:23:37 -0500 (CDT) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <14747.25702.435148.549678@beluga.mojam.com> References: <14747.25702.435148.549678@beluga.mojam.com> Message-ID: <14747.26825.977663.599413@beluga.mojam.com> Skip> os.path.commonprefix(["/home/swen", "/home/swenson"]) Skip> would yield the most likely invalid path "/home/sw" as the common Skip> prefix. Ack! I meant to use this example: os.path.commonprefix(["/home/swen", "/home/swanson"]) which would yield "/home/sw"... S From m.favas@per.dem.csiro.au Thu Aug 17 05:27:20 2000 From: m.favas@per.dem.csiro.au (Mark Favas) Date: Thu, 17 Aug 2000 12:27:20 +0800 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements Message-ID: <399B69A8.4A94337C@per.dem.csiro.au> [Trent} -------------- snipped from current thread_pthread.h --------------- long PyThread_get_thread_ident(void) { volatile pthread_t threadid; if (!initialized) PyThread_init_thread(); /* Jump through some hoops for Alpha OSF/1 */ threadid = pthread_self(); return (long) *(long *) &threadid; } ------------------------------------------------------------------- WHAT IS UP WITH THAT return STATEMENT? return (long) *(long *) &threadid; My *guess* is that this is an attempt to just cast 'threadid' (a pthread_t) to a long and go through hoops to avoid compiler warnings. I dont' know what else it could be. Is that what the "Alpha OSF/1" comment is about? Anybody have an Alpha OSF/1 hanging around. The problem is that when sizeof(pthread_t) != sizeof(long) this line is just broken. Could this be changed to return threadid; safely? This is a DEC-threads thing... (and I'm not a DEC-threads savant). Making the suggested change gives the compiler warning: cc -O -Olimit 1500 -I./../Include -I.. -DHAVE_CONFIG_H -c thread.c -o thread.o cc: Warning: thread_pthread.h, line 182: In this statement, "threadid" of type " pointer to struct __pthread_t", is being converted to "long". (cvtdiftypes) return threadid; ---------------^ The threads test still passes with this change. From the pthread.h file, typedef struct __pthreadTeb_t { __pthreadLongAddr_p _Pfield(reserved1); /* Reserved to DECthreads */ __pthreadLongAddr_p _Pfield(reserved2); /* Reserved to DECthreads */ unsigned short _Pfield(size); /* V1: Size of TEB */ unsigned char _Pfield(version); /* TEB version */ unsigned char _Pfield(reserved3); /* Reserved to DECthreads */ unsigned char _Pfield(external); /* V1: PTHREAD_TEB_EFLG_ flgs */ unsigned char _Pfield(reserved4)[2]; /* RESERVED */ unsigned char _Pfield(creator); /* V1: PTHREAD_TEB_CREATOR_* */ __pthreadLongUint_t _Pfield(sequence); /* V0: Thread sequence */ __pthreadLongUint_t _Pfield(reserved5)[2]; /* Reserved to DECthreads */ __pthreadLongAddr_t _Pfield(per_kt_area); /* V0: Reserved */ __pthreadLongAddr_t _Pfield(stack_base); /* V0: Initial SP */ __pthreadLongAddr_t _Pfield(stack_reserve); /* V0: reserved stack */ __pthreadLongAddr_t _Pfield(stack_yellow); /* V0: yellow zone */ __pthreadLongAddr_t _Pfield(stack_guard); /* V0: guard (red) zone */ __pthreadLongUint_t _Pfield(stack_size); /* V0: total stack size */ __pthreadTsd_t _Pfield(tsd_values); /* V0: TSD array (void *) */ unsigned long _Pfield(tsd_count); /* V0: TSD array size */ unsigned int _Pfield(reserved6); /* Reserved to DECthreads */ unsigned int _Pfield(reserved7); /* Reserved to DECthreads */ unsigned int _Pfield(thread_flags); /* Reserved to DECthreads */ int _Pfield(thd_errno); /* V1: thread's errno */ __pthreadLongAddr_t _Pfield(stack_hiwater); /* V1: lowest known SP */ } pthreadTeb_t, *pthreadTeb_p; # if defined (_PTHREAD_ALLOW_MIXED_PROTOS_) && defined (__INITIAL_POINTER_SIZE) typedef pthreadTeb_p pthread_t; /* Long pointer if possible */ # pragma __required_pointer_size __restore # elif defined (_PTHREAD_ENV_ALPHA) && defined (_PTHREAD_ENV_VMS) typedef unsigned __int64 pthread_t; /* Force 64 bits anyway */ # else typedef pthreadTeb_p pthread_t; /* Pointers is pointers */ # endif #endif -- Mark From greg@cosc.canterbury.ac.nz Thu Aug 17 05:28:19 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 17 Aug 2000 16:28:19 +1200 (NZST) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <14747.25702.435148.549678@beluga.mojam.com> Message-ID: <200008170428.QAA15385@s454.cosc.canterbury.ac.nz> Skip: > Also, should the behavior be > consistent across platforms or should it do what is correct for each > platform on which it's implemented (dospath, ntpath, macpath)? Obviously it should do what's correct for each platform, although more than one thing can be correct for a given platform -- e.g Unix doesn't care whether there's a trailing slash on a pathname. In the Unix case it's probably less surprising if the trailing slash is removed, because it's redundant. The "broken" code referred to in the original message highlights another problem, however: there is no platform-independent way provided to remove a prefix from a pathname, given the prefix as returned by one of the other platform-independent path munging functions. So maybe there should be an os.path.removeprefix(prefix, path) function. While we're on the subject, another thing that's missing is a platform-independent way of dealing with the notion of "up one directory". Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Thu Aug 17 05:34:01 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 17 Aug 2000 16:34:01 +1200 (NZST) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <14747.25702.435148.549678@beluga.mojam.com> Message-ID: <200008170434.QAA15389@s454.cosc.canterbury.ac.nz> Skip: > The previous behavior was clearly broken, however, because it was > advancing character-by-character instead of directory-by-directory. I've just looked at the 1.5.2 docs and realised that this is what it *says* it does! So it's right according to the docs, although it's obviously useless as a pathname manipulating function. The question now is, do we change both the specification and the behaviour, which could break existing code, or leave it be and add a new function which does the right thing? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From skip@mojam.com (Skip Montanaro) Thu Aug 17 05:41:59 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 16 Aug 2000 23:41:59 -0500 (CDT) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <14747.26112.609255.338170@cj42289-a.reston1.va.home.com> References: <14747.25702.435148.549678@beluga.mojam.com> <14747.26112.609255.338170@cj42289-a.reston1.va.home.com> Message-ID: <14747.27927.170223.873328@beluga.mojam.com> Fred> I'd guess that the path separator should only be appended if it's Fred> part of the passed-in strings; that would make it a legitimate Fred> part of the prefix. If it isn't present for all of them, it Fred> shouldn't be part of the result: >>> os.path.commonprefix(["foo", "foo/bar"]) 'foo' Hmmm... I think you're looking at it character-by-character again. I see three possibilities: * it's invalid to have a path with a trailing separator * it's okay to have a path with a trailing separator * it's required to have a path with a trailing separator In the first and third cases, you have no choice. In the second you have to decide which would be best. On Unix my preference would be to not include the trailing "/" for aesthetic reasons. The shell's pwd command, the os.getcwd function and the os.path.normpath function all return directories without the trailing slash. Also, while Python may not have this problem (and os.path.join seems to normalize things), some external tools will interpret doubled "/" characters as single characters while others (most notably Emacs), will treat the second slash as "erase the prefix and start from /". In fact, the more I think of it, the more I think that Mark's reliance on the trailing slash is a bug waiting to happen (in fact, it just happened ;-). There's certainly nothing wrong (on Unix anyway) with paths that don't contain a trailing slash, so if you're going to join paths together, you ought to be using os.path.join. To whack off prefixes, perhaps we need something more general than os.path.split, so instead of prefix = os.path.commonprefix(files) for file in files: tail_portion = file[len(prefix):] Mark would have used prefix = os.path.commonprefix(files) for file in files: tail_portion = os.path.splitprefix(prefix, file)[1] The assumption being that os.path.splitprefix("/home", "/home/beluga/skip") would return ["/home", "beluga/skip"] Alternatively, how about os.path.suffixes? It would work similar to os.path.commonprefix, but instead of returning the prefix of a group of files, return a list of the suffixes resulting in the application of the common prefix: >>> files = ["/home/swen", "/home/swanson", "/home/jules"] >>> prefix = os.path.commonprefix(files) >>> print prefix "/home" >>> suffixes = os.path.suffixes(prefix, files) >>> print suffixes ["swen", "swanson", "jules"] Skip From fdrake@beopen.com Thu Aug 17 05:49:24 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Thu, 17 Aug 2000 00:49:24 -0400 (EDT) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <200008170434.QAA15389@s454.cosc.canterbury.ac.nz> References: <14747.25702.435148.549678@beluga.mojam.com> <200008170434.QAA15389@s454.cosc.canterbury.ac.nz> Message-ID: <14747.28372.771170.783868@cj42289-a.reston1.va.home.com> Greg Ewing writes: > I've just looked at the 1.5.2 docs and realised that this is > what it *says* it does! So it's right according to the docs, > although it's obviously useless as a pathname manipulating > function. I think we should now fix the docs; Skip's right about the desired functionality. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From greg@cosc.canterbury.ac.nz Thu Aug 17 05:53:05 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Thu, 17 Aug 2000 16:53:05 +1200 (NZST) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <14747.27927.170223.873328@beluga.mojam.com> Message-ID: <200008170453.QAA15394@s454.cosc.canterbury.ac.nz> Skip: > Alternatively, how about os.path.suffixes? It would work similar to > os.path.commonprefix, but instead of returning the prefix of a group of > files, return a list of the suffixes resulting in the application of the > common prefix: To avoid duplication of effort, how about a single function that does both: >>> files = ["/home/swen", "/home/swanson", "/home/jules"] >>> os.path.factorize(files) ("/home", ["swen", "swanson", "jules"]) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From nowonder@nowonder.de Thu Aug 17 08:13:08 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Thu, 17 Aug 2000 07:13:08 +0000 Subject: [Python-Dev] timeout support for socket.py? (was: [ANN] TCP socket timeout module --> timeoutsocket.py) References: <300720002054234840%timo@alum.mit.edu> Message-ID: <399B9084.C068DCE3@nowonder.de> As the socketmodule is now exported as _socket and a seperate socket.py file wrapping _socket is now available in the standard library, wouldn't it be possible to include timeout capabilities like in http://www.timo-tasi.org/python/timeoutsocket.py If the default behaviour would be "no timeout", I would think this would not break any code. But it would give an easy(and documentable) solution) to people who e.g. have their urllib.urlopen("http://spam.org").read() hang on them. (Actually the approach should work for all streaming socket connections, as far as I understand it.) Are there any efficiency concerns? If so, would it be possible to include a second socket class timeoutsocket in socket.py, so that this could be used instead of the normal socket class? [In this case a different default timeout than "None" could be chosen.] Peter P.S.: For your convenience a quote of the announcement on c.l.py, for module documentation (== endlessly long doc string) look in http://www.timo-tasi.org/python/timeoutsocket.py Timothy O'Malley wrote: > > Numerous times I have seen people request a solution for TCP socket > timeouts in conjunction with urllib. Recently, I found myself in the > same boat. I wrote a server that would connect to skytel.com and send > automated pages. Periodically, the send thread in the server would > hang for a long(!) time. Yup -- I was bit by a two hour timeout built > into tcp sockets. > > Thus, I wrote timeoutsocket.py > > With timeoutsocket.py, you can force *all* TCP sockets to have a > timeout. And, this is all accomplished without interfering with the > standard python library! > > Here's how to put a 20 second timeout on all TCP sockets for urllib: > > import timeoutsock > import urllib > timeoutsocket.setDefaultSocketTimeout(20) > > Just like that, any TCP connection made by urllib will have a 20 second > timeout. If a connect(), read(), or write() blocks for more than 20 > seconds, then a socket.Timeout error will be raised. > > Want to see how to use this in ftplib? > > import ftplib > import timeoutsocket > timeoutsocket.setDefaultSocketTimeout(20) > > Wasn't that easy! > The timeoutsocket.py module acts as a shim on top of the standard > socket module. Whenever a TCP socket is requested, an instance of > TimeoutSocket is returned. This wrapper class adds timeout support to > the standard TCP socket. > > Where can you get this marvel of modern engineering? > > http://www.timo-tasi.org/python/timeoutsocket.py > > And it will very soon be found on the Vaults of Parnassus. > > Good Luck! > > -- > -- > happy daze > -tim O > -- > http://www.python.org/mailman/listinfo/python-list -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From Moshe Zadka Thu Aug 17 07:16:29 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 17 Aug 2000 09:16:29 +0300 (IDT) Subject: [Python-Dev] PEP 214, extended print statement In-Reply-To: <14747.22851.266303.28877@anthem.concentric.net> Message-ID: On Wed, 16 Aug 2000, Barry A. Warsaw wrote: > > >>>>> "GE" == Greg Ewing writes: > > GE> Looks reasonably good. Not entirely sure I like the look > GE> of >> though -- a bit too reminiscent of C++. > > GE> How about > > GE> print to myfile, x, y, z > > Not bad at all. Seems quite Pythonic to me. Ummmmm...... print to myfile (print a newline on myfile) print to, myfile (print to+" "+myfile to stdout) Perl has similar syntax, and I always found it horrible. -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From thomas@xs4all.net Thu Aug 17 07:30:23 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 17 Aug 2000 08:30:23 +0200 Subject: [Python-Dev] PEP 214, extended print statement In-Reply-To: ; from moshez@math.huji.ac.il on Thu, Aug 17, 2000 at 09:16:29AM +0300 References: <14747.22851.266303.28877@anthem.concentric.net> Message-ID: <20000817083023.J376@xs4all.nl> On Thu, Aug 17, 2000 at 09:16:29AM +0300, Moshe Zadka wrote: > On Wed, 16 Aug 2000, Barry A. Warsaw wrote: > > >>>>> "GE" == Greg Ewing writes: > > GE> How about > > GE> print to myfile, x, y, z > > Not bad at all. Seems quite Pythonic to me. > print to myfile (print a newline on myfile) > print to, myfile (print to+" "+myfile to stdout) > Perl has similar syntax, and I always found it horrible. Agreed. It might be technically unambiguous, but I think it's too hard for a *human* to parse this correctly. The '>>' version might seem more C++ish and less pythonic, but it also stands out a lot more. The 'print from' statement could easily (and more consistently, IMHO ;) be written as 'print <<' (not that I like the 'print from' idea, though.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From tim_one@email.msn.com Thu Aug 17 07:41:29 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 17 Aug 2000 02:41:29 -0400 Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <14747.28372.771170.783868@cj42289-a.reston1.va.home.com> Message-ID: [Greg Ewing] > I've just looked at the 1.5.2 docs and realised that this is > what it *says* it does! So it's right according to the docs, > although it's obviously useless as a pathname manipulating > function. [Fred Drake] > I think we should now fix the docs; Skip's right about the desired > functionality. Oddly enough, I don't: commonprefix worked exactly as documented for at least 6 years and 5 months (which is when CVS shows Guido checking in ntpath.py with the character-based functionality), and got out of synch with the docs about 5 weeks ago when Skip changed to this other algorithm. Since the docs *did* match the code, there's no reason to believe the original author was confused, and no reason to believe users aren't relying on it (they've had over 6 years to gripe ). I think it's wrong to change what released code or docs do or say in non-trivial ways when they weren't ever in conflict. We have no idea who may be relying on the old behavior! Bitch all you like about MarkH's test case, it used to work, it doesn't now, and that sucks for the user. I appreciate that some other behavior may be more useful more often, but if you can ever agree on what that is across platforms, it should be spelled via a new function name ("commonpathprefix" comes to mind), or optional flag (defaulting to "old behavior") on commonprefix (yuck!). BTW, the presence or absence of a trailing path separator makes a *big* difference to many cmds on Windows, and you can't tell me nobody isn't currently doing e.g. commonprefix(["blah.o", "blah", "blah.cpp"]) on Unix either. From thomas@xs4all.net Thu Aug 17 07:55:41 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 17 Aug 2000 08:55:41 +0200 Subject: [Python-Dev] autoconf question: howto add to CFLAGS and LDFLAGS? In-Reply-To: <20000816165542.D29260@ActiveState.com>; from trentm@ActiveState.com on Wed, Aug 16, 2000 at 04:55:42PM -0700 References: <20000816165542.D29260@ActiveState.com> Message-ID: <20000817085541.K376@xs4all.nl> On Wed, Aug 16, 2000 at 04:55:42PM -0700, Trent Mick wrote: > I am currently trying to port Python to Monterey (64-bit AIX) and I need > to add a couple of Monterey specific options to CFLAGS and LDFLAGS (or to > whatever appropriate variables for all 'cc' and 'ld' invocations) but it > is not obvious *at all* how to do that in configure.in. Can anybody helpme > on that? You'll have to write a shell 'case' for AIX Monterey, checking to make sure it is monterey, and setting LDFLAGS accordingly. If you look around in configure.in, you'll see a few other 'special cases', all to tune the way the compiler is called. Depending on what you need to do to detect monterey, you could fit it in one of those. Just search for 'Linux' or 'bsdos' to find a couple of those cases. > ANother issue that I am having. This is how the python executable is linked > on Linux with gcc: > gcc -Xlinker -export-dynamic python.o ../libpython2.0.a -lpthread -ldl -lutil -lm -o python > It, of course, works fine, but shouldn't the proper (read "portable") > invocation to include the python2.0 library be > gcc -Xlinker -export-dynamic python.o -L.. -lpython2.0 -lpthread -ldl -lutil -lm -o python > That invocation form (i.e. with the '-L.. -lpython2.0') works on Linux, and > is *required* on Monterey. Does this problem not show up with other Unix > compilers. My hunch is that simply listing library (*.a) arguments on the gcc > command line is a GNU gcc/ld shortcut to the more portable usage of -L and > -l. Any opinions. I would either like to change the form to the latter or > I'll have to special case the invocation for Monterey. ANy opinions on which > is worse. Well, as far as I know, '-L.. -lpython2.0' does something *different* than just '../libpython2.0.a' ! When supplying the static library on the command line, the library is always statically linked. When using -L/-l, it is usually dynamically linked, unless a dynamic library doesn't exist. We currently don't have a libpython2.0.so, but a patch to add it is on Barry's plate. Also, I'm not entirely sure about the search order in such a case: gcc's docs seem to suggest that the systemwide library directories are searched before the -L directories. I'm not sure on that, though. Also, listing the library on the command line is not a gcc shortcut, but other people already said that :) I'd be suprised if AIX removed it (but not especially so; my girlfriend works with AIX machines a lot, and she already showed me some suprising things ;) but perhaps there is another workaround ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From gstein@lyra.org Thu Aug 17 08:01:22 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 17 Aug 2000 00:01:22 -0700 Subject: [Python-Dev] os.path.commonprefix breakage In-Reply-To: ; from MarkH@ActiveState.com on Thu, Aug 17, 2000 at 01:32:25PM +1000 References: Message-ID: <20000817000122.L17689@lyra.org> >>> os.path.split('/foo/bar/') ('/foo/bar', '') >>> Jamming a trailing slash on the end is a bit wonky. I'm with Skip on saying that the slash should probably *not* be appended. It gives funny behavior with the split. Users should use .join() to combine the resulting with something else. The removal of a prefix is an interesting issue. No opinions there. Cheers, -g On Thu, Aug 17, 2000 at 01:32:25PM +1000, Mark Hammond wrote: > Hi, > I believe that Skip recently made a patch to os.path.commonprefix to only > return the portion of the common prefix that corresponds to a directory. > > I have just dicovered some code breakage from this change. On 1.5.2, the > behaviour was: > > >>> os.path.commonprefix(["../foo/bar", "../foo/spam"]) > '../foo/' > > While since the change we have: > '../foo' > > Note that the trailing slash has been dropped. > > The code this broke did similar to: > > prefix = os.path.commonprefix(files) > for file in files: > tail_portion = file[len(prefix):] > > In 1.6, the "tail_portion" result looks like an absolute path "/bar" and > "/spam", respectively. The intent was obviously to get absolute path names > back ("bar" and "spam") > > The code that broke is not mine, so you can safely be horrified at how > broken it is :-) The point, however, is that code like this does exist out > there. > > I'm obviously going to change the code that broke, and don't have time to > look into the posixpath.py code - but is this level of possible breakage > acceptable? > > Thanks, > > Mark. > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://www.python.org/mailman/listinfo/python-dev -- Greg Stein, http://www.lyra.org/ From thomas@xs4all.net Thu Aug 17 08:09:42 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 17 Aug 2000 09:09:42 +0200 Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <200008170428.QAA15385@s454.cosc.canterbury.ac.nz>; from greg@cosc.canterbury.ac.nz on Thu, Aug 17, 2000 at 04:28:19PM +1200 References: <14747.25702.435148.549678@beluga.mojam.com> <200008170428.QAA15385@s454.cosc.canterbury.ac.nz> Message-ID: <20000817090942.L376@xs4all.nl> On Thu, Aug 17, 2000 at 04:28:19PM +1200, Greg Ewing wrote: > given platform -- e.g Unix doesn't care whether there's a > trailing slash on a pathname. Bzzzt. This is unfortunately not true. Observe: daemon2:~/python > mkdir perl daemon2:~/python > rm perl/ rm: perl/: is a directory daemon2:~/python > rmdir perl/ rmdir: perl/: Is a directory daemon2:~/python > rm -rf perl/ rm: perl/: Is a directory daemon2:~/python > su # rmdir perl/ rmdir: perl/: Is a directory # rm -rf perl/ rm: perl/: Is a directory # ^D daemon2:~/python > rmdir perl daemon2:~/python > Note that the trailing slash is added by all tab-completing shells that I know. And the problem *really* is that trailing slash, I shit you not. Needless to say, every one of us ran into this at one time or another, and spent an hour figuring out *why* the rmdir wouldn't remove a directory. Consequently, I'm all for removing trailing slashes, but not enough to break existing code. I wonder howmuch breakage there really is, though. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From tim_one@email.msn.com Thu Aug 17 08:49:33 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 17 Aug 2000 03:49:33 -0400 Subject: [Python-Dev] Pending patches for 2.0 In-Reply-To: <20000816225552.H376@xs4all.nl> Message-ID: [Thomas Wouters, needs a well-earned vacation!] > ... > and if those decisions are made before, say, August 30th, I think > I can move them into the CVS tree before leaving and just shove > the responsibility for them on the entire dev team ;) > > This isn't a push to get them accepted ! Just a warning that if > they aren't accepted before then, someone will have to take over > the breastfeeding ;) Guido will be back from his travels next week, and PythonLabs will have an intense 2.0 release meeting on Tuesday or Wednesday (depending also on exactly when Jeremy gets back). I expect all go/nogo decisions will be made then. Part of deciding on a patch that isn't fully complete is deciding whether others can take up the slack in time. That's just normal release business as usual -- nothing to worry about. Well, not for *you*, anyway. BTW, there's a trick few have learned: get the doc patches in *first*, and then we look like idiots if we release without code to implement it. And since this trick has hardly ever been tried, I bet you can sneak it by Fred Drake for at least a year before anyone at PythonLabs catches on to it <0.9 wink>. my-mouth-is-sealed-ly y'rs - tim From tim_one@email.msn.com Thu Aug 17 08:29:05 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 17 Aug 2000 03:29:05 -0400 Subject: [Python-Dev] PEP 214, extended print statement In-Reply-To: Message-ID: [Moshe Zadka] > Ummmmm...... > > print to myfile (print a newline on myfile) > print to, myfile (print to+" "+myfile to stdout) Like I (and Greg too) clearly said all along, -1 on changing ">>" to "to"! From mal@lemburg.com Thu Aug 17 08:31:55 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 17 Aug 2000 09:31:55 +0200 Subject: [Python-Dev] Re: os.path.commonprefix breakage References: <14747.25702.435148.549678@beluga.mojam.com> <14747.26112.609255.338170@cj42289-a.reston1.va.home.com> <14747.27927.170223.873328@beluga.mojam.com> Message-ID: <399B94EB.E95260EE@lemburg.com> Skip Montanaro wrote: > > Fred> I'd guess that the path separator should only be appended if it's > Fred> part of the passed-in strings; that would make it a legitimate > Fred> part of the prefix. If it isn't present for all of them, it > Fred> shouldn't be part of the result: > > >>> os.path.commonprefix(["foo", "foo/bar"]) > 'foo' > > Hmmm... I think you're looking at it character-by-character again. I see > three possibilities: > > * it's invalid to have a path with a trailing separator > > * it's okay to have a path with a trailing separator > > * it's required to have a path with a trailing separator > > In the first and third cases, you have no choice. In the second you have to > decide which would be best. > > On Unix my preference would be to not include the trailing "/" for aesthetic > reasons. Wait, Skip :-) By dropping the trailing slash from the path you are removing important information from the path information. This information can only be regained by performing an .isdir() check and then only of the directory exists somewhere. If it doesn't you are loosing valid information here. Another aspect: Since posixpath is also used by URL handling code, I would suspect that this results in some nasty problems too. You'd have to actually ask the web server to give you back the information you already had. Note that most web-servers send back a redirect in case a directory is queried without ending slash in the URL. They do this for exactly the reason stated above: to add the .isdir() information to the path itself. Conclusion: Please don't remove the slash -- at least not in posixpath. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From thomas@xs4all.net Thu Aug 17 10:54:16 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 17 Aug 2000 11:54:16 +0200 Subject: [Python-Dev] PEP 214, extended print statement In-Reply-To: ; from tim_one@email.msn.com on Thu, Aug 17, 2000 at 03:29:05AM -0400 References: Message-ID: <20000817115416.M376@xs4all.nl> On Thu, Aug 17, 2000 at 03:29:05AM -0400, Tim Peters wrote: > [Moshe Zadka] > > Ummmmm...... > > > > print to myfile (print a newline on myfile) > > print to, myfile (print to+" "+myfile to stdout) > > Like I (and Greg too) clearly said all along, -1 on changing ">>" to "to"! Really ? Hmmmm... [Greg Ewing] > Looks reasonably good. Not entirely sure I like the look > of >> though -- a bit too reminiscent of C++. > > How about > > print to myfile, x, y, z [Barry Warsaw] > Not bad at all. Seems quite Pythonic to me. [Tim Peters] > Me too! +1 on changing ">>" to "to" here. Then we can introduce [print from etc] I guessed I missed the sarcasm ;-P Gullib-ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From gmcm@hypernet.com Thu Aug 17 12:58:26 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Thu, 17 Aug 2000 07:58:26 -0400 Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <200008170428.QAA15385@s454.cosc.canterbury.ac.nz> References: <14747.25702.435148.549678@beluga.mojam.com> Message-ID: <1245608987-154490918@hypernet.com> Greg Ewing wrote: [snip] > While we're on the subject, another thing that's missing is > a platform-independent way of dealing with the notion of > "up one directory". os.chdir(os.pardir) - Gordon From paul@prescod.net Thu Aug 17 13:56:23 2000 From: paul@prescod.net (Paul Prescod) Date: Thu, 17 Aug 2000 08:56:23 -0400 Subject: [Python-Dev] Winreg update References: <3993FEC7.4E38B4F1@prescod.net> <20000815195751.A16100@ludwig.cnri.reston.va.us> Message-ID: <399BE0F7.F00765DA@prescod.net> Greg Ward wrote: > > ... > I'm all in favour of high-level interfaces, and I'm also in favour of > speaking the local tongue -- when in Windows, follow the Windows API (at > least for features that are totally Windows-specific, like the > registry). At this point, the question is not whether to follow the Microsoft API or not (any more). It is whether to follow the early 1990s Microsoft API for C programmers or the new Microsoft API for Visual Basic, C#, Eiffel and Javascript programmers. -- Paul Prescod - Not encumbered by corporate consensus Simplicity does not precede complexity, but follows it. - http://www.cs.yale.edu/homes/perlis-alan/quotes.html From paul@prescod.net Thu Aug 17 13:57:08 2000 From: paul@prescod.net (Paul Prescod) Date: Thu, 17 Aug 2000 08:57:08 -0400 Subject: [Python-Dev] Lockstep iteration - eureka! References: Message-ID: <399BE124.9920B0B6@prescod.net> Tim Peters wrote: > > ... > > If you want a more efficient way to do it, it's available (just not as > > syntactically beautiful -- same as range/xrangel). > > Which way would that be? I don't know of one, "efficient" either in the > sense of runtime speed or of directness of expression. One of the reasons for adding range literals was for efficiency. So for x in [:len(seq)]: ... should be efficient. > The "loop index" isn't an accident of the way Python happens to implement > "for" today, it's the very basis of Python's thing.__getitem__(i)/IndexError > iteration protocol. Exposing it is natural, because *it* is natural. I don't think of iterators as indexing in terms of numbers. Otherwise I could do this: >>> a={0:"zero",1:"one",2:"two",3:"three"} >>> for i in a: ... print i ... So from a Python user's point of view, for-looping has nothing to do with integers. From a Python class/module creator's point of view it does have to do with integers. I wouldn't be either surprised nor disappointed if that changed one day. > Sorry, but seq.keys() just makes me squirm. It's a little step down the > Lispish path of making everything look the same. I don't want to see > float.write() either . You'll have to explain your squeamishness better if you expect us to channel you in the future. Why do I use the same syntax for indexing sequences and dictionaries and for deleting sequence and dictionary items? Is the rule: "syntax can work across types but method names should never be shared"? -- Paul Prescod - Not encumbered by corporate consensus Simplicity does not precede complexity, but follows it. - http://www.cs.yale.edu/homes/perlis-alan/quotes.html From paul@prescod.net Thu Aug 17 13:58:00 2000 From: paul@prescod.net (Paul Prescod) Date: Thu, 17 Aug 2000 08:58:00 -0400 Subject: [Python-Dev] Winreg update References: <3993FEC7.4E38B4F1@prescod.net> <045901c00414$27a67010$8119fea9@neil> Message-ID: <399BE158.C2216D34@prescod.net> Neil Hodgson wrote: > > ... > > The registry is just not important enough to have this much attention or > work. I remain unreconstructed. My logic is as follows: * The registry is important enough to be in the standard library ... unlike, let's say, functions to operate the Remote Access Service. * The registry is important enough that the interface to it is documented (partially) * Therefore, the registry is important enough to have a decent API with complete documentation. You know the old adage: "anything worth doing..." If the registry is just supposed to expose one or two functions for distutils then it could expose one or two functions for distutils, be called _distreg and be undocumented and formally unsupported. > The Microsoft.Win32.Registry* API appears to be a hacky legacy API to me. > Its there for compatibility during the transition to the > System.Configuration API. Read the blurb for ConfigManager to understand the > features of System.Configuration. Its all based on XML files. What a > surprise. Nobody on Windows is going to migrate to XML configuration files this year or next year. The change-over is going to be too difficult. Predicting Microsoft configuration ideology in 2002 is highly risky. If we need to do the registry today then we can do the registry right today. -- Paul Prescod - Not encumbered by corporate consensus Simplicity does not precede complexity, but follows it. - http://www.cs.yale.edu/homes/perlis-alan/quotes.html From skip@mojam.com (Skip Montanaro) Thu Aug 17 13:50:28 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 17 Aug 2000 07:50:28 -0500 (CDT) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: References: <14747.28372.771170.783868@cj42289-a.reston1.va.home.com> Message-ID: <14747.57236.264324.165612@beluga.mojam.com> Tim> Oddly enough, I don't: commonprefix worked exactly as documented Tim> for at least 6 years and 5 months (which is when CVS shows Guido Tim> checking in ntpath.py with the character-based functionality), and Tim> got out of synch with the docs about 5 weeks ago when Skip changed Tim> to this other algorithm. Since the docs *did* match the code, Tim> there's no reason to believe the original author was confused, and Tim> no reason to believe users aren't relying on it (they've had over 6 Tim> years to gripe ). I don't realize that because a bug wasn't noticed for a long time was any reason not to fix it. Guido was also involved in the repair of the bug, and had no objections to the fix I eventually arrived at. Also, when I announced my original patch the subject of the message was patch for os.path.commonprefix (changes semantics - pls review) In the body of the message I said Since my patch changes the semantics of the function, I submitted a patch via SF that implements what I believe to be the correct behavior instead of just checking in the change, so people could comment on it. I don't think I could have done much more to alert people to the change than I did. I didn't expect the patch to go into 1.6. (Did it? It shouldn't have.) I see nothing wrong with correcting the semantics of a function that is broken when we increment the major version number of the code. Tim> I appreciate that some other behavior may be more useful more Tim> often, but if you can ever agree on what that is across platforms, Tim> it should be spelled via a new function name ("commonpathprefix" Tim> comes to mind), or optional flag (defaulting to "old behavior") on Tim> commonprefix (yuck!). BTW, the presence or absence of a trailing Tim> path separator makes a *big* difference to many cmds on Windows, Tim> and you can't tell me nobody isn't currently doing e.g. Tim> commonprefix(["blah.o", "blah", "blah.cpp"]) Tim> on Unix either. Fine. Let's preserve the broken implementation and not break any broken usage. Switch it back then. Taking a look at the copious documentation for posixpath.commonprefix: Return the longest string that is a prefix of all strings in list. If list is empty, return the empty string ''. I see no mention of anything in this short bit of documentation taken completely out of context that suggests that posixpath.commonprefix has anything to do with paths, so maybe we should move it to some other module that has no directory path implications. That way nobody can make the mistake of trying to assume it operates on paths. Perhaps string? Oh, that's deprecated. Maybe we should undeprecate it or make commonprefix a string method. Maybe I'll just reopen the patch and assign it to Barry since he's the string methods guru. On a more realistic note, perhaps I should submit a patch that corrects the documentation. Skip From skip@mojam.com (Skip Montanaro) Thu Aug 17 13:19:46 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 17 Aug 2000 07:19:46 -0500 (CDT) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <200008170453.QAA15394@s454.cosc.canterbury.ac.nz> References: <14747.27927.170223.873328@beluga.mojam.com> <200008170453.QAA15394@s454.cosc.canterbury.ac.nz> Message-ID: <14747.55394.783997.167234@beluga.mojam.com> Greg> To avoid duplication of effort, how about a single function that Greg> does both: >>> files = ["/home/swen", "/home/swanson", "/home/jules"] >>> os.path.factorize(files) ("/home", ["swen", "swanson", "jules"]) Since we already have os.path.commonprefix and it's not going away, it seemed to me that just adding a complementary function to return the suffixes made sense. Also, there's nothing in the name factorize that suggests that it would split the paths at the common prefix. It could easily be written in terms of the two: def factorize(files): pfx = os.path.commonprefix(files) suffixes = os.path.suffixes(pfx, files) return (pfx, suffixes) Skip From bwarsaw@beopen.com Thu Aug 17 15:35:03 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Thu, 17 Aug 2000 10:35:03 -0400 (EDT) Subject: [Python-Dev] PEP 214, extended print statement References: <14747.22851.266303.28877@anthem.concentric.net> <20000817083023.J376@xs4all.nl> Message-ID: <14747.63511.725610.771162@anthem.concentric.net> >>>>> "TW" == Thomas Wouters writes: TW> Agreed. It might be technically unambiguous, but I think it's TW> too hard for a *human* to parse this correctly. The '>>' TW> version might seem more C++ish and less pythonic, but it also TW> stands out a lot more. The 'print from' statement could easily TW> (and more consistently, IMHO ;) be written as 'print <<' (not TW> that I like the 'print from' idea, though.) I also played around with trying to get the grammar and parser to recognize 'print to' and variants, and it seemed difficult and complicated. So I'm back to -0 on 'print to' and +1 on 'print >>'. -Barry From bwarsaw@beopen.com Thu Aug 17 15:43:02 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Thu, 17 Aug 2000 10:43:02 -0400 (EDT) Subject: [Python-Dev] PEP 214, extended print statement References: <20000817115416.M376@xs4all.nl> Message-ID: <14747.63990.296049.566791@anthem.concentric.net> >>>>> "TW" == Thomas Wouters writes: TW> Really ? Hmmmm... TW> [Tim Peters] >> Me too! +1 on changing ">>" to "to" here. Then we can >> introduce TW> I guessed I missed the sarcasm ;-P No, Tim just forgot to twist the blue knob while he was pressing the shiny pedal on Guido's time machine. I've made the same mistake myself before -- the VRTM can be as inscrutable as the BDFL himself at times. Sadly, changing those opinions now would cause an irreparable time paradox, the outcome of which would force Python to be called Bacon and require you to type `albatross' instead of colons to start every block. good-thing-tim-had-the-nose-plugs-in-or-Python-would-only-work-on- 19-bit-architectures-ly y'rs, -Barry From Vladimir.Marangozov@inrialpes.fr Thu Aug 17 16:09:44 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Thu, 17 Aug 2000 17:09:44 +0200 (CEST) Subject: [Python-Dev] PyErr_NoMemory Message-ID: <200008171509.RAA20891@python.inrialpes.fr> The current PyErr_NoMemory() function reads: PyObject * PyErr_NoMemory(void) { /* raise the pre-allocated instance if it still exists */ if (PyExc_MemoryErrorInst) PyErr_SetObject(PyExc_MemoryError, PyExc_MemoryErrorInst); else /* this will probably fail since there's no memory and hee, hee, we have to instantiate this class */ PyErr_SetNone(PyExc_MemoryError); return NULL; } thus overriding any previous exceptions unconditionally. This is a problem when the current exception already *is* PyExc_MemoryError, notably when we have a chain (cascade) of memory errors. It is a problem because the original memory error and eventually its error message is lost. I suggest to make this code look like: PyObject * PyErr_NoMemory(void) { if (PyErr_ExceptionMatches(PyExc_MemoryError)) /* already current */ return NULL; /* raise the pre-allocated instance if it still exists */ if (PyExc_MemoryErrorInst) PyErr_SetObject(PyExc_MemoryError, PyExc_MemoryErrorInst); ... If nobody sees a problem with this, I'm very tempted to check it in. Any objections? -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From gmcm@hypernet.com Thu Aug 17 16:22:27 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Thu, 17 Aug 2000 11:22:27 -0400 Subject: [Python-Dev] PEP 214, extended print statement In-Reply-To: <14747.63990.296049.566791@anthem.concentric.net> Message-ID: <1245596748-155226852@hypernet.com> > No, Tim just forgot to twist the blue knob while he was pressing > the shiny pedal on Guido's time machine. I've made the same > mistake myself before -- the VRTM can be as inscrutable as the > BDFL himself at times. Sadly, changing those opinions now would > cause an irreparable time paradox, the outcome of which would > force Python to be called Bacon and require you to type > `albatross' instead of colons to start every block. That accounts for the strange python.ba (mtime 1/1/70) I stumbled across this morning: #!/usr/bin/env bacon # released to the public domain at least one Tim Peters import sys, string, tempfile txt = string.replace(open(sys.argv[1]).read(), ':', ' albatross') fnm = tempfile.mktemp() + '.ba' open(fnm, 'w').write(txt) os.system('bacon %s %s' % (fnm, string.join(sys.argv[2:])) - Gordon From nowonder@nowonder.de Thu Aug 17 20:30:13 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Thu, 17 Aug 2000 19:30:13 +0000 Subject: [Python-Dev] PyErr_NoMemory References: <200008171509.RAA20891@python.inrialpes.fr> Message-ID: <399C3D45.95ED79D8@nowonder.de> Vladimir Marangozov wrote: > > If nobody sees a problem with this, I'm very tempted to check it in. > Any objections? This change makes sense to me. I can't see any harm in checking it in. Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From tim_one@email.msn.com Thu Aug 17 18:58:25 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 17 Aug 2000 13:58:25 -0400 Subject: [Python-Dev] PEP 214, extended print statement In-Reply-To: <14747.63990.296049.566791@anthem.concentric.net> Message-ID: > >>>>> "TW" == Thomas Wouters writes: > > TW> Really ? Hmmmm... > > TW> [Tim Peters] > >> Me too! +1 on changing ">>" to "to" here. Then we can > >> introduce > TW> I guessed I missed the sarcasm ;-P [Barry A. Warsaw] > No, Tim just forgot to twist the blue knob while he was pressing the > shiny pedal on Guido's time machine. I've made the same mistake > myself before -- the VRTM can be as inscrutable as the BDFL himself at > times. Sadly, changing those opinions now would cause an irreparable > time paradox, the outcome of which would force Python to be called > Bacon and require you to type `albatross' instead of colons to start > every block. > > good-thing-tim-had-the-nose-plugs-in-or-Python-would-only-work-on- > 19-bit-architectures-ly y'rs, I have no idea what this is about. I see an old msg from Barry voting "-1" on changing ">>" to "to", but don't believe any such suggestion was ever made. And I'm sure that had such a suggestion ever been made, it would have been voted down at once by everyone. OTOH, there is *some* evidence that an amateur went mucking with the time machine! No 19-bit architectures, but somewhere in a reality distortion field around Vancouver, it appears that AIX actually survived long enough to see the 64-bit world, and that some yahoo vendor decided to make a version of C where sizeof(void*) > sizeof(long). There's no way either of those could have happened naturally. even-worse-i-woke-up-today-*old*!-ly y'rs - tim From trentm@ActiveState.com Thu Aug 17 19:21:22 2000 From: trentm@ActiveState.com (Trent Mick) Date: Thu, 17 Aug 2000 11:21:22 -0700 Subject: screwin' with the time machine in Canada, eh (was: Re: [Python-Dev] PEP 214, extended print statement) In-Reply-To: ; from tim_one@email.msn.com on Thu, Aug 17, 2000 at 01:58:25PM -0400 References: <14747.63990.296049.566791@anthem.concentric.net> Message-ID: <20000817112122.A27284@ActiveState.com> On Thu, Aug 17, 2000 at 01:58:25PM -0400, Tim Peters wrote: > > OTOH, there is *some* evidence that an amateur went mucking with the time > machine! No 19-bit architectures, but somewhere in a reality distortion > field around Vancouver, it appears that AIX actually survived long enough to > see the 64-bit world, and that some yahoo vendor decided to make a version > of C where sizeof(void*) > sizeof(long). There's no way either of those > could have happened naturally. > And though this place is supposed to be one the more successful pot havens on the planet I just can't seem to compete with the stuff those "vendors" in Austin (AIX) and Seattle must have been smokin'. - -if-i-wasn't-seeing-flying-bunnies-i-would-swear-that-compiler is-from-SCO-ly-y'rs - trent > even-worse-i-woke-up-today-*old*!-ly y'rs - tim Come on up for a visit and we'll make you feel young again. :) Trent -- Trent Mick From akuchlin@mems-exchange.org Thu Aug 17 21:40:35 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Thu, 17 Aug 2000 16:40:35 -0400 Subject: [Python-Dev] Cookie.py module, and Web PEP Message-ID: Tim O'Malley finally mailed me the correct URL for the latest version of the cookie module: http://www.timo-tasi.org/python/Cookie.py *However*... I think the Web support in Python needs more work in generally, and certainly more than can be done for 2.0. One of my plans for the not-too-distant future is to start writing a Python/CGI guide, and the process of writing it is likely to shake out more ugliness that should be fixed. I'd like to propose a 'Web Library Enhancement PEP', and offer to champion and write it. Its goal would be to identify missing features and specify them, and list other changes to improve Python as a Web/CGI language. Possibly the PEP would also drop backward-compatibility cruft. (Times like this I wish the Web-SIG hadn't been killed...) --amk From akuchlin@mems-exchange.org Thu Aug 17 21:43:45 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Thu, 17 Aug 2000 16:43:45 -0400 Subject: [Python-Dev] Cookie.py module, and Web PEP Message-ID: Tim O'Malley finally mailed me the correct URL for the latest version of the cookie module: http://www.timo-tasi.org/python/Cookie.py *However*... I think the Web support in Python needs more work in generally, and certainly more than can be done for 2.0. One of my plans for the not-too-distant future is to start writing a Python/CGI guide, and the process of writing it is likely to shake out more ugliness that should be fixed. I'd like to propose a 'Web Library Enhancement PEP', and offer to champion and write it. Its goal would be to identify missing features and specify them, and list other changes to improve Python as a Web/CGI language. Possibly the PEP would also drop backward-compatibility cruft. (Times like this I wish the Web-SIG hadn't been killed...) --amk From bwarsaw@beopen.com Thu Aug 17 22:05:10 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Thu, 17 Aug 2000 17:05:10 -0400 (EDT) Subject: [Python-Dev] Cookie.py module, and Web PEP References: Message-ID: <14748.21382.305979.784637@anthem.concentric.net> >>>>> "AK" == Andrew Kuchling writes: AK> Tim O'Malley finally mailed me the correct URL for the latest AK> version of the cookie module: AK> http://www.timo-tasi.org/python/Cookie.py AK> *However*... I think the Web support in Python needs more AK> work in generally, and certainly more than can be done for AK> 2.0. I agree, but I still think Cookie.py should go in the stdlib for 2.0. -Barry From akuchlin@mems-exchange.org Thu Aug 17 22:13:52 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Thu, 17 Aug 2000 17:13:52 -0400 Subject: [Python-Dev] Cookie.py module, and Web PEP In-Reply-To: <14748.21382.305979.784637@anthem.concentric.net>; from bwarsaw@beopen.com on Thu, Aug 17, 2000 at 05:05:10PM -0400 References: <14748.21382.305979.784637@anthem.concentric.net> Message-ID: <20000817171352.B26730@kronos.cnri.reston.va.us> On Thu, Aug 17, 2000 at 05:05:10PM -0400, Barry A. Warsaw wrote: >I agree, but I still think Cookie.py should go in the stdlib for 2.0. Fine. Shall I just add it as-is? (Opinion was generally positive as I recall, unless the BDFL wants to exercise his veto for some reason.) --amk From thomas@xs4all.net Thu Aug 17 22:19:42 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 17 Aug 2000 23:19:42 +0200 Subject: [Python-Dev] 'import as' Message-ID: <20000817231942.O376@xs4all.nl> I have two remaining issues regarding the 'import as' statement, which I'm just about ready to commit. The first one is documentation: I have documentation patches, to the ref and the libdis sections, but I can't really test them :P I *think* they are fine, though, and they aren't really complicated. Should I upload a patch for them, so Fred or someone else can look at them, or just check them in ? The other issue is the change in semantics for 'from-import'. Currently, 'IMPORT_FROM' is a single operation that retrieves a name (possible '*') from the module object at TOS, and stores it directly in the local namespace. This is contrary to 'import ', which pushes it onto the stack and uses a normal STORE operation to store it. It's also necessary for 'from ... import *', which can load any number of objects. After the patch, 'IMPORT_FROM' is only used to load normal names, and a new opcode, 'IMPORT_STAR' (with no argument) is used for 'from module import *'. 'IMPORT_FROM' pushes the result on the stack, instead of modifying the local namespace directly, so that it's possible to store it to a different name. This also means that a 'global' statement now has effect on objects 'imported from' a module, *except* those imported by '*'. I don't think that's a big issue. 'global' is not that heavily used, and old code mixing 'import from' and 'global' statements on the same identifier would not have been doing what the programmer intended. However, if it *is* a big issue, I can revert to an older version of the patch, that added a new bytecode to handle 'from x import y as z', and leave the bytecode for the currently valid cases unchanged. That would mean that only the '... as z' would be effected by 'global' statements. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From trentm@ActiveState.com Thu Aug 17 22:22:07 2000 From: trentm@ActiveState.com (Trent Mick) Date: Thu, 17 Aug 2000 14:22:07 -0700 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: ; from tim_one@email.msn.com on Wed, Aug 16, 2000 at 11:34:12PM -0400 References: <20000816172425.A32338@ActiveState.com> Message-ID: <20000817142207.A5592@ActiveState.com> On Wed, Aug 16, 2000 at 11:34:12PM -0400, Tim Peters wrote: > [Trent Mick] > > I am porting Python to Monterey (64-bit AIX) and have a small > > (hopefully) question about POSIX threads. > > POSIX threads. "small question". HAHAHAHAHAHA. Thanks, that felt good > . Happy to provide you with cheer. > > Does the POSIX threads spec specify a C type or minimum size for > > pthread_t? > > or user-space arrays of structs. So I think it's *safe* to assume it will > always fit in an integral type large enough to hold a pointer, but not > guaranteed. Plain "long" certainly isn't safe in theory. Not for pthread ports to Win64 anyway. But that is not my concern right now. I'll let the pthreads-on-Windows fans worry about that when the time comes. > > this up. On Linux (mine at least): > > /usr/include/bits/pthreadtypes.h:120:typedef unsigned long int > > pthread_t; > > And this is a 32- or 64-bit Linux? That was 32-bit Linux. My 64-bit Linux box is down right now, I can tell later if you really want to know. > > WHAT IS UP WITH THAT return STATEMENT? > > return (long) *(long *) &threadid; > > > So, here's the scoop: > Thanks for trolling the cvs logs, Tim! > > So one of two things can be done: > > 1. Bite the bullet and do it correctly. For example, maintain a static > dict mapping the native pthread_self() return value to Python ints, > and return the latter as Python's thread.get_ident() value. Much > better would to implement a x-platform thread-local storage > abstraction, and use that to hold a Python-int ident value. > > 2. Continue in the tradition already established , and #ifdef the > snot out of it for Monterey. > > In favor of #2, the code is already so hosed that making it hosier won't be > a significant relative increase in its inherent hosiness. > > spoken-like-a-true-hoser-ly y'rs - tim > I'm all for being a hoser then. #ifdef's a-comin' down the pipe. One thing, the only #define that I know I have a handle on for Monterey is '_LP64'. Do you have an objection to that (seeing at is kind of misleading)? I will accompany it with an explicative comment of course. take-off-you-hoser-ly y'rs - wannabe Bob & Doug fan -- Trent Mick TrentM@ActiveState.com From fdrake@beopen.com Thu Aug 17 22:35:14 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Thu, 17 Aug 2000 17:35:14 -0400 (EDT) Subject: [Python-Dev] 'import as' In-Reply-To: <20000817231942.O376@xs4all.nl> References: <20000817231942.O376@xs4all.nl> Message-ID: <14748.23186.372772.48426@cj42289-a.reston1.va.home.com> Thomas Wouters writes: > really test them :P I *think* they are fine, though, and they aren't really > complicated. Should I upload a patch for them, so Fred or someone else can > look at them, or just check them in ? Just check them in; I'll catch problems before anyone else tries to format the stuff at any rate. With regard to your semantics question, I think your proposed solution is fine. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From thomas@xs4all.net Thu Aug 17 22:38:21 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 17 Aug 2000 23:38:21 +0200 Subject: [Python-Dev] 'import as' In-Reply-To: <20000817231942.O376@xs4all.nl>; from thomas@xs4all.net on Thu, Aug 17, 2000 at 11:19:42PM +0200 References: <20000817231942.O376@xs4all.nl> Message-ID: <20000817233821.P376@xs4all.nl> On Thu, Aug 17, 2000 at 11:19:42PM +0200, Thomas Wouters wrote: > This also means that a 'global' statement now has effect on objects > 'imported from' a module, *except* those imported by '*'. And while I was checking my documentation patches, I found this: Names bound by \keyword{import} statements may not occur in \keyword{global} statements in the same scope. \stindex{global} But there doesn't seem to be anything to prevent it ! On my RedHat supplied Python 1.5.2: >>> def test(): ... global sys ... import sys ... >>> test() >>> sys And on a few weeks old CVS Python: >>> def test(): ... global sys ... import sys ... >>> test() >>> sys Also, mixing 'global' and 'from-import' wasn't illegal, it was just ineffective. (That is, it didn't make the variable 'global', but it didn't raise an exception either!) How about making 'from module import *' a special case in this regard, and letting 'global' operate fine on normal 'import' and 'from-import' statements ? I can definately see a use for it, anyway. Is this workable (and relevant) for JPython / #Py ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From trentm@ActiveState.com Thu Aug 17 22:41:04 2000 From: trentm@ActiveState.com (Trent Mick) Date: Thu, 17 Aug 2000 14:41:04 -0700 Subject: [Python-Dev] [Fwd: segfault in sre on 64-bit plats] In-Reply-To: <399B3D36.6921271@per.dem.csiro.au>; from m.favas@per.dem.csiro.au on Thu, Aug 17, 2000 at 09:17:42AM +0800 References: <399B3D36.6921271@per.dem.csiro.au> Message-ID: <20000817144104.B7658@ActiveState.com> On Thu, Aug 17, 2000 at 09:17:42AM +0800, Mark Favas wrote: > [Trent] > > This test on Win32 and Linux32 hits the recursion limit check of 10000 in > > SRE_MATCH(). However, on Linux64 the segfault occurs at a recursion depth of > > 7500. I don't want to just willy-nilly drop the recursion limit down to make > > the problem go away. > > > > Sorry for the delay - yes, I had these segfaults due to exceeding the > stack size on Tru64 Unix (which, by default, is 2048 kbytes) before > Fredrick introduced the recusrion limit of 10000 in _sre.c. You'd expect > a 64-bit OS to use a bit more bytes of the stack when handling recursive > calls, but your 7500 down from 10000 sounds a bit much - unless the Actually with pointers being twice the size the stack will presumably get comsumed more quickly (right?), so all other things being equal the earlier stack overflow is expected. > stack size limit you're using on Linux64 is smaller than that for > Linux32 - what are they? ------------------- snip --------- snip ---------------------- #include #include #include int main(void) { struct rlimit lims; if (getrlimit(RLIMIT_STACK, &lims) != 0) { printf("error in getrlimit\n"); exit(1); } printf("cur stack limit = %d, max stack limit = %d\n", lims.rlim_cur, lims.rlim_max); return 0; } ------------------- snip --------- snip ---------------------- On Linux32: cur stack limit = 8388608, max stack limit = 2147483647 On Linux64: cur stack limit = 8388608, max stack limit = -1 Trent -- Trent Mick TrentM@ActiveState.com From cgw@fnal.gov Thu Aug 17 22:43:38 2000 From: cgw@fnal.gov (Charles G Waldman) Date: Thu, 17 Aug 2000 16:43:38 -0500 (CDT) Subject: [Python-Dev] Include/config.h in CVS In-Reply-To: <20000817125903.2C29E1D0F5@dinsdale.python.org> References: <20000817125903.2C29E1D0F5@dinsdale.python.org> Message-ID: <14748.23690.632808.944375@buffalo.fnal.gov> This has probably been noted by somebody else already - somehow a config.h showed up in the Include directory when I did a cvs update today. I assume this is an error. It certainly keeps Python from building on my system! From thomas@xs4all.net Thu Aug 17 22:46:07 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Thu, 17 Aug 2000 23:46:07 +0200 Subject: [Python-Dev] 'import as' In-Reply-To: <20000817233821.P376@xs4all.nl>; from thomas@xs4all.net on Thu, Aug 17, 2000 at 11:38:21PM +0200 References: <20000817231942.O376@xs4all.nl> <20000817233821.P376@xs4all.nl> Message-ID: <20000817234607.Q376@xs4all.nl> On Thu, Aug 17, 2000 at 11:38:21PM +0200, Thomas Wouters wrote: > On Thu, Aug 17, 2000 at 11:19:42PM +0200, Thomas Wouters wrote: > > > This also means that a 'global' statement now has effect on objects > > 'imported from' a module, *except* those imported by '*'. > > And while I was checking my documentation patches, I found this: > Names bound by \keyword{import} statements may not occur in > \keyword{global} statements in the same scope. > \stindex{global} And about five lines lower, I saw this: (The current implementation does not enforce the latter two restrictions, but programs should not abuse this freedom, as future implementations may enforce them or silently change the meaning of the program.) My only excuse is that all that TeX stuff confuzzles my eyes ;) In any case, my point still stands: 1) can we change this behaviour even if it's documented to be impossible, and 2) should it be documented differently, allowing mixing of 'global' and 'import' ? Multip-ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From fdrake@beopen.com Thu Aug 17 22:52:28 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Thu, 17 Aug 2000 17:52:28 -0400 (EDT) Subject: [Python-Dev] Include/config.h in CVS In-Reply-To: <14748.23690.632808.944375@buffalo.fnal.gov> References: <20000817125903.2C29E1D0F5@dinsdale.python.org> <14748.23690.632808.944375@buffalo.fnal.gov> Message-ID: <14748.24220.666086.9128@cj42289-a.reston1.va.home.com> Charles G Waldman writes: > This has probably been noted by somebody else already - somehow a > config.h showed up in the Include directory when I did a cvs update > today. I assume this is an error. It certainly keeps Python from > building on my system! This doesn't appear to be in CVS. If you delete the file and the do a CVS update, does it reappear? -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From cgw@fnal.gov Thu Aug 17 22:56:55 2000 From: cgw@fnal.gov (Charles G Waldman) Date: Thu, 17 Aug 2000 16:56:55 -0500 (CDT) Subject: [Python-Dev] Include/config.h in CVS In-Reply-To: <14748.24220.666086.9128@cj42289-a.reston1.va.home.com> References: <20000817125903.2C29E1D0F5@dinsdale.python.org> <14748.23690.632808.944375@buffalo.fnal.gov> <14748.24220.666086.9128@cj42289-a.reston1.va.home.com> Message-ID: <14748.24487.903334.663705@buffalo.fnal.gov> And it's not that sticky date, either (no idea how that got set!) buffalo:Include$ cvs update -A cvs server: Updating . U config.h buffalo:Include$ cvs status config.h =================================================================== File: config.h Status: Up-to-date Working revision: 2.1 Repository revision: 2.1 /cvsroot/python/python/dist/src/Include/Attic/config.h,v Sticky Tag: (none) Sticky Date: (none) Sticky Options: (none) From cgw@fnal.gov Thu Aug 17 22:58:40 2000 From: cgw@fnal.gov (Charles G Waldman) Date: Thu, 17 Aug 2000 16:58:40 -0500 (CDT) Subject: [Python-Dev] Include/config.h in CVS In-Reply-To: <14748.24220.666086.9128@cj42289-a.reston1.va.home.com> References: <20000817125903.2C29E1D0F5@dinsdale.python.org> <14748.23690.632808.944375@buffalo.fnal.gov> <14748.24220.666086.9128@cj42289-a.reston1.va.home.com> Message-ID: <14748.24592.448009.515511@buffalo.fnal.gov> Fred L. Drake, Jr. writes: > > This doesn't appear to be in CVS. If you delete the file and the do > a CVS update, does it reappear? > Yes. buffalo:src$ pwd /usr/local/src/Python-CVS/python/dist/src buffalo:src$ cd Include/ buffalo:Include$ cvs update cvs server: Updating . U config.h buffalo:Include$ cvs status config.h =================================================================== File: config.h Status: Up-to-date Working revision: 2.1 Repository revision: 2.1 /cvsroot/python/python/dist/src/Include/Attic/config.h,v Sticky Tag: (none) Sticky Date: 2000.08.17.05.00.00 Sticky Options: (none) From fdrake@beopen.com Thu Aug 17 23:02:39 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Thu, 17 Aug 2000 18:02:39 -0400 (EDT) Subject: [Python-Dev] Include/config.h in CVS In-Reply-To: <14748.24487.903334.663705@buffalo.fnal.gov> References: <20000817125903.2C29E1D0F5@dinsdale.python.org> <14748.23690.632808.944375@buffalo.fnal.gov> <14748.24220.666086.9128@cj42289-a.reston1.va.home.com> <14748.24487.903334.663705@buffalo.fnal.gov> Message-ID: <14748.24831.313742.340896@cj42289-a.reston1.va.home.com> Charles G Waldman writes: > And it's not that sticky date, either (no idea how that got set!) -sigh- Is there an entry for config.h in the CVS/entries file? If so, surgically remove it, then delete the config.h, then try the update again. *This* is getting mysterious. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From cgw@fnal.gov Thu Aug 17 23:07:28 2000 From: cgw@fnal.gov (Charles G Waldman) Date: Thu, 17 Aug 2000 17:07:28 -0500 (CDT) Subject: [Python-Dev] Include/config.h in CVS In-Reply-To: <14748.24831.313742.340896@cj42289-a.reston1.va.home.com> References: <20000817125903.2C29E1D0F5@dinsdale.python.org> <14748.23690.632808.944375@buffalo.fnal.gov> <14748.24220.666086.9128@cj42289-a.reston1.va.home.com> <14748.24487.903334.663705@buffalo.fnal.gov> <14748.24831.313742.340896@cj42289-a.reston1.va.home.com> Message-ID: <14748.25120.807735.628798@buffalo.fnal.gov> Fred L. Drake, Jr. writes: > -sigh- Is there an entry for config.h in the CVS/entries file? If > so, surgically remove it, then delete the config.h, then try the > update again. Yes, this entry was present, I removed it as you suggested. Now, when I do cvs update the config.h doesn't reappear, but I still see "needs checkout" if I ask for cvs status: buffalo:Include$ cvs status config.h =================================================================== File: no file config.h Status: Needs Checkout Working revision: No entry for config.h Repository revision: 2.1 /cvsroot/python/python/dist/src/Include/Attic/config.h,v I keep my local CVS tree updated daily, I never use any kind of sticky tags, and haven't seen this sort of problem at all, up until today. Today I also noticed the CVS server responding very slowly, so I suspect that something may be wrong with the server. From fdrake@beopen.com Thu Aug 17 23:13:28 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Thu, 17 Aug 2000 18:13:28 -0400 (EDT) Subject: [Python-Dev] Include/config.h in CVS In-Reply-To: <14748.25120.807735.628798@buffalo.fnal.gov> References: <20000817125903.2C29E1D0F5@dinsdale.python.org> <14748.23690.632808.944375@buffalo.fnal.gov> <14748.24220.666086.9128@cj42289-a.reston1.va.home.com> <14748.24487.903334.663705@buffalo.fnal.gov> <14748.24831.313742.340896@cj42289-a.reston1.va.home.com> <14748.25120.807735.628798@buffalo.fnal.gov> Message-ID: <14748.25480.976849.825016@cj42289-a.reston1.va.home.com> Charles G Waldman writes: > Now, when I do cvs update the config.h doesn't reappear, but I still > see "needs checkout" if I ask for cvs status: [...output elided...] I get exactly the same output from "cvs status", and "cvs update" doesn't produce the file. Now, if I say "cvs update config.h", it shows up and doesn't get deleted by "cvs update", but after removing the line from CVS/Entries and removing the file, it doesn't reappear. So you're probably set for now. > I keep my local CVS tree updated daily, I never use any kind of sticky > tags, and haven't seen this sort of problem at all, up until today. > Today I also noticed the CVS server responding very slowly, so I > suspect that something may be wrong with the server. This is weird, but that doesn't sound like the problem; the SF servers can be very slow some days, but we suspect it's just load. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From trentm@ActiveState.com Thu Aug 17 23:15:08 2000 From: trentm@ActiveState.com (Trent Mick) Date: Thu, 17 Aug 2000 15:15:08 -0700 Subject: [Python-Dev] autoconf question: howto add to CFLAGS and LDFLAGS? In-Reply-To: <20000817085541.K376@xs4all.nl>; from thomas@xs4all.net on Thu, Aug 17, 2000 at 08:55:41AM +0200 References: <20000816165542.D29260@ActiveState.com> <20000817085541.K376@xs4all.nl> Message-ID: <20000817151508.C7658@ActiveState.com> On Thu, Aug 17, 2000 at 08:55:41AM +0200, Thomas Wouters wrote: > On Wed, Aug 16, 2000 at 04:55:42PM -0700, Trent Mick wrote: > > > I am currently trying to port Python to Monterey (64-bit AIX) and I need > > to add a couple of Monterey specific options to CFLAGS and LDFLAGS (or to > > whatever appropriate variables for all 'cc' and 'ld' invocations) but it > > is not obvious *at all* how to do that in configure.in. Can anybody helpme > > on that? > > You'll have to write a shell 'case' for AIX Monterey, checking to make sure > it is monterey, and setting LDFLAGS accordingly. If you look around in > configure.in, you'll see a few other 'special cases', all to tune the > way the compiler is called. Depending on what you need to do to detect > monterey, you could fit it in one of those. Just search for 'Linux' or > 'bsdos' to find a couple of those cases. Right, thanks. I was looking at first to modify CFLAGS and LDFLAGS (as I thought would be cleaner) but I have got it working by just modifying CC and LINKCC instead (following the crowd on that one). [Trent blames placing *.a on the cc command line for his problems and Thomas and Barry, etc. tell Trent that that cannot be] Okay, I don't know what I was on. I think I was flailing for things to blame. I have got it working with simply listing the .a on the command line. Thanks, Trent -- Trent Mick TrentM@ActiveState.com From bwarsaw@beopen.com Thu Aug 17 23:26:42 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Thu, 17 Aug 2000 18:26:42 -0400 (EDT) Subject: [Python-Dev] Cookie.py module, and Web PEP References: <14748.21382.305979.784637@anthem.concentric.net> <20000817171352.B26730@kronos.cnri.reston.va.us> Message-ID: <14748.26274.949428.733639@anthem.concentric.net> >>>>> "AK" == Andrew Kuchling writes: AK> Fine. Shall I just add it as-is? (Opinion was generally AK> positive as I recall, unless the BDFL wants to exercise his AK> veto for some reason.) Could you check and see if there are any substantial differences between the version you've got and the version in the Mailman tree? If there are none, then I'm +1. Let me know if you want me to email it to you. -Barry From MarkH@ActiveState.com Fri Aug 18 00:07:38 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Fri, 18 Aug 2000 09:07:38 +1000 Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <14747.57236.264324.165612@beluga.mojam.com> Message-ID: > I don't realize that because a bug wasn't noticed for a long time was any > reason not to fix it. Guido was also involved in the repair of > the bug, and I think most people agreed that the new semantics were preferable to the old. I believe Tim was just having a dig at the fact the documentation was not changed, and also wearing his grumpy-conservative hat (well, it is election fever in the US!) But remember - the original question was if the new semantics should return the trailing "\\" as part of the common prefix, due to the demonstrated fact that at least _some_ code out there depends on it. Tim wanted a bug filed, but a few other people have chimed in saying nothing needs fixing. So what is it? Do I file the bug as Tim requested? Maybe I should just do it, and assign the bug to Guido - at least that way he can make a quick decision? At-least-my-code-works-again ly, Mark. From akuchlin@mems-exchange.org Fri Aug 18 00:27:06 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Thu, 17 Aug 2000 19:27:06 -0400 Subject: [Python-Dev] Cookie.py module, and Web PEP In-Reply-To: <14748.26274.949428.733639@anthem.concentric.net>; from bwarsaw@beopen.com on Thu, Aug 17, 2000 at 06:26:42PM -0400 References: <14748.21382.305979.784637@anthem.concentric.net> <20000817171352.B26730@kronos.cnri.reston.va.us> <14748.26274.949428.733639@anthem.concentric.net> Message-ID: <20000817192706.A28225@newcnri.cnri.reston.va.us> On Thu, Aug 17, 2000 at 06:26:42PM -0400, Barry A. Warsaw wrote: >Could you check and see if there are any substantial differences >between the version you've got and the version in the Mailman tree? >If there are none, then I'm +1. If you're referring to misc/Cookie.py in Mailman, the two files are vastly different (though not necessarily incompatible). The Mailman version derives from a version of Cookie.py dating from 1998, according to the CVS tree. Timo's current version has three different flavours of cookie, the Mailman version doesn't, so you wind up with a 1000-line long diff between the two. --amk From tim_one@email.msn.com Fri Aug 18 00:29:16 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 17 Aug 2000 19:29:16 -0400 Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: Message-ID: [Skip, as quoted by MarkH] > I don't realize that because a bug wasn't noticed for a long > time was any reason not to fix it. Guido was also involved in the > repair of the bug, and [MarkH] > I think most people agreed that the new semantics were preferable to the > old. I believe Tim was just having a dig at the fact the documentation > was not changed, and also wearing his grumpy-conservative hat (well, it is > election fever in the US!) Not at all, I meant it. When the code and the docs have matched for more than 6 years, there is no bug by any rational definition of the term, and you can be certain that changing the library semantics then will break existing code. Presuming to change it anyway is developer arrogance of the worst kind, no matter how many developers cheer it on. The docs are a contract, and if they were telling the truth, we have a responsibility to stand by them -- and whether we like it or not (granted, I am overly sensitive to contractual matters these days <0.3 wink>). The principled solution is to put the new functionality in a new function. Then nobody's code breaks, no user feels abused, and everyone gets what they want. If you despise what the old function did, that's fine too, deprecate it -- but don't screw people who were using it happily for what it was documented to do. > But remember - the original question was if the new semantics > should return the trailing "\\" as part of the common prefix, due > to the demonstrated fact that at least _some_ code out there > depends on it. > > Tim wanted a bug filed, but a few other people have chimed in saying > nothing needs fixing. > > So what is it? Do I file the bug as Tim requested? Maybe I should just > do it, and assign the bug to Guido - at least that way he can make a quick > decision? By my count, Unix and Windows people have each voted for both answers, and the Mac contingent is silently laughing . hell-stick-in-fifty-new-functions-if-that's-what-it-takes-but-leave- the-old-one-alone-ly y'rs - tim From gstein@lyra.org Fri Aug 18 00:41:37 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 17 Aug 2000 16:41:37 -0700 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: ; from tim_one@email.msn.com on Wed, Aug 16, 2000 at 11:34:12PM -0400 References: <20000816172425.A32338@ActiveState.com> Message-ID: <20000817164137.U17689@lyra.org> On Wed, Aug 16, 2000 at 11:34:12PM -0400, Tim Peters wrote: >... > So one of two things can be done: > > 1. Bite the bullet and do it correctly. For example, maintain a static > dict mapping the native pthread_self() return value to Python ints, > and return the latter as Python's thread.get_ident() value. Much > better would to implement a x-platform thread-local storage > abstraction, and use that to hold a Python-int ident value. > > 2. Continue in the tradition already established , and #ifdef the > snot out of it for Monterey. > > In favor of #2, the code is already so hosed that making it hosier won't be > a significant relative increase in its inherent hosiness. The x-plat thread-local storage idea is the best thing to do. That will be needed for some of the free-threading work in Python. IOW, an x-plat TLS is going to be done at some point. If you need it now, then please do it now. That will help us immeasurably in the long run. Cheers, -g -- Greg Stein, http://www.lyra.org/ From MarkH@ActiveState.com Fri Aug 18 00:59:18 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Fri, 18 Aug 2000 09:59:18 +1000 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: <20000817164137.U17689@lyra.org> Message-ID: > IOW, an x-plat TLS is going to be done at some point. If you need it now, > then please do it now. That will help us immeasurably in the long run. I just discovered the TLS code in the Mozilla source tree. This could be a good place to start. The definitions are in mozilla/nsprpub/pr/include/prthread.h, and I include some of this file below... I can confirm this code works _with_ Python - but I have no idea how hard it would be to distill it _into_ Python! Then-we-just-need-Tim-to-check-the-license-for-us ly, Mark. /* * The contents of this file are subject to the Netscape Public License * Version 1.1 (the "NPL"); you may not use this file except in * compliance with the NPL. You may obtain a copy of the NPL at * http://www.mozilla.org/NPL/ * [MarkH - it looks good to me - very open license] ... /* ** This routine returns a new index for per-thread-private data table. ** The index is visible to all threads within a process. This index can ** be used with the PR_SetThreadPrivate() and PR_GetThreadPrivate() routines ** to save and retrieve data associated with the index for a thread. ** ** Each index is associationed with a destructor function ('dtor'). The function ** may be specified as NULL when the index is created. If it is not NULL, the ** function will be called when: ** - the thread exits and the private data for the associated index ** is not NULL, ** - new thread private data is set and the current private data is ** not NULL. ** ** The index independently maintains specific values for each binding thread. ** A thread can only get access to its own thread-specific-data. ** ** Upon a new index return the value associated with the index for all threads ** is NULL, and upon thread creation the value associated with all indices for ** that thread is NULL. ** ** Returns PR_FAILURE if the total number of indices will exceed the maximun ** allowed. */ typedef void (PR_CALLBACK *PRThreadPrivateDTOR)(void *priv); NSPR_API(PRStatus) PR_NewThreadPrivateIndex( PRUintn *newIndex, PRThreadPrivateDTOR destructor); /* ** Define some per-thread-private data. ** "tpdIndex" is an index into the per-thread private data table ** "priv" is the per-thread-private data ** ** If the per-thread private data table has a previously registered ** destructor function and a non-NULL per-thread-private data value, ** the destructor function is invoked. ** ** This can return PR_FAILURE if the index is invalid. */ NSPR_API(PRStatus) PR_SetThreadPrivate(PRUintn tpdIndex, void *priv); /* ** Recover the per-thread-private data for the current thread. "tpdIndex" is ** the index into the per-thread private data table. ** ** The returned value may be NULL which is indistinguishable from an error ** condition. ** ** A thread can only get access to its own thread-specific-data. */ NSPR_API(void*) PR_GetThreadPrivate(PRUintn tpdIndex); From gstein@lyra.org Fri Aug 18 01:19:17 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 17 Aug 2000 17:19:17 -0700 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: ; from MarkH@ActiveState.com on Fri, Aug 18, 2000 at 09:59:18AM +1000 References: <20000817164137.U17689@lyra.org> Message-ID: <20000817171917.V17689@lyra.org> On Fri, Aug 18, 2000 at 09:59:18AM +1000, Mark Hammond wrote: > > IOW, an x-plat TLS is going to be done at some point. If you need it now, > > then please do it now. That will help us immeasurably in the long run. > > I just discovered the TLS code in the Mozilla source tree. This could be a > good place to start. > > The definitions are in mozilla/nsprpub/pr/include/prthread.h, and I include > some of this file below... I can confirm this code works _with_ Python - > but I have no idea how hard it would be to distill it _into_ Python! > > Then-we-just-need-Tim-to-check-the-license-for-us ly, The NPL is not compatible with the Python license. While we could use their API as a guide for our own code, we cannot use their code. The real question is whether somebody has the time/inclination to sit down now and write an x-plat TLS for Python. Always the problem :-) Cheers, -g -- Greg Stein, http://www.lyra.org/ From tim_one@email.msn.com Fri Aug 18 01:18:08 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 17 Aug 2000 20:18:08 -0400 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: Message-ID: [MarkH] > I just discovered the TLS code in the Mozilla source tree. This > could be a good place to start. > ... > Then-we-just-need-Tim-to-check-the-license-for-us ly, Jesus, Mark, I haven't even been able to figure what the license means by "you" yet: 1. Definitions ... 1.12. "You'' (or "Your") means an individual or a legal entity exercising rights under, and complying with all of the terms of, this License or a future version of this License issued under Section 6.1. For legal entities, "You'' includes any entity which controls, is controlled by, or is under common control with You. For purposes of this definition, "control'' means (a) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (b) ownership of more than fifty percent (50%) of the outstanding shares or beneficial ownership of such entity. at-least-they-left-little-doubt-about-the-meaning-of-"fifty-percent"-ly y'rs - tim (tee eye em, and neither you nor You. I think.) From bwarsaw@beopen.com Fri Aug 18 01:18:34 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Thu, 17 Aug 2000 20:18:34 -0400 (EDT) Subject: [Python-Dev] Cookie.py module, and Web PEP References: <14748.21382.305979.784637@anthem.concentric.net> <20000817171352.B26730@kronos.cnri.reston.va.us> <14748.26274.949428.733639@anthem.concentric.net> <20000817192706.A28225@newcnri.cnri.reston.va.us> Message-ID: <14748.32986.835733.255687@anthem.concentric.net> >>>>> "AK" == Andrew Kuchling writes: >> Could you check and see if there are any substantial >> differences between the version you've got and the version in >> the Mailman tree? If there are none, then I'm +1. AK> If you're referring to misc/Cookie.py in Mailman, That's the one. AK> the two files are vastly different (though not necessarily AK> incompatible). The Mailman version derives from a version of AK> Cookie.py dating from 1998, according to the CVS tree. Timo's AK> current version has three different flavours of cookie, the AK> Mailman version doesn't, so you wind up with a 1000-line long AK> diff between the two. Okay, don't sweat it. If the new version makes sense to you, I'll just be sure to make any Mailman updates that are necessary. I'll take a look once it's been checked in. -Barry From tim_one@email.msn.com Fri Aug 18 01:24:04 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 17 Aug 2000 20:24:04 -0400 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: <20000817171917.V17689@lyra.org> Message-ID: [Gret Stein] > The NPL is not compatible with the Python license. Or human comprehensibility either, far as I can tell. > While we could use their API as a guide for our own code, we cannot > use their code. > > The real question is whether somebody has the time/inclination to sit > down now and write an x-plat TLS for Python. Always the problem :-) The answer to Trent's original question is determined by whether he wants to get a Monterey hack in as a bugfix for 2.0, or can wait a few years <0.9 wink> (the 2.0 feature set is frozen now). If somebody wants to *buy* the time/inclination to get x-plat TLS, I'm sure BeOpen or ActiveState would be keen to cash the check. Otherwise ... don't know. all-it-takes-is-50-people-to-write-50-one-platform-packages-and- then-50-years-to-iron-out-their-differences-ly y'rs - tim From bwarsaw@beopen.com Fri Aug 18 01:26:10 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Thu, 17 Aug 2000 20:26:10 -0400 (EDT) Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements References: <20000817164137.U17689@lyra.org> <20000817171917.V17689@lyra.org> Message-ID: <14748.33442.7609.588513@anthem.concentric.net> >>>>> "GS" == Greg Stein writes: GS> The NPL is not compatible with the Python license. While we GS> could use their API as a guide for our own code, we cannot use GS> their code. >>>>> "TP" == Tim Peters writes: TP> Jesus, Mark, I haven't even been able to figure what the TP> license means by "you" yet: Is the NPL compatible with /anything/? :) -Barry From trentm@ActiveState.com Fri Aug 18 01:41:37 2000 From: trentm@ActiveState.com (Trent Mick) Date: Thu, 17 Aug 2000 17:41:37 -0700 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: <14748.33442.7609.588513@anthem.concentric.net>; from bwarsaw@beopen.com on Thu, Aug 17, 2000 at 08:26:10PM -0400 References: <20000817164137.U17689@lyra.org> <20000817171917.V17689@lyra.org> <14748.33442.7609.588513@anthem.concentric.net> Message-ID: <20000817174137.B18811@ActiveState.com> On Thu, Aug 17, 2000 at 08:26:10PM -0400, Barry A. Warsaw wrote: > > Is the NPL compatible with /anything/? :) > Mozilla will be dual licenced with the GPL. But you already read that. Trent -- Trent Mick TrentM@ActiveState.com From gstein@lyra.org Fri Aug 18 01:55:56 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 17 Aug 2000 17:55:56 -0700 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: <14748.33442.7609.588513@anthem.concentric.net>; from bwarsaw@beopen.com on Thu, Aug 17, 2000 at 08:26:10PM -0400 References: <20000817164137.U17689@lyra.org> <20000817171917.V17689@lyra.org> <14748.33442.7609.588513@anthem.concentric.net> Message-ID: <20000817175556.Y17689@lyra.org> On Thu, Aug 17, 2000 at 08:26:10PM -0400, Barry A. Warsaw wrote: > > >>>>> "GS" == Greg Stein writes: > > GS> The NPL is not compatible with the Python license. While we > GS> could use their API as a guide for our own code, we cannot use > GS> their code. > > >>>>> "TP" == Tim Peters writes: > > TP> Jesus, Mark, I haven't even been able to figure what the > TP> license means by "you" yet: > > Is the NPL compatible with /anything/? :) All kinds of stuff. It is effectively a non-viral GPL. Any changes to the NPL/MPL licensed stuff must be released. It does not affect the stuff that it is linked/dist'd with. However, I was talking about the Python source code base. The Python license and the NPL/MPL are definitely compatible. I mean that we don't want both licenses in the Python code base. Hmm. Should have phrased that differently. And one nit: the NPL is very different from the MPL. NPL x.x is nasty, while MPL 1.1 is very nice. Note the whole MPL/GPL dual-license stuff that you see (Expat and now Mozilla) is not because they are trying to be nice, but because they are trying to compensate for the GPL's nasty viral attitude. You cannot use MPL code in a GPL product because the *GPL* says so. The MPL would be perfectly happy, but no... Therefore, people dual-license so that you can choose the GPL when linking with GPL code. Ooops. I'll shut up now. :-) Cheers, -g -- Greg Stein, http://www.lyra.org/ From bwarsaw@beopen.com Fri Aug 18 01:49:17 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Thu, 17 Aug 2000 20:49:17 -0400 (EDT) Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements References: <20000817164137.U17689@lyra.org> <20000817171917.V17689@lyra.org> <14748.33442.7609.588513@anthem.concentric.net> <20000817174137.B18811@ActiveState.com> Message-ID: <14748.34829.130052.124407@anthem.concentric.net> >>>>> "TM" == Trent Mick writes: TM> Mozilla will be dual licenced with the GPL. But you already TM> read that. Yup, but it'll still be a big hurdle to include any GPL'd code in Python. -Barry From MarkH@ActiveState.com Fri Aug 18 01:55:02 2000 From: MarkH@ActiveState.com (Mark Hammond) Date: Fri, 18 Aug 2000 10:55:02 +1000 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: <20000817175556.Y17689@lyra.org> Message-ID: [Greg] > However, I was talking about the Python source code base. The > Python license > and the NPL/MPL are definitely compatible. Phew. Obviously IANAL, but I thought I was going senile. I didn't seek clarification for fear of further demonstrating my ignorance :-) > I mean that we don't want both licenses in the Python code base. That makes much more sense to me! Thanks for the clarification. Mark. From greg@cosc.canterbury.ac.nz Fri Aug 18 02:01:17 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Fri, 18 Aug 2000 13:01:17 +1200 (NZST) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <20000817090942.L376@xs4all.nl> Message-ID: <200008180101.NAA15496@s454.cosc.canterbury.ac.nz> Thomas Wouters: > Bzzzt. This is unfortunately not true. Observe: > > daemon2:~/python > rmdir perl/ > rmdir: perl/: Is a directory I'd say that's a bug in rmdir in whatever Unix you're using. Solaris doesn't have the problem: s454% cd ~/tmp s454% mkdir foo s454% rmdir foo/ s454% There's always room for a particular program to screw up. However, the usual principle in Unices is that trailing slashes are optional. > Note that the trailing slash is added by all tab-completing shells that I > know. This is for the convenience of the user, who is probably going to type another pathname component, and also to indicate that the object found is a directory. It makes sense in an interactive tool, but not necessarily in other places. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Fri Aug 18 02:27:33 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Fri, 18 Aug 2000 13:27:33 +1200 (NZST) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <1245608987-154490918@hypernet.com> Message-ID: <200008180127.NAA15502@s454.cosc.canterbury.ac.nz> Gordon: > os.chdir(os.pardir) Ah, I missed that somehow. Probably I was looking in os.path instead of os. Shouldn't everything to do with pathname semantics be in os.path? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Fri Aug 18 02:52:32 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Fri, 18 Aug 2000 13:52:32 +1200 (NZST) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <14747.55394.783997.167234@beluga.mojam.com> Message-ID: <200008180152.NAA15507@s454.cosc.canterbury.ac.nz> Skip: > Since we already have os.path.commonprefix and it's not going away, If it's to stay the way it is, we need another function to do what it should have been designed to do in the first place. That means two new functions, one to find a common prefix, and one to remove a given prefix. But it's not clear exactly what a function such as removeprefix(prefix, path) should do. What happens, for instance, if 'prefix' is not actually a prefix of 'path', or only part of it is a prefix? A reasonable definition might be that however much of 'prefix' is a prefix of 'path' is removed. But that requires finding the common prefix of the prefix and the path, which is intruding on commonprefix's territory! This is what led me to think of combining the two operations into one, which would have a clear, unambiguous definition covering all cases. > there's nothing in the name factorize that suggests that it would > split the paths at the common prefix. I'm not particularly attached to that name. Call it 'splitcommonprefix' or something if you like. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Fri Aug 18 03:02:09 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Fri, 18 Aug 2000 14:02:09 +1200 (NZST) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <14747.57236.264324.165612@beluga.mojam.com> Message-ID: <200008180202.OAA15511@s454.cosc.canterbury.ac.nz> Skip: > maybe we should move it to some other module > that has no directory path implications. I agree! > Perhaps string? Oh, that's deprecated. Is the whole string module deprecated, or only those parts which are now available as string methods? I think trying to eliminate the string module altogether would be a mistake, since it would leave nowhere for string operations that don't make sense as methods of a string. The current version of commonprefix is a case in point, since it operates symmetrically on a collection of strings. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From gmcm@hypernet.com Fri Aug 18 03:07:04 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Thu, 17 Aug 2000 22:07:04 -0400 Subject: [Python-Dev] 'import as' In-Reply-To: <20000817231942.O376@xs4all.nl> Message-ID: <1245558070-157553278@hypernet.com> Thomas Wouters wrote: > The other issue is the change in semantics for 'from-import'. Um, maybe I'm not seeing something, but isn't the effect of "import goom.bah as snarf" the same as "from goom import bah as snarf"? Both forms mean that we don't end up looking for (the aliased) bah in another namespace, (thus both forms fall prey to the circular import problem). Why not just disallow "from ... import ... as ..."? - Gordon From fdrake@beopen.com Fri Aug 18 03:13:25 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Thu, 17 Aug 2000 22:13:25 -0400 (EDT) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <200008180202.OAA15511@s454.cosc.canterbury.ac.nz> References: <14747.57236.264324.165612@beluga.mojam.com> <200008180202.OAA15511@s454.cosc.canterbury.ac.nz> Message-ID: <14748.39877.3411.744665@cj42289-a.reston1.va.home.com> Skip: > Perhaps string? Oh, that's deprecated. Greg Ewing writes: > Is the whole string module deprecated, or only those parts > which are now available as string methods? I think trying to I wasn't aware of any actual deprecation, just a shift of preference. There's not a notice of the deprecation in the docs. ;) In fact, there are things that are in the module that are not available as string methods. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From fdrake@beopen.com Fri Aug 18 03:38:06 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Thu, 17 Aug 2000 22:38:06 -0400 (EDT) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <200008180127.NAA15502@s454.cosc.canterbury.ac.nz> References: <1245608987-154490918@hypernet.com> <200008180127.NAA15502@s454.cosc.canterbury.ac.nz> Message-ID: <14748.41358.61606.202184@cj42289-a.reston1.va.home.com> Greg Ewing writes: > Gordon: > > > os.chdir(os.pardir) > > Ah, I missed that somehow. Probably I was looking in os.path > instead of os. > > Shouldn't everything to do with pathname semantics be in os.path? Should be, yes. I'd vote that curdir, pardir, sep, altsep, and pathsep be added to the *path modules, and os could pick them up from there. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From akuchlin@mems-exchange.org Fri Aug 18 03:46:32 2000 From: akuchlin@mems-exchange.org (A.M. Kuchling) Date: Thu, 17 Aug 2000 22:46:32 -0400 Subject: [Python-Dev] Request for help w/ bsddb module Message-ID: <20000817224632.A525@207-172-146-154.s154.tnt3.ann.va.dialup.rcn.com> [CC'ed to python-dev, python-list] I've started writing a straight C version of Greg Smith's BSDDB3 module (http://electricrain.com/greg/python/bsddb3/), which currently uses SWIG. The code isn't complete enough to do anything yet, though it does compile. Now I'm confronted with writing around 60 different methods for 3 different types; the job doesn't look difficult, but it does look tedious and lengthy. Since the task will parallelize well, I'm asking if anyone wants to help out by writing the code for one of the types. If you want to help, grab Greg's code from the above URL, and my incomplete module from ftp://starship.python.net/pub/crew/amk/new/_bsddb.c. Send me an e-mail telling me which set of methods (those for the DB, DBC, DB_Env types) you want to implement before starting to avoid duplicating work. I'll coordinate, and will debug the final product. (Can this get done in time for Python 2.0? Probably. Can it get tested in time for 2.0? Ummm....) --amk From greg@cosc.canterbury.ac.nz Fri Aug 18 03:45:46 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Fri, 18 Aug 2000 14:45:46 +1200 (NZST) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: Message-ID: <200008180245.OAA15517@s454.cosc.canterbury.ac.nz> Tim Peters: > The principled solution is to put the new functionality in a new > function. I agree with that. > By my count, Unix and Windows people have each voted for both answers, and > the Mac contingent is silently laughing . The Mac situation is somewhat complicated. Most of the time a single trailing colon makes no difference, but occasionally it does. For example, "abc" is a relative pathname, but "abc:" is an absolute pathname! The best way to resolve this, I think, is to decree that it should do the same as what os.path.split does, on all platforms. That function seems to know how to deal with all the tricky cases correctly. Don't-even-think-of-asking-about-VMS-ly, Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From fdrake@beopen.com Fri Aug 18 03:55:59 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Thu, 17 Aug 2000 22:55:59 -0400 (EDT) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <200008180245.OAA15517@s454.cosc.canterbury.ac.nz> References: <200008180245.OAA15517@s454.cosc.canterbury.ac.nz> Message-ID: <14748.42431.165537.946022@cj42289-a.reston1.va.home.com> Greg Ewing writes: > Don't-even-think-of-asking-about-VMS-ly, Really! I looked at some docs for the path names on that system, and didn't come away so much as convinced DEC/Compaq knew what they looked like. Or where they stopped. Or started. I think a fully general path algebra will be *really* hard to do, but it's something I've thought about a little. Don't know when I'll have time to dig back into it. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From greg@cosc.canterbury.ac.nz Fri Aug 18 03:57:34 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Fri, 18 Aug 2000 14:57:34 +1200 (NZST) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <399B94EB.E95260EE@lemburg.com> Message-ID: <200008180257.OAA15523@s454.cosc.canterbury.ac.nz> M.-A. Lemburg: > By dropping the trailing slash from the path > you are removing important information from the path information. No, you're not. A trailing slash on a Unix pathname doesn't tell you anything about whether it refers to a directory. Actually, it doesn't tell you anything at all. Slashes simply delimit pathname components, nothing more. A demonstration of this: s454% cat > foo/ asdf s454% cat foo/ asdf s454% A few utilites display pathnames with trailing slashes in order to indicate that they refer to directories, but that's a special convention confined to those tools. It doesn't apply in general. The only sure way to find out whether a given pathname refers to a directory or not is to ask the filesystem. And if the object referred to doesn't exist, the question of whether it's a directory is meaningless. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg@cosc.canterbury.ac.nz Fri Aug 18 04:34:57 2000 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Fri, 18 Aug 2000 15:34:57 +1200 (NZST) Subject: [Python-Dev] 'import as' In-Reply-To: <1245558070-157553278@hypernet.com> Message-ID: <200008180334.PAA15543@s454.cosc.canterbury.ac.nz> Gordon McMillan : > isn't the effect of "import goom.bah as snarf" the same as "from goom > import bah as snarf"? Only if goom.bah is a submodule or subpackage, I think. Otherwise "import goom.bah" doesn't work in the first place. I'm not sure that "import goom.bah as snarf" should be allowed, even if goom.bah is a module. Should the resulting object be referred to as snarf, snarf.bah or goom.snarf? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim_one@email.msn.com Fri Aug 18 04:39:29 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 17 Aug 2000 23:39:29 -0400 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: <20000817142207.A5592@ActiveState.com> Message-ID: [Trent Mick] > ... > I'm all for being a hoser then. Canadian . > #ifdef's a-comin' down the pipe. > One thing, the only #define that I know I have a handle on for > Monterey is '_LP64'. Do you have an objection to that (seeing at > is kind of misleading)? I will accompany it with an explicative > comment of course. Hmm! I hate "mystery #defines", even when they do make sense. In my last commerical project, we had a large set of #defines in its equivalent of pyport.h, along the lines of Py_COMPILER_MSVC, Py_COMPILER_GCC, Py_ARCH_X86, Py_ARCH_KATMAI, etc etc. Over time, *nobody* can remember what goofy combinations of mystery preprocessor symbols vendors define, and vendors come and go, and you're left with piles of code you can't make head or tail of. "#ifdef __SC__" -- what? So there was A Rule that vendor-supplied #defines could *only* appear in (that project's version of) pyport.h, used there to #define symbols whose purpose was clear from extensive comments and naming conventions. That proved to be an excellent idea over years of practice! So I'll force Python to do that someday too. In the meantime, _LP64 is a terrible name for this one, because its true *meaning* (the way you want to use it) appears to be "sizeof(pthread_t) < sizeof(long)", and that's certainly not a property of all LP64 platforms. So how about a runtime test for what's actually important (and it's not Monterey!)? if (sizeof(threadid) <= sizeof(long)) return (long)threadid; End of problem, right? It's a cheap runtime test in a function whose speed isn't critical anyway. And it will leave the God-awful casting to the one platform where it appears to be needed -- while also (I hope) making it clearer that that's absolutely the wrong thing to be doing on that platform (throwing away half the bits in the threadid value is certain to make get_ident return the same value for two distinct threads sooner or later ...). less-preprocessor-more-sense-ly y'rs - tim From tim_one@email.msn.com Fri Aug 18 04:58:13 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 17 Aug 2000 23:58:13 -0400 Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <200008180257.OAA15523@s454.cosc.canterbury.ac.nz> Message-ID: [Greg Ewing] > ... > A trailing slash on a Unix pathname doesn't tell you anything > about whether it refers to a directory. It does if it's also the only character in the pathname <0.5 wink>. The same thing bites people on Windows, except even worse, because in UNC pathnames the leading \\machine\volume "acts like a root", and the presence or absence of a trailing backslash there makes a world of difference too. > ... > The only sure way to find out whether a given pathname refers > to a directory or not is to ask the filesystem. On Windows again, >>> from os import path >>> path.exists("/python16") 1 >>> path.exists("/python16/") 0 >>> This insane behavior is displayed by the MS native APIs too, but isn't documented (at least not last time I peed away hours looking for it). just-more-evidence-that-windows-weenies-shouldn't-get-a-vote!-ly y'rs - tim From Moshe Zadka Fri Aug 18 05:39:18 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 18 Aug 2000 07:39:18 +0300 (IDT) Subject: [Python-Dev] Cookie.py module, and Web PEP In-Reply-To: Message-ID: On Thu, 17 Aug 2000, Andrew Kuchling wrote: > Tim O'Malley finally mailed me the correct URL for the latest version > of the cookie module: http://www.timo-tasi.org/python/Cookie.py > > *However*... I think the Web support in Python needs more work in > generally, and certainly more than can be done for 2.0. This is certainly true, but is that reason enough to keep Cookie.py out of 2.0? (+1 on enhancing the Python standard library, of course) -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From tim_one@email.msn.com Fri Aug 18 06:26:51 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 18 Aug 2000 01:26:51 -0400 Subject: indexing, indices(), irange(), list.items() (was RE: [Python-Dev] Lockstep iteration - eureka!) In-Reply-To: <399BE124.9920B0B6@prescod.net> Message-ID: Note that Guido rejected all the loop-gimmick proposals ("indexing", indices(), irange(), and list.items()) on Thursday, so let's stifle this debate until after 2.0 (or, even better, until after I'm dead ). hope-springs-eternal-ly y'rs - tim From tim_one@email.msn.com Fri Aug 18 06:43:14 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 18 Aug 2000 01:43:14 -0400 Subject: [Python-Dev] PyErr_NoMemory In-Reply-To: <200008171509.RAA20891@python.inrialpes.fr> Message-ID: [Vladimir Marangozov] > The current PyErr_NoMemory() function reads: > > PyObject * > PyErr_NoMemory(void) > { > /* raise the pre-allocated instance if it still exists */ > if (PyExc_MemoryErrorInst) > PyErr_SetObject(PyExc_MemoryError, PyExc_MemoryErrorInst); > else > /* this will probably fail since there's no > memory and hee, > hee, we have to instantiate this class > */ > PyErr_SetNone(PyExc_MemoryError); > > return NULL; > } > > thus overriding any previous exceptions unconditionally. This is a > problem when the current exception already *is* PyExc_MemoryError, > notably when we have a chain (cascade) of memory errors. It is a > problem because the original memory error and eventually its error > message is lost. > > I suggest to make this code look like: > > PyObject * > PyErr_NoMemory(void) > { > if (PyErr_ExceptionMatches(PyExc_MemoryError)) > /* already current */ > return NULL; > > /* raise the pre-allocated instance if it still exists */ > if (PyExc_MemoryErrorInst) > PyErr_SetObject(PyExc_MemoryError, PyExc_MemoryErrorInst); > ... > > > If nobody sees a problem with this, I'm very tempted to check it in. > Any objections? Looks good to me. And if it breaks something, it will be darned hard to tell . From nowonder@nowonder.de Fri Aug 18 09:06:23 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Fri, 18 Aug 2000 08:06:23 +0000 Subject: [Python-Dev] Re: indexing, indices(), irange(), list.items() (was RE: [Python-Dev] Lockstep iteration - eureka!) References: Message-ID: <399CEE7F.F2B865D2@nowonder.de> Tim Peters wrote: > > Note that Guido rejected all the loop-gimmick proposals ("indexing", > indices(), irange(), and list.items()) on Thursday, so let's stifle this > debate until after 2.0 (or, even better, until after I'm dead ). That's sad. :-/ One of the reasons I implemented .items() is that I wanted to increase the probability that at least *something* is available instead of: for i in range(len(list): e = list[i] ... or for i, e in zip(range(len(list)), list): ... I'm going to teach Python to a lot of newbies (ca. 30) in October. From my experience (I already tried my luck on two individuals from that group) 'range(len(list))' is one of the harder concepts to get across. Even indices(list) would help here. Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From Moshe Zadka Fri Aug 18 07:12:39 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 18 Aug 2000 09:12:39 +0300 (IDT) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <200008180127.NAA15502@s454.cosc.canterbury.ac.nz> Message-ID: On Fri, 18 Aug 2000, Greg Ewing wrote: > Gordon: > > > os.chdir(os.pardir) > > Ah, I missed that somehow. Probably I was looking in os.path > instead of os. > > Shouldn't everything to do with pathname semantics be in os.path? Nope, os.path is just for "abstract pathname algebra". Anything dealing with real paths on a real machine belongs elsewhere (os, probably) -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From tim_one@email.msn.com Fri Aug 18 07:30:40 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 18 Aug 2000 02:30:40 -0400 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: <20000817164137.U17689@lyra.org> Message-ID: [Greg Stein] > ... > IOW, an x-plat TLS is going to be done at some point. If you need it now, > then please do it now. That will help us immeasurably in the long run. It appears that a correct thread.get_ident() for DEC threads needed it 6 years ago (or at least that would have been-- and remains --the most elegant way to solve it). Trent doesn't need it to fix Monterey, though -- his only problem there is that the Alpha hack doesn't work on his platform, due to the former's utter bogosity. From Trent's POV, I bet the one-liner workaround sounds more appealing. From cgw@fnal.gov Fri Aug 18 08:01:59 2000 From: cgw@fnal.gov (Charles G Waldman) Date: Fri, 18 Aug 2000 02:01:59 -0500 (CDT) Subject: [Python-Dev] [Bug #111620] lots of use of send() without verifying amount of data sent Message-ID: <14748.57191.25642.168078@buffalo.fnal.gov> I'm jumping in late to this discussion to mention to mention that even for sockets in blocking mode, you can do sends with the MSG_DONTWAIT flag: sock.send(msg, socket.MSG_DONTWAIT) and this will send only as much data as can be written immediately. I.E., a per-message non-blocking write, without putting the socket into blocking mode. So if somebody decides to raise an exception on short TCP writes, they need to be aware of this. Personally I think it's a bad idea to be raising an exception at all for short writes. From thomas@xs4all.net Fri Aug 18 08:07:43 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 18 Aug 2000 09:07:43 +0200 Subject: [Python-Dev] 'import as' In-Reply-To: <1245558070-157553278@hypernet.com>; from gmcm@hypernet.com on Thu, Aug 17, 2000 at 10:07:04PM -0400 References: <20000817231942.O376@xs4all.nl> <1245558070-157553278@hypernet.com> Message-ID: <20000818090743.S376@xs4all.nl> On Thu, Aug 17, 2000 at 10:07:04PM -0400, Gordon McMillan wrote: > Thomas Wouters wrote: > > The other issue is the change in semantics for 'from-import'. > Um, maybe I'm not seeing something, but isn't the effect of "import > goom.bah as snarf" the same as "from goom import bah as snarf"? I don't understand what you're saying here. 'import goom.bah' imports goom, then bah, and the resulting module in the local namespace is 'goom'. That's existing behaviour (which I find perplexing, but I've never ran into before ;) which has changed in a reliable way: the local name being stored, whatever it would have been in a normal import, is changed into the "as-name" by "as ". If you're saying that 'import goom.bah.baz as b' won't do what people expect, I agree. (But neither does 'import goom.bah.baz', I think :-) > Both forms mean that we don't end up looking for (the aliased) bah in > another namespace, (thus both forms fall prey to the circular import > problem). Maybe it's the early hour, but I really don't understand the problem here. Ofcourse we end up looking 'bah' in the other namespace, we have to import it. And I don't know what it has to do with circular import either ;P > Why not just disallow "from ... import ... as ..."? That would kind of defeat the point of this change. I don't see any unexpected behaviour with 'from .. import .. as ..'; the object mentioned after 'import' and before 'as' is the object stored with the local name which follows 'as'. Why would we want to disallow that ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Fri Aug 18 08:17:03 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 18 Aug 2000 09:17:03 +0200 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: ; from tim_one@email.msn.com on Thu, Aug 17, 2000 at 11:39:29PM -0400 References: <20000817142207.A5592@ActiveState.com> Message-ID: <20000818091703.T376@xs4all.nl> On Thu, Aug 17, 2000 at 11:39:29PM -0400, Tim Peters wrote: > So how about a runtime test for what's actually important (and it's not > Monterey!)? > > if (sizeof(threadid) <= sizeof(long)) > return (long)threadid; > > End of problem, right? It's a cheap runtime test in a function whose speed > isn't critical anyway. Note that this is what autoconf is for. It also helps to group all that behaviour-testing code together, in one big lump noone pretends to understand ;) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From Fredrik Lundh" Message-ID: <001901c008e6$dc222760$f2a6b5d4@hagrid> thomas wrote: > I have two remaining issues regarding the 'import as' statement, which I'm > just about ready to commit. has this been tested with import hooks? what's passed to the __import__ function's fromlist argument if you do "from spam import egg as bacon"? From thomas@xs4all.net Fri Aug 18 08:30:49 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 18 Aug 2000 09:30:49 +0200 Subject: [Python-Dev] 'import as' In-Reply-To: <001901c008e6$dc222760$f2a6b5d4@hagrid>; from effbot@telia.com on Fri, Aug 18, 2000 at 09:35:17AM +0200 References: <20000817231942.O376@xs4all.nl> <001901c008e6$dc222760$f2a6b5d4@hagrid> Message-ID: <20000818093049.I27945@xs4all.nl> On Fri, Aug 18, 2000 at 09:35:17AM +0200, Fredrik Lundh wrote: > thomas wrote: > > I have two remaining issues regarding the 'import as' statement, which I'm > > just about ready to commit. > has this been tested with import hooks? Not really, I'm afraid. I don't know how to use import hooks ;-P But nothing substantial changed, and I took care to make sure 'find_from_args' gave the same information, still. For what it's worth, the test suite passed fine, but I don't know if there's a test for import hooks in there. > what's passed to the __import__ function's fromlist argument > if you do "from spam import egg as bacon"? The same as 'from spam import egg', currently. Better ideas are welcome, of course, especially if you know how to use import hooks, and how they generally are used. Pointers towards the right sections are also welcome. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From tim_one@email.msn.com Fri Aug 18 09:02:40 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 18 Aug 2000 04:02:40 -0400 Subject: indexing, indices(), irange(), list.items() (was RE: [Python-Dev] Lockstep iteration - eureka!) In-Reply-To: <399CEE7F.F2B865D2@nowonder.de> Message-ID: I'm stifling it, but, FWIW, I've been trying to sell "indexing" for most of my adult life . > -----Original Message----- > From: nowonder@stud.ntnu.no [mailto:nowonder@stud.ntnu.no]On Behalf Of > Peter Schneider-Kamp > Sent: Friday, August 18, 2000 4:06 AM > To: Tim Peters > Cc: python-dev@python.org > Subject: Re: indexing, indices(), irange(), list.items() (was RE: > [Python-Dev] Lockstep iteration - eureka!) > > > Tim Peters wrote: > > > > Note that Guido rejected all the loop-gimmick proposals ("indexing", > > indices(), irange(), and list.items()) on Thursday, so let's stifle this > > debate until after 2.0 (or, even better, until after I'm dead ). > > That's sad. :-/ > > One of the reasons I implemented .items() is that I wanted > to increase the probability that at least *something* is > available instead of: > > for i in range(len(list): > e = list[i] > ... > > or > > for i, e in zip(range(len(list)), list): > ... > > I'm going to teach Python to a lot of newbies (ca. 30) in > October. From my experience (I already tried my luck on two > individuals from that group) 'range(len(list))' is one > of the harder concepts to get across. Even indices(list) > would help here. > > Peter > -- > Peter Schneider-Kamp ++47-7388-7331 > Herman Krags veg 51-11 mailto:peter@schneider-kamp.de > N-7050 Trondheim http://schneider-kamp.de From mal@lemburg.com Fri Aug 18 09:05:30 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 18 Aug 2000 10:05:30 +0200 Subject: [Python-Dev] Re: os.path.commonprefix breakage References: <200008180101.NAA15496@s454.cosc.canterbury.ac.nz> Message-ID: <399CEE49.8E646DC3@lemburg.com> Greg Ewing wrote: > > > Note that the trailing slash is added by all tab-completing shells that I > > know. > > This is for the convenience of the user, who is probably going to type > another pathname component, and also to indicate that the object found > is a directory. It makes sense in an interactive tool, but not > necessarily in other places. Oh, C'mon Greg... haven't you read my reply to this ? The trailing slash contains important information which might otherwise not be regainable or only using explicit queries to the storage system. The "/" tells the program that the last path component is a directory. Removing the slash will also remove that information from the path (and yes: files without extension are legal). Now, since e.g. posixpath is also used as basis for fiddling with URLs and other tools using Unix style paths, removing the slash will result in problems... just look at what your browser does when you request http://www.python.org/search ... the server redirects you to search/ to make sure that the links embedded in the page are relative to search/ and not www.python.org/. Skip, have you already undone that change in CVS ? -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From tim_one@email.msn.com Fri Aug 18 09:10:01 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 18 Aug 2000 04:10:01 -0400 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: <20000818091703.T376@xs4all.nl> Message-ID: -0 on autoconf for this. I doubt that Trent ever needs to know more than in this one place the relative sizes of threadid and a long, and this entire function is braindead (hence will be gutted someday) anyway. Using the explicit test makes it obvious to everyone; winding back thru layers of autoconf crust makes it A Project and yet another goofy preprocessor symbol cluttering the code. > -----Original Message----- > From: Thomas Wouters [mailto:thomas@xs4all.net] > Sent: Friday, August 18, 2000 3:17 AM > To: Tim Peters > Cc: Trent Mick; python-dev@python.org > Subject: Re: [Python-Dev] pthreads question: typedef ??? pthread_t and > hacky return statements > > > On Thu, Aug 17, 2000 at 11:39:29PM -0400, Tim Peters wrote: > > > So how about a runtime test for what's actually important (and it's not > > Monterey!)? > > > > if (sizeof(threadid) <= sizeof(long)) > > return (long)threadid; > > > > End of problem, right? It's a cheap runtime test in a function > > whose speed isn't critical anyway. > > Note that this is what autoconf is for. It also helps to group all that > behaviour-testing code together, in one big lump noone pretends to > understand ;) > > -- > Thomas Wouters > > Hi! I'm a .signature virus! copy me into your .signature file to > help me spread! From mal@lemburg.com Fri Aug 18 09:30:51 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 18 Aug 2000 10:30:51 +0200 Subject: [Python-Dev] Re: indexing, indices(), irange(), list.items() (was RE: [Python-Dev] Lockstep iteration - eureka!) References: Message-ID: <399CF43A.478D7165@lemburg.com> Tim Peters wrote: > > Note that Guido rejected all the loop-gimmick proposals ("indexing", > indices(), irange(), and list.items()) on Thursday, so let's stifle this > debate until after 2.0 (or, even better, until after I'm dead ). Hey, we still have mxTools which gives you most of those goodies and lots more ;-) -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From nowonder@nowonder.de Fri Aug 18 12:07:43 2000 From: nowonder@nowonder.de (Peter Schneider-Kamp) Date: Fri, 18 Aug 2000 11:07:43 +0000 Subject: [Python-Dev] Re: indexing, indices(), irange(), list.items() (was RE: [Python-Dev] Lockstep iteration - eureka!) References: Message-ID: <399D18FF.BD807ED5@nowonder.de> What about 'indexing' xor 'in' ? Like that: for i indexing sequence: # good for e in sequence: # good for i indexing e in sequence: # BAD! This might help Guido to understand what it does in the 'indexing' case. I admit that the third one may be a bit harder to parse, so why not *leave it out*? But then I'm sure this has also been discussed before. Nevertheless I'll mail Barry and volunteer for a PEP on this. [Tim Peters about his life] > I've been trying to sell "indexing" for most of my adult life then-I'll-have-to-spend-another-life-on-it-ly y'rs Peter -- Peter Schneider-Kamp ++47-7388-7331 Herman Krags veg 51-11 mailto:peter@schneider-kamp.de N-7050 Trondheim http://schneider-kamp.de From sjoerd@oratrix.nl Fri Aug 18 10:42:38 2000 From: sjoerd@oratrix.nl (Sjoerd Mullender) Date: Fri, 18 Aug 2000 11:42:38 +0200 Subject: [Python-Dev] serious bug in new import X as Y code Message-ID: <20000818094239.A3A1931047C@bireme.oratrix.nl> Your changes for the import X as Y feature introduced a serious bug: I can no longer run Python at all. The problem is that in line 2150 of compile.c com_addopname is called with a NULL last argument, and the firat thing com_addopname does is indirect off of that very argument. On my machine (and on many other machines) that results in a core dump. In case it helps, here is the stack trace. The crash happens when importing site.py. I have not made any changes to my site.py. > 0 com_addopname(c = 0x7fff1e20, op = 90, n = (nil)) ["/ufs/sjoerd/src/Python/dist/src/Python/compile.c":738, 0x1006cb58] 1 com_import_stmt(c = 0x7fff1e20, n = 0x101e2ad0) ["/ufs/sjoerd/src/Python/dist/src/Python/compile.c":2150, 0x10071ecc] 2 com_node(c = 0x7fff1e20, n = 0x101e2ad0) ["/ufs/sjoerd/src/Python/dist/src/Python/compile.c":2903, 0x10074764] 3 com_node(c = 0x7fff1e20, n = 0x101eaf68) ["/ufs/sjoerd/src/Python/dist/src/Python/compile.c":2855, 0x10074540] 4 com_node(c = 0x7fff1e20, n = 0x101e2908) ["/ufs/sjoerd/src/Python/dist/src/Python/compile.c":2864, 0x100745b0] 5 com_node(c = 0x7fff1e20, n = 0x1020d450) ["/ufs/sjoerd/src/Python/dist/src/Python/compile.c":2855, 0x10074540] 6 com_file_input(c = 0x7fff1e20, n = 0x101e28f0) ["/ufs/sjoerd/src/Python/dist/src/Python/compile.c":3137, 0x10075324] 7 compile_node(c = 0x7fff1e20, n = 0x101e28f0) ["/ufs/sjoerd/src/Python/dist/src/Python/compile.c":3241, 0x100759c0] 8 jcompile(n = 0x101e28f0, filename = 0x7fff2430 = "./../Lib/site.py", base = (nil)) ["/ufs/sjoerd/src/Python/dist/src/Python/compile.c":3400, 0x10076058] 9 PyNode_Compile(n = 0x101e28f0, filename = 0x7fff2430 = "./../Lib/site.py") ["/ufs/sjoerd/src/Python/dist/src/Python/compile.c":3378, 0x10075f7c] 10 parse_source_module(pathname = 0x7fff2430 = "./../Lib/site.py", fp = 0xfb563c8) ["/ufs/sjoerd/src/Python/dist/src/Python/import.c":632, 0x100151a4] 11 load_source_module(name = 0x7fff28d8 = "site", pathname = 0x7fff2430 = "./../Lib/site.py", fp = 0xfb563c8) ["/ufs/sjoerd/src/Python/dist/src/Python/import.c":722, 0x100154c8] 12 load_module(name = 0x7fff28d8 = "site", fp = 0xfb563c8, buf = 0x7fff2430 = "./../Lib/site.py", type = 1) ["/ufs/sjoerd/src/Python/dist/src/Python/import.c":1199, 0x1001629c] 13 import_submodule(mod = 0x101b8478, subname = 0x7fff28d8 = "site", fullname = 0x7fff28d8 = "site") ["/ufs/sjoerd/src/Python/dist/src/Python/import.c":1727, 0x10017dc4] 14 load_next(mod = 0x101b8478, altmod = 0x101b8478, p_name = 0x7fff2d04, buf = 0x7fff28d8 = "site", p_buflen = 0x7fff28d0) ["/ufs/sjoerd/src/Python/dist/src/Python/import.c":1583, 0x100174c0] 15 import_module_ex(name = (nil), globals = (nil), locals = (nil), fromlist = (nil)) ["/ufs/sjoerd/src/Python/dist/src/Python/import.c":1434, 0x10016d04] 16 PyImport_ImportModuleEx(name = 0x101d9450 = "site", globals = (nil), locals = (nil), fromlist = (nil)) ["/ufs/sjoerd/src/Python/dist/src/Python/import.c":1475, 0x10016fe0] 17 PyImport_ImportModule(name = 0x101d9450 = "site") ["/ufs/sjoerd/src/Python/dist/src/Python/import.c":1408, 0x10016c64] 18 initsite() ["/ufs/sjoerd/src/Python/dist/src/Python/pythonrun.c":429, 0x10053148] 19 Py_Initialize() ["/ufs/sjoerd/src/Python/dist/src/Python/pythonrun.c":166, 0x100529c8] 20 Py_Main(argc = 1, argv = 0x7fff2ec4) ["/ufs/sjoerd/src/Python/dist/src/Modules/main.c":229, 0x10013690] 21 main(argc = 1, argv = 0x7fff2ec4) ["/ufs/sjoerd/src/Python/dist/src/Modules/python.c":10, 0x10012f24] 22 __start() ["/xlv55/kudzu-apr12/work/irix/lib/libc/libc_n32_M4/csu/crt1text.s":177, 0x10012ec8] -- Sjoerd Mullender From fredrik@pythonware.com Fri Aug 18 11:42:54 2000 From: fredrik@pythonware.com (Fredrik Lundh) Date: Fri, 18 Aug 2000 12:42:54 +0200 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements References: <20000816172425.A32338@ActiveState.com> Message-ID: <003001c00901$11fd8ae0$0900a8c0@SPIFF> trent mick wrote: > return (long) *(long *) &threadid; from what I can tell, pthread_t is a pointer under OSF/1. I've been using OSF/1 since the early days, and as far as I can remember, you've never needed to use stupid hacks like that to convert a pointer to a long integer. an ordinary (long) cast should be sufficient. > Could this be changed to > return threadid; > safely? safely, yes. but since it isn't a long on all platforms, you might get warnings from the compiler (see Mark's mail). ::: from what I can tell, it's compatible with a long on all sane plat- forms (Win64 doesn't support pthreads anyway ;-), so I guess the right thing here is to remove volatile and simply use: return (long) threadid; (Mark: can you try this out on your box? setting up a Python 2.0 environment on our alphas would take more time than I can spare right now...) From gstein@lyra.org Fri Aug 18 12:00:34 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 18 Aug 2000 04:00:34 -0700 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: ; from tim_one@email.msn.com on Fri, Aug 18, 2000 at 04:10:01AM -0400 References: <20000818091703.T376@xs4all.nl> Message-ID: <20000818040034.F17689@lyra.org> That is a silly approach, Tim. This is exactly what autoconf is for. Using run-time logic to figure out something that is compile-time is Badness. And the "but it will eventually be fixed" rationale is bogus. Gee, should we just start loading bogus patches into Python, knowing that everything will be fixed in the next version? Whoops. We forgot some. Oh, we can't change those now. Well, gee. Maybe Py3K will fix it. I realize that you're only -0 on this, but it should be at least +0... Cheers, -g On Fri, Aug 18, 2000 at 04:10:01AM -0400, Tim Peters wrote: > -0 on autoconf for this. > > I doubt that Trent ever needs to know more than in this one place the > relative sizes of threadid and a long, and this entire function is braindead > (hence will be gutted someday) anyway. Using the explicit test makes it > obvious to everyone; winding back thru layers of autoconf crust makes it A > Project and yet another goofy preprocessor symbol cluttering the code. > > > -----Original Message----- > > From: Thomas Wouters [mailto:thomas@xs4all.net] > > Sent: Friday, August 18, 2000 3:17 AM > > To: Tim Peters > > Cc: Trent Mick; python-dev@python.org > > Subject: Re: [Python-Dev] pthreads question: typedef ??? pthread_t and > > hacky return statements > > > > > > On Thu, Aug 17, 2000 at 11:39:29PM -0400, Tim Peters wrote: > > > > > So how about a runtime test for what's actually important (and it's not > > > Monterey!)? > > > > > > if (sizeof(threadid) <= sizeof(long)) > > > return (long)threadid; > > > > > > End of problem, right? It's a cheap runtime test in a function > > > whose speed isn't critical anyway. > > > > Note that this is what autoconf is for. It also helps to group all that > > behaviour-testing code together, in one big lump noone pretends to > > understand ;) > > > > -- > > Thomas Wouters > > > > Hi! I'm a .signature virus! copy me into your .signature file to > > help me spread! > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://www.python.org/mailman/listinfo/python-dev -- Greg Stein, http://www.lyra.org/ From gmcm@hypernet.com Fri Aug 18 13:35:42 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Fri, 18 Aug 2000 08:35:42 -0400 Subject: [Python-Dev] 'import as' In-Reply-To: <20000818090743.S376@xs4all.nl> References: <1245558070-157553278@hypernet.com>; from gmcm@hypernet.com on Thu, Aug 17, 2000 at 10:07:04PM -0400 Message-ID: <1245520353-159821909@hypernet.com> Thomas Wouters wrote: > On Thu, Aug 17, 2000 at 10:07:04PM -0400, Gordon McMillan wrote: > > Um, maybe I'm not seeing something, but isn't the effect of > > "import goom.bah as snarf" the same as "from goom import bah as > > snarf"? > > I don't understand what you're saying here. 'import goom.bah' > imports goom, then bah, and the resulting module in the local > namespace is 'goom'. That's existing behaviour (which I find > perplexing, but I've never ran into before ;) which has changed > in a reliable way: the local name being stored, whatever it would > have been in a normal import, is changed into the "as-name" by > "as ". A whole lot rides on what you mean by "resulting" above. If by "resulting" you mean "goom", then "import goom.bah as snarf" would result in my namespace having "snarf" as an alias for "goom", and I would use "bah" as "snarf.bah". In which case Greg Ewing is right, and it's "import as ..." that should be outlawed, (since that's not what anyone would expect). OTOH, if by "resulting" you meant "bah", things are much worse, because it means you must patched code you didn't understand ;-b. > If you're saying that 'import goom.bah.baz as b' won't do what > people expect, I agree. (But neither does 'import goom.bah.baz', > I think :-) I disagree with paranthetical comment. Maybe some don't understand the first time they see it, but it has precedent (Java), and it's the only form that works in circular imports. > Maybe it's the early hour, but I really don't understand the > problem here. Ofcourse we end up looking 'bah' in the other > namespace, we have to import it. And I don't know what it has to > do with circular import either ;P "goom.bah" ends up looking in "goom" when *used*. If, in a circular import situation, "goom" is already being imported, an "import goom.bah" will succeed, even though it can't access "bah" in "goom". The code sees it in sys.modules, sees that it's being imported, and says, "Oh heck, lets keep going, it'll be there before it gets used". But "from goom import bah" will fail with an import error because "goom" is an empty shell, so there's no way to grab "bah" and bind it into the local namespace. - Gordon From bwarsaw@beopen.com Fri Aug 18 14:27:05 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Fri, 18 Aug 2000 09:27:05 -0400 (EDT) Subject: [Python-Dev] 'import as' References: <1245558070-157553278@hypernet.com> <1245520353-159821909@hypernet.com> Message-ID: <14749.14761.275554.898385@anthem.concentric.net> >>>>> "Gordo" == Gordon McMillan writes: Gordo> A whole lot rides on what you mean by "resulting" above. If Gordo> by "resulting" you mean "goom", then "import goom.bah as Gordo> snarf" would result in my namespace having "snarf" as an Gordo> alias for "goom", and I would use "bah" as "snarf.bah". In Gordo> which case Greg Ewing is right, and it's "import name> as ..." that should be outlawed, (since that's not Gordo> what anyone would expect). Right. Gordo> OTOH, if by "resulting" you meant "bah", things are much Gordo> worse, because it means you must patched code you didn't Gordo> understand ;-b. But I think it /is/ useful behavior for "import as" to bind the rightmost attribute to the local name. I agree though that if that can't be done in a sane way, it has to raise an exception. But that will frustrate users. -Barry From gmcm@hypernet.com Fri Aug 18 14:28:00 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Fri, 18 Aug 2000 09:28:00 -0400 Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <399CEE49.8E646DC3@lemburg.com> Message-ID: <1245517214-160010723@hypernet.com> M.-A. Lemburg wrote: > ... just look at what your browser does > when you request http://www.python.org/search ... the server > redirects you to search/ to make sure that the links embedded in > the page are relative to search/ and not www.python.org/. While that seems to be what Apache does, I get 40x's from IIS and Netscape server. Greg Ewing's demonstrated a Unix where the trailing slash indicates nothing useful, Tim's demonstrated that Windows gets confused by a trailing slash unless we're talking about the root directory on a drive (and BTW, same results if you use backslash). On WIndows, os.path.commonprefix doesn't use normcase and normpath, so it's completely useless anyway. (That is, it's really a "string" function and has nothing to do with paths). - Gordon From nascheme@enme.ucalgary.ca Fri Aug 18 14:55:41 2000 From: nascheme@enme.ucalgary.ca (Neil Schemenauer) Date: Fri, 18 Aug 2000 07:55:41 -0600 Subject: [Python-Dev] Re: indexing, indices(), irange(), list.items() (was RE: [Python-Dev] Lockstep iteration - eureka!) In-Reply-To: <399CF43A.478D7165@lemburg.com>; from M.-A. Lemburg on Fri, Aug 18, 2000 at 10:30:51AM +0200 References: <399CF43A.478D7165@lemburg.com> Message-ID: <20000818075541.A14919@keymaster.enme.ucalgary.ca> On Fri, Aug 18, 2000 at 10:30:51AM +0200, M.-A. Lemburg wrote: > Hey, we still have mxTools which gives you most of those goodies > and lots more ;-) Yes, I don't understand what's wrong with a function. It would be nice if it was a builtin. IMHO, all this new syntax is a bad idea. Neil From fdrake@beopen.com Fri Aug 18 15:12:24 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Fri, 18 Aug 2000 10:12:24 -0400 (EDT) Subject: indexing, indices(), irange(), list.items() (was RE: [Python-Dev] Lockstep iteration - eureka!) In-Reply-To: References: <399CEE7F.F2B865D2@nowonder.de> Message-ID: <14749.17480.153311.549655@cj42289-a.reston1.va.home.com> Tim Peters writes: > I'm stifling it, but, FWIW, I've been trying to sell "indexing" for most of > my adult life extraordinarly hard to get across to newbies at first; and I *expect* > [:len(seq)] to be at least as hard>. And "for i indexing o in ...:" is the best proposal I've seen to resolve the whole problem in what *I* would describe as a Pythonic way. And it's not a new proposal. I haven't read the specific patch, but bugs can be fixed. I guess a lot of us will just have to disagree with the Guido on this one. ;-( Linguistic coup, anyone? ;-) -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From thomas@xs4all.net Fri Aug 18 15:17:46 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 18 Aug 2000 16:17:46 +0200 Subject: [Python-Dev] serious bug in new import X as Y code In-Reply-To: <20000818094239.A3A1931047C@bireme.oratrix.nl>; from sjoerd@oratrix.nl on Fri, Aug 18, 2000 at 11:42:38AM +0200 References: <20000818094239.A3A1931047C@bireme.oratrix.nl> Message-ID: <20000818161745.U376@xs4all.nl> On Fri, Aug 18, 2000 at 11:42:38AM +0200, Sjoerd Mullender wrote: > Your changes for the import X as Y feature introduced a serious bug: > I can no longer run Python at all. > The problem is that in line 2150 of compile.c com_addopname is called > with a NULL last argument, and the firat thing com_addopname does is > indirect off of that very argument. On my machine (and on many other > machines) that results in a core dump. Hm. That's very strange. Line 2150 of compile.c calls com_addopname with 'CHILD(CHILD(subn, 0), 0)' as argument. 'subn' is supposed to be a 'dotted_as_name', which always has at least one child (a dotted_name), which also always has at least one child (a NAME). I don't see how dotted_as_name and dotted_name can be valid nodes, but the first child of dotted_name be NULL. Can you confirm that the tree is otherwise unmodified ? If you have local patches, can you try to compile and test a 'clean' tree ? I can't reproduce this on the machines I have access to, so if you could find out what statement exactly is causing this behaviour, I'd be very grateful. Something like this should do the trick, changing: } else com_addopname(c, STORE_NAME, CHILD(CHILD(subn, 0),0)); into } else { if (CHILD(CHILD(subn, 0), 0) == NULL) { com_error(c, PyExc_SyntaxError, "NULL name for import"); return; } com_addopname(c, STORE_NAME, CHILD(CHILD(subn, 0),0)); } And then recompile, and remove site.pyc if there is one. (Unlikely, if a crash occured while compiling site.py, but possible.) This should raise a SyntaxError on or about the appropriate line, at least identifying what the problem *could* be ;) If that doesn't yield anything obvious, and you have the time for it (and want to do it), some 'print' statements in the debugger might help. (I'm assuming it works more or less like GDB, in which case 'print n', 'print n->n_child[1]', 'print subn', 'print subn->n_child[0]' and 'print subn->n_child[1]' would be useful. I'm also assuming there isn't an easier way to debug this, like you sending me a corefile, because corefiles normally aren't very portable :P If it *is* portable, that'd be great.) > In case it helps, here is the stack trace. The crash happens when > importing site.py. I have not made any changes to my site.py. Oh, it's probably worth it to re-make the Grammar (just to be sure) and remove Lib/*.pyc. The bytecode magic changes in the patch, so that last measure should be unecessary, but who knows :P breaky-breaky-ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From fdrake@beopen.com Fri Aug 18 15:21:20 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Fri, 18 Aug 2000 10:21:20 -0400 (EDT) Subject: [Python-Dev] Re: indexing, indices(), irange(), list.items() (was RE: [Python-Dev] Lockstep iteration - eureka!) In-Reply-To: <399D18FF.BD807ED5@nowonder.de> References: <399D18FF.BD807ED5@nowonder.de> Message-ID: <14749.18016.323403.295212@cj42289-a.reston1.va.home.com> Peter Schneider-Kamp writes: > What about 'indexing' xor 'in' ? Like that: > > for i indexing sequence: # good > for e in sequence: # good > for i indexing e in sequence: # BAD! > > This might help Guido to understand what it does in the > 'indexing' case. I admit that the third one may be a > bit harder to parse, so why not *leave it out*? I hadn't considered *not* using an "in" clause, but that is actually pretty neat. I'd like to see all of these allowed; disallowing "for i indexing e in ...:" reduces the intended functionality substantially. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From gmcm@hypernet.com Fri Aug 18 15:42:20 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Fri, 18 Aug 2000 10:42:20 -0400 Subject: [Python-Dev] 'import as' In-Reply-To: <14749.14761.275554.898385@anthem.concentric.net> Message-ID: <1245512755-160278942@hypernet.com> Barry "5 String" Warsaw wrote: > But I think it /is/ useful behavior for "import as" > to bind the rightmost attribute to the local name. That is certainly what I would expect (and thus the confusion created by my original post). > I agree > though that if that can't be done in a sane way, it has to raise > an exception. But that will frustrate users. "as" is minorly useful in dealing with name clashes between packages, and with reallyreallylongmodulename. Unfortunately, it's also yet another way of screwing up circular imports and reloads, (unless you go whole hog, and use Jim Fulton's idea of an assoctiation object). Then there's all the things that can go wrong with relative imports (loading the same module twice; masking anything outside the package with the same name). It's not surprising that most import problems posted to c.l.py get more wrong answers than right. Unfortunately, there's no way to fix it in a backwards compatible way. So I'm -0: it just adds complexity to an already overly complex area. - Gordon From bwarsaw@beopen.com Fri Aug 18 15:55:18 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Fri, 18 Aug 2000 10:55:18 -0400 (EDT) Subject: [Python-Dev] Re: indexing, indices(), irange(), list.items() (was RE: [Python-Dev] Lockstep iteration - eureka!) References: <399CF43A.478D7165@lemburg.com> <20000818075541.A14919@keymaster.enme.ucalgary.ca> Message-ID: <14749.20054.495550.467507@anthem.concentric.net> >>>>> "NS" == Neil Schemenauer writes: NS> On Fri, Aug 18, 2000 at 10:30:51AM +0200, M.-A. Lemburg wrote: >> Hey, we still have mxTools which gives you most of those >> goodies and lots more ;-) NS> Yes, I don't understand what's wrong with a function. It NS> would be nice if it was a builtin. IMHO, all this new syntax NS> is a bad idea. I agree, but Guido nixed even the builtin. Let's move on; there's always Python 2.1. -Barry From akuchlin@mems-exchange.org Fri Aug 18 16:00:37 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Fri, 18 Aug 2000 11:00:37 -0400 Subject: [Python-Dev] Adding insint() function Message-ID: <20000818110037.C27419@kronos.cnri.reston.va.us> Four modules define insint() functions to insert an integer into a dictionary in order to initialize constants in their module dictionaries: kronos Modules>grep -l insint *.c pcremodule.c shamodule.c socketmodule.c zlibmodule.c kronos Modules> (Hm... I was involved with 3 of them...) Other modules don't use a helper function, but just do PyDict_SetItemString(d, "foo", PyInt_FromLong(...)) directly. This duplication bugs me. Shall I submit a patch to add an API convenience function to do this, and change the modules to use it? Suggested prototype and name: PyDict_InsertInteger( dict *, string, long) --amk From bwarsaw@beopen.com Fri Aug 18 16:06:11 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Fri, 18 Aug 2000 11:06:11 -0400 (EDT) Subject: [Python-Dev] 'import as' References: <14749.14761.275554.898385@anthem.concentric.net> <1245512755-160278942@hypernet.com> Message-ID: <14749.20707.347217.763385@anthem.concentric.net> >>>>> "Gordo" == Gordon "Punk Cellist" McMillan writes: Gordo> So I'm -0: it just adds complexity to an already overly Gordo> complex area. I agree, -0 from me too. From sjoerd@oratrix.nl Fri Aug 18 16:06:37 2000 From: sjoerd@oratrix.nl (Sjoerd Mullender) Date: Fri, 18 Aug 2000 17:06:37 +0200 Subject: [Python-Dev] serious bug in new import X as Y code In-Reply-To: Your message of Fri, 18 Aug 2000 16:17:46 +0200. <20000818161745.U376@xs4all.nl> References: <20000818094239.A3A1931047C@bireme.oratrix.nl> <20000818161745.U376@xs4all.nl> Message-ID: <20000818150639.6685C31047C@bireme.oratrix.nl> Ok, problem solved. The problem was that because of your (I think it was you :-) earlier change to have a Makefile in Grammar, I had an old graminit.c lying around in my build directory. I don't build in the source directory and the changes for a Makefile in Grammar resulted in a file graminit.c in the wrong place. My subsequent change to this part of the build process resulted in a different place for graminit.c and I never removed the bogus graminit.c (because I didn't know about it). However, the compiler found the bogus one so that's why python crashed. On Fri, Aug 18 2000 Thomas Wouters wrote: > On Fri, Aug 18, 2000 at 11:42:38AM +0200, Sjoerd Mullender wrote: > > > Your changes for the import X as Y feature introduced a serious bug: > > I can no longer run Python at all. > > > The problem is that in line 2150 of compile.c com_addopname is called > > with a NULL last argument, and the firat thing com_addopname does is > > indirect off of that very argument. On my machine (and on many other > > machines) that results in a core dump. > > Hm. That's very strange. Line 2150 of compile.c calls com_addopname with > 'CHILD(CHILD(subn, 0), 0)' as argument. 'subn' is supposed to be a > 'dotted_as_name', which always has at least one child (a dotted_name), which > also always has at least one child (a NAME). I don't see how dotted_as_name > and dotted_name can be valid nodes, but the first child of dotted_name be > NULL. > > Can you confirm that the tree is otherwise unmodified ? If you have local > patches, can you try to compile and test a 'clean' tree ? I can't reproduce > this on the machines I have access to, so if you could find out what > statement exactly is causing this behaviour, I'd be very grateful. Something > like this should do the trick, changing: > > } else > com_addopname(c, STORE_NAME, > CHILD(CHILD(subn, 0),0)); > > into > > } else { > if (CHILD(CHILD(subn, 0), 0) == NULL) { > com_error(c, PyExc_SyntaxError, > "NULL name for import"); > return; > } > com_addopname(c, STORE_NAME, > CHILD(CHILD(subn, 0),0)); > } > > And then recompile, and remove site.pyc if there is one. (Unlikely, if a > crash occured while compiling site.py, but possible.) This should raise a > SyntaxError on or about the appropriate line, at least identifying what the > problem *could* be ;) > > If that doesn't yield anything obvious, and you have the time for it (and > want to do it), some 'print' statements in the debugger might help. (I'm > assuming it works more or less like GDB, in which case 'print n', 'print > n->n_child[1]', 'print subn', 'print subn->n_child[0]' and 'print > subn->n_child[1]' would be useful. I'm also assuming there isn't an easier > way to debug this, like you sending me a corefile, because corefiles > normally aren't very portable :P If it *is* portable, that'd be great.) > > > In case it helps, here is the stack trace. The crash happens when > > importing site.py. I have not made any changes to my site.py. > > Oh, it's probably worth it to re-make the Grammar (just to be sure) and > remove Lib/*.pyc. The bytecode magic changes in the patch, so that last > measure should be unecessary, but who knows :P > > breaky-breaky-ly y'rs, > -- > Thomas Wouters > > Hi! I'm a .signature virus! copy me into your .signature file to help me spread! > -- Sjoerd Mullender From bwarsaw@beopen.com Fri Aug 18 16:27:41 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Fri, 18 Aug 2000 11:27:41 -0400 (EDT) Subject: [Python-Dev] Adding insint() function References: <20000818110037.C27419@kronos.cnri.reston.va.us> Message-ID: <14749.21997.872741.463566@anthem.concentric.net> >>>>> "AK" == Andrew Kuchling writes: AK> Four modules define insint() functions to insert an integer AK> into a dictionary in order to initialize constants in their AK> module dictionaries: | kronos Modules>grep -l insint *.c | pcremodule.c | shamodule.c | socketmodule.c | zlibmodule.c | kronos Modules> AK> (Hm... I was involved with 3 of them...) Other modules don't AK> use a helper function, but just do PyDict_SetItemString(d, AK> "foo", PyInt_FromLong(...)) directly. AK> This duplication bugs me. Shall I submit a patch to add an AK> API convenience function to do this, and change the modules to AK> use it? Suggested prototype and name: PyDict_InsertInteger( AK> dict *, string, long) +0, but it should probably be called PyDict_SetItemSomething(). It seems more related to the other PyDict_SetItem*() functions, even though in those cases the `*' refers to the type of the key, not the value. -Barry From akuchlin@mems-exchange.org Fri Aug 18 16:29:47 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Fri, 18 Aug 2000 11:29:47 -0400 Subject: [Python-Dev] Adding insint() function In-Reply-To: <14749.21997.872741.463566@anthem.concentric.net>; from bwarsaw@beopen.com on Fri, Aug 18, 2000 at 11:27:41AM -0400 References: <20000818110037.C27419@kronos.cnri.reston.va.us> <14749.21997.872741.463566@anthem.concentric.net> Message-ID: <20000818112947.F27419@kronos.cnri.reston.va.us> On Fri, Aug 18, 2000 at 11:27:41AM -0400, Barry A. Warsaw wrote: >+0, but it should probably be called PyDict_SetItemSomething(). It >seems more related to the other PyDict_SetItem*() functions, even >though in those cases the `*' refers to the type of the key, not the >value. PyDict_SetItemInteger seems misleading; PyDict_SetItemStringToInteger is simply long. PyDict_SetIntegerItem, maybe? :) Anyway, I'll start working on a patch and change the name later once there's a good suggestion. --amk From mal@lemburg.com Fri Aug 18 16:41:14 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 18 Aug 2000 17:41:14 +0200 Subject: [Python-Dev] Re: os.path.commonprefix breakage References: <1245517214-160010723@hypernet.com> Message-ID: <399D591A.F909CF9D@lemburg.com> Gordon McMillan wrote: > > M.-A. Lemburg wrote: > > > ... just look at what your browser does > > when you request http://www.python.org/search ... the server > > redirects you to search/ to make sure that the links embedded in > > the page are relative to search/ and not www.python.org/. > > While that seems to be what Apache does, I get 40x's from > IIS and Netscape server. Greg Ewing's demonstrated a Unix > where the trailing slash indicates nothing useful, Tim's > demonstrated that Windows gets confused by a trailing slash > unless we're talking about the root directory on a drive (and > BTW, same results if you use backslash). > > On WIndows, os.path.commonprefix doesn't use normcase > and normpath, so it's completely useless anyway. (That is, it's > really a "string" function and has nothing to do with paths). I still don't get it: what's the point in carelessly dropping valid and useful information for no obvious reason at all ? Besides the previous behaviour was documented and most probably used in some apps. Why break those ? And last not least: what if the directory in question doesn't even exist anywhere and is only encoded in the path by the fact that there is a slash following it ? Puzzled by needless discussions ;-), -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From fdrake@beopen.com Fri Aug 18 16:42:59 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Fri, 18 Aug 2000 11:42:59 -0400 (EDT) Subject: [Python-Dev] Adding insint() function In-Reply-To: <14749.21997.872741.463566@anthem.concentric.net> References: <20000818110037.C27419@kronos.cnri.reston.va.us> <14749.21997.872741.463566@anthem.concentric.net> Message-ID: <14749.22915.717712.613834@cj42289-a.reston1.va.home.com> Barry A. Warsaw writes: > +0, but it should probably be called PyDict_SetItemSomething(). It > seems more related to the other PyDict_SetItem*() functions, even > though in those cases the `*' refers to the type of the key, not the > value. Hmm... How about PyDict_SetItemStringInt() ? It's still long, but I don't think that's actually a problem. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From thomas@xs4all.net Fri Aug 18 17:22:46 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 18 Aug 2000 18:22:46 +0200 Subject: [Python-Dev] 'import as' In-Reply-To: <14749.20707.347217.763385@anthem.concentric.net>; from bwarsaw@beopen.com on Fri, Aug 18, 2000 at 11:06:11AM -0400 References: <14749.14761.275554.898385@anthem.concentric.net> <1245512755-160278942@hypernet.com> <14749.20707.347217.763385@anthem.concentric.net> Message-ID: <20000818182246.V376@xs4all.nl> On Fri, Aug 18, 2000 at 11:06:11AM -0400, Barry A. Warsaw wrote: > Gordo> So I'm -0: it just adds complexity to an already overly > Gordo> complex area. > I agree, -0 from me too. What are we voting on, here ? import as (in general) or import . + as (where localname turns out an alias for name1) or import . *. as (where localname turns out an alias for nameX, that is, the last part of the dotted name that's being imported) ? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From gmcm@hypernet.com Fri Aug 18 17:28:49 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Fri, 18 Aug 2000 12:28:49 -0400 Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <399D591A.F909CF9D@lemburg.com> Message-ID: <1245506365-160663281@hypernet.com> M.-A. Lemburg wrote: > Gordon McMillan wrote: > > > > M.-A. Lemburg wrote: > > > > > ... just look at what your browser does > > > when you request http://www.python.org/search ... the server > > > redirects you to search/ to make sure that the links embedded > > > in the page are relative to search/ and not www.python.org/. > > > > While that seems to be what Apache does, I get 40x's from > > IIS and Netscape server. Greg Ewing's demonstrated a Unix > > where the trailing slash indicates nothing useful, Tim's > > demonstrated that Windows gets confused by a trailing slash > > unless we're talking about the root directory on a drive (and > > BTW, same results if you use backslash). > > > > On WIndows, os.path.commonprefix doesn't use normcase > > and normpath, so it's completely useless anyway. (That is, it's > > really a "string" function and has nothing to do with paths). > > I still don't get it: what's the point in carelessly dropping > valid and useful information for no obvious reason at all ? I wasn't advocating anything. I was pointing out that it's not necessarily "valid" or "useful" information in all contexts. > Besides the previous behaviour was documented and most probably > used in some apps. Why break those ? I don't think commonprefix should be changed, precisely because it might break apps. I also think it should not live in os.path, because it is not an abstract path operation. It's just a string operation. But it's there, so the best I can advise is not to use it. > And last not least: what if the directory in question doesn't > even exist anywhere and is only encoded in the path by the fact > that there is a slash following it ? If it doesn't exist, it's not a directory with or without a slash following it. The fact that Python largely successfully reuses os.path code to deal with URLs does not mean that the syntax of URLs should be mistaken for the syntax of file systems, even at an abstract level. At the level where commonprefix operates, abstraction isn't even a concept. - Gordon From gmcm@hypernet.com Fri Aug 18 17:33:12 2000 From: gmcm@hypernet.com (Gordon McMillan) Date: Fri, 18 Aug 2000 12:33:12 -0400 Subject: [Python-Dev] 'import as' In-Reply-To: <20000818182246.V376@xs4all.nl> References: <14749.20707.347217.763385@anthem.concentric.net>; from bwarsaw@beopen.com on Fri, Aug 18, 2000 at 11:06:11AM -0400 Message-ID: <1245506103-160679086@hypernet.com> Thomas Wouters wrote: > On Fri, Aug 18, 2000 at 11:06:11AM -0400, Barry A. Warsaw wrote: > > > Gordo> So I'm -0: it just adds complexity to an already > > overly Gordo> complex area. > > > I agree, -0 from me too. > > What are we voting on, here ? > > import as (in general) -0 > import . + as (where localname turns out > an alias for name1) -1000 > import . *. as (where localname > turns out an alias for nameX, that is, the last part of the > dotted name that's being imported) -0 - Gordon From thomas@xs4all.net Fri Aug 18 17:41:31 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 18 Aug 2000 18:41:31 +0200 Subject: [Python-Dev] serious bug in new import X as Y code In-Reply-To: <20000818150639.6685C31047C@bireme.oratrix.nl>; from sjoerd@oratrix.nl on Fri, Aug 18, 2000 at 05:06:37PM +0200 References: <20000818094239.A3A1931047C@bireme.oratrix.nl> <20000818161745.U376@xs4all.nl> <20000818150639.6685C31047C@bireme.oratrix.nl> Message-ID: <20000818184131.W376@xs4all.nl> On Fri, Aug 18, 2000 at 05:06:37PM +0200, Sjoerd Mullender wrote: > Ok, problem solved. > The problem was that because of your (I think it was you :-) earlier > change to have a Makefile in Grammar, I had an old graminit.c lying > around in my build directory. Right. That patch was mine, and I think we should remove it again :P We aren't changing Grammar *that* much, and we'll just have to 'make Grammar' individually. Grammar now also gets re-made much too often (though that doesn't really present any problems, it's just a tad sloppy.) Do we really want that in the released package ? The Grammar dependency can't really be solved until dependencies in general are handled better (or at all), especially between directories. It'll only create more Makefile spaghetti :P -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From fdrake@beopen.com Fri Aug 18 17:41:35 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Fri, 18 Aug 2000 12:41:35 -0400 (EDT) Subject: [Python-Dev] Re: os.path.commonprefix breakage In-Reply-To: <1245506365-160663281@hypernet.com> References: <399D591A.F909CF9D@lemburg.com> <1245506365-160663281@hypernet.com> Message-ID: <14749.26431.198802.970572@cj42289-a.reston1.va.home.com> Gordon McMillan writes: > I don't think commonprefix should be changed, precisely > because it might break apps. I also think it should not live in > os.path, because it is not an abstract path operation. It's just > a string operation. But it's there, so the best I can advise is > not to use it. This works. Let's accept (some variant) or Skip's desired functionality as os.path.splitprefix(); this avoid breaking existing code and uses a name that's consistent with the others. The result can be (prefix, [list of suffixes]). Trailing slashes should be handled so that os.path.join(prefix, suffix) does the "right thing". -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From fdrake@beopen.com Fri Aug 18 17:37:02 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Fri, 18 Aug 2000 12:37:02 -0400 (EDT) Subject: [Python-Dev] 'import as' In-Reply-To: <20000818182246.V376@xs4all.nl> References: <14749.14761.275554.898385@anthem.concentric.net> <1245512755-160278942@hypernet.com> <14749.20707.347217.763385@anthem.concentric.net> <20000818182246.V376@xs4all.nl> Message-ID: <14749.26158.777771.458507@cj42289-a.reston1.va.home.com> Thomas Wouters writes: > What are we voting on, here ? We should be really clear about this, since it is confusing. > import as (in general) +1 for this basic usage. > import . + as (where localname turns out an alias > for name1) -1, because it's confusing for users > import . *. as (where localname turns out an > alias for nameX, that is, the last part of the dotted name that's being > imported) +1 on the idea, but the circular import issue is very real and I'm not sure of the best way to solve it. For now, let's support: import name1 as localname where neither name1 nor localname can be dotted. The dotted-name1 case can be added when the circular import issue can be dealt with. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From trentm@ActiveState.com Fri Aug 18 17:54:12 2000 From: trentm@ActiveState.com (Trent Mick) Date: Fri, 18 Aug 2000 09:54:12 -0700 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: ; from tim_one@email.msn.com on Fri, Aug 18, 2000 at 02:30:40AM -0400 References: <20000817164137.U17689@lyra.org> Message-ID: <20000818095412.C11316@ActiveState.com> On Fri, Aug 18, 2000 at 02:30:40AM -0400, Tim Peters wrote: > [Greg Stein] > > ... > > IOW, an x-plat TLS is going to be done at some point. If you need it now, > > then please do it now. That will help us immeasurably in the long run. > > the former's utter bogosity. From Trent's POV, I bet the one-liner > workaround sounds more appealing. > Yes. Trent -- Trent Mick TrentM@ActiveState.com From trentm@ActiveState.com Fri Aug 18 17:56:24 2000 From: trentm@ActiveState.com (Trent Mick) Date: Fri, 18 Aug 2000 09:56:24 -0700 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: <20000818040034.F17689@lyra.org>; from gstein@lyra.org on Fri, Aug 18, 2000 at 04:00:34AM -0700 References: <20000818091703.T376@xs4all.nl> <20000818040034.F17689@lyra.org> Message-ID: <20000818095624.D11316@ActiveState.com> On Fri, Aug 18, 2000 at 04:00:34AM -0700, Greg Stein wrote: > That is a silly approach, Tim. This is exactly what autoconf is for. Using > run-time logic to figure out something that is compile-time is Badness. > > > > On Thu, Aug 17, 2000 at 11:39:29PM -0400, Tim Peters wrote: > > > > > > > So how about a runtime test for what's actually important (and it's not > > > > Monterey!)? > > > > > > > > if (sizeof(threadid) <= sizeof(long)) > > > > return (long)threadid; > > > > > > > > End of problem, right? It's a cheap runtime test in a function > > > > whose speed isn't critical anyway. > > > I am inclined to agrre with Thomas and Greg on this one. Why not check for sizeof(pthread_t) if pthread.h exists and test: #if SIZEOF_PTHREAD_T < SIZEOF_LONG return (long)threadid; #endif Trent -- Trent Mick TrentM@ActiveState.com From tim_one@email.msn.com Fri Aug 18 18:09:05 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 18 Aug 2000 13:09:05 -0400 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: <20000818040034.F17689@lyra.org> Message-ID: [Greg Stein] > That is a silly approach, Tim. This is exactly what autoconf is for. > Using run-time logic to figure out something that is compile-time > is Badness. Remain -0. autoconf may work slick as snot on Unix derivatives, but each new symbol it introduces also serves to make people on other platforms scratch their heads about what it means and what they're supposed to do with it in their manual config files. In this case, the alternative is an obvious and isolated 1-liner that's transparent on inspection regardless of the reader's background. You haven't noted a *downside* to that approach that I can see, and your technical objection is incorrect: sizeof is not a compile-time operation (try it in an #if, but make very sure it does what you think it does ). From tim_one@email.msn.com Fri Aug 18 18:09:07 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 18 Aug 2000 13:09:07 -0400 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: <003001c00901$11fd8ae0$0900a8c0@SPIFF> Message-ID: [/F] > from what I can tell, it's compatible with a long on all sane plat- > forms (Win64 doesn't support pthreads anyway ;-), so I guess the > right thing here is to remove volatile and simply use: > > return (long) threadid; That's what the code originally did, and the casting was introduced in version 2.5. As for the "volatile", Vladimir reported that he needed that. This isn't worth the brain cell it's getting. Put in the hack and move on already! From trentm@ActiveState.com Fri Aug 18 18:23:39 2000 From: trentm@ActiveState.com (Trent Mick) Date: Fri, 18 Aug 2000 10:23:39 -0700 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects object.c,2.95,2.96 In-Reply-To: <200008180501.WAA28237@slayer.i.sourceforge.net>; from bwarsaw@users.sourceforge.net on Thu, Aug 17, 2000 at 10:01:22PM -0700 References: <200008180501.WAA28237@slayer.i.sourceforge.net> Message-ID: <20000818102339.E11316@ActiveState.com> On Thu, Aug 17, 2000 at 10:01:22PM -0700, Barry Warsaw wrote: > Update of /cvsroot/python/python/dist/src/Objects > In directory slayer.i.sourceforge.net:/tmp/cvs-serv28173 > > Modified Files: > object.c > Log Message: > make_pair(): When comparing the pointers, they must be cast to integer > types (i.e. Py_uintptr_t, our spelling of C9X's uintptr_t). ANSI > specifies that pointer compares other than == and != to non-related > structures are undefined. This quiets an Insure portability warning. > > > Index: object.c > =================================================================== > RCS file: /cvsroot/python/python/dist/src/Objects/object.c,v > retrieving revision 2.95 > retrieving revision 2.96 > diff -C2 -r2.95 -r2.96 > *** object.c 2000/08/16 12:24:51 2.95 > --- object.c 2000/08/18 05:01:19 2.96 > *************** > *** 372,375 **** > --- 372,377 ---- > { > PyObject *pair; > + Py_uintptr_t iv = (Py_uintptr_t)v; > + Py_uintptr_t iw = (Py_uintptr_t)w; > > pair = PyTuple_New(2); > *************** > *** 377,381 **** > return NULL; > } > ! if (v <= w) { > PyTuple_SET_ITEM(pair, 0, PyLong_FromVoidPtr((void *)v)); > PyTuple_SET_ITEM(pair, 1, PyLong_FromVoidPtr((void *)w)); > --- 379,383 ---- > return NULL; > } > ! if (iv <= iw) { > PyTuple_SET_ITEM(pair, 0, PyLong_FromVoidPtr((void *)v)); > PyTuple_SET_ITEM(pair, 1, PyLong_FromVoidPtr((void *)w)); > *************** > *** 488,492 **** > } > if (vtp->tp_compare == NULL) { > ! return (v < w) ? -1 : 1; > } > _PyCompareState_nesting++; > --- 490,496 ---- > } > if (vtp->tp_compare == NULL) { > ! Py_uintptr_t iv = (Py_uintptr_t)v; > ! Py_uintptr_t iw = (Py_uintptr_t)w; > ! return (iv < iw) ? -1 : 1; > } > _PyCompareState_nesting++; > Can't you just do the cast for the comparison instead of making new variables? Trent -- Trent Mick TrentM@ActiveState.com From bwarsaw@beopen.com Fri Aug 18 18:41:50 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Fri, 18 Aug 2000 13:41:50 -0400 (EDT) Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects object.c,2.95,2.96 References: <200008180501.WAA28237@slayer.i.sourceforge.net> <20000818102339.E11316@ActiveState.com> Message-ID: <14749.30046.345520.779328@anthem.concentric.net> >>>>> "TM" == Trent Mick writes: TM> Can't you just do the cast for the comparison instead of TM> making new variables? Does it matter? From trentm@ActiveState.com Fri Aug 18 18:47:52 2000 From: trentm@ActiveState.com (Trent Mick) Date: Fri, 18 Aug 2000 10:47:52 -0700 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects object.c,2.95,2.96 In-Reply-To: <14749.30046.345520.779328@anthem.concentric.net>; from bwarsaw@beopen.com on Fri, Aug 18, 2000 at 01:41:50PM -0400 References: <200008180501.WAA28237@slayer.i.sourceforge.net> <20000818102339.E11316@ActiveState.com> <14749.30046.345520.779328@anthem.concentric.net> Message-ID: <20000818104752.A15002@ActiveState.com> On Fri, Aug 18, 2000 at 01:41:50PM -0400, Barry A. Warsaw wrote: > > >>>>> "TM" == Trent Mick writes: > > TM> Can't you just do the cast for the comparison instead of > TM> making new variables? > > Does it matter? No, I guess not. Just being a nitpicker first thing in the morning. Revving up for real work. :) Trent -- Trent Mick TrentM@ActiveState.com From tim_one@email.msn.com Fri Aug 18 18:52:20 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 18 Aug 2000 13:52:20 -0400 Subject: [Python-Dev] Adding insint() function In-Reply-To: <20000818110037.C27419@kronos.cnri.reston.va.us> Message-ID: [Andrew Kuchling] > Four modules define insint() functions to insert an integer into a Five or macro > dictionary in order to initialize constants in their module > dictionaries: > > kronos Modules>grep -l insint *.c > pcremodule.c > shamodule.c > socketmodule.c > zlibmodule.c > kronos Modules> It's actually a macro in shamodule. Missing is _winreg.c (in the PC directory). The perils of duplication manifest in subtle differences among these guys (like _winreg's inserts a long while the others insert an int -- and _winreg is certainly more correct here because a Python int *is* a C long; and they differ in treatment of errors, and it's not at all clear that's intentional). > ... > This duplication bugs me. Shall I submit a patch to add an API > convenience function to do this, and change the modules to use it? > Suggested prototype and name: PyDict_InsertInteger( dict *, string, > long) +1, provided the treatment of errors is clearly documented. From akuchlin@mems-exchange.org Fri Aug 18 18:58:33 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Fri, 18 Aug 2000 13:58:33 -0400 Subject: [Python-Dev] Adding insint() function In-Reply-To: ; from tim_one@email.msn.com on Fri, Aug 18, 2000 at 01:52:20PM -0400 References: <20000818110037.C27419@kronos.cnri.reston.va.us> Message-ID: <20000818135833.K27419@kronos.cnri.reston.va.us> On Fri, Aug 18, 2000 at 01:52:20PM -0400, Tim Peters wrote: >+1, provided the treatment of errors is clearly documented. The treatment of errors in module init functions seems to be simply charge ahead and do the inserts, and then do 'if (PyErr_Occurred()) Py_FatalError())'. The new function will probably return NULL if there's an error, but I doubt anyone will check it; it's too ungainly to write if ( (PyDict_SetItemStringInt(d, "foo", FOO)) == NULL || (PyDict_SetItemStringInt(d, "bar", BAR)) == NULL || ... repeat for 187 more constants ... --amk From Vladimir.Marangozov@inrialpes.fr Fri Aug 18 19:17:53 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 18 Aug 2000 20:17:53 +0200 (CEST) Subject: [Python-Dev] Adding insint() function In-Reply-To: <20000818135833.K27419@kronos.cnri.reston.va.us> from "Andrew Kuchling" at Aug 18, 2000 01:58:33 PM Message-ID: <200008181817.UAA07799@python.inrialpes.fr> Andrew Kuchling wrote: > > On Fri, Aug 18, 2000 at 01:52:20PM -0400, Tim Peters wrote: > >+1, provided the treatment of errors is clearly documented. > > The treatment of errors in module init functions seems to be simply > charge ahead and do the inserts, and then do 'if (PyErr_Occurred()) > Py_FatalError())'. The new function will probably return NULL if > there's an error, but I doubt anyone will check it; it's too ungainly > to write > if ( (PyDict_SetItemStringInt(d, "foo", FOO)) == NULL || > (PyDict_SetItemStringInt(d, "bar", BAR)) == NULL || > ... repeat for 187 more constants ... :-) So name it PyModule_AddConstant(module, name, constant), which fails with "can't add constant to module" err msg. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From tim_one@email.msn.com Fri Aug 18 19:24:57 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 18 Aug 2000 14:24:57 -0400 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: <20000818095624.D11316@ActiveState.com> Message-ID: [Trent Mick] > I am inclined to agrre with Thomas and Greg on this one. Why not check for > sizeof(pthread_t) if pthread.h exists and test: > > #if SIZEOF_PTHREAD_T < SIZEOF_LONG > return (long)threadid; > #endif Change "<" to "<=" and I won't gripe. From fdrake@beopen.com Fri Aug 18 19:40:48 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Fri, 18 Aug 2000 14:40:48 -0400 (EDT) Subject: [Python-Dev] Adding insint() function In-Reply-To: <200008181817.UAA07799@python.inrialpes.fr> References: <20000818135833.K27419@kronos.cnri.reston.va.us> <200008181817.UAA07799@python.inrialpes.fr> Message-ID: <14749.33584.683341.684523@cj42289-a.reston1.va.home.com> Vladimir Marangozov writes: > So name it PyModule_AddConstant(module, name, constant), > which fails with "can't add constant to module" err msg. Even better! I expect there should be at least a couple of these; one for ints, one for strings. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From tim_one@email.msn.com Fri Aug 18 19:37:19 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 18 Aug 2000 14:37:19 -0400 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Objects object.c,2.95,2.96 In-Reply-To: <20000818102339.E11316@ActiveState.com> Message-ID: [Trent Mick] > > ... > > if (vtp->tp_compare == NULL) { > > ! Py_uintptr_t iv = (Py_uintptr_t)v; > > ! Py_uintptr_t iw = (Py_uintptr_t)w; > > ! return (iv < iw) ? -1 : 1; > > } > Can't you just do the cast for the comparison instead of making new > variables? Any compiler worth beans will optimize them out of existence. In the meantime, it makes each line (to my eyes) short, clear, and something I can set a debugger breakpoint on in debug mode if I suspect the cast isn't working as intended. From Fredrik Lundh" <20000818161745.U376@xs4all.nl> <20000818150639.6685C31047C@bireme.oratrix.nl> Message-ID: <000001c00945$a8d37e40$f2a6b5d4@hagrid> sjoerd wrote: > The problem was that because of your (I think it was you :-) earlier > change to have a Makefile in Grammar, I had an old graminit.c lying > around in my build directory. I don't build in the source directory > and the changes for a Makefile in Grammar resulted in a file > graminit.c in the wrong place. is the Windows build system updated to generate new graminit files if the Grammar are updated? or is python development a unix-only thingie these days? From tim_one@email.msn.com Fri Aug 18 20:05:29 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 18 Aug 2000 15:05:29 -0400 Subject: [Python-Dev] serious bug in new import X as Y code In-Reply-To: <000001c00945$a8d37e40$f2a6b5d4@hagrid> Message-ID: [/F] > is the Windows build system updated to generate new > graminit files if the Grammar are updated? No, not yet. > or is python development a unix-only thingie these days? It pretty much always has been! Just ask Jack . It's unreasonable to expect Unix(tm) weenies to keep the other builds working (although vital that they tell Python-Dev when they do something "new & improved"), and counterproductive to imply that their inability to do so should deter them from improving the build process on their platform. In some ways, building is more pleasant under Windows, and if turnabout is fair play the Unix droids could insist we build them a honking powerful IDE . From m.favas@per.dem.csiro.au Fri Aug 18 20:08:36 2000 From: m.favas@per.dem.csiro.au (Mark Favas) Date: Sat, 19 Aug 2000 03:08:36 +0800 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements References: <20000816172425.A32338@ActiveState.com> <003001c00901$11fd8ae0$0900a8c0@SPIFF> Message-ID: <399D89B4.476FB5EF@per.dem.csiro.au> OK - return (long) threadid; compiles without warnings, and all tests pass on OSF/1 (aka Tru64 Unix). Removing the "volatile" is also fine for me, but may affect Vladimir. I'm still a bit (ha!) confused by Tim's comments that the function is bogus for OSF/1 because it throws away half the bits, and will therefore result in id collisions - this will only happen on platforms where sizeof(long) is less than sizeof(pointer), which is not OSF/1 (but is Win64). Also, one of the suggested tests only cast the pointer to a long SIZEOF_PTHREAD_T < SIZEOF_LONG - that should surely be <= ... In summary, whatever issue there was for OSF/1 six (or so) years ago appears to be no longer relevant - but there will be the truncation issue for Win64-like platforms. Mark Fredrik Lundh wrote: > > trent mick wrote: > > return (long) *(long *) &threadid; > > from what I can tell, pthread_t is a pointer under OSF/1. > > I've been using OSF/1 since the early days, and as far as I can > remember, you've never needed to use stupid hacks like that > to convert a pointer to a long integer. an ordinary (long) cast > should be sufficient. > > > Could this be changed to > > return threadid; > > safely? > > safely, yes. but since it isn't a long on all platforms, you might > get warnings from the compiler (see Mark's mail). > > ::: > > from what I can tell, it's compatible with a long on all sane plat- > forms (Win64 doesn't support pthreads anyway ;-), so I guess the > right thing here is to remove volatile and simply use: > > return (long) threadid; > > (Mark: can you try this out on your box? setting up a Python 2.0 > environment on our alphas would take more time than I can spare > right now...) > > -- Email - m.favas@per.dem.csiro.au Mark C Favas Phone - +61 8 9333 6268, 0418 926 074 CSIRO Exploration & Mining Fax - +61 8 9383 9891 Private Bag No 5, Wembley WGS84 - 31.95 S, 115.80 E Western Australia 6913 From Vladimir.Marangozov@inrialpes.fr Fri Aug 18 20:09:48 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 18 Aug 2000 21:09:48 +0200 (CEST) Subject: [Python-Dev] Introducing memprof (was PyErr_NoMemory) In-Reply-To: from "Tim Peters" at Aug 18, 2000 01:43:14 AM Message-ID: <200008181909.VAA08003@python.inrialpes.fr> [Tim, on PyErr_NoMemory] > > Looks good to me. And if it breaks something, it will be darned hard to > tell . It's easily demonstrated with the memprof.c module I'd like to introduce quickly here. Note: I'll be out of town next week and if someone wants to play with this, tell me what to do quickly: upload a (postponed) patch which goes in pair with obmalloc.c, put it in a web page or remain quiet. The object allocator is well tested, the memory profiler is not so thouroughly tested... The interface needs more work IMHO, but it's already quite useful and fun in it's current state . Demo: ~/python/dev>python -p Python 2.0b1 (#9, Aug 18 2000, 20:11:29) [GCC egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)] on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam Copyright 1995-2000 Corporation for National Research Initiatives (CNRI) >>> >>> # Note the -p option -- it starts any available profilers through ... # a newly introduced Py_ProfileFlag. Otherwise you'll get funny results ... # if you start memprof in the middle of an execution ... >>> import memprof >>> memprof.__doc__ 'This module provides access to the Python memory profiler.' >>> >>> dir(memprof) ['ALIGNMENT', 'ERROR_ABORT', 'ERROR_IGNORE', 'ERROR_RAISE', 'ERROR_REPORT', 'ERROR_STOP', 'MEM_CORE', 'MEM_OBCLASS', 'MEM_OBJECT', '__doc__', '__name__', 'geterrlevel', 'getpbo', 'getprofile', 'getthreshold', 'isprofiling', 'seterrlevel', 'setpbo', 'setproftype', 'setthreshold', 'start', 'stop'] >>> >>> memprof.isprofiling() 1 >>> # It's running -- cool. We're now ready to get the current memory profile ... >>> print memprof.getprofile.__doc__ getprofile([type]) -> object Return a snapshot of the current memory profile of the interpreter. An optional type argument may be provided to request the profile of a specific memory layer. It must be one of the following constants: MEM_CORE - layer 1: Python core memory MEM_OBJECT - layer 2: Python object memory MEM_OBCLASS - layer 3: Python object-specific memory If a type argument is not specified, the default profile is returned. The default profile type can be set with the setproftype() function. >>> >>> mp = memprof.getprofile() >>> mp >>> >>> # now see how much mem we're using, it's a 3 tuple ... # (requested mem, minimum allocated mem, estimated mem) ... >>> mp.memory (135038, 142448, 164792) >>> mp.peakmemory (137221, 144640, 167032) >>> >>> # indeed, peak values are important. Now let's see what this gives in ... # terms of memory blocks ... >>> mp.blocks (2793, 2793) >>> mp.peakblocks (2799, 2799) >>> >>> # Again this is a 2-tuple (requested blocks, allocated blocks) ... # Now let's see the stats of the calls to the allocator. ... >>> mp.malloc (4937, 0, 0) >>> mp.calloc (0, 0, 0) >>> mp.realloc (43, 0, 0) >>> mp.free (2144, 0, 0) >>> >>> # A 3-tuple (nb of calls, nb of errors, nb of warnings by memprof) ... # ... # Good. Now let's see the memory profile detailed by size classes ... they're memory profile objects too, similar to the global profile: >>> >>> mp.sizeclass[0] >>> mp.sizeclass[1] >>> mp.sizeclass[2] >>> len(mp.sizeclass) 33 >>> mp.sizeclass[-1] >>> >>> # The last one is for big blocks: 257 bytes and up. ... # Now let's see ithe detailed memory picture: >>> >>> for s in mp.sizeclass: ... print "%.2d - " % s.sizeclass, "%8d %8d %8d" % s.memory ... 00 - 0 0 0 01 - 3696 3776 5664 02 - 116 120 160 03 - 31670 34464 43080 04 - 30015 32480 38976 05 - 10736 11760 13720 06 - 10846 11200 12800 07 - 2664 2816 3168 08 - 1539 1584 1760 09 - 1000 1040 1144 10 - 2048 2112 2304 11 - 1206 1248 1352 12 - 598 624 672 13 - 109 112 120 14 - 575 600 640 15 - 751 768 816 16 - 407 408 432 17 - 144 144 152 18 - 298 304 320 19 - 466 480 504 20 - 656 672 704 21 - 349 352 368 22 - 542 552 576 23 - 188 192 200 24 - 392 400 416 25 - 404 416 432 26 - 640 648 672 27 - 441 448 464 28 - 0 0 0 29 - 236 240 248 30 - 491 496 512 31 - 501 512 528 32 - 31314 31480 31888 >>> >>> for s in mp.sizeclass: ... print "%.2d - " % s.sizeclass, "%8d %8d" % s.blocks ... 00 - 0 0 01 - 236 236 02 - 5 5 03 - 1077 1077 04 - 812 812 05 - 245 245 06 - 200 200 07 - 44 44 08 - 22 22 09 - 13 13 10 - 24 24 11 - 13 13 12 - 6 6 13 - 1 1 14 - 5 5 15 - 6 6 16 - 3 3 17 - 1 1 18 - 2 2 19 - 3 3 20 - 4 4 21 - 2 2 22 - 3 3 23 - 1 1 24 - 2 2 25 - 2 2 26 - 3 3 27 - 2 2 28 - 0 0 29 - 1 1 30 - 2 2 31 - 2 2 32 - 51 51 >>> >>> # Note that just started the interpreter and analysed it's initial ... # memory profile. You can repeat this game at any point of time, ... # look at the stats and enjoy a builtin memory profiler. ... # ... # Okay, now to the point on PyErr_NoMemory: but we need to restart ... # Python without "-p" >>> ~/python/dev>python Python 2.0b1 (#9, Aug 18 2000, 20:11:29) [GCC egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)] on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam Copyright 1995-2000 Corporation for National Research Initiatives (CNRI) >>> >>> import memprof >>> memprof.isprofiling() 0 >>> memprof.start() memprof: freeing unknown block (0x40185e60) memprof: freeing unknown block (0x40175098) memprof: freeing unknown block (0x40179288) >>> >>> # See? We're freeing unknown blocks for memprof. ... # Okay, enough. See the docs for more: ... >>> print memprof.seterrlevel.__doc__ seterrlevel(flags) -> None Set the error level of the profiler. The provided argument instructs the profiler on how tolerant it should be against any detected simple errors or memory corruption. The following non-exclusive values are recognized: ERROR_IGNORE - ignore silently any detected errors ERROR_REPORT - report all detected errors to stderr ERROR_STOP - stop the profiler on the first detected error ERROR_RAISE - raise a MemoryError exception for all detected errors ERROR_ABORT - report the first error as fatal and abort immediately The default error level is ERROR_REPORT. >>> >>> # So here's you're PyErr_NoMemory effect: ... >>> memprof.seterrlevel(memprof.ERROR_REPORT | memprof.ERROR_RAISE) >>> >>> import test.regrtest memprof: resizing unknown block (0x82111b0) memprof: raised MemoryError. Traceback (most recent call last): File " ", line 1, in ? File "./Lib/test/regrtest.py", line 39, in ? import random File "./Lib/random.py", line 23, in ? import whrandom File "./Lib/whrandom.py", line 40, in ? class whrandom: MemoryError: memprof: resizing unknown block (0x82111b0) >>> >>> # Okay, gotta run. There are no docs for the moment. Just the source ... and function docs. (and to avoid another exception...) >>> >>> memprof.seterrlevel(memprof.ERROR_IGNORE) >>> >>> for i in dir(memprof): ... x = memprof.__dict__[i] ... if hasattr(x, "__doc__"): ... print ">>>>>>>>>>>>>>>>>>>>>>>>>>>>> [%s]" % i ... print x.__doc__ ... print '='*70 ... >>>>>>>>>>>>>>>>>>>>>>>>>>>>> [geterrlevel] geterrlevel() -> errflags Get the current error level of the profiler. ====================================================================== >>>>>>>>>>>>>>>>>>>>>>>>>>>>> [getpbo] getpbo() -> int Return the fixed per block overhead (pbo) used for estimations. ====================================================================== >>>>>>>>>>>>>>>>>>>>>>>>>>>>> [getprofile] getprofile([type]) -> object Return a snapshot of the current memory profile of the interpreter. An optional type argument may be provided to request the profile of a specific memory layer. It must be one of the following constants: MEM_CORE - layer 1: Python core memory MEM_OBJECT - layer 2: Python object memory MEM_OBCLASS - layer 3: Python object-specific memory If a type argument is not specified, the default profile is returned. The default profile type can be set with the setproftype() function. ====================================================================== >>>>>>>>>>>>>>>>>>>>>>>>>>>>> [getthreshold] getthreshold() -> int Return the size threshold (in bytes) between small and big blocks. ====================================================================== >>>>>>>>>>>>>>>>>>>>>>>>>>>>> [isprofiling] isprofiling() -> 1 if profiling is currently in progress, 0 otherwise. ====================================================================== >>>>>>>>>>>>>>>>>>>>>>>>>>>>> [seterrlevel] seterrlevel(flags) -> None Set the error level of the profiler. The provided argument instructs the profiler on how tolerant it should be against any detected simple errors or memory corruption. The following non-exclusive values are recognized: ERROR_IGNORE - ignore silently any detected errors ERROR_REPORT - report all detected errors to stderr ERROR_STOP - stop the profiler on the first detected error ERROR_RAISE - raise a MemoryError exception for all detected errors ERROR_ABORT - report the first error as fatal and abort immediately The default error level is ERROR_REPORT. ====================================================================== >>>>>>>>>>>>>>>>>>>>>>>>>>>>> [setpbo] setpbo(int) -> None Set the fixed per block overhead (pbo) used for estimations. ====================================================================== >>>>>>>>>>>>>>>>>>>>>>>>>>>>> [setproftype] setproftype(type) -> None Set the default profile type returned by getprofile() without arguments. ====================================================================== >>>>>>>>>>>>>>>>>>>>>>>>>>>>> [setthreshold] setthreshold(int) -> None Set the size threshold (in bytes) between small and big blocks. The maximum is 256. The argument is rounded up to the ALIGNMENT. ====================================================================== >>>>>>>>>>>>>>>>>>>>>>>>>>>>> [start] start() -> None Start the profiler. If it has been started, this function has no effect. ====================================================================== >>>>>>>>>>>>>>>>>>>>>>>>>>>>> [stop] stop() -> None Stop the profiler. If it has been stopped, this function has no effect. ====================================================================== -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From tim_one@email.msn.com Fri Aug 18 20:11:11 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 18 Aug 2000 15:11:11 -0400 Subject: [Python-Dev] RE: Introducing memprof (was PyErr_NoMemory) In-Reply-To: <200008181909.VAA08003@python.inrialpes.fr> Message-ID: [Tim, on PyErr_NoMemory] > Looks good to me. And if it breaks something, it will be darned hard to > tell . [Vladimir Marangozov] > It's easily demonstrated with the memprof.c module I'd like to introduce > quickly here. > > Note: I'll be out of town next week and if someone wants to > play with this, tell me what to do quickly: upload a (postponed) patch > which goes in pair with obmalloc.c, put it in a web page or remain > quiet. > > The object allocator is well tested, the memory profiler is not so > thouroughly tested... The interface needs more work IMHO, but it's > already quite useful and fun in it's current state . > ... My bandwidth is consumed by 2.0 issues, so I won't look at it. On the chance that Guido gets hit by a bus, though, and I have time to kill at his funeral, it would be nice to have it available on SourceForge. Uploading a postponed patch sounds fine! From tim_one@email.msn.com Fri Aug 18 20:26:31 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 18 Aug 2000 15:26:31 -0400 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: <399D89B4.476FB5EF@per.dem.csiro.au> Message-ID: [Mark Favas] > return (long) threadid; > > compiles without warnings, and all tests pass on OSF/1 (aka Tru64 Unix). > Removing the "volatile" is also fine for me, but may affect Vladimir. > I'm still a bit (ha!) confused by Tim's comments that the function is > bogus for OSF/1 because it throws away half the bits, and will therefore > result in id collisions - this will only happen on platforms where > sizeof(long) is less than sizeof(pointer), which is not OSF/1 Pure guess on my part -- couldn't imagine why a compiler would warn *unless* bits were being lost. Are you running this on an Alpha? The comment in the code specifically names "Alpha OSF/1" as the culprit. I don't know anything about OSF/1; perhaps "Alpha" is implied. > ... > In summary, whatever issue there was for OSF/1 six (or so) years ago > appears to be no longer relevant - but there will be the truncation > issue for Win64-like platforms. And there's Vladimir's "volatile" hack. From Fredrik Lundh" Message-ID: <00e501c0094b$c9ee2ac0$f2a6b5d4@hagrid> tim peters wrote: > [/F] > > is the Windows build system updated to generate new > > graminit files if the Grammar are updated? > > No, not yet. > > > or is python development a unix-only thingie these days? > > It pretty much always has been! Just ask Jack . It's unreasonable to > expect Unix(tm) weenies to keep the other builds working (although vital > that they tell Python-Dev when they do something "new & improved"), and > counterproductive to imply that their inability to do so should deter them > from improving the build process on their platform. well, all I expect from them is that the repository should be in a consistent state at all time. (in other words, never assume that just because generated files are rebuilt by the unix makefiles, they don't have to be checked into the repository). for a moment, sjoerd's problem report made me think that someone had messed up here... but I just checked, and he hadn't ;-) From thomas@xs4all.net Fri Aug 18 20:43:34 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 18 Aug 2000 21:43:34 +0200 Subject: [Python-Dev] serious bug in new import X as Y code In-Reply-To: <000001c00945$a8d37e40$f2a6b5d4@hagrid>; from effbot@telia.com on Fri, Aug 18, 2000 at 08:42:34PM +0200 References: <20000818094239.A3A1931047C@bireme.oratrix.nl> <20000818161745.U376@xs4all.nl> <20000818150639.6685C31047C@bireme.oratrix.nl> <000001c00945$a8d37e40$f2a6b5d4@hagrid> Message-ID: <20000818214333.X376@xs4all.nl> On Fri, Aug 18, 2000 at 08:42:34PM +0200, Fredrik Lundh wrote: > sjoerd wrote: > > The problem was that because of your (I think it was you :-) earlier > > change to have a Makefile in Grammar, I had an old graminit.c lying > > around in my build directory. I don't build in the source directory > > and the changes for a Makefile in Grammar resulted in a file > > graminit.c in the wrong place. > is the Windows build system updated to generate new > graminit files if the Grammar are updated? No, and that's one more reason to reverse my patch ! :-) Note that I didn't *add* the Makefile, I only added Grammar to the directories-to-run-make-in-by-default. If the Grammar is changed, you already need a way to run pgen (of which the source rests in Parser/) to generate the new graminit.c/graminit.h files. I have no way of knowing whether that is the case for the windows build files. The CVS tree should always contain up to date graminit.c/.h files! The reason it was added was the multitude of Grammar-changing patches on SF, and the number of people that forgot to run 'make' in Grammar/ after applying them. I mentioned adding Grammar/ to the directories to be made, Guido said it was a good idea, and noone complained to it until after it was added ;P I think we can drop the idea, though, at least for (alpha, beta, final) releases. > or is python development a unix-only thingie these days? Well, *my* python development is a unix-only thingie, mostly because I don't have a compiler for under Windows. If anyone wants to send me or point me to the canonical Windows compiler & debugger and such, in a way that won't set me back a couple of megabucks, I'd be happy to test & debug windows as well. Gives me *two* uses for Windows: games and Python ;) My-boss-doesn't-pay-me-to-work-on-Windows-ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From m.favas@per.dem.csiro.au Fri Aug 18 21:33:21 2000 From: m.favas@per.dem.csiro.au (Mark Favas) Date: Sat, 19 Aug 2000 04:33:21 +0800 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements References: Message-ID: <399D9D91.3E76ED8D@per.dem.csiro.au> Tim Peters wrote: > > [Mark Favas] > > return (long) threadid; > > > > compiles without warnings, and all tests pass on OSF/1 (aka Tru64 Unix). > > Removing the "volatile" is also fine for me, but may affect Vladimir. > > I'm still a bit (ha!) confused by Tim's comments that the function is > > bogus for OSF/1 because it throws away half the bits, and will therefore > > result in id collisions - this will only happen on platforms where > > sizeof(long) is less than sizeof(pointer), which is not OSF/1 > > Pure guess on my part -- couldn't imagine why a compiler would warn *unless* > bits were being lost. Are you running this on an Alpha? The comment in the > code specifically names "Alpha OSF/1" as the culprit. I don't know anything > about OSF/1; perhaps "Alpha" is implied. Yep - I'm running on an Alpha. The name of the OS has undergone a couple of, um, appellation transmogrifications since the first Alpha was produced by DEC: OSF/1 -> Digital Unix -> Tru64 Unix, although uname has always reported "OSF1". (I don't think that there's any other implementation of OSF/1 left these days... not that uses the name, anyway.) > > > ... > > In summary, whatever issue there was for OSF/1 six (or so) years ago > > appears to be no longer relevant - but there will be the truncation > > issue for Win64-like platforms. > > And there's Vladimir's "volatile" hack. Wonder if that also is still relevant (was it required because of the long * long * cast?)... -- Email - m.favas@per.dem.csiro.au Mark C Favas Phone - +61 8 9333 6268, 0418 926 074 CSIRO Exploration & Mining Fax - +61 8 9383 9891 Private Bag No 5, Wembley WGS84 - 31.95 S, 115.80 E Western Australia 6913 From bwarsaw@beopen.com Fri Aug 18 21:45:03 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Fri, 18 Aug 2000 16:45:03 -0400 (EDT) Subject: [Python-Dev] serious bug in new import X as Y code References: <20000818094239.A3A1931047C@bireme.oratrix.nl> <20000818161745.U376@xs4all.nl> <20000818150639.6685C31047C@bireme.oratrix.nl> <000001c00945$a8d37e40$f2a6b5d4@hagrid> <20000818214333.X376@xs4all.nl> Message-ID: <14749.41039.166847.942483@anthem.concentric.net> >>>>> "TW" == Thomas Wouters writes: TW> No, and that's one more reason to reverse my patch ! :-) Note TW> that I didn't *add* the Makefile, I only added Grammar to the TW> directories-to-run-make-in-by-default. If the Grammar is TW> changed, you already need a way to run pgen (of which the TW> source rests in Parser/) to generate the new TW> graminit.c/graminit.h files. I have no way of knowing whether TW> that is the case for the windows build files. The CVS tree TW> should always contain up to date graminit.c/.h files! I don't think you need to reverse your patch because of this, although I haven't been following this thread closely. Just make sure that if you commit a Grammar change, you must commit the newly generated graminit.c and graminit.h files. This is no different than if you change configure.in; you must commit both that file and the generated configure file. -Barry From akuchlin@mems-exchange.org Fri Aug 18 21:48:37 2000 From: akuchlin@mems-exchange.org (Andrew Kuchling) Date: Fri, 18 Aug 2000 16:48:37 -0400 Subject: [Python-Dev] Re: [Patches] [Patch #101055] Cookie.py In-Reply-To: <14749.38716.228254.649957@cj42289-a.reston1.va.home.com>; from fdrake@beopen.com on Fri, Aug 18, 2000 at 04:06:20PM -0400 References: <200008181951.MAA30358@bush.i.sourceforge.net> <14749.38716.228254.649957@cj42289-a.reston1.va.home.com> Message-ID: <20000818164837.A8423@kronos.cnri.reston.va.us> [Overquoting for the sake of python-dev readers] On Fri, Aug 18, 2000 at 04:06:20PM -0400, Fred L. Drake, Jr. wrote: >amk writes: > > I have a copy of Tim O'Malley's Cookie.py,v file (in order to preserve the > > version history). I can either ask the SF admins to drop it into > > the right place in the CVS repository, but will that screw up the > > Python 1.6 tagging in some way? (I would expect not, but who > > knows?) > > That would have no effect on any of the Python tagging. It's >probably worthwhile making sure there are no tags in the ,v file, but >that can be done after it gets dropped in place. > Now, Greg Stein will tell us that dropping this into place is the >wrong thing to do. What it *will* screw up is people asking for the >state of Python at a specific date before the file was actually added; >they'll get this file even for when it wasn't in the Python CVS tree. >I can live with that, but we should make a policy decision for the >Python tree regarding this sort of thing. Excellent point. GvR's probably the only person whose ruling matters on this point; I'll try to remember it and bring it up whenever he gets back (next week, right?). > Don't -- it's not worth it. I hate throwing away information that stands even a tiny chance of being useful; good thing the price of disk storage keeps dropping, eh? The version history might contain details that will be useful in future debugging or code comprehension, so I dislike losing it forever. (My minimalist side is saying that the enhanced Web tools should be a separately downloadable package. But you probably guessed that already...) --amk From thomas@xs4all.net Fri Aug 18 21:56:07 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 18 Aug 2000 22:56:07 +0200 Subject: [Python-Dev] serious bug in new import X as Y code In-Reply-To: <14749.41039.166847.942483@anthem.concentric.net>; from bwarsaw@beopen.com on Fri, Aug 18, 2000 at 04:45:03PM -0400 References: <20000818094239.A3A1931047C@bireme.oratrix.nl> <20000818161745.U376@xs4all.nl> <20000818150639.6685C31047C@bireme.oratrix.nl> <000001c00945$a8d37e40$f2a6b5d4@hagrid> <20000818214333.X376@xs4all.nl> <14749.41039.166847.942483@anthem.concentric.net> Message-ID: <20000818225607.Z376@xs4all.nl> On Fri, Aug 18, 2000 at 04:45:03PM -0400, Barry A. Warsaw wrote: > This is no different than if you change configure.in; you must commit > both that file and the generated configure file. Yes, but more critically so, since it'll screw up more than a couple of defines on a handful of systems :-) However, this particular change in the make process doesn't adress this at all. It would merely serve to mask this problem, in the event of someone commiting a change to Grammar but not to graminit.*. The reasoning behind the change was "if you change Grammar/Grammar, and then type 'make', graminit.* should be regenerated automatically, before they are used in other files." I thought the change was a small and reasonable one, but now I don't think so, anymore ;P On the other hand, perhaps the latest changes (not mine) fixed it for real. But I still think that if this particular makefile setup is used in releases, 'pgen' should at least be made a tad less verbose. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From thomas@xs4all.net Fri Aug 18 22:04:01 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 18 Aug 2000 23:04:01 +0200 Subject: [Python-Dev] [Patch #101055] Cookie.py In-Reply-To: <20000818164837.A8423@kronos.cnri.reston.va.us>; from akuchlin@mems-exchange.org on Fri, Aug 18, 2000 at 04:48:37PM -0400 References: <200008181951.MAA30358@bush.i.sourceforge.net> <14749.38716.228254.649957@cj42289-a.reston1.va.home.com> <20000818164837.A8423@kronos.cnri.reston.va.us> Message-ID: <20000818230401.A376@xs4all.nl> On Fri, Aug 18, 2000 at 04:48:37PM -0400, Andrew Kuchling wrote: [ About adding Cookie.py including CVS history ] > I hate throwing away information that stands even a tiny chance of > being useful; good thing the price of disk storage keeps dropping, eh? > The version history might contain details that will be useful in > future debugging or code comprehension, so I dislike losing it > forever. It would be moderately nice to have the versioning info, though I think Fred has a point when he says that it might be confusing for people: it would look like the file had been in the CVS repository the whole time, and it would be very hard to see where the file had been added to Python. And what about new versions ? Would this file be adopted by Python, would changes by the original author be incorporated ? How about version history for those changes ? The way it usually goes (as far as my experience goes) is that such files are updated only periodically. Would those updates incorporate the history of those changes as well ? Unless Cookie.py is really split off, and we're going to maintain a separate version, I don't think it's worth worrying about the version history as such. Pointing to the 'main' version and it's history should be enough. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From bwarsaw@beopen.com Fri Aug 18 22:13:31 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Fri, 18 Aug 2000 17:13:31 -0400 (EDT) Subject: [Python-Dev] gettext in the standard library Message-ID: <14749.42747.411862.940207@anthem.concentric.net> Apologies for duplicates to those of you already on python-dev... I've been working on merging all the various implementations of Python interfaces to the GNU gettext libraries. I've worked from code contributed by Martin, James, and Peter. I now have something that seems to work fairly well so I thought I'd update you all. After looking at all the various wizzy and experimental stuff in these implementations, I opted for simplicity, mostly just so I could get my head around what was needed. My goal was to build a fast C wrapper module around the C library, and to provide a pure Python implementation of an identical API for platforms without GNU gettext. I started with Martin's libintlmodule, renamed it _gettext and cleaned up the C code a bit. This provides gettext(), dgettext(), dcgettext(), textdomain(), and bindtextdomain() functions. The gettext.py module imports these, and if it succeeds, it's done. If that fails, then there's a bunch of code, mostly derived from Peter's fintl.py module, that reads the binary .mo files and does the look ups itself. Note that Peter's module only supported the GNU gettext binary format, and that's all mine does too. It should be easy to support other binary formats (Solaris?) by overriding one method in one class, and contributions are welcome. James's stuff looked cool too, what I grokked of it :) but I think those should be exported as higher level features. I didn't include the ability to write .mo files or the exported Catalog objects. I haven't used the I18N services enough to know whether these are useful. I added one convenience function, gettext.install(). If you call this, it inserts the gettext.gettext() function into the builtins namespace as `_'. You'll often want to do this, based on the I18N translatable strings marking conventions. Note that importing gettext does /not/ install by default! And since (I think) you'll almost always want to call bindtextdomain() and textdomain(), you can pass the domain and localedir in as arguments to install. Thus, the simple and quick usage pattern is: import gettext gettext.install('mydomain', '/my/locale/dir') print _('this is a localized message') I think it'll be easier to critique this stuff if I just check it in. Before I do, I still need to write up a test module and hack together docos. In the meantime, here's the module docstring for gettext.py. Talk amongst yourselves. :) -Barry """Internationalization and localization support. This module provides internationalization (I18N) and localization (L10N) support for your Python programs by providing an interface to the GNU gettext message catalog library. I18N refers to the operation by which a program is made aware of multiple languages. L10N refers to the adaptation of your program, once internationalized, to the local language and cultural habits. In order to provide multilingual messages for your Python programs, you need to take the following steps: - prepare your program by specially marking translatable strings - run a suite of tools over your marked program files to generate raw messages catalogs - create language specific translations of the message catalogs - use this module so that message strings are properly translated In order to prepare your program for I18N, you need to look at all the strings in your program. Any string that needs to be translated should be marked by wrapping it in _('...') -- i.e. a call to the function `_'. For example: filename = 'mylog.txt' message = _('writing a log message') fp = open(filename, 'w') fp.write(message) fp.close() In this example, the string `writing a log message' is marked as a candidate for translation, while the strings `mylog.txt' and `w' are not. The GNU gettext package provides a tool, called xgettext that scans C and C++ source code looking for these specially marked strings. xgettext generates what are called `.pot' files, essentially structured human readable files which contain every marked string in the source code. These .pot files are copied and handed over to translators who write language-specific versions for every supported language. For I18N Python programs however, xgettext won't work; it doesn't understand the myriad of string types support by Python. The standard Python distribution provides a tool called pygettext that does though (usually in the Tools/i18n directory). This is a command line script that supports a similar interface as xgettext; see its documentation for details. Once you've used pygettext to create your .pot files, you can use the standard GNU gettext tools to generate your machine-readable .mo files, which are what's used by this module and the GNU gettext libraries. In the simple case, to use this module then, you need only add the following bit of code to the main driver file of your application: import gettext gettext.install() This sets everything up so that your _('...') function calls Just Work. In other words, it installs `_' in the builtins namespace for convenience. You can skip this step and do it manually by the equivalent code: import gettext import __builtin__ __builtin__['_'] = gettext.gettext Once you've done this, you probably want to call bindtextdomain() and textdomain() to get the domain set up properly. Again, for convenience, you can pass the domain and localedir to install to set everything up in one fell swoop: import gettext gettext.install('mydomain', '/my/locale/dir') """ From tim_one@email.msn.com Fri Aug 18 22:13:29 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 18 Aug 2000 17:13:29 -0400 Subject: [Python-Dev] serious bug in new import X as Y code In-Reply-To: <20000818214333.X376@xs4all.nl> Message-ID: [Thomas Wouters] > No, and that's one more reason to reverse my patch ! :-) Note > that I didn't *add* the Makefile, I only added Grammar to the > directories-to-run-make-in-by-default. > ... > The reason it was added was the multitude of Grammar-changing > patches on SF, and the number of people that forgot to run 'make' > in Grammar/ after applying them. I mentioned adding Grammar/ to > the directories to be made, Guido said it was a good idea, and > noone complained to it until after it was added ;P And what exactly is the complaint? It's nice to build things that are out of date; I haven't used Unix(tm) for some time, but I vaguely recall that was "make"'s purpose in life . Or is it that the grammar files are getting rebuilt unnecessarily? > ... > My-boss-doesn't-pay-me-to-work-on-Windows-ly y'rs, Your boss *pays* you?! Wow. No wonder you get so much done. From bwarsaw@beopen.com Fri Aug 18 22:16:07 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Fri, 18 Aug 2000 17:16:07 -0400 (EDT) Subject: [Python-Dev] serious bug in new import X as Y code References: <20000818094239.A3A1931047C@bireme.oratrix.nl> <20000818161745.U376@xs4all.nl> <20000818150639.6685C31047C@bireme.oratrix.nl> <000001c00945$a8d37e40$f2a6b5d4@hagrid> <20000818214333.X376@xs4all.nl> <14749.41039.166847.942483@anthem.concentric.net> <20000818225607.Z376@xs4all.nl> Message-ID: <14749.42903.342401.245594@anthem.concentric.net> >>>>> "TW" == Thomas Wouters writes: TW> Yes, but more critically so, since it'll screw up more than a TW> couple of defines on a handful of systems :-) Yes, but a "cvs update" should always clue you in that those files needs committing too. Every always does a "cvs update" before committing any files, right? :) TW> But I still think that if this particular makefile setup is TW> used in releases, 'pgen' should at least be made a tad less TW> verbose. pgen also leaks like a sieve, but it's not worth patching. ;} -Barry From bwarsaw@beopen.com Fri Aug 18 22:17:14 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Fri, 18 Aug 2000 17:17:14 -0400 (EDT) Subject: [Python-Dev] Re: [Patches] [Patch #101055] Cookie.py References: <200008181951.MAA30358@bush.i.sourceforge.net> <14749.38716.228254.649957@cj42289-a.reston1.va.home.com> <20000818164837.A8423@kronos.cnri.reston.va.us> Message-ID: <14749.42970.845587.90980@anthem.concentric.net> >>>>> "AK" == Andrew Kuchling writes: AK> I hate throwing away information that stands even a tiny AK> chance of being useful; good thing the price of disk storage AK> keeps dropping, eh? The version history might contain details AK> that will be useful in future debugging or code comprehension, AK> so I dislike losing it forever. I agree. Let's try to keep the revision history for Cookie.py. -Barry From tim_one@email.msn.com Fri Aug 18 22:18:56 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 18 Aug 2000 17:18:56 -0400 Subject: [Python-Dev] [PEP 214] FW: sys.stndout syntax Message-ID: For those of you who stopped reading comp.lang.python, note that this is *typical* of the advice users get in the absence of syntactic "print" sugar. it's-not-exactly-intuitive-that-a-"print"-stmt-isn't-meant-to- do-output -ly y'rs - tim -----Original Message----- From: python-list-admin@python.org [mailto:python-list-admin@python.org]On Behalf Of Jeff Kunce Sent: Friday, August 18, 2000 3:41 PM To: python-list@python.org Subject: Re: sys.stndout syntax > on win32, to get printed output to a file i use > >>> sys.stndout = open('file.txt', 'w') > how do i get it to go back to the command line #Here is the specific solution for those who remember the secret password: sys.stdout = open('file.txt', 'w') print 'this is written to file.txt' sys.stdout.close() sys.stdout = sys.__stdout__ print 'this is written to console' #Here is the general solution if you can't be bothered with remembering secret passwords: original_stdout = sys.stdout sys.stdout = open('file.txt', 'w') print 'this is written to file.txt' sys.stdout.close() sys.stdout = original_stdout print 'this is written to console' --Jeff -- http://www.python.org/mailman/listinfo/python-list From mal@lemburg.com Fri Aug 18 22:21:23 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 18 Aug 2000 23:21:23 +0200 Subject: [Python-Dev] Re: gettext in the standard library References: <14749.42747.411862.940207@anthem.concentric.net> Message-ID: <399DA8D3.70E85C58@lemburg.com> "Barry A. Warsaw" wrote: > > Apologies for duplicates to those of you already on python-dev... > > I've been working on merging all the various implementations of Python > interfaces to the GNU gettext libraries. I've worked from code > contributed by Martin, James, and Peter. I now have something that > seems to work fairly well so I thought I'd update you all. > > After looking at all the various wizzy and experimental stuff in these > implementations, I opted for simplicity, mostly just so I could get my > head around what was needed. My goal was to build a fast C wrapper > module around the C library, and to provide a pure Python > implementation of an identical API for platforms without GNU gettext. Sounds cool. I know that gettext is a standard, but from a technology POV I would have implemented this as codec wich can then be plugged wherever l10n is needed, since strings have the new .encode() method which could just as well be used to convert not only the string into a different encoding, but also a different language. Anyway, just a thought... What I'm missing in your doc-string is a reference as to how well gettext works together with Unicode. After all, i18n is among other things about international character sets. Have you done any experiments in this area ? -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From bwarsaw@beopen.com Fri Aug 18 22:19:12 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Fri, 18 Aug 2000 17:19:12 -0400 (EDT) Subject: [Python-Dev] [Patch #101055] Cookie.py References: <200008181951.MAA30358@bush.i.sourceforge.net> <14749.38716.228254.649957@cj42289-a.reston1.va.home.com> <20000818164837.A8423@kronos.cnri.reston.va.us> <20000818230401.A376@xs4all.nl> Message-ID: <14749.43088.855537.355621@anthem.concentric.net> >>>>> "TW" == Thomas Wouters writes: TW> It would be moderately nice to have the versioning info, TW> though I think Fred has a point when he says that it might be TW> confusing for people: it would look like the file had been in TW> the CVS repository the whole time, and it would be very hard TW> to see where the file had been added to Python. I don't think that's true, because the file won't have the tag information in it. That could be a problem in and of itself, but I dunno. -Barry From tim_one@email.msn.com Fri Aug 18 22:41:18 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 18 Aug 2000 17:41:18 -0400 Subject: [Python-Dev] serious bug in new import X as Y code In-Reply-To: <14749.42903.342401.245594@anthem.concentric.net> Message-ID: > TW> But I still think that if this particular makefile setup is > TW> used in releases, 'pgen' should at least be made a tad less > TW> verbose. [Barry] > pgen also leaks like a sieve, but it's not worth patching. ;} Huh! And here I thought Thomas was suggesting we shorten its name to "pge". or-even-"p"-if-we-wanted-it-a-lot-less-verbose-ly y'rs - tim From bwarsaw@beopen.com Fri Aug 18 22:49:23 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Fri, 18 Aug 2000 17:49:23 -0400 (EDT) Subject: [Python-Dev] Re: gettext in the standard library References: <14749.42747.411862.940207@anthem.concentric.net> <399DA8D3.70E85C58@lemburg.com> Message-ID: <14749.44899.573649.483154@anthem.concentric.net> >>>>> "M" == M writes: M> I know that gettext is a standard, but from a technology POV I M> would have implemented this as codec wich can then be plugged M> wherever l10n is needed, since strings have the new .encode() M> method which could just as well be used to convert not only the M> string into a different encoding, but also a different M> language. Anyway, just a thought... That might be cool to play with, but I haven't done anything with Python's Unicode stuff (and painfully little with gettext too) so right now I don't see how they'd fit together. My gut reaction is that gettext could be the lower level interface to string.encode(language). M> What I'm missing in your doc-string is a reference as to how M> well gettext works together with Unicode. After all, i18n is M> among other things about international character sets. M> Have you done any experiments in this area ? No, but I've thought about it, and I don't think the answer is good. The GNU gettext functions take and return char*'s, which probably isn't very compatible with Unicode. _gettext therefore takes and returns PyStringObjects. We could do better with the pure-Python implementation, and that might be a good reason to forgo any performance gains or platform-dependent benefits you'd get with _gettext. Of course the trick is using the Unicode-unaware tools to build .mo files containing Unicode strings. I don't track GNU gettext developement close enough to know whether they are addressing Unicode issues or not. -Barry From Fredrik Lundh" <399D9D91.3E76ED8D@per.dem.csiro.au> Message-ID: <006801c00960$944da200$f2a6b5d4@hagrid> tim wrote: > > Pure guess on my part -- couldn't imagine why a compiler would warn *unless* > > bits were being lost. the compiler doesn't warn about bits being lost -- it complained because the code was returning a pointer from a function declared to return a long integer. (explicitly casting the pthread_t to a long gets rid of the warning). mark wrote: > > > In summary, whatever issue there was for OSF/1 six (or so) years ago > > > appears to be no longer relevant - but there will be the truncation > > > issue for Win64-like platforms. > > > > And there's Vladimir's "volatile" hack. > > Wonder if that also is still relevant (was it required because of the > long * long * cast?)... probably. messing up when someone abuses pointer casts is one thing, but if the AIX compiler cannot cast a long to a long, it's broken beyond repair ;-) frankly, the code is just plain broken. instead of adding even more dumb hacks, just fix it. here's how it should be done: return (long) pthread_self(); /* look! no variables! */ or change /* Jump through some hoops for Alpha OSF/1 */ to /* Jump through some hoops because Tim Peters wants us to ;-) */ From bwarsaw@beopen.com Fri Aug 18 23:03:24 2000 From: bwarsaw@beopen.com (Barry A. Warsaw) Date: Fri, 18 Aug 2000 18:03:24 -0400 (EDT) Subject: [Python-Dev] [PEP 214] FW: sys.stndout syntax References: Message-ID: <14749.45740.432586.615745@anthem.concentric.net> >>>>> "TP" == Tim Peters writes: TP> For those of you who stopped reading comp.lang.python, note TP> that this is *typical* of the advice users get in the absence TP> of syntactic "print" sugar. Which is of course broken, if say, you print an object that has a str() that raises an exception. From tim_one@email.msn.com Fri Aug 18 23:08:55 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 18 Aug 2000 18:08:55 -0400 Subject: [Python-Dev] pthreads question: typedef ??? pthread_t and hacky return statements In-Reply-To: <006801c00960$944da200$f2a6b5d4@hagrid> Message-ID: [/F] > the compiler doesn't warn about bits being lost -- it complained > because the code was returning a pointer from a function declared > to return a long integer. > > (explicitly casting the pthread_t to a long gets rid of the warning). For the umpty-umpth time, the code with the simple cast to long is what was there originally. The convoluted casting was added later to stop "Alpha OSF/1" compiler complaints. Perhaps the compiler no longer complains, though, or perhaps the one or two people who have tried it since don't have a version of the compiler that cares about it. > ... > frankly, the code is just plain broken. instead of adding even more dumb > hacks, just fix it. here's how it should be done: > > return (long) pthread_self(); /* look! no variables! */ Fine by me, provided that works on all current platforms, and it's understood that the function is inherently hosed anyway (the cast to long is inherently unsafe, and we're still doing nothing to meet the promise in the docs that this function returns a non-zero integer). > or change > > /* Jump through some hoops for Alpha OSF/1 */ > > to > > /* Jump through some hoops because Tim Peters wants us to ;-) */ Also fine by me, provided that works on all current platforms, and it's understood that the function is inherently hosed anyway (the cast to long is inherently unsafe, and we're still doing nothing to meet the promise in the docs that this function returns a non-zero integer). From tim_one@email.msn.com Fri Aug 18 23:14:25 2000 From: tim_one@email.msn.com (Tim Peters) Date: Fri, 18 Aug 2000 18:14:25 -0400 Subject: [Python-Dev] [PEP 214] FW: sys.stndout syntax In-Reply-To: <14749.45740.432586.615745@anthem.concentric.net> Message-ID: > TP> For those of you who stopped reading comp.lang.python, note > TP> that this is *typical* of the advice users get in the absence > TP> of syntactic "print" sugar. [Barry] > Which is of course broken, if say, you print an object that has a > str() that raises an exception. Yes, and if you follow that thread on c.l.py, you'll find that it's also typical for the suggestions to get more and more convoluted (for that and other reasons). From barry@scottb.demon.co.uk Fri Aug 18 23:36:28 2000 From: barry@scottb.demon.co.uk (Barry Scott) Date: Fri, 18 Aug 2000 23:36:28 +0100 Subject: [Python-Dev] Preventing recursion core dumps In-Reply-To: <20000814094440.0BC7F303181@snelboot.oratrix.nl> Message-ID: <000501c00964$c00e0de0$060210ac@private> > -----Original Message----- > From: python-dev-admin@python.org [mailto:python-dev-admin@python.org]On > Behalf Of Jack Jansen > Sent: 14 August 2000 10:45 > To: Guido van Rossum > Cc: Vladimir Marangozov; Python core developers > Subject: Re: [Python-Dev] Preventing recursion core dumps > > > Isn't the solution to this problem to just implement PyOS_CheckStack() for > unix? And for Windows... I still want to control the recursion depth for other reasons than preventing crashes. Especially when I have embedded Python inside my app. (CUrrently I have to defend against a GPF under windows when def x(): x() is called.) Barry From barry@scottb.demon.co.uk Fri Aug 18 23:50:39 2000 From: barry@scottb.demon.co.uk (Barry Scott) Date: Fri, 18 Aug 2000 23:50:39 +0100 Subject: [Python-Dev] Terminal programs (was: Python-dev summary: Jul 1-15) In-Reply-To: <20000718124144.M29590@lyra.org> Message-ID: <000601c00966$bac6f890$060210ac@private> I can second Tera Term Pro. It is one of the few VT100 emulators that gets the emulation right. Many term program get the emulation wrong, often badly. If you do not have the docs for the VT series terminals a devo will not know the details of how the escape sequences should work and apps will fail. BArry (Ex DEC VT expert) > -----Original Message----- > From: python-dev-admin@python.org [mailto:python-dev-admin@python.org]On > Behalf Of Greg Stein > Sent: 18 July 2000 20:42 > To: python-dev@python.org > Subject: [Python-Dev] Terminal programs (was: Python-dev summary: Jul > 1-15) > > > On Tue, Jul 18, 2000 at 10:13:21AM -0400, Andrew Kuchling wrote: > > Thanks to everyone who made some suggestions. The more minor > > edits have been made, but I haven't added any of the threads I missed > > because doing a long stretch of Emacs editing in this lame Windows terminal > > program will drive me insane, so I just posted the summary to python-list. > > > > How is it possible for Microsoft to not get a VT100-compatible > > terminal program working correctly? VT100s have been around since, > > when, the 1970s? Can anyone suggest a Windows terminal program that > > *doesn't* suck dead bunnies through a straw? > > yes. > > I use "Tera Term Pro" with the SSH extensions. That gives me an excellent > Telnet app, and it gives me SSH. I have never had a problem with it. > > [ initially, there is a little tweakiness with creating the "known_hosts" > file, but you just hit "continue" and everything is fine after that. ] > > Tera Term Pro can be downloaded from some .jp address. I think there is a > 16-bit vs 32-bit program. I use the latter. The SSL stuff is located in Oz, > me thinks. > > I've got it on the laptop. Great stuff. > > Cheers, > -g > > -- > Greg Stein, http://www.lyra.org/ > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://www.python.org/mailman/listinfo/python-dev > From james@daa.com.au Sat Aug 19 01:54:30 2000 From: james@daa.com.au (James Henstridge) Date: Sat, 19 Aug 2000 08:54:30 +0800 (WST) Subject: [Python-Dev] Re: gettext in the standard library In-Reply-To: <399DA8D3.70E85C58@lemburg.com> Message-ID: On Fri, 18 Aug 2000, M.-A. Lemburg wrote: > What I'm missing in your doc-string is a reference as to how > well gettext works together with Unicode. After all, i18n is > among other things about international character sets. > Have you done any experiments in this area ? At the C level, the extent to which gettext supports unicode is if the catalog was encoded in the UTF8 encoding. As an example, GTK (a GUI toolkit) is moving to pango (a library used to allow display of multiple languages at once), all the catalogs for GTK programs will have to be reencoded in UTF8. I don't know if it is worth adding explicit support to the python gettext module though. James. -- Email: james@daa.com.au WWW: http://www.daa.com.au/~james/ From fdrake@beopen.com Sat Aug 19 02:16:33 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Fri, 18 Aug 2000 21:16:33 -0400 (EDT) Subject: [Python-Dev] [Patch #101055] Cookie.py In-Reply-To: <14749.43088.855537.355621@anthem.concentric.net> References: <200008181951.MAA30358@bush.i.sourceforge.net> <14749.38716.228254.649957@cj42289-a.reston1.va.home.com> <20000818164837.A8423@kronos.cnri.reston.va.us> <20000818230401.A376@xs4all.nl> <14749.43088.855537.355621@anthem.concentric.net> Message-ID: <14749.57329.966314.171906@cj42289-a.reston1.va.home.com> Barry A. Warsaw writes: > I don't think that's true, because the file won't have the tag > information in it. That could be a problem in and of itself, but I > dunno. The confusion isn't from the tags, but the dates; if the ,v was created 2 years ago, asking for the python tree as of a year ago (using -D ) will include the file, even though it wasn't part of our repository then. Asking for a specific tag (using -r ) will properly not include it unless there's a matching tag there. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From james@daa.com.au Sat Aug 19 02:26:44 2000 From: james@daa.com.au (James Henstridge) Date: Sat, 19 Aug 2000 09:26:44 +0800 (WST) Subject: [Python-Dev] Re: gettext in the standard library In-Reply-To: <14749.42747.411862.940207@anthem.concentric.net> Message-ID: On Fri, 18 Aug 2000, Barry A. Warsaw wrote: > > Apologies for duplicates to those of you already on python-dev... > > I've been working on merging all the various implementations of Python > interfaces to the GNU gettext libraries. I've worked from code > contributed by Martin, James, and Peter. I now have something that > seems to work fairly well so I thought I'd update you all. > > After looking at all the various wizzy and experimental stuff in these > implementations, I opted for simplicity, mostly just so I could get my > head around what was needed. My goal was to build a fast C wrapper > module around the C library, and to provide a pure Python > implementation of an identical API for platforms without GNU gettext. Sounds good. Most of the experimental stuff in my module turned out to not be very useful. Having a simple gettext module plus your pyxgettext script should be enough. > > I started with Martin's libintlmodule, renamed it _gettext and cleaned > up the C code a bit. This provides gettext(), dgettext(), > dcgettext(), textdomain(), and bindtextdomain() functions. The > gettext.py module imports these, and if it succeeds, it's done. > > If that fails, then there's a bunch of code, mostly derived from > Peter's fintl.py module, that reads the binary .mo files and does the > look ups itself. Note that Peter's module only supported the GNU > gettext binary format, and that's all mine does too. It should be > easy to support other binary formats (Solaris?) by overriding one > method in one class, and contributions are welcome. I think support for Solaris big endian format .po files would probably be a good idea. It is not very difficult and doesn't really add to the complexity. > > James's stuff looked cool too, what I grokked of it :) but I think > those should be exported as higher level features. I didn't include > the ability to write .mo files or the exported Catalog objects. I > haven't used the I18N services enough to know whether these are > useful. As I said above, most of that turned out not to be very useful. Did you include any of the language selection code in the last version of my gettext module? It gave behaviour very close to C gettext in this respect. It expands the locale name given by the user using the locale.alias files found on the systems, then decomposes that into the simpler forms. For instance, if LANG=en_GB, then my gettext module would search for catalogs by names: ['en_GB.ISO8859-1', 'en_GB', 'en.ISO8859-1', 'en', 'C'] This also allows things like expanding LANG=catalan to: ['ca_ES.ISO-8859-1', 'ca_ES', 'ca.ISO-8859-1', 'ca', 'C'] (provided the appropriate locale.alias files are found) If you missed that that version I sent you I can send it again. It stripped out a lot of the experimental code giving a much simpler module. > > I added one convenience function, gettext.install(). If you call > this, it inserts the gettext.gettext() function into the builtins > namespace as `_'. You'll often want to do this, based on the I18N > translatable strings marking conventions. Note that importing gettext > does /not/ install by default! That sounds like a good idea that will make things a lot easier in the common case. > > And since (I think) you'll almost always want to call bindtextdomain() > and textdomain(), you can pass the domain and localedir in as > arguments to install. Thus, the simple and quick usage pattern is: > > import gettext > gettext.install('mydomain', '/my/locale/dir') > > print _('this is a localized message') > > I think it'll be easier to critique this stuff if I just check it in. > Before I do, I still need to write up a test module and hack together > docos. In the meantime, here's the module docstring for gettext.py. > Talk amongst yourselves. :) > > -Barry James. -- Email: james@daa.com.au WWW: http://www.daa.com.au/~james/ From Vladimir.Marangozov@inrialpes.fr Sat Aug 19 04:27:20 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Sat, 19 Aug 2000 05:27:20 +0200 (CEST) Subject: [Python-Dev] Adding insint() function In-Reply-To: <14749.33584.683341.684523@cj42289-a.reston1.va.home.com> from "Fred L. Drake, Jr." at Aug 18, 2000 02:40:48 PM Message-ID: <200008190327.FAA10001@python.inrialpes.fr> Fred L. Drake, Jr. wrote: > > > Vladimir Marangozov writes: > > So name it PyModule_AddConstant(module, name, constant), > > which fails with "can't add constant to module" err msg. > > Even better! I expect there should be at least a couple of these; > one for ints, one for strings. > What about something like this (untested): ------------------------------------------------------------------------ int PyModule_AddObject(PyObject *m, char *name, PyObject *o) { if (!PyModule_Check(m) || o == NULL) return -1; if (PyDict_SetItemString(((PyModuleObject *)m)->md_dict, name, o)) return -1; Py_DECREF(o); return 0; } #define PyModule_AddConstant(m, x) \ PyModule_AddObject(m, #x, PyInt_FromLong(x)) #define PyModule_AddString(m, x) \ PyModule_AddObject(m, x, PyString_FromString(x)) ------------------------------------------------------------------------ void initmymodule(void) { int CONSTANT = 123456; char *STR__doc__ = "Vlad"; PyObject *m = Py_InitModule4("mymodule"...); if (PyModule_AddString(m, STR__doc__) || PyModule_AddConstant(m, CONSTANT) || ... { Py_FatalError("can't init mymodule"); } } -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From cgw@fnal.gov Sat Aug 19 04:55:21 2000 From: cgw@fnal.gov (Charles G Waldman) Date: Fri, 18 Aug 2000 22:55:21 -0500 (CDT) Subject: [Python-Dev] RE: compile.c: problem with duplicate argument bugfix Message-ID: <14750.1321.978274.117748@buffalo.fnal.gov> I'm catching up on the python-dev archives and see your message. Note that I submitted a patch back in May to fix this same problem: http://www.python.org/pipermail/patches/2000-May/000638.html There you will find a working patch, and a detailed discussion which explains why your approach results in a core-dump. I submitted this patch back before Python moved over to SourceForge, there was a small amount of discussion about it and then the word from Guido was "I'm too busy to look at this now", and the patch got dropped on the floor. From tim_one@email.msn.com Sat Aug 19 05:11:41 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 19 Aug 2000 00:11:41 -0400 Subject: [Python-Dev] RE: [Patches] [Patch #101055] Cookie.py In-Reply-To: <14749.38716.228254.649957@cj42289-a.reston1.va.home.com> Message-ID: Moving this over from patches to python-dev. My 2 cents: The primary job of a source control system is to maintain an accurate and easily retrieved historical record of a project. Tim O'Malley's ,v file records the history of his project, Python's should record the history of its. While a handful of people at CNRI have been able to (or could, if they were of a common mind to) keep track of a handful of exceptions in their heads, Python's CVS tree is available to the world now, and approximately nobody looking at it will have any knowledge of this discussion. If they ask CVS for a date-based snapshot of the past, they're using CVS for what it's *designed* for, and they should get back what they asked for. Have these kinds of tricks already been played in the CVS tree? I'm mildly concerned about that too, because whenever license or copyright issues are in question, an accurate historical record is crucial ("Now, Mr. Kuchling, isn't it true that you deliberately sabotaged the history of the Python project in order to obscure your co-conspirators' theft of my client's intellectual property?" <0.9 wink>). let's-honor-the-past-by-keeping-it-honest-ly y'rs - tim > -----Original Message----- > From: patches-admin@python.org [mailto:patches-admin@python.org]On > Behalf Of Fred L. Drake, Jr. > Sent: Friday, August 18, 2000 4:06 PM > To: noreply@sourceforge.net > Cc: akuchlin@mems-exchange.org; patches@python.org > Subject: Re: [Patches] [Patch #101055] Cookie.py > > > > noreply@sourceforge.net writes: > > I have a copy of Tim O'Malley's ,v file (in order to preserve the > > version history). I can either ask the SF admins to drop it into > > the right place in the CVS repository, but will that screw up the > > Python 1.6 tagging in some way? (I would expect not, but who > > knows?) > > That would have no effect on any of the Python tagging. It's > probably worthwhile making sure there are no tags in the ,v file, but > that can be done after it gets dropped in place. > Now, Greg Stein will tell us that dropping this into place is the > wrong thing to do. What it *will* screw up is people asking for the > state of Python at a specific date before the file was actually added; > they'll get this file even for when it wasn't in the Python CVS tree. > I can live with that, but we should make a policy decision for the > Python tree regarding this sort of thing. > > > The second option would be for me to retrace Cookie.py's > > development -- add revision 1.1, check in revision 1.2 with the > > right log message, check in revision 1.3, &c. Obviously I'd prefer > > to not do this. > > Don't -- it's not worth it. > > > -Fred > > -- > Fred L. Drake, Jr. > BeOpen PythonLabs Team Member > > > _______________________________________________ > Patches mailing list > Patches@python.org > http://www.python.org/mailman/listinfo/patches From cgw@fnal.gov Sat Aug 19 05:31:06 2000 From: cgw@fnal.gov (Charles G Waldman) Date: Fri, 18 Aug 2000 23:31:06 -0500 (CDT) Subject: [Python-Dev] Eureka! (Re: test_fork fails --with-thread) Message-ID: <14750.3466.34096.504552@buffalo.fnal.gov> Last month there was a flurry of discussion, around http://www.python.org/pipermail/python-dev/2000-July/014208.html about problems arising when combining threading and forking. I've been reading through the python-dev archives and as far as I can tell this problem has not yet been resolved. Well, I think I understand what's going on and I have a patch that fixes the problem. Contrary to some folklore, you *can* use fork() in threaded code; you just have to be a bit careful about locks... Rather than write up a long-winded explanation myself, allow me to quote: ----------------------------------------------------------------- from "man pthread_atfork": ... recall that fork(2) duplicates the whole memory space, including mutexes in their current locking state, but only the calling thread: other threads are not running in the child process. Thus, if a mutex is locked by a thread other than the thread calling fork, that mutex will remain locked forever in the child process, possibly blocking the execu- tion of the child process. and from http://www.lambdacs.com/newsgroup/FAQ.html#Q120 Q120: Calling fork() from a thread > Can I fork from within a thread ? Absolutely. > If that is not explicitly forbidden, then what happens to the > other threads in the child process ? There ARE no other threads in the child process. Just the one that forked. If your application/library has background threads that need to exist in a forked child, then you should set up an "atfork" child handler (by calling pthread_atfork) to recreate them. And if you use mutexes, and want your application/library to be "fork safe" at all, you also need to supply an atfork handler set to pre-lock all your mutexes in the parent, then release them in the parent and child handlers. Otherwise, ANOTHER thread might have a mutex locked when one thread forks -- and because the owning thread doesn't exist in the child, the mutex could never be released. (And, worse, whatever data is protected by the mutex is in an unknown and inconsistent state.) ------------------------------------------------------------------- Below is a patch (I will also post this to SourceForge) Notes on the patch: 1) I didn't make use of pthread_atfork, because I don't know how portable it is. So, if somebody uses "fork" in a C extension there will still be trouble. 2) I'm deliberately not cleaning up the old lock before creating the new one, because the lock destructors also do error-checking. It might be better to add a PyThread_reset_lock function to all the thread_*.h files, but I'm hesitant to do this because of the amount of testing required. Patch: Index: Modules/signalmodule.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/signalmodule.c,v retrieving revision 2.53 diff -c -r2.53 signalmodule.c *** Modules/signalmodule.c 2000/08/03 02:34:44 2.53 --- Modules/signalmodule.c 2000/08/19 03:37:52 *************** *** 667,672 **** --- 667,673 ---- PyOS_AfterFork(void) { #ifdef WITH_THREAD + PyEval_ReInitThreads(); main_thread = PyThread_get_thread_ident(); main_pid = getpid(); #endif Index: Parser/intrcheck.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Parser/intrcheck.c,v retrieving revision 2.39 diff -c -r2.39 intrcheck.c *** Parser/intrcheck.c 2000/07/31 15:28:04 2.39 --- Parser/intrcheck.c 2000/08/19 03:37:54 *************** *** 206,209 **** --- 206,212 ---- void PyOS_AfterFork(void) { + #ifdef WITH_THREAD + PyEval_ReInitThreads(); + #endif } Index: Python/ceval.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/ceval.c,v retrieving revision 2.191 diff -c -r2.191 ceval.c *** Python/ceval.c 2000/08/18 19:53:25 2.191 --- Python/ceval.c 2000/08/19 03:38:06 *************** *** 142,147 **** --- 142,165 ---- Py_FatalError("PyEval_ReleaseThread: wrong thread state"); PyThread_release_lock(interpreter_lock); } + + /* This function is called from PyOS_AfterFork to ensure that newly + created child processes don't hold locks referring to threads which + are not running in the child process. (This could also be done using + pthread_atfork mechanism, at least for the pthreads implementation) */ + void + PyEval_ReInitThreads(void) + { + if (!interpreter_lock) + return; + /*XXX Can't use PyThread_free_lock here because it does too + much error-checking. Doing this cleanly would require + adding a new function to each thread_*.h. Instead, just + create a new lock and waste a little bit of memory */ + interpreter_lock = PyThread_allocate_lock(); + PyThread_acquire_lock(interpreter_lock, 1); + main_thread = PyThread_get_thread_ident(); + } #endif /* Functions save_thread and restore_thread are always defined so From esr@thyrsus.com Sat Aug 19 06:17:17 2000 From: esr@thyrsus.com (Eric S. Raymond) Date: Sat, 19 Aug 2000 01:17:17 -0400 Subject: [Python-Dev] Request for help w/ bsddb module In-Reply-To: <20000817224632.A525@207-172-146-154.s154.tnt3.ann.va.dialup.rcn.com>; from amk@s154.tnt3.ann.va.dialup.rcn.com on Thu, Aug 17, 2000 at 10:46:32PM -0400 References: <20000817224632.A525@207-172-146-154.s154.tnt3.ann.va.dialup.rcn.com> Message-ID: <20000819011717.L835@thyrsus.com> A.M. Kuchling : > (Can this get done in time for Python 2.0? Probably. Can it get > tested in time for 2.0? Ummm....) I have zero experience with writing C extensions, so I'm probably not best deployed on this. But I'm willing to help with testing. -- Eric S. Raymond "As to the species of exercise, I advise the gun. While this gives [only] moderate exercise to the body, it gives boldness, enterprise, and independence to the mind. Games played with the ball and others of that nature, are too violent for the body and stamp no character on the mind. Let your gun, therefore, be the constant companion to your walks." -- Thomas Jefferson, writing to his teenaged nephew. From tim_one@email.msn.com Sat Aug 19 06:11:28 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 19 Aug 2000 01:11:28 -0400 Subject: [Python-Dev] Who can make test_fork1 fail? Message-ID: Note that a patch has been posted to SourceForge that purports to solve *some* thread vs fork problems: http://sourceforge.net/patch/?func=detailpatch&patch_id=101226&group_id=5470 Since nobody has made real progress on figuring out why test_fork1 fails on some systems, would somebody who is able to make it fail please just try this patch & see what happens? understanding-is-better-than-a-fix-but-i'll-settle-for-the-latter-ly y'rs - tim From cgw@fnal.gov Sat Aug 19 06:26:33 2000 From: cgw@fnal.gov (Charles G Waldman) Date: Sat, 19 Aug 2000 00:26:33 -0500 (CDT) Subject: [Python-Dev] Who can make test_fork1 fail? In-Reply-To: References: Message-ID: <14750.6793.342815.211141@buffalo.fnal.gov> Tim Peters writes: > Since nobody has made real progress on figuring out why test_fork1 > fails on some systems, would somebody who is able to make it fail > please just try this patch & see what happens? Or try this program (based on Neil's example), which will fail almost immediately unless you apply my patch: import thread import os, sys import time def doit(name): while 1: if os.fork()==0: print name, 'forked', os.getpid() os._exit(0) r = os.wait() for x in range(50): name = 't%s'%x print 'starting', name thread.start_new_thread(doit, (name,)) time.sleep(300) From tim_one@email.msn.com Sat Aug 19 06:59:12 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 19 Aug 2000 01:59:12 -0400 Subject: [Python-Dev] Who can make test_fork1 fail? In-Reply-To: <14750.6793.342815.211141@buffalo.fnal.gov> Message-ID: [Tim] > Since nobody has made real progress on figuring out why test_fork1 > fails on some systems, would somebody who is able to make it fail > please just try this patch & see what happens? [Charles G Waldman] > Or try this program (based on Neil's example), which will fail almost > immediately unless you apply my patch: Not "or", please, "both". Without understanding the problem in detail, we have no idea how many bugs are lurking here. For example, Python allocates at least two locks besides "the global lock", and "doing something" about the latter alone may not help with all the failing test cases. Note too that the pthread_atfork docs were discussed earlier, and neither Guido nor I were able to dream up a scenario that accounted for the details of most failures people *saw*: we both stumbled into another (and the same) failing scenario, but it didn't match the stacktraces people posted (which showed deadlocks/hangs in the *parent* thread; but at a fork, only the state of the locks in the child "can" get screwed up). The patch you posted should plug the "deadlock in the child" scenario we did understand, but that scenario didn't appear to be relevant in most cases. The more info the better, let's just be careful to test *everything* that failed before writing this off. From ping@lfw.org Sat Aug 19 07:43:18 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Fri, 18 Aug 2000 23:43:18 -0700 (PDT) Subject: [Python-Dev] 'import as' In-Reply-To: <20000818182246.V376@xs4all.nl> Message-ID: My $0.02. +1 on: import as import as +1 on: from import as from import as +1 on: from . import as from . import as -1 on *either* meaning of: import . as ...as it's not clear what the correct meaning is. If the intent of this last form is to import a sub-module of a package into the local namespace with an aliased name, then you can just say from import as and the meaning is then quite clear. -- ?!ng From ping@lfw.org Sat Aug 19 07:38:10 2000 From: ping@lfw.org (Ka-Ping Yee) Date: Fri, 18 Aug 2000 23:38:10 -0700 (PDT) Subject: [Python-Dev] Re: indexing, indices(), irange(), list.items() In-Reply-To: <14749.18016.323403.295212@cj42289-a.reston1.va.home.com> Message-ID: On Fri, 18 Aug 2000, Fred L. Drake, Jr. wrote: > I hadn't considered *not* using an "in" clause, but that is actually > pretty neat. I'd like to see all of these allowed; disallowing "for i > indexing e in ...:" reduces the intended functionality substantially. I like them all as well (and had previously assumed that the "indexing" proposal included the "for i indexing sequence" case!). While we're sounding off on the issue, i'm quite happy (+1) on both of: for el in seq: for i indexing seq: for i indexing el in seq: and for el in seq: for i in indices(seq): for i, el in irange(seq): with a slight preference for the former. -- ?!ng From loewis@informatik.hu-berlin.de Sat Aug 19 08:25:20 2000 From: loewis@informatik.hu-berlin.de (Martin von Loewis) Date: Sat, 19 Aug 2000 09:25:20 +0200 (MET DST) Subject: [Python-Dev] Re: gettext in the standard library In-Reply-To: <399DA8D3.70E85C58@lemburg.com> (mal@lemburg.com) References: <14749.42747.411862.940207@anthem.concentric.net> <399DA8D3.70E85C58@lemburg.com> Message-ID: <200008190725.JAA26022@pandora.informatik.hu-berlin.de> > What I'm missing in your doc-string is a reference as to how > well gettext works together with Unicode. After all, i18n is > among other things about international character sets. > Have you done any experiments in this area ? I have, to some degree. As others pointed out, gettext maps byte arrays to byte arrays. However, in the GNU internationalization project, it is convention to put an entry like msgid "" msgstr "" "Project-Id-Version: GNU grep 2.4\n" "POT-Creation-Date: 1999-11-13 11:33-0500\n" "PO-Revision-Date: 1999-12-07 10:10+01:00\n" "Last-Translator: Martin von L=F6wis \n" "Language-Team: German \n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=3DISO-8859-1\n" "Content-Transfer-Encoding: 8-bit\n" into the catalog, which can be accessed as translation of the empty string. It typically has a charset=3D element, which allows to analyse what character set is used in the catalog. Of course, this is a convention only, so it may not be present. If it is absent, and conversion to Unicode is requested, it is probably a good idea to assume UTF-8 (as James indicated, that will be the GNOME coded character set for catalogs, for example). In any case, I think it is a good idea to support retrieval of translated strings as Unicode objects. I can think of two alternative interfaces: gettext.gettext(msgid, unicode=3D1) #or gettext.unigettext(msgid) Of course, if applications install _, they'd know whether they want unicode or byte strings, so _ would still take a single argument. However, I don't think that this feature must be there at the first checkin; I'd volunteer to work on a patch after Barry has installed his code, and after I got some indication what the interface should be. Regards, Martin From tim_one@email.msn.com Sat Aug 19 10:19:23 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 19 Aug 2000 05:19:23 -0400 Subject: [Python-Dev] RE: Call for reviewer! In-Reply-To: Message-ID: [David Goodger] > I thought the "backwards compatibility" issue might be a sticking point ;> > And I *can* see why. > > So, if I were to rework the patch to remove the incompatibility, would it > fly or still be shot down? I'm afraid "shot down" is the answer, but it's no reflection on the quality of your work. Guido simply doesn't want any enhancements of any kind to getopt to be distributed in the standard library. He made that very clear in a conference call with the PythonLabs team, and as the interim 2.0 release manager du jour I pass that on in his absence. This wasn't a surprise to me, as there's a very long history of rejected getopt patches. There are certainly users who *want* fancier getopt capabilities! The problem in making them happy is threefold: (1) most users don't (as the lack of positive response in this thread on Python-Dev confirms); (2) users who do want them seem unable to agree on how they should work (witness the bifurcation in your own patch set); and, (3) Guido actively wants to keep the core getopt simple in the absence of both demand for, and consensus on, more than it offers already. This needn't mean your work is dead. It will find users if it you make it available on the web, and even in the core Andrew Kuchling pointed out that the Distutils folks are keen to have a fancier getopt for their own purposes: [Andrew] > Note that there's Lib/distutils/fancy_getopt.py. The docstring reads: > > Wrapper around the standard getopt module that provides the following > additional features: > * short and long options are tied together > * options have help strings, so fancy_getopt could potentially > create a complete usage summary > * options set attributes of a passed-in object So you might want to talk to Gred Ward about that too (Greg is the Distuils Dood). [back to David] > ... > BUT WAIT, THERE'S MORE! As part of the deal, you get a free > test_getopt.py regression test module! Act now; vote +1! (Actually, > you'll get that no matter what you vote. I'll remove the getoptdict- > specific stuff and resubmit it if this patch is rejected.) We don't have to ask Guido abut *that*: a test module for getopt would be accepted with extreme (yet intangible ) gratitude. Thank you! From mal@lemburg.com Sat Aug 19 10:28:32 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Sat, 19 Aug 2000 11:28:32 +0200 Subject: [Python-Dev] Re: gettext in the standard library References: <14749.42747.411862.940207@anthem.concentric.net> <399DA8D3.70E85C58@lemburg.com> <200008190725.JAA26022@pandora.informatik.hu-berlin.de> Message-ID: <399E5340.B00811EF@lemburg.com> Martin von Loewis wrote: > > In any case, I think it is a good idea to support retrieval of > translated strings as Unicode objects. I can think of two alternative > interfaces: > > gettext.gettext(msgid, unicode=1) > #or > gettext.unigettext(msgid) > > Of course, if applications install _, they'd know whether they want > unicode or byte strings, so _ would still take a single argument. Hmm, if your catalogs are encoded in UTF-8 and use non-ASCII chars then the traditional API would have to raise encoding errors -- probably not a good idea since the errors would be hard to deal with in large applications. Perhaps the return value type of .gettext() should be given on the .install() call: e.g. encoding='utf-8' would have .gettext() return a string using UTF-8 while encoding='unicode' would have it return Unicode objects. [Which makes me think: perhaps I should add a new codec which does pretty much the same as the unicode() call: convert the input data to Unicode ?!] > However, I don't think that this feature must be there at the first > checkin; I'd volunteer to work on a patch after Barry has installed > his code, and after I got some indication what the interface should > be. Right. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mal@lemburg.com Sat Aug 19 10:37:28 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Sat, 19 Aug 2000 11:37:28 +0200 Subject: [Python-Dev] Re: gettext in the standard library References: <14749.42747.411862.940207@anthem.concentric.net> <399DA8D3.70E85C58@lemburg.com> <14749.44899.573649.483154@anthem.concentric.net> Message-ID: <399E5558.C7B6029B@lemburg.com> "Barry A. Warsaw" wrote: > > >>>>> "M" == M writes: > > M> I know that gettext is a standard, but from a technology POV I > M> would have implemented this as codec wich can then be plugged > M> wherever l10n is needed, since strings have the new .encode() > M> method which could just as well be used to convert not only the > M> string into a different encoding, but also a different > M> language. Anyway, just a thought... > > That might be cool to play with, but I haven't done anything with > Python's Unicode stuff (and painfully little with gettext too) so > right now I don't see how they'd fit together. My gut reaction is > that gettext could be the lower level interface to > string.encode(language). Oh, codecs are not just about Unicode. Normal string objects also have an .encode() method which can be used for these purposes as well. > M> What I'm missing in your doc-string is a reference as to how > M> well gettext works together with Unicode. After all, i18n is > M> among other things about international character sets. > M> Have you done any experiments in this area ? > > No, but I've thought about it, and I don't think the answer is good. > The GNU gettext functions take and return char*'s, which probably > isn't very compatible with Unicode. _gettext therefore takes and > returns PyStringObjects. Martin mentioned the possibility of using UTF-8 for the catalogs and then decoding them into Unicode. That should be a reasonable way of getting .gettext() to talk Unicode :-) > We could do better with the pure-Python implementation, and that might > be a good reason to forgo any performance gains or platform-dependent > benefits you'd get with _gettext. Of course the trick is using the > Unicode-unaware tools to build .mo files containing Unicode strings. > I don't track GNU gettext developement close enough to know whether > they are addressing Unicode issues or not. Just dreaming a little here: I would prefer that we use some form of XML to write the catalogs. XML comes with Unicode support and tools for writing XML are available too. We'd only need a way to transform XML into catalog files of some Python specific platform independent format (should be possible to create .mo files from XML too). -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mal@lemburg.com Sat Aug 19 10:44:19 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Sat, 19 Aug 2000 11:44:19 +0200 Subject: [Python-Dev] Re: gettext in the standard library References: Message-ID: <399E56F3.53799860@lemburg.com> James Henstridge wrote: > > On Fri, 18 Aug 2000, Barry A. Warsaw wrote: > > > I started with Martin's libintlmodule, renamed it _gettext and cleaned > > up the C code a bit. This provides gettext(), dgettext(), > > dcgettext(), textdomain(), and bindtextdomain() functions. The > > gettext.py module imports these, and if it succeeds, it's done. > > > > If that fails, then there's a bunch of code, mostly derived from > > Peter's fintl.py module, that reads the binary .mo files and does the > > look ups itself. Note that Peter's module only supported the GNU > > gettext binary format, and that's all mine does too. It should be > > easy to support other binary formats (Solaris?) by overriding one > > method in one class, and contributions are welcome. > > I think support for Solaris big endian format .po files would probably be > a good idea. It is not very difficult and doesn't really add to the > complexity. > > > > > James's stuff looked cool too, what I grokked of it :) but I think > > those should be exported as higher level features. I didn't include > > the ability to write .mo files or the exported Catalog objects. I > > haven't used the I18N services enough to know whether these are > > useful. > > As I said above, most of that turned out not to be very useful. Did you > include any of the language selection code in the last version of my > gettext module? It gave behaviour very close to C gettext in this > respect. It expands the locale name given by the user using the > locale.alias files found on the systems, then decomposes that into the > simpler forms. For instance, if LANG=en_GB, then my gettext module would > search for catalogs by names: > ['en_GB.ISO8859-1', 'en_GB', 'en.ISO8859-1', 'en', 'C'] > > This also allows things like expanding LANG=catalan to: > ['ca_ES.ISO-8859-1', 'ca_ES', 'ca.ISO-8859-1', 'ca', 'C'] > (provided the appropriate locale.alias files are found) > > If you missed that that version I sent you I can send it again. It > stripped out a lot of the experimental code giving a much simpler module. Uhm, can't you make some use of the new APIs in locale.py for this ? locale.py has a whole new set of encoding aware support for LANG variables. It supports Unix and Windows (thanks to /F). -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mwh21@cam.ac.uk Sat Aug 19 10:52:00 2000 From: mwh21@cam.ac.uk (Michael Hudson) Date: 19 Aug 2000 10:52:00 +0100 Subject: [Python-Dev] [Patch #101055] Cookie.py In-Reply-To: "Fred L. Drake, Jr."'s message of "Fri, 18 Aug 2000 21:16:33 -0400 (EDT)" References: <200008181951.MAA30358@bush.i.sourceforge.net> <14749.38716.228254.649957@cj42289-a.reston1.va.home.com> <20000818164837.A8423@kronos.cnri.reston.va.us> <20000818230401.A376@xs4all.nl> <14749.43088.855537.355621@anthem.concentric.net> <14749.57329.966314.171906@cj42289-a.reston1.va.home.com> Message-ID: "Fred L. Drake, Jr." writes: > Barry A. Warsaw writes: > > I don't think that's true, because the file won't have the tag > > information in it. That could be a problem in and of itself, but I > > dunno. > > The confusion isn't from the tags, but the dates; if the ,v was > created 2 years ago, asking for the python tree as of a year ago > (using -D ) will include the file, even though it wasn't part of > our repository then. Asking for a specific tag (using -r ) will > properly not include it unless there's a matching tag there. Is it feasible to hack the dates in the ,v file so that it looks like all the revisions happened between say 2000.08.19.10.50.00 and 2000.08.19.10.51.00 ? This probably has problems too, but they will be more subtle... Cheers, M. -- That's why the smartest companies use Common Lisp, but lie about it so all their competitors think Lisp is slow and C++ is fast. (This rumor has, however, gotten a little out of hand. :) -- Erik Naggum, comp.lang.lisp From Vladimir.Marangozov@inrialpes.fr Sat Aug 19 11:23:12 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Sat, 19 Aug 2000 12:23:12 +0200 (CEST) Subject: [Python-Dev] RE: Introducing memprof (was PyErr_NoMemory) In-Reply-To: from "Tim Peters" at Aug 18, 2000 03:11:11 PM Message-ID: <200008191023.MAA11071@python.inrialpes.fr> Tim Peters wrote: > > My bandwidth is consumed by 2.0 issues, so I won't look at it. On the > chance that Guido gets hit by a bus, though, and I have time to kill at his > funeral, it would be nice to have it available on SourceForge. Uploading a > postponed patch sounds fine! Done. Both patches are updated and relative to current CVS: Optional object malloc: http://sourceforge.net/patch/?func=detailpatch&patch_id=101104&group_id=5470 Optional memory profiler: http://sourceforge.net/patch/?func=detailpatch&patch_id=101229&group_id=5470 Let me insist again that these are totally optional and off by default (lately, a recurrent wish of mine regarding proposed features). Since they're optional, off by default, and consitute a solid base for further work on mem + GC, and despite the tiny imperfections I see in the profiler, I think I'm gonna push a bit, given that I'm pretty confident in the code and that it barely affects anything. So while I'm out of town, my mailbox would be happy to register any opinions that the python-dev crowd might have (I'm thinking of Barry and Neil Schemenauer in particular). Also, when BDFL is back from Palo Alto, give him a chance to emit a statement (although I know he's not a memory fan ). I'll *try* to find some time for docs and test cases, but I'd like to get some preliminary feedback first (especially if someone care to try this on a 64 bit machine). That's it for now. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From fdrake@beopen.com Sat Aug 19 13:44:00 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Sat, 19 Aug 2000 08:44:00 -0400 (EDT) Subject: [Python-Dev] 'import as' In-Reply-To: References: <20000818182246.V376@xs4all.nl> Message-ID: <14750.33040.285051.600113@cj42289-a.reston1.va.home.com> Ka-Ping Yee writes: > If the intent of this last form is to import a sub-module of a > package into the local namespace with an aliased name, then you > can just say > > from import as I could live with this restriction, and this expression is unambiguous (a good thing for Python!). -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From Moshe Zadka Sat Aug 19 14:54:21 2000 From: Moshe Zadka (Moshe Zadka) Date: Sat, 19 Aug 2000 16:54:21 +0300 (IDT) Subject: [Python-Dev] Intent to document: Cookie.py Message-ID: This is just a notice that I'm currently in the middle of documenting Cookie. I should be finished sometime today. This is just to stop anyone else from wasting his time -- if you got time to kill, you can write a test suite -- Moshe Zadka There is no IGLU cabal. http://advogato.org/person/moshez From james@daa.com.au Sat Aug 19 15:14:12 2000 From: james@daa.com.au (James Henstridge) Date: Sat, 19 Aug 2000 22:14:12 +0800 (WST) Subject: [Python-Dev] Re: gettext in the standard library In-Reply-To: <399E56F3.53799860@lemburg.com> Message-ID: On Sat, 19 Aug 2000, M.-A. Lemburg wrote: > > As I said above, most of that turned out not to be very useful. Did you > > include any of the language selection code in the last version of my > > gettext module? It gave behaviour very close to C gettext in this > > respect. It expands the locale name given by the user using the > > locale.alias files found on the systems, then decomposes that into the > > simpler forms. For instance, if LANG=en_GB, then my gettext module would > > search for catalogs by names: > > ['en_GB.ISO8859-1', 'en_GB', 'en.ISO8859-1', 'en', 'C'] > > > > This also allows things like expanding LANG=catalan to: > > ['ca_ES.ISO-8859-1', 'ca_ES', 'ca.ISO-8859-1', 'ca', 'C'] > > (provided the appropriate locale.alias files are found) > > > > If you missed that that version I sent you I can send it again. It > > stripped out a lot of the experimental code giving a much simpler module. > > Uhm, can't you make some use of the new APIs in locale.py > for this ? > > locale.py has a whole new set of encoding aware support for > LANG variables. It supports Unix and Windows (thanks to /F). Well, it can do a little more than that. It will also handle the case of a number of locales listed in the LANG environment variable. It also doesn't look like it handles decomposition of a locale like ll_CC.encoding@modifier into other matching encodings in the correct precedence order. Maybe something to do this sort of decomposition would fit better in locale.py though. This sort of thing is very useful for people who know more than one language, and doesn't seem to be handled by plain setlocale() > > -- > Marc-Andre Lemburg James. -- Email: james@daa.com.au WWW: http://www.daa.com.au/~james/ From fdrake@beopen.com Sat Aug 19 15:14:27 2000 From: fdrake@beopen.com (Fred L. Drake, Jr.) Date: Sat, 19 Aug 2000 10:14:27 -0400 (EDT) Subject: [Python-Dev] Intent to document: Cookie.py In-Reply-To: References: Message-ID: <14750.38467.274688.274349@cj42289-a.reston1.va.home.com> Moshe Zadka writes: > This is just a notice that I'm currently in the middle of documenting > Cookie. I should be finished sometime today. This is just to stop anyone > else from wasting his time -- if you got time to kill, you can write a > test suite Great, thanks! Just check it in as libcookie.tex when you're ready, and I'll check the markup for details. Someone familiar with the module can proof it for content. -Fred -- Fred L. Drake, Jr. BeOpen PythonLabs Team Member From m.favas@per.dem.csiro.au Sat Aug 19 15:24:18 2000 From: m.favas@per.dem.csiro.au (Mark Favas) Date: Sat, 19 Aug 2000 22:24:18 +0800 Subject: [Python-Dev] [Fwd: Who can make test_fork1 fail?] Message-ID: <399E9892.35A1AC79@per.dem.csiro.au> This is a multi-part message in MIME format. --------------24871638E708E90DACDF8A87 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit --------------24871638E708E90DACDF8A87 Content-Type: message/rfc822 Content-Disposition: inline Message-ID: <399E5A71.C54C8055@per.dem.csiro.au> Date: Sat, 19 Aug 2000 17:59:13 +0800 From: Mark Favas Organization: CSIRO Exploration & Mining X-Mailer: Mozilla 4.73 [en] (X11; U; OSF1 V4.0 alpha) X-Accept-Language: en MIME-Version: 1.0 To: cgw@fnal.gov Subject: Who can make test_fork1 fail? Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit {Charles] >Or try this program (based on Neil's example), which will fail almost >immediately unless you apply my patch: Just a data point: - said program runs happily on ... -- Mark --------------24871638E708E90DACDF8A87-- From tim_one@email.msn.com Sat Aug 19 18:34:28 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 19 Aug 2000 13:34:28 -0400 Subject: [Python-Dev] New anal crusade Message-ID: Has anyone tried compiling Python under gcc with -Wmissing-prototypes -Wstrict-prototypes ? Someone on Python-Help just complained about warnings under that mode, but it's unclear to me which version of Python they were talking about. From Vladimir.Marangozov@inrialpes.fr Sat Aug 19 19:01:52 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Sat, 19 Aug 2000 20:01:52 +0200 (CEST) Subject: [Python-Dev] New anal crusade In-Reply-To: from "Tim Peters" at Aug 19, 2000 01:34:28 PM Message-ID: <200008191801.UAA17999@python.inrialpes.fr> Tim Peters wrote: > > Has anyone tried compiling Python under gcc with > > -Wmissing-prototypes -Wstrict-prototypes > > ? Someone on Python-Help just complained about warnings under that mode, > but it's unclear to me which version of Python they were talking about. Just tried it. Indeed, there are a couple of warnings. Wanna list? -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From tim_one@email.msn.com Sat Aug 19 19:33:57 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sat, 19 Aug 2000 14:33:57 -0400 Subject: [Python-Dev] New anal crusade In-Reply-To: <200008191801.UAA17999@python.inrialpes.fr> Message-ID: [Tim, on gcc -Wmissing-prototypes -Wstrict-prototypes] [Vladimir] > Just tried it. Indeed, there are a couple of warnings. Wanna list? Not me personally, no. The very subtle implied request in that was that someone who *can* run gcc this way actually commit to doing so as a matter of course, and fix warnings as they pop up. But, in the absence of joy, the occasional one-shot list is certainly better than nothing. From Vladimir.Marangozov@inrialpes.fr Sat Aug 19 19:58:18 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Sat, 19 Aug 2000 20:58:18 +0200 (CEST) Subject: [Python-Dev] New anal crusade In-Reply-To: from "Tim Peters" at Aug 19, 2000 02:33:57 PM Message-ID: <200008191858.UAA31550@python.inrialpes.fr> Tim Peters wrote: > > [Tim, on gcc -Wmissing-prototypes -Wstrict-prototypes] > > [Vladimir] > > Just tried it. Indeed, there are a couple of warnings. Wanna list? > > Not me personally, no. The very subtle implied request in that was > that someone who *can* run gcc this way actually commit to doing so as a > matter of course, and fix warnings as they pop up. But, in the absence of > joy, the occasional one-shot list is certainly better than nothing. Sorry, I'm running after my plane (and I need to run fast ) so please find another volunteer. They're mostly ansification thingies, as expected. Here's the list from a the default ./configure, make, on Linux, so that even someone without gcc can fix them : ---------------------------------------------------------------------------- pgenmain.c:43: warning: no previous prototype for `Py_Exit' pgenmain.c:169: warning: no previous prototype for `PyOS_Readline' myreadline.c:66: warning: no previous prototype for `PyOS_StdioReadline' intrcheck.c:138: warning: no previous prototype for `PyErr_SetInterrupt' intrcheck.c:191: warning: no previous prototype for `PyOS_FiniInterrupts' fileobject.c:253: warning: function declaration isn't a prototype fileobject.c:302: warning: function declaration isn't a prototype floatobject.c:242: warning: no previous prototype for `PyFloat_AsStringEx' floatobject.c:285: warning: no previous prototype for `PyFloat_AsString' unicodeobject.c:548: warning: no previous prototype for `_PyUnicode_AsDefaultEncodedString' unicodeobject.c:5142: warning: no previous prototype for `_PyUnicode_Init' unicodeobject.c:5159: warning: no previous prototype for `_PyUnicode_Fini' codecs.c:423: warning: no previous prototype for `_PyCodecRegistry_Init' codecs.c:434: warning: no previous prototype for `_PyCodecRegistry_Fini' frozenmain.c:34: warning: no previous prototype for `Py_FrozenMain' getmtime.c:30: warning: no previous prototype for `PyOS_GetLastModificationTime' import.c:2269: warning: no previous prototype for `initimp' marshal.c:771: warning: no previous prototype for `PyMarshal_Init' pyfpe.c:21: warning: no previous prototype for `PyFPE_dummy' pythonrun.c: In function `run_pyc_file': pythonrun.c:880: warning: function declaration isn't a prototype dynload_shlib.c:49: warning: no previous prototype for `_PyImport_GetDynLoadFunc' In file included from thread.c:125: thread_pthread.h:205: warning: no previous prototype for `PyThread__exit_thread' getopt.c:48: warning: no previous prototype for `getopt' ./threadmodule.c:389: warning: no previous prototype for `initthread' ./gcmodule.c:698: warning: no previous prototype for `initgc' ./regexmodule.c:661: warning: no previous prototype for `initregex' ./pcremodule.c:633: warning: no previous prototype for `initpcre' ./posixmodule.c:3698: warning: no previous prototype for `posix_strerror' ./posixmodule.c:5456: warning: no previous prototype for `initposix' ./signalmodule.c:322: warning: no previous prototype for `initsignal' ./_sre.c:2301: warning: no previous prototype for `init_sre' ./arraymodule.c:792: warning: function declaration isn't a prototype ./arraymodule.c:1511: warning: no previous prototype for `initarray' ./cmathmodule.c:412: warning: no previous prototype for `initcmath' ./mathmodule.c:254: warning: no previous prototype for `initmath' ./stropmodule.c:1205: warning: no previous prototype for `initstrop' ./structmodule.c:1225: warning: no previous prototype for `initstruct' ./timemodule.c:571: warning: no previous prototype for `inittime' ./operator.c:251: warning: no previous prototype for `initoperator' ./_codecsmodule.c:628: warning: no previous prototype for `init_codecs' ./unicodedata.c:277: warning: no previous prototype for `initunicodedata' ./ucnhash.c:107: warning: no previous prototype for `getValue' ./ucnhash.c:179: warning: no previous prototype for `initucnhash' ./_localemodule.c:408: warning: no previous prototype for `init_locale' ./fcntlmodule.c:322: warning: no previous prototype for `initfcntl' ./pwdmodule.c:129: warning: no previous prototype for `initpwd' ./grpmodule.c:136: warning: no previous prototype for `initgrp' ./errnomodule.c:74: warning: no previous prototype for `initerrno' ./mmapmodule.c:940: warning: no previous prototype for `initmmap' ./selectmodule.c:339: warning: no previous prototype for `initselect' ./socketmodule.c:2366: warning: no previous prototype for `init_socket' ./md5module.c:275: warning: no previous prototype for `initmd5' ./shamodule.c:550: warning: no previous prototype for `initsha' ./rotormodule.c:621: warning: no previous prototype for `initrotor' ./newmodule.c:205: warning: no previous prototype for `initnew' ./binascii.c:1014: warning: no previous prototype for `initbinascii' ./parsermodule.c:2637: warning: no previous prototype for `initparser' ./cStringIO.c:643: warning: no previous prototype for `initcStringIO' ./cPickle.c:358: warning: no previous prototype for `cPickle_PyMapping_HasKey' ./cPickle.c:2287: warning: no previous prototype for `Pickler_setattr' ./cPickle.c:4518: warning: no previous prototype for `initcPickle' main.c:33: warning: function declaration isn't a prototype main.c:79: warning: no previous prototype for `Py_Main' main.c:292: warning: no previous prototype for `Py_GetArgcArgv' getbuildinfo.c:34: warning: no previous prototype for `Py_GetBuildInfo' ./Modules/getbuildinfo.c:34: warning: no previous prototype for `Py_GetBuildInfo' -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From guido@python.org Fri Aug 18 20:13:14 2000 From: guido@python.org (Guido van Rossum) Date: Fri, 18 Aug 2000 15:13:14 -0400 Subject: [Python-Dev] Re: os.path.commonprefix breakage References: Message-ID: <011301c00a1f$927e7980$7aa41718@beopen.com> I'm reading this tread off-line. I'm feeling responsible because I gave Skip the green light. I admit that that was a mistake: I didn't recall the purpose of commonprefix() correctrly, and didn't refresh my memory by reading the docs. I think Tim is right: as the docs say, the function was *intended* to work on a character basis. This doesn't mean that it doesn't belong in os.path! Note that os.dirname() will reliably return the common directory, exactly because the trailing slash is kept. I propose: - restore the old behavior on all platforms - add to the docs that to get the common directory you use dirname() - add testcases that check that this works on all platforms - don't add commonpathprefix(), because dirname() already does it Note that I've only read email up to Thursday morning. If this has been superseded by more recent resolution, I'll reconsider; but if it's still up in the air this should be it. It doesn't look like the change made it into 1.6. --Guido From guido@python.org Fri Aug 18 20:20:06 2000 From: guido@python.org (Guido van Rossum) Date: Fri, 18 Aug 2000 15:20:06 -0400 Subject: [Python-Dev] PEP 214, extended print statement References: <14747.22851.266303.28877@anthem.concentric.net> <20000817083023.J376@xs4all.nl> <14747.63511.725610.771162@anthem.concentric.net> Message-ID: <011401c00a1f$92db8da0$7aa41718@beopen.com> I'm still reading my email off-line on the plane. I've now read PEP 214 and think I'll reverse my opinion: it's okay. Barry, check it in! (And change the SF PM status to 'Accepted'.) I think I'll start using it for error messages: errors should go to stderr, but it's often inconvenient, so in minor scripts instead of doing sys.stderr.write("Error: can't open %s\n" % filename) I often write print "Error: can't open", filename which is incorrect but more convenient. I can now start using print >>sys.stderr, "Error: can't open", filename --Guido From guido@python.org Fri Aug 18 20:23:37 2000 From: guido@python.org (Guido van Rossum) Date: Fri, 18 Aug 2000 15:23:37 -0400 Subject: [Python-Dev] PyErr_NoMemory References: <200008171509.RAA20891@python.inrialpes.fr> Message-ID: <011501c00a1f$939bd060$7aa41718@beopen.com> > The current PyErr_NoMemory() function reads: > > PyObject * > PyErr_NoMemory(void) > { > /* raise the pre-allocated instance if it still exists */ > if (PyExc_MemoryErrorInst) > PyErr_SetObject(PyExc_MemoryError, PyExc_MemoryErrorInst); > else > /* this will probably fail since there's no memory and hee, > hee, we have to instantiate this class > */ > PyErr_SetNone(PyExc_MemoryError); > > return NULL; > } > > thus overriding any previous exceptions unconditionally. This is a > problem when the current exception already *is* PyExc_MemoryError, > notably when we have a chain (cascade) of memory errors. It is a > problem because the original memory error and eventually its error > message is lost. > > I suggest to make this code look like: > > PyObject * > PyErr_NoMemory(void) > { > if (PyErr_ExceptionMatches(PyExc_MemoryError)) > /* already current */ > return NULL; > > /* raise the pre-allocated instance if it still exists */ > if (PyExc_MemoryErrorInst) > PyErr_SetObject(PyExc_MemoryError, PyExc_MemoryErrorInst); > ... > > If nobody sees a problem with this, I'm very tempted to check it in. > Any objections? +1. The cascading memory error seems a likely scenario indeed: something returns a memory error, the error handling does some more stuff, and hits more memory errors. --Guido From guido@python.org Fri Aug 18 21:57:15 2000 From: guido@python.org (Guido van Rossum) Date: Fri, 18 Aug 2000 16:57:15 -0400 Subject: [Python-Dev] iterators References: <399BE124.9920B0B6@prescod.net> Message-ID: <011601c00a1f$9923d460$7aa41718@beopen.com> Paul Prescod wrote: > I don't think of iterators as indexing in terms of numbers. Otherwise I > could do this: > > >>> a={0:"zero",1:"one",2:"two",3:"three"} > >>> for i in a: > ... print i > ... > > So from a Python user's point of view, for-looping has nothing to do > with integers. From a Python class/module creator's point of view it > does have to do with integers. I wouldn't be either surprised nor > disappointed if that changed one day. Bingo! I've long had an idea for generalizing 'for' loops using iterators. This is more a Python 3000 thing, but I'll explain it here anyway because I think it's relevant. Perhaps this should become a PEP? (Maybe we should have a series of PEPs with numbers in the 3000 range for Py3k ideas?) The statement for in
RetroSearch is an open source project built by @garambo
| Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4