and(where newlines do matter)? Skip From guido at python.org Sun Sep 11 17:24:53 2005 From: guido at python.org (Guido van Rossum) Date: Sun, 11 Sep 2005 08:24:53 -0700 Subject: [Python-Dev] PEP 3000 and iterators In-Reply-To: <3F65B538-4D54-4DC9-9136-9F6BAB941EC8@fuhm.net> References: <3F65B538-4D54-4DC9-9136-9F6BAB941EC8@fuhm.net> Message-ID: On 9/10/05, James Y Knight wrote: > No, that cannot work. However, there is a very obvious and trivial > solution. Do not remove dict.iteritems in Py 3.0. Py2.X programs > wishing forward compat can avoid dict.items and use instead > dict.iteritems. In Py3.0, dict.items becomes a synonym for > dict.iteritems and programs that don't care about compat with 2.X can > just use dict.items from then on. And everybody can be happy. A small > number of redundant methods is a small price to pay for compatibility. But it breaks the desire to keep the Python 3.0 language clean from deprecated features. Given that I don't expect there will be much compatibility *anyway*, I don't want to promise this. I expect that we'll have to write a source-level translator -- which could replace all iteritems() calls to items(), for example. Such a source-level translator may not be able to reach perfection, but it should take care of the tedious tasks and leave the rest up to manual polishing. This doesn't mean that there's no point in trying to introduce certain 3.0 features in 2.x; it's always good to have early experience with a new feature, and in some cases it *will* improve forward compatibility. But just installing python3.0 as python and expecting nothing will break is not a goal -- it would be too constraining. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From foom at fuhm.net Sun Sep 11 18:09:26 2005 From: foom at fuhm.net (James Y Knight) Date: Sun, 11 Sep 2005 12:09:26 -0400 Subject: [Python-Dev] PEP 3000 and iterators In-Reply-To: References: <3F65B538-4D54-4DC9-9136-9F6BAB941EC8@fuhm.net> Message-ID: <9C872F06-AED6-49FC-BFF1-AD56C84493D5@fuhm.net> On Sep 11, 2005, at 11:24 AM, Guido van Rossum wrote: > But it breaks the desire to keep the Python 3.0 language clean from > deprecated features. That is a nice goal, another nice goal is to not unnecessarily break things. > But just installing python3.0 as python and expecting > nothing will break is not a goal -- it would be too constraining. > Just to be clear, I do not want nor expect this. I wish to be able to specifically modify code with full knowledge of what has changed in Py3.0 such that it will work with both Py2.X and Py3.0. And, now is probably not really the right time to discuss such minor issues as whether to keep iteritems in Py3.0, but, if it is kept, it becomes easier to write such code. It is of course still possible to write compatible code without keeping iteritems, you just have to replace all the method calls with a function wrapper which calls one of items or iteritems depending on the version. James From martin.blais at gmail.com Sun Sep 11 18:35:08 2005 From: martin.blais at gmail.com (Martin Blais) Date: Sun, 11 Sep 2005 12:35:08 -0400 Subject: [Python-Dev] pygettext() without newlines (Was: Re: Replacement for print in Python 3.0) In-Reply-To: <17188.17794.574463.31979@montanaro.dyndns.org> References: <874q8xzexl.fsf@tleepslib.sk.tsukuba.ac.jp> <87wtltxmd7.fsf@tleepslib.sk.tsukuba.ac.jp> <1126094808.12806.2.camel@presto.wooz.org> <877jdryik1.fsf@tleepslib.sk.tsukuba.ac.jp> <1126180081.882.9.camel@p-dvsi-418-1.rd.francetelecom.fr> <8393fff05090913353f6133dc@mail.gmail.com> <17188.17794.574463.31979@montanaro.dyndns.org> Message-ID: <8393fff0509110935b12cbb9@mail.gmail.com> On 9/11/05, skip at pobox.com wrote: > > (Maybe someone else has already raised this point. I'm a bit behind.) > > Martin> Here goes something: for applications targeted to the web, where > Martin> newlines don't matter, the line breaks in _()'ed strings are > Martin> superfluous. > > How will you know you're generating output that goes between
and >(where newlines do matter)? The great majority of the strings are _(), but in my web app framework I use __() for that specific purpose you mention, which invokes the "normal" behaviour of pygettext (which my patch doesn't remove, BTW). Here is how I invoke pygettext from my Makefile: POTCMD = pygettext.py \ --verbose --extract-all --no-default-keywords \ --keyword-single=_ --keyword-single=N_ \ --keyword=__ I use _() and N_() for single-line'd strings (runtime and marker-only versions), and __() for the usual multi-line strings. It works well, I've been using this for a while on a web application I'm building. cheers, From orent at hishome.net Sun Sep 11 19:19:00 2005 From: orent at hishome.net (Oren Tirosh) Date: Sun, 11 Sep 2005 20:19:00 +0300 Subject: [Python-Dev] Python 3 executable name (was: Re: PEP 3000 and iterators) In-Reply-To: References: <3F65B538-4D54-4DC9-9136-9F6BAB941EC8@fuhm.net> Message-ID: <7168d65a0509111019a697cb2@mail.gmail.com> On 9/11/05, Guido van Rossum wrote: ... > But just installing python3.0 as python and expecting > nothing will break is not a goal -- it would be too constraining. It should be expected that many users will keep both 2.x and 3 side by side for quite a long time. Instead of having distributions choosing their own naming schemes (like the python/python2 redhat fiasco) perhaps the Python 3 executable should have a different name as part of the standard distribution? I suggest "py" / "py.exe" Oren From tdelaney at avaya.com Mon Sep 12 01:28:07 2005 From: tdelaney at avaya.com (Delaney, Timothy (Tim)) Date: Mon, 12 Sep 2005 09:28:07 +1000 Subject: [Python-Dev] PEP 3000 and iterators Message-ID: <2773CAC687FD5F4689F526998C7E4E5F4DB5F7@au3010avexu1.global.avaya.com> James Y Knight wrote: > Just to be clear, I do not want nor expect this. I wish to be able to > specifically modify code with full knowledge of what has changed in > Py3.0 such that it will work with both Py2.X and Py3.0. If you want these things to work in 2.x and 3.0, just use iter(dict_instance) and list(dict_instance) as appropriate. Tim Delaney From greg.ewing at canterbury.ac.nz Mon Sep 12 07:22:46 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 12 Sep 2005 17:22:46 +1200 Subject: [Python-Dev] Python 3 executable name In-Reply-To: <7168d65a0509111019a697cb2@mail.gmail.com> References: <3F65B538-4D54-4DC9-9136-9F6BAB941EC8@fuhm.net> <7168d65a0509111019a697cb2@mail.gmail.com> Message-ID: <432510A6.902@canterbury.ac.nz> Oren Tirosh wrote: > perhaps the Python 3 executable should have a different name as part > of the standard distribution? I suggest "py" / "py.exe" Or "python3"? EIBTI :-) -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Mon Sep 12 07:24:34 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 12 Sep 2005 17:24:34 +1200 Subject: [Python-Dev] PEP 3000 and iterators In-Reply-To: <43224221.1090506@gmail.com> References: <43224221.1090506@gmail.com> Message-ID: <43251112.4010401@canterbury.ac.nz> Nick Coghlan wrote: > However, such a "future_builtins" module could still include modified versions > of those standard objects - such "future-proofed" code would simply still need > to deal with the fact that other libraries or clients may pass in the > old-style components (e.g. just as unicode-aware code needs to deal with the > fact that other libraries or clients may produce 8-bit strings rather than > unicode text). And be careful not to pass them to old code which expects the traditional versions of these objects. Sounds far too tricky and error-prone to be worth the trouble to me. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Mon Sep 12 07:23:53 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 12 Sep 2005 17:23:53 +1200 Subject: [Python-Dev] unintentional and unsafe use of realpath() In-Reply-To: <1126380571.21655.18.camel@localhost.localdomain> References: <1126380571.21655.18.camel@localhost.localdomain> Message-ID: <432510E9.80905@canterbury.ac.nz> Peter Jones wrote: > Another problem (which I have not fixed) is that when realpath() is > used, in some cases MAXPATHLEN is smaller than the system's > PATH_MAX/pathconf(path, _PC_PATH_MAX). The linux man page for realpath() has this at the bottom: BUGS Never use this function. It is broken by design since it is impossible to determine a suitable size for the output buffer. According to POSIX a buffer of size PATH_MAX suffices, but PATH_MAX need not be a defined constant, and may have to be obtained using pathconf(). And asking pathconf() does not really help, since on the one hand POSIX warns that the result of pathconf() may be huge and unsuitable for mallocing mem- ory. And on the other hand pathconf() may return -1 to signify that PATH_MAX is not bounded. So maybe it shouldn't be using realpath() at all? -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Mon Sep 12 07:28:46 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 12 Sep 2005 17:28:46 +1200 Subject: [Python-Dev] Replacement for print in Python 3.0 In-Reply-To: <432144D8.8020903@gmail.com> References: <20050908051402.m7j55nypuvwg448k@login.werra.lunarpages.com> <4320EFA3.8070607@canterbury.ac.nz> <432144D8.8020903@gmail.com> Message-ID: <4325120E.4040802@canterbury.ac.nz> Nick Coghlan wrote: > Not to mention the annoyingly large number of fonts that make '`' and ''' look > virtually identical :( Well, you need to be careful about choice of font for programming anyway, for 0/O, 1/l, etc. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From orent at hishome.net Mon Sep 12 08:51:41 2005 From: orent at hishome.net (Oren Tirosh) Date: Mon, 12 Sep 2005 09:51:41 +0300 Subject: [Python-Dev] Python 3 executable name In-Reply-To: <432510A6.902@canterbury.ac.nz> References: <3F65B538-4D54-4DC9-9136-9F6BAB941EC8@fuhm.net> <7168d65a0509111019a697cb2@mail.gmail.com> <432510A6.902@canterbury.ac.nz> Message-ID: <7168d65a0509112351d7afe9e@mail.gmail.com> On 9/12/05, Greg Ewing wrote: > Oren Tirosh wrote: > > > perhaps the Python 3 executable should have a different name as part > > of the standard distribution? I suggest "py" / "py.exe" > > Or "python3"? EIBTI :-) Generally, each distribution makes its own decision about when to make the default "python" the new version. Any damage is usually limited to third party extension modules because python versions are source compatible. But this time it isn't. So do you keep the name "python3" forever? Do you keep unqualified "python" as 2.x forever? I expect many installations to keep 2.x around for many years. How do you keep different distributions from making their own incompatible decisions about naming conventions? Using version numbers in the executable name is just asking for this to happen. I suggest an explicitly and permanently different name for the interpreter executable of this new and incompatible branch of the language. I want Python 3 scripts starting with #! to have an average shelf life longer than 18 months. Oren From greg.ewing at canterbury.ac.nz Mon Sep 12 10:57:46 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 12 Sep 2005 20:57:46 +1200 Subject: [Python-Dev] Python 3 executable name In-Reply-To: <7168d65a0509112351d7afe9e@mail.gmail.com> References: <3F65B538-4D54-4DC9-9136-9F6BAB941EC8@fuhm.net> <7168d65a0509111019a697cb2@mail.gmail.com> <432510A6.902@canterbury.ac.nz> <7168d65a0509112351d7afe9e@mail.gmail.com> Message-ID: <4325430A.80808@canterbury.ac.nz> Oren Tirosh wrote: > I suggest an explicitly and permanently different name for the > interpreter executable of this new and incompatible branch of the > language. I want Python 3 scripts starting with #! to have an average > shelf life longer than 18 months. That makes sense, but I'd prefer something less arbitrary than just chopping the name back from "python" to "py". Going that way, Python 4.0 is going to just be called "p", and by the time we get to Python 5.0, the name will have vanished altogether! Greg From p.f.moore at gmail.com Mon Sep 12 11:29:54 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 12 Sep 2005 10:29:54 +0100 Subject: [Python-Dev] Python 3 executable name In-Reply-To: <4325430A.80808@canterbury.ac.nz> References: <3F65B538-4D54-4DC9-9136-9F6BAB941EC8@fuhm.net> <7168d65a0509111019a697cb2@mail.gmail.com> <432510A6.902@canterbury.ac.nz> <7168d65a0509112351d7afe9e@mail.gmail.com> <4325430A.80808@canterbury.ac.nz> Message-ID: <79990c6b0509120229366e054b@mail.gmail.com> On 9/12/05, Greg Ewing wrote: > Oren Tirosh wrote: > > > I suggest an explicitly and permanently different name for the > > interpreter executable of this new and incompatible branch of the > > language. I want Python 3 scripts starting with #! to have an average > > shelf life longer than 18 months. > > That makes sense, but I'd prefer something less arbitrary > than just chopping the name back from "python" to "py". > Going that way, Python 4.0 is going to just be called > "p", and by the time we get to Python 5.0, the name > will have vanished altogether! That's OK, by the time Python 5.0 comes out, it will have taken over the world and be the default language for everything. So omitting the name is exactly right :-) Paul. From reinhold-birkenfeld-nospam at wolke7.net Mon Sep 12 11:26:56 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Mon, 12 Sep 2005 11:26:56 +0200 Subject: [Python-Dev] Tools directory (Was RE: Replacement for print in Python 3.0) In-Reply-To: References: Message-ID: Brett Cannon wrote: > On 9/8/05, Tony Meyer wrote: >> [finding Tools/i18n/pygettext.py] >> > You're right, I think Tools is probably a bad place for >> > anything. If it's not part of the stdlib, I'll likely never >> > find it. >> >> Agreed. Maybe with the introduction of -m in Python 2.4, some of the Tools/ >> scripts could be put in __main__ sections of appropriate modules? So that >> "python -m gettext" would be equivilant to "python Tools/i18n/pygettext.py"? Questionable. Most scripts don't correspond to a single library module. >> (However, pyggettext.py is 22KB, which is a big addition to the module; not >> everything in Tools/Scripts might be used enough for this, or have an >> appopriate module to be put in either). >> >> Are there other ideas about how Tools/ could be improved? Either moving >> things, or making it more likely that people will look there for scripts? >> > > I assume that the Windows installer includes the Tools/ directory. If > it doesn't that is one problem. =) > > Otherwise it is mostly a lack of advertisement and them not being > installed by ``make install``. If you just download the soure and > install you will never know the directory even exists. It needs to be > made obvious to people that it is even there. +1. Most non-Windows users with distribution-supplied Pythons will never get the Tools directory on their installs though there is a number of really useful scripts there. Question is, if ``make install`` should install it, where? Has the time come for /usr/share/python? Or /usr/lib/pythonX.Y/Tools (without __init__.py)? > Probably the only way is to document the directory. I think so, too. The tools are worth a top-level documentation entry. Reinhold -- Mail address is perfectly valid! From guido at python.org Mon Sep 12 16:37:30 2005 From: guido at python.org (Guido van Rossum) Date: Mon, 12 Sep 2005 07:37:30 -0700 Subject: [Python-Dev] PEP 3000 and iterators In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F4DB5F7@au3010avexu1.global.avaya.com> References: <2773CAC687FD5F4689F526998C7E4E5F4DB5F7@au3010avexu1.global.avaya.com> Message-ID: On 9/11/05, Delaney, Timothy (Tim) wrote: > James Y Knight wrote: > > > Just to be clear, I do not want nor expect this. I wish to be able to > > specifically modify code with full knowledge of what has changed in > > Py3.0 such that it will work with both Py2.X and Py3.0. > > If you want these things to work in 2.x and 3.0, just use > iter(dict_instance) and list(dict_instance) as appropriate. Simpler still, just use di.keys() but make sure you're only using the result to iterate over once without modifying the dict's key set. Or if you *have* to avoid creating a list in Py2.x, write your code to iterate over the dict itself even if you'd like itervalues or iteritems; you can always get the value explicitly by indexing the dict. IOW use the API whose name will remain but don't rely on the functionality that will change. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pjones at redhat.com Mon Sep 12 17:55:59 2005 From: pjones at redhat.com (Peter Jones) Date: Mon, 12 Sep 2005 11:55:59 -0400 Subject: [Python-Dev] unintentional and unsafe use of realpath() In-Reply-To: <432510E9.80905@canterbury.ac.nz> References: <1126380571.21655.18.camel@localhost.localdomain> <432510E9.80905@canterbury.ac.nz> Message-ID: <1126540559.9198.8.camel@localhost.localdomain> On Mon, 2005-09-12 at 17:23 +1200, Greg Ewing wrote: > Peter Jones wrote: > > > Another problem (which I have not fixed) is that when realpath() is > > used, in some cases MAXPATHLEN is smaller than the system's > > PATH_MAX/pathconf(path, _PC_PATH_MAX). > > The linux man page for realpath() has this at the bottom: > > BUGS > Never use this function. It is broken by design since it is impossible > to determine a suitable size for the output buffer. According to POSIX > a buffer of size PATH_MAX suffices, but PATH_MAX need not be a defined > constant, and may have to be obtained using pathconf(). And asking > pathconf() does not really help, since on the one hand POSIX warns that > the result of pathconf() may be huge and unsuitable for mallocing mem- > ory. And on the other hand pathconf() may return -1 to signify that > PATH_MAX is not bounded. > > So maybe it shouldn't be using realpath() at all? Well, the intent is clearly that OSes that have a better option, we shouldn't be using it. But that wasn't really my second point. When use of realpath is unavoidable, it is vitally important that MAXPATHLEN is set to the same value that realpath() got when it was built. If it's any smaller, it's a straightforward overflow, with something like $SYSTEM_MAXPATHLEN - $PYTHON_MAXPATHLEN of space to write nasty bits into. At least on one box I looked at, system's was 4096 and python wound up doing the fallback of 256 (I'm not entirely sure why) so that's 3768 bytes of stack potentially overwritten. In this case, it's not realistically exploitable, because it means a user has to trick root into running "python foo" where foo is a symlink that's built terrifyingly weirdly. So since the user is supplying the symlink, there are much more trivial attacks. But I haven't checked all the uses of realpath in python, some of them could be dangerous. -- Peter From janssen at parc.com Mon Sep 12 20:29:00 2005 From: janssen at parc.com (Bill Janssen) Date: Mon, 12 Sep 2005 11:29:00 PDT Subject: [Python-Dev] Python 3 executable name (was: Re: PEP 3000 and iterators) In-Reply-To: Your message of "Sun, 11 Sep 2005 10:19:00 PDT." <7168d65a0509111019a697cb2@mail.gmail.com> Message-ID: <05Sep12.112908pdt."58617"@synergy1.parc.xerox.com> > perhaps the Python 3 executable should have a different name as part > of the standard distribution? I suggest "py" / "py.exe" Perhaps the renaming should be more radical, to indicate the extent of the change. I suggest "second", to commemorate "Second City" in Chicago, another famous comedy troupe. The Python3 (that is, Second) version of IDLE could be DICK, after Andy Dick, an alumnus who somewhat resembles Eric Idle, or perhaps MURRAY, after Bill Murray. Bill From janssen at parc.com Mon Sep 12 20:40:44 2005 From: janssen at parc.com (Bill Janssen) Date: Mon, 12 Sep 2005 11:40:44 PDT Subject: [Python-Dev] Python 3 executable name (was: Re: PEP 3000 and iterators) In-Reply-To: Your message of "Mon, 12 Sep 2005 11:29:00 PDT." <05Sep12.112908pdt."58617"@synergy1.parc.xerox.com> Message-ID: <05Sep12.114053pdt."58617"@synergy1.parc.xerox.com> > Perhaps the renaming should be more radical, to indicate the extent of > the change. Another possiblity would be to use the new name to indicate a position in a spectrum of programming languages. For instance, "beryl", "topaz", "steel" and "jade" are (on the Mohs scale) all sturdier than pearl but not as hard as ruby :-). Bill From adurdin at gmail.com Tue Sep 13 08:50:23 2005 From: adurdin at gmail.com (Andrew Durdin) Date: Tue, 13 Sep 2005 16:50:23 +1000 Subject: [Python-Dev] Simplify the file-like-object interface In-Reply-To: <1126012360.5469.13.camel@p-dvsi-418-1.rd.francetelecom.fr> References: <79990c6b05090212453f3b7c77@mail.gmail.com> <431D7BFB.70201@canterbury.ac.nz> <79990c6b050906054943999631@mail.gmail.com> <1126012360.5469.13.camel@p-dvsi-418-1.rd.francetelecom.fr> Message-ID: <59e9fd3a05091223506d5c45ef@mail.gmail.com> On 9/6/05, Antoine Pitrou wrote: > > One could use "class decorators". For example if you want to define the > method foo() in a file-like class, you could use code like: I like the sound of this. Suppose there were a function textstream() that decorated a file-like object (supporting read() and write()), so as to add all of __iter__(), next(), readline(), readlines(), and writeline() that it did not already implement. Then you could wrap any file-like object easily to give it convenient text-handling: mytextsocketstream = textstream(mysocketstream) for line in mytextsocketstream: print line Another area where I think this approach can help is with the text/binary file distinction. file() could always open files as binary, and there could be a convenience function textfile(name, mode) which would simply return textstream(file(name, mode)). This would remove the need for "a" or "b" in the mode parameter, and make it easier to keep text- and binary-file access separate in one's mind: tf = textfile("log.txt", "w") tf.writelines(loglist) bf = file("img.jpg", "r") process_image_file(bf) Finally, I think this could nicely tie into the print statement/function/method debate, as the print() method could be defined by the textstream() wrapper, allowing any text stream to support the convenient print() behaviour, but not requiring it for binary streams, for which such a function makes little sense anyway. And then textstream() could be used (in the sys module, preferably) to ensure that sys.stdout and sys.stderr both support print(). So you'd have the builtin: # Actual signature depending on which variant proposal is taken up def print(*args, **kwargs): sys.stdout.print(*args, **kwargs) And using print() could be a simple as: print("Hello, World") sys.stdout.print("This is normal output") sys.stderr.print("This is an error message") I have a vague idea that a wrapper like this (or another similar wrapper) could provide for convenient, transparent handling of Unicode text files also, but I don't know Unicode well enough to be certain. Andrew. From adurdin at gmail.com Tue Sep 13 08:58:57 2005 From: adurdin at gmail.com (Andrew Durdin) Date: Tue, 13 Sep 2005 16:58:57 +1000 Subject: [Python-Dev] Replacement for print in Python 3.0 In-Reply-To: References: <17176.26475.644454.492490@montanaro.dyndns.org> <4318F633.6050501@gmail.com> <79990c6b05090306355891f450@mail.gmail.com> <4319AE7E.8020803@gmail.com> <4319C0ED.4060608@libero.it> Message-ID: <59e9fd3a0509122358cf48cee@mail.gmail.com> On 9/6/05, Guido van Rossum wrote: > > My hypothesis is that there are actually only two use cases that > matter enough to be supported directly: > > (a) quickly print a bunch of items with spaces in between them and a > trailing newline > > (b) print one or more items with precise control over each character Doesn't the write() method of file-like objects already cover (b), except that it only takes a single argument? If the need to print multiple arguments without any separator is so common, could the write() method be extended to take multiple arguments and just write them all out? It'd certainly be backward compatible with old code that called the write() method... Andrew. From hyeshik at gmail.com Tue Sep 13 10:56:32 2005 From: hyeshik at gmail.com (Hye-Shik Chang) Date: Tue, 13 Sep 2005 17:56:32 +0900 Subject: [Python-Dev] Python code.interact() and UTF-8 locale In-Reply-To: <1126408583.12608.34.camel@haypopc> References: <1126408583.12608.34.camel@haypopc> Message-ID: <4f0b69dc05091301564229d4d7@mail.gmail.com> On 9/11/05, Victor STINNER wrote: > Hi, > > I found a bug in Python interactive command line (program python alone: > looks to be code.interact() function in code.py). With UTF-8 locale, the > command << u"?" >> returns << u'\xc3\xa9' >> and not << u'\xE9' >>. > Remember: the french e with acute is Unicode 233 (0xE9), encoded \xC3 > \xA9 in UTF-8. Which version of python do you use? From 2.4, the interactive mode respects locale as a source code encoding and it falls back to latin-1 when decoding fails. Python 2.4.1 (#2, Jul 31 2005, 04:45:53) [GCC 3.4.2 [FreeBSD] 20040728] on freebsd5 Type "help", "copyright", "credits" or "license" for more information. >>> u"?" u'\xe9' Hye-Shik From hyeshik at gmail.com Tue Sep 13 11:20:41 2005 From: hyeshik at gmail.com (Hye-Shik Chang) Date: Tue, 13 Sep 2005 18:20:41 +0900 Subject: [Python-Dev] Python code.interact() and UTF-8 locale In-Reply-To: <4f0b69dc05091301564229d4d7@mail.gmail.com> References: <1126408583.12608.34.camel@haypopc> <4f0b69dc05091301564229d4d7@mail.gmail.com> Message-ID: <4f0b69dc0509130220ac9b781@mail.gmail.com> On 9/13/05, Hye-Shik Chang wrote: > On 9/11/05, Victor STINNER wrote: > > > > I found a bug in Python interactive command line (program python alone: > > looks to be code.interact() function in code.py). With UTF-8 locale, the > > command << u"?" >> returns << u'\xc3\xa9' >> and not << u'\xE9' >>. > > Remember: the french e with acute is Unicode 233 (0xE9), encoded \xC3 > > \xA9 in UTF-8. > > Which version of python do you use? From 2.4, the interactive mode > respects locale as a source code encoding and it falls back to latin-1 > when decoding fails. > Aah, code.interact() and IDLE behaviors different from the real interactive mode currently. I think it needs to be fixed before the next release. For IDLE, I filed a patch on SF #1061803. But it may need some discussion because of its trickiness. :) Hye-Shik From ironfroggy at gmail.com Tue Sep 13 14:45:50 2005 From: ironfroggy at gmail.com (Calvin Spealman) Date: Tue, 13 Sep 2005 08:45:50 -0400 Subject: [Python-Dev] Replacement for print in Python 3.0 In-Reply-To: References: <87wtltxmd7.fsf@tleepslib.sk.tsukuba.ac.jp> <1126094808.12806.2.camel@presto.wooz.org> <877jdryik1.fsf@tleepslib.sk.tsukuba.ac.jp> <1126180081.882.9.camel@p-dvsi-418-1.rd.francetelecom.fr> <87psrjv7j3.fsf@tleepslib.sk.tsukuba.ac.jp> <4320F1AF.6090802@canterbury.ac.nz> <878xy6ycs6.fsf@tleepslib.sk.tsukuba.ac.jp> Message-ID: <76fd5acf050913054564a39403@mail.gmail.com> On 9/9/05, Guido van Rossum wrote: > While I laugh at the naive view of people who write things like > "Interface equality and neutrality would be a good thing in the > language" and seriously (? I didn't see a smiley) use this argument to > plead for not making print() a built-in, I do think that avoiding the > 'print' name would be a good thing if it could be done without ticking > off the old-timers. Oh, no! I've been misrepresented! I can be a little unclear sometimes, and for that I apologize. What I was saying is that there are essential to ends to the spectrum: you either elevate text console IO to a status above other forms of interface with the applications written in the language, or you don't build any interface mechanisms into the lanuage at all. Python currently is at the former end of that spectrum, and the current discussions seem to be pushing towards the later. My disagreement is more with consistancy than where it actually stands in that spectrum. So, I'm saying if it has to be in the language directly, keep a statement for it. If if really shouldn't be a statement, then make me import it first. Yes, I know that no one wants to import a module just to output text, but I don't see how or why it is any different than importing GUI modules. From ironfroggy at gmail.com Tue Sep 13 14:51:20 2005 From: ironfroggy at gmail.com (Calvin Spealman) Date: Tue, 13 Sep 2005 08:51:20 -0400 Subject: [Python-Dev] Simplify the file-like-object interface In-Reply-To: <59e9fd3a05091223506d5c45ef@mail.gmail.com> References: <431D7BFB.70201@canterbury.ac.nz> <79990c6b050906054943999631@mail.gmail.com> <1126012360.5469.13.camel@p-dvsi-418-1.rd.francetelecom.fr> <59e9fd3a05091223506d5c45ef@mail.gmail.com> Message-ID: <76fd5acf050913055175723a46@mail.gmail.com> On 9/13/05, Andrew Durdin wrote: > On 9/6/05, Antoine Pitrou wrote: > > > > One could use "class decorators". For example if you want to define the > > method foo() in a file-like class, you could use code like: > > I like the sound of this. Suppose there were a function textstream() > that decorated a file-like object (supporting read() and write()), so > as to add all of __iter__(), next(), readline(), readlines(), and > writeline() that it did not already implement. Then you could wrap any > file-like object easily to give it convenient text-handling: Yes, this isn't perl and text, although still important, is not worth its wait in gold these days. And, have you even tried to weigh digital content to begin with? Not much there anyway. From mcherm at mcherm.com Tue Sep 13 15:44:39 2005 From: mcherm at mcherm.com (Michael Chermside) Date: Tue, 13 Sep 2005 06:44:39 -0700 Subject: [Python-Dev] Simplify the file-like-object interface Message-ID: <20050913064439.xmcqf3zfjqrwoowg@login.werra.lunarpages.com> Andrew Durdin writes: > Another area where I think this approach can help is with the > text/binary file distinction. file() could always open files as > binary, and there could be a convenience function textfile(name, mode) > which would simply return textstream(file(name, mode)). This would > remove the need for "a" or "b" in the mode parameter, and make it > easier to keep text- and binary-file access separate in one's mind: I think you are suffering from the (rather common) misconception that all files are binary, and the definition of "text file" is a binary file which should be interpreted as containing characters in some encoding. In unix, the above is true. One of the fundamental decisions in Unix was to treat all files (and lots of other vaguely file-like things) as undiferentiated streams of bytes. But this is NOT true on many other operating systems. It is not, for example, true on Windows. Many operating systems make a distinction between two basic types of files... "text files" which are line-oriented and contain "text", and "binary files" which contain streams of bytes. This distinction is supported by the basic file operations in the C library. To open a text file in binary mode is technically an error (although in many OSs you'll get away with it). -- Michael Chermside From victor.stinner-linux at haypocalc.com Tue Sep 13 15:53:29 2005 From: victor.stinner-linux at haypocalc.com (Victor STINNER) Date: Tue, 13 Sep 2005 15:53:29 +0200 Subject: [Python-Dev] Python code.interact() and UTF-8 locale In-Reply-To: <4f0b69dc05091301564229d4d7@mail.gmail.com> References: <1126408583.12608.34.camel@haypopc> <4f0b69dc05091301564229d4d7@mail.gmail.com> Message-ID: <1126619609.3184.18.camel@haypopc> Le mardi 13 septembre 2005 ? 17:56 +0900, Hye-Shik Chang a ?crit : > On 9/11/05, Victor STINNER wrote: > > Hi, > > > > I found a bug in Python interactive command line (program python alone: > > looks to be code.interact() function in code.py). With UTF-8 locale, the > > command << u"?" >> returns << u'\xc3\xa9' >> and not << u'\xE9' >>. > > Remember: the french e with acute is Unicode 233 (0xE9), encoded \xC3 > > \xA9 in UTF-8. > > Which version of python do you use? From 2.4, the interactive mode > respects locale as a source code encoding and it falls back to latin-1 > when decoding fails. > > Python 2.4.1 (#2, Jul 31 2005, 04:45:53) > [GCC 3.4.2 [FreeBSD] 20040728] on freebsd5 > Type "help", "copyright", "credits" or "license" for more information. > >>> u"?" > u'\xe9' I installed my own Python 2.4 in /opt/python/. I don't know if the right code.py is loaded, but here is the output : =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- $ ./python2.4 Python 2.4.1 (#1, Sep 11 2005, 01:37:26) [GCC 4.0.2 20050821 (prerelease) (Debian 4.0.1-6)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> u"?" u'\xe9' >>> import code >>> code.interact() Python 2.4.1 (#1, Sep 11 2005, 01:37:26) [GCC 4.0.2 20050821 (prerelease) (Debian 4.0.1-6)] on linux2 Type "help", "copyright", "credits" or "license" for more information. (InteractiveConsole) >>> u"?" u'\xc3\xa9' =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Well, that works better :-) For code.interact(), you can read my attached patch. I don't know if it the best way to fix the but. But, the following code still bug in Python 2.4 : =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- $ cat python_unicode_eval_bug.py #*- coding: UTF-8 -*- print "One Unicode character: %u" % len(u"?") print "One Unicode character (using eval) : %u" % eval('len(u"?")') $ python2.4 python_unicode_eval_bug.py One Unicode character: 1 One Unicode character (using eval) : 2 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- RexFi explains me that Python can't guess eval('len(u"?")') charset. Yep, that's difficult: locale? charset encoding? This test doesn't matter. @+, Haypo -------------- next part -------------- A non-text attachment was scrubbed... Name: code-interact.patch Type: text/x-patch Size: 407 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050913/c620d813/code-interact.bin From nathan_kent_bullock at yahoo.ca Tue Sep 13 05:11:57 2005 From: nathan_kent_bullock at yahoo.ca (Nathan Bullock) Date: Mon, 12 Sep 2005 23:11:57 -0400 (EDT) Subject: [Python-Dev] os.path.diff(path1, path2) Message-ID: <20050913031157.46635.qmail@web51809.mail.yahoo.com> Just wondering if a function such as this has ever been considered? I find that I quite often want a function that will give me a relative path from path A to path B. I have created such a function, but it would be nice if it was in the standard library. This function would take two paths: A and B and give the relation between them. Here are a few of examples. os.path.diff("/A/C/D/", "/A/D/F/") ==> "../../D/F" os.path.diff("/A/", "/A/B/C/") ==> "B/C" os.path.diff("/A/B/C/", "/A/") ==> "../.." I suppose it would also be nice if you could use path + file. For example: os.path.diff("/A/C/D/xyz.html", "/A/D/F/zlf.html") ==> "../../D/F/zlf.html" I am not subscribed to the list so if anyone thinks this is useful please CC my email address. I also want to say thank you to everyone who has made Python what it is today. Nathan Bullock Visit my website at http://www.nathanbullock.org __________________________________________________________ Find your next car at http://autos.yahoo.ca From ajsiegel at optonline.net Tue Sep 13 16:16:36 2005 From: ajsiegel at optonline.net (Arthur) Date: Tue, 13 Sep 2005 10:16:36 -0400 Subject: [Python-Dev] IDLE development Message-ID: <0IMR004IMEC2LK50@mta9.srv.hcvlny.cv.net> Moam writes - >Hello, >More than a year and a half ago, I posted a big patch to IDLE which >adds support for completion and much better calltips, along with some >other improvements. I had also tried to have a little input to the IDLE development process. Suggesting on the idle-dev list it seemed to me that a trivial patch to the existing code would provide functionality to allow customization via user- defined domain-specific syntax highlighting files in a user's home directory. So that if one wished one could add appropriate syntax highlighting for say, Numeric. I left open the possibility that I was mistaken in my opinion that it was trivial or necessary or desirable. I was unable to assess whether the lack of my ability to get a yes, no or sideways was a result of the fact my suggestion and analysis was absurd, or something else. One of the "something else" possibilities was a broken process. Still can't assess it. Art From p.f.moore at gmail.com Tue Sep 13 17:14:35 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 13 Sep 2005 16:14:35 +0100 Subject: [Python-Dev] Simplify the file-like-object interface In-Reply-To: <20050913064439.xmcqf3zfjqrwoowg@login.werra.lunarpages.com> References: <20050913064439.xmcqf3zfjqrwoowg@login.werra.lunarpages.com> Message-ID: <79990c6b050913081441d6eaaa@mail.gmail.com> On 9/13/05, Michael Chermside wrote: > In unix, the above is true. One of the fundamental decisions in Unix > was to treat all files (and lots of other vaguely file-like things) > as undiferentiated streams of bytes. But this is NOT true on many > other operating systems. It is not, for example, true on Windows. Actually, on Windows, it *is* true. At the OS API level, all files are streams of bytes (it's not as uniform as Unix - many things that Unix forces into a file-like mould don't look exactly like OS-level files on Windows, consoles and sockets being particular examples). However, at the C library level, text files are "special" insofar as the C stdio routines handle CRLF issues and the like differently. The problem is twofold: (1) that Python works with the C runtime, so stdio behaviour gets involved, and (2) on Windows, the familiar Unix/C conventions of "\n" for "newline" don't work without translation - so if you write text to a binary file, your output doesn't conform to the *conventions* used by other applications (notably notepad - it's surprising how many Windows programs actually work fine with LF-delimited lines in text files...) Paul. From trentm at ActiveState.com Tue Sep 13 18:15:39 2005 From: trentm at ActiveState.com (Trent Mick) Date: Tue, 13 Sep 2005 09:15:39 -0700 Subject: [Python-Dev] os.path.diff(path1, path2) In-Reply-To: <20050913031157.46635.qmail@web51809.mail.yahoo.com> References: <20050913031157.46635.qmail@web51809.mail.yahoo.com> Message-ID: <20050913161539.GA32440@ActiveState.com> [Nathan Bullock wrote] > Just wondering if a function such as this has ever > been considered? I find that I quite often want a > function that will give me a relative path from path A > to path B. I have created such a function, but it > would be nice if it was in the standard library. > > This function would take two paths: A and B and give > the relation between them. Here are a few of examples. > > os.path.diff("/A/C/D/", "/A/D/F/") > ==> "../../D/F" > > os.path.diff("/A/", "/A/B/C/") > ==> "B/C" > > os.path.diff("/A/B/C/", "/A/") > ==> "../.." Look around for functions/recipes called "relpath". E.g.: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/302594 http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/208993 http://www.jorendorff.com/articles/python/path/ Trent -- Trent Mick TrentM at ActiveState.com From janssen at parc.com Wed Sep 14 02:52:16 2005 From: janssen at parc.com (Bill Janssen) Date: Tue, 13 Sep 2005 17:52:16 PDT Subject: [Python-Dev] Simplify the file-like-object interface In-Reply-To: Your message of "Tue, 13 Sep 2005 06:44:39 PDT." <20050913064439.xmcqf3zfjqrwoowg@login.werra.lunarpages.com> Message-ID: <05Sep13.175225pdt."58617"@synergy1.parc.xerox.com> > This [text/binary] distinction is > supported by the basic file operations in the C library. To open a > text file in binary mode is technically an error (although in many OSs > you'll get away with it). It's one of those "technical" errors that really isn't an error (from Python). On the other hand, opening a file in text mode will cause real data damage to binary files on Windows. Let some platform-specific library for that platform preserve it, if it cares to. I see no point in keeping this distinction in Python, even for those platforms bone-headed enough to preserve it. And the default certainly shouldn't be to the mode which loses data. Bill From nnorwitz at gmail.com Wed Sep 14 07:50:17 2005 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 13 Sep 2005 22:50:17 -0700 Subject: [Python-Dev] speeding up list append calls Message-ID: Tim made me do it! http://groups.google.com/group/comp.lang.python/msg/9075a3bc59c334c9 For whatever reason, I was just curious how his code could be sped up. I kept seeing this append method being called and I thought, "there's an opcode for that." What happens if you replace var.append() with the LIST_APPEND opcode. # standard opcodes $ ./python ./Lib/timeit.py -n 1000 -s 'import foo' 'foo.foo1(10000)' 1000 loops, best of 3: 3.66 msec per loop # hacked version $ ./python ./Lib/timeit.py -n 1000 -s 'import foo' 'foo.foo2(10000)' 1000 loops, best of 3: 1.74 msec per loop The patch and foo.py are attached. This code doesn't really work in general. It assumes that any append function call is a list method, which is obviously invalid. But if a variable is known to be a list (ie, local and assigned as list (BUILD_LIST) or a list comprehension), could we do something like this as a peephole optimization? I'm not familiar enough with some dynamic tricks to know if it there are conditions that could break this. Probably useless, but it was interesting to me at the time. n -------------- next part -------------- A non-text attachment was scrubbed... Name: LIST_APPEND.diff Type: text/x-patch Size: 1353 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050913/a32201e1/LIST_APPEND.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: foo.py Type: text/x-python Size: 209 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050913/a32201e1/foo.py From martin at v.loewis.de Wed Sep 14 18:34:46 2005 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 14 Sep 2005 18:34:46 +0200 Subject: [Python-Dev] Python code.interact() and UTF-8 locale In-Reply-To: <1126619609.3184.18.camel@haypopc> References: <1126408583.12608.34.camel@haypopc> <4f0b69dc05091301564229d4d7@mail.gmail.com> <1126619609.3184.18.camel@haypopc> Message-ID: <43285126.6030109@v.loewis.de> Victor STINNER wrote: > RexFi explains me that Python can't guess eval('len(u"?")') charset. I personally like to see a charset argument for eval() and compile(). exec would not directly support other charsets; you would have to compile() first to specify a charset. I have a patch somewhere that does that, but did not get to publish it yet. Regards, Martin From blais at furius.ca Wed Sep 14 18:40:38 2005 From: blais at furius.ca (Martin Blais) Date: Wed, 14 Sep 2005 12:40:38 -0400 Subject: [Python-Dev] os.path.diff(path1, path2) In-Reply-To: <20050913031157.46635.qmail@web51809.mail.yahoo.com> References: <20050913031157.46635.qmail@web51809.mail.yahoo.com> Message-ID: <8393fff050914094071c9fa93@mail.gmail.com> On 9/12/05, Nathan Bullock wrote: > Just wondering if a function such as this has ever > been considered? I find that I quite often want a > function that will give me a relative path from path A > to path B. I have created such a function, but it > would be nice if it was in the standard library. I also often have to plop a snippet of Python in my code to do the same and always wonder why there isn't anything like that in the standard library. +1 for adding this to os.path, it would be very handy. From martin at v.loewis.de Wed Sep 14 18:55:12 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 14 Sep 2005 18:55:12 +0200 Subject: [Python-Dev] speeding up list append calls In-Reply-To: References: Message-ID: <432855F0.6000906@v.loewis.de> Neal Norwitz wrote: > This code doesn't really work in general. It assumes that any append > function call is a list method, which is obviously invalid. But if a > variable is known to be a list (ie, local and assigned as list > (BUILD_LIST) or a list comprehension), could we do something like this > as a peephole optimization? Alternatively, couldn't LIST_APPEND check that this really is a list, and, if it isn't, fall back to PyObject_CallMethod? Not sure which .append call is performed most frequently, but the traditional trick of caching x.append in a local variable might give you most of the speedup. Regards, Martin From reinhold-birkenfeld-nospam at wolke7.net Wed Sep 14 19:14:39 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Wed, 14 Sep 2005 19:14:39 +0200 Subject: [Python-Dev] speeding up list append calls In-Reply-To: <432855F0.6000906@v.loewis.de> References: <432855F0.6000906@v.loewis.de> Message-ID: Martin v. L?wis wrote: > Neal Norwitz wrote: >> This code doesn't really work in general. It assumes that any append >> function call is a list method, which is obviously invalid. But if a >> variable is known to be a list (ie, local and assigned as list >> (BUILD_LIST) or a list comprehension), could we do something like this >> as a peephole optimization? > > Alternatively, couldn't LIST_APPEND check that this really is a list, > and, if it isn't, fall back to PyObject_CallMethod? Are there any other optimizations which solely act on the name of a method? This seems a step too far. Reinhold -- Mail address is perfectly valid! From fredrik at pythonware.com Wed Sep 14 19:23:37 2005 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 14 Sep 2005 19:23:37 +0200 Subject: [Python-Dev] speeding up list append calls References: <432855F0.6000906@v.loewis.de> Message-ID: Martin v. Löwis wrote: > Alternatively, couldn't LIST_APPEND check that this really is a list, > and, if it isn't, fall back to PyObject_CallMethod? that's the obvious solution, of course. cf. existing shortcuts: $ grep -n INLINE Python-2.4.1/Python/ceval.c 1103: /* INLINE: int + int */ 1133: /* INLINE: int - int */ 1156: /* INLINE: list[int] */ 1305: /* INLINE: int + int */ 1335: /* INLINE: int - int */ 1967: /* INLINE: cmp(int, int) */ From pje at telecommunity.com Wed Sep 14 19:34:09 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 14 Sep 2005 13:34:09 -0400 Subject: [Python-Dev] speeding up list append calls In-Reply-To: <432855F0.6000906@v.loewis.de> References: Message-ID: <5.1.1.6.0.20050914130458.01b693d8@mail.telecommunity.com> At 06:55 PM 9/14/2005 +0200, Martin v. L?wis wrote: >Neal Norwitz wrote: > > This code doesn't really work in general. It assumes that any append > > function call is a list method, which is obviously invalid. But if a > > variable is known to be a list (ie, local and assigned as list > > (BUILD_LIST) or a list comprehension), could we do something like this > > as a peephole optimization? > >Alternatively, couldn't LIST_APPEND check that this really is a list, >and, if it isn't, fall back to PyObject_CallMethod? That's an interesting idea - the opcodes for some math operators check if the operands are integers and then have a fast path, so this would sort of be the reverse. >Not sure which .append call is performed most frequently, but the >traditional trick of caching x.append in a local variable might give >you most of the speedup. Maybe the VM could actually do that caching for you, if we had a CALL_METHOD opcode that cached the attribute in a local variable if it was a C method or Python instance method, and reused the cached attribute as long as the target object is of the same type/class as the last invocation at that point. I think this is called a "polymorphic inline cache", although it doesn't seem very polymorphic if you're only caching one type. :) The downside, alas, would be that modifying the class, or using dynamically generated methods would break this, unless there was some sort of "version counter" on the class that could also be checked, and the caching mechanism only cached in the first place if the attribute was found in a class dictionary. Or perhaps the same mechanism that notifies subclasses of changes to the class could be used to notify frame objects of cache invalidation. The interesting question, of course, is whether all the extra complexity would be worth it. I think other polymorphic inline caches actually cache per call site, not per frame invocation, so it might be that the code object would actually be the place to cache this, allowing the program as a whole to gain. From falcon at intercable.ru Wed Sep 14 19:12:36 2005 From: falcon at intercable.ru (Sokolov Yura) Date: Wed, 14 Sep 2005 21:12:36 +0400 Subject: [Python-Dev] Skiping searching throw dictionaries of mro() members. Message-ID: <43285A04.5050605@intercable.ru> Excuse my english. I have a complex Idea. It can be not worth, but just look at. It touches the classes and inheritance. Base point: We change classes not too often. But every look at a member leads to search throw a __dict__ of all bases in ierarhy till member found. We could cache this search in class.__dict__ itself and check only bases versions. Lets see: we have some classes: __metaclass__=object class a: ....def meth_a(self): ........pass class b: ....def meth_b(self): ........pass class c(a,b): ....pass Every class has mro. >>>c.mro() [ , , , ] if we call methon from a base class, we look at c.__dict__ and at dict of every inherited class till found the method itself. so c.meth_a() leads to search throw c.__dict__ and a.__dict__ c.meth_b() leads to search throw c.__dict__ , a.__dict__ and b.__dict__ We could cash looks by this way: Store with a class its version. Every time after creation when we change a class (add,remove or chage class member, including __base__, __bases__ and mro) , we increase the version number. Lets call it VERSION Also store a tuple MRO_VERSION=tuple(base.VERSION for base in self.mro()). It changes, when we touches __base__, __bases__ or self.mro) Store with every class member a tuple : MEMBER_VERSION=( , ) When we add member to (or change member of) class __dict__ directly, we store in this tuple (MRO_VERSION,class.mro().index(class)). When we search a class member: a) and have not found it in class.__dict__, we search it throw mro() by a classic way. If we've found it (in a class _BASE_), we store it in class.__dict__ with a tuple MEMBER_VERSION=(MRO_VERSION, class.mro().index(_BASE_)). Also we check all of seen base.VERSION to match their cached in MRO_VERSION values. If some is not match, we should update MRO_VERSION (berfore storing founded member in a class.__dict__) and adjust VERSION. If it was not found, we could store a (MRO_VERSION, class.mro().index(class)) with an internal value NotFound or do not store anything, and report that we have not found anything (so we search in an object itself). b) When we've found a class member in class.__dict__ we check versions if MEMBER_VERSION[0]==class.MRO_VERSION (the same object) then we just compare MRO_VERSION[i]==class.mro()[i].VERSION for i in range(MEMBER_VERSION[1]+1) and not search throw mro()[i].__dict__ what we are doing now. If all versions concur, we return found member (or, if we've found previously stored NotFound, we report that we have not found anything) if version of MRO_VERSION[j]!=class.mro()[j].VERSION and class!=class.mro()[j] (here can be a bug, one must think about it more) then we revert to a) point, but search throw [base.__dict__ for base in class.mro()[j:]], (and, of cause, update MRO_VERSION) if MEMBER_VERSION[0]!=class.MRO_VERSION, we reverts to a) point. That's all. PS. We can subclass module from a __builtin__, so we leave one dict lookup. From pje at telecommunity.com Wed Sep 14 19:49:42 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 14 Sep 2005 13:49:42 -0400 Subject: [Python-Dev] Skiping searching throw dictionaries of mro() members. In-Reply-To: <43285A04.5050605@intercable.ru> Message-ID: <5.1.1.6.0.20050914134751.01b766e8@mail.telecommunity.com> At 09:12 PM 9/14/2005 +0400, Sokolov Yura wrote: >We could cash looks by this way: >Store with a class its version. Every time after creation when we change >a class >(add,remove or chage class member, including __base__, __bases__ and mro) , >we increase the version number. Lets call it VERSION FYI, there is already a mechanism for new-style classes that notifies subclasses of changes to the base class; this mechanism could be used to push cache flushes downward, rather than doing version checking upward. From guido at python.org Wed Sep 14 20:18:32 2005 From: guido at python.org (Guido van Rossum) Date: Wed, 14 Sep 2005 11:18:32 -0700 Subject: [Python-Dev] Fwd: [Python-checkins] python/dist/src/Misc NEWS, 1.1193.2.94, 1.1193.2.95 In-Reply-To: <20050914181506.EAADC1E400E@bag.python.org> References: <20050914181506.EAADC1E400E@bag.python.org> Message-ID: I have no way to test this code on Windows (or on other UNIX platforms besides Linux and AIX). Hopefully someone can verify whether it breaks anything. ---------- Forwarded message ---------- From: gvanrossum at users.sourceforge.net Date: Sep 14, 2005 11:15 AM Subject: [Python-checkins] python/dist/src/Misc NEWS, 1.1193.2.94, 1.1193.2.95 To: python-checkins at python.org Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv24791/Misc Modified Files: Tag: release24-maint NEWS Log Message: - Changes donated by Elemental Security to make it work on AIX 5.3 with IBM's 64-bit compiler (SF patch #1284289). This also closes SF bug #105470: test_pwd fails on 64bit system (Opteron). Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1193.2.94 retrieving revision 1.1193.2.95 diff -u -d -r1.1193.2.94 -r1.1193.2.95 --- NEWS 14 Sep 2005 17:54:39 -0000 1.1193.2.94 +++ NEWS 14 Sep 2005 18:15:03 -0000 1.1193.2.95 @@ -12,6 +12,10 @@ Core and builtins ----------------- +- Changes donated by Elemental Security to make it work on AIX 5.3 + with IBM's 64-bit compiler (SF patch #1284289). This also closes SF + bug #105470: test_pwd fails on 64bit system (Opteron). + - Changes donated by Elemental Security to make it work on HP-UX 11 on Itanium2 with HP's 64-bit compiler (SF patch #1225212). _______________________________________________ Python-checkins mailing list Python-checkins at python.org http://mail.python.org/mailman/listinfo/python-checkins -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nnorwitz at gmail.com Wed Sep 14 21:01:09 2005 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 14 Sep 2005 12:01:09 -0700 Subject: [Python-Dev] [Python-checkins] python/dist/src/Lib/test regrtest.py, 1.171, 1.172 test_ioctl.py, 1.2, 1.3 In-Reply-To: <20050914180944.848421E4003@bag.python.org> References: <20050914180944.848421E4003@bag.python.org> Message-ID: > Index: test_ioctl.py > =================================================================== > RCS file: /cvsroot/python/python/dist/src/Lib/test/test_ioctl.py,v > retrieving revision 1.2 > retrieving revision 1.3 > diff -u -d -r1.2 -r1.3 > --- test_ioctl.py 20 Mar 2003 04:33:16 -0000 1.2 > +++ test_ioctl.py 14 Sep 2005 18:09:41 -0000 1.3 > @@ -16,19 +16,23 @@ > > class IoctlTests(unittest.TestCase): > def test_ioctl(self): > - pgrp = os.getpgrp() > + # If this process has been put into the background, TIOCGPGRP returns > + # the session ID instead of the process group id. > + ids = (os.getpgrp(), os.getsid(0)) > tty = open("/dev/tty", "r") > r = fcntl.ioctl(tty, termios.TIOCGPGRP, " ") > - self.assertEquals(pgrp, struct.unpack("i", r)[0]) > + rpgrp = struct.unpack("i", r)[0] > + self.assert_(rpgrp in ids, "%s not in %s" % (rpgrp, ids)) With the change to use unsigned ints in pwd and grp modules, should the struct.unpack() use "I" (capital i) instead of "i"? n From fredrik at pythonware.com Wed Sep 14 20:59:05 2005 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 14 Sep 2005 20:59:05 +0200 Subject: [Python-Dev] speeding up list append calls References: <432855F0.6000906@v.loewis.de> <5.1.1.6.0.20050914130458.01b693d8@mail.telecommunity.com> Message-ID: Phillip J. Eby wrote: > I think this is called a "polymorphic inline cache", although it doesn't > seem very polymorphic if you're only caching one type. :) if it's not polymorphic, it's an ordinary inline cache (aka "call-site cache"). (see various Self papers for more on polymorphic caches) From pjones at redhat.com Wed Sep 14 21:25:08 2005 From: pjones at redhat.com (Peter Jones) Date: Wed, 14 Sep 2005 15:25:08 -0400 Subject: [Python-Dev] unintentional and unsafe use of realpath() In-Reply-To: <43276823.2000603@levkowetz.com> References: <1126380571.21655.18.camel@localhost.localdomain> <43276823.2000603@levkowetz.com> Message-ID: <1126725908.23071.11.camel@localhost.localdomain> [re-adding Python-Dev] On Wed, 2005-09-14 at 02:00 +0200, Henrik Levkowetz wrote: > Hi Peter, > > on 2005-09-10 21:29 Peter Jones said the following: > > Hi, > > > > In Python 2.4.1, Python/sysmodule.c includes a function PySys_SetArgv(). > > One of the things it does is attempt to resolve symbolic links into > > absolute paths. Currently, it uses readlink() if configure found that > > your system supports it, and then it tries to do the same thing again > > using realpath() if you system supports that. > > > > This seems wrong; there's really no reason to do both. So here's a > > patch to move the realpath() usage into a #else following the > > HAVE_READLINK test: > > > > If a path component above the basename is a link, it will not be > resolved by only doing readlink() on the basename, while realpath() will > resolve it. So it seems to me that your proposed patch will accomplish > less than the current code for systems which have realpath(). What the current code is doing is a buffer overrun. Nevertheless, you're right that it would behave differently than the current code if that code worked as intended. Here's a different patch; it uses canonicalize_file_name instead, and if you don't have that it just uses the same codepath that's already in the tree. canonicalize_file_name is identical to realpath() when using the glibc extension of allocating a buffer when you pass in a NULL target, as in: fullpath = realpath(path, NULL); I chose to use canonicalize_file_name simply because the autoconf test for it is easier than testing to see if the extension works. Here's the new patch: --- Python-2.4.1/pyconfig.h.in.canonicalize 2005-09-14 11:47:04.000000000 -0400 +++ Python-2.4.1/pyconfig.h.in 2005-09-14 11:47:02.000000000 -0400 @@ -58,6 +58,9 @@ /* Define if pthread_sigmask() does not work on your system. */ #undef HAVE_BROKEN_PTHREAD_SIGMASK +/* Define to 1 if you have the `canonicalize_file_name' function. */ +#undef HAVE_CANONICALIZE_FILE_NAME + /* Define to 1 if you have the `chown' function. */ #undef HAVE_CHOWN --- Python-2.4.1/Python/sysmodule.c.canonicalize 2005-09-14 11:53:30.000000000 -0400 +++ Python-2.4.1/Python/sysmodule.c 2005-09-14 11:52:04.000000000 -0400 @@ -1184,6 +1184,9 @@ char *p = NULL; int n = 0; PyObject *a; +#ifdef HAVE_CANONICALIZE_FILE_NAME + argv0 = canonicalize_file_name(argv0); +#else /* ! HAVE_CANONICALIZE_FILE_NAME */ #ifdef HAVE_READLINK char link[MAXPATHLEN+1]; char argv0copy[2*MAXPATHLEN+1]; @@ -1256,6 +1259,7 @@ #endif /* Unix */ } #endif /* All others */ +#endif /* ! HAVE_CANONICALIZE_FILE_NAME */ a = PyString_FromStringAndSize(argv0, n); if (a == NULL) Py_FatalError("no mem for sys.path insertion"); --- Python-2.4.1/configure.in.canonicalize 2005-09-14 11:46:00.000000000 -0400 +++ Python-2.4.1/configure.in 2005-09-14 11:47:22.000000000 -0400 @@ -2096,8 +2096,8 @@ AC_MSG_RESULT(MACHDEP_OBJS) # checks for library functions -AC_CHECK_FUNCS(alarm bind_textdomain_codeset chown clock confstr ctermid \ - execv fork fpathconf ftime ftruncate \ +AC_CHECK_FUNCS(alarm bind_textdomain_codeset canonicalize_file_name chown \ + clock confstr ctermid execv fork fpathconf ftime ftruncate \ gai_strerror getgroups getlogin getloadavg getpeername getpgid getpid \ getpriority getpwent getsid getwd \ kill killpg lchown lstat mkfifo mknod mktime \ -- Peter From reinhold-birkenfeld-nospam at wolke7.net Wed Sep 14 21:27:36 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Wed, 14 Sep 2005 21:27:36 +0200 Subject: [Python-Dev] Term unification Message-ID: Hi, looking at bug #1283289, I saw that the term "keyword parameter" is used in Python/getargs.c, mixed with "keyword argument". Grepping through the source, "keyword parameter" had 43 matches, while "keyword argument" had 430. Should the "parameter" form be extinguished? Reinhold (And BTW, should bug #1283289 be fixed as the poster suggests?) -- Mail address is perfectly valid! From guido at python.org Wed Sep 14 21:46:16 2005 From: guido at python.org (Guido van Rossum) Date: Wed, 14 Sep 2005 12:46:16 -0700 Subject: [Python-Dev] Term unification In-Reply-To: References: Message-ID: Correct usage is argument for the call site but parameter for the function/method definition. So you can't just count occurrences. On 9/14/05, Reinhold Birkenfeld wrote: > Hi, > > looking at bug #1283289, I saw that the term "keyword parameter" is used in > Python/getargs.c, mixed with "keyword argument". > > Grepping through the source, "keyword parameter" had 43 matches, while > "keyword argument" had 430. Should the "parameter" form be extinguished? > > Reinhold > > (And BTW, should bug #1283289 be fixed as the poster suggests?) > > > -- > Mail address is perfectly valid! > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From reinhold-birkenfeld-nospam at wolke7.net Wed Sep 14 22:05:25 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Wed, 14 Sep 2005 22:05:25 +0200 Subject: [Python-Dev] Term unification In-Reply-To: References: Message-ID: Ah, ok. Then I guess everything's fine. It was just the /* make sure there are no duplicate values for an argument; its not clear when to use the term "keyword argument vs. keyword parameter in messages */ that disturbed me. Reinhold Guido van Rossum wrote: > Correct usage is argument for the call site but parameter for the > function/method definition. So you can't just count occurrences. > > On 9/14/05, Reinhold Birkenfeld wrote: >> Hi, >> >> looking at bug #1283289, I saw that the term "keyword parameter" is used in >> Python/getargs.c, mixed with "keyword argument". >> >> Grepping through the source, "keyword parameter" had 43 matches, while >> "keyword argument" had 430. Should the "parameter" form be extinguished? >> >> Reinhold >> >> (And BTW, should bug #1283289 be fixed as the poster suggests?) -- Mail address is perfectly valid! From reinhold-birkenfeld-nospam at wolke7.net Thu Sep 15 00:09:24 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Thu, 15 Sep 2005 00:09:24 +0200 Subject: [Python-Dev] str.dedent Message-ID: Hi, some time ago, I proposed a string method "dedent" (which currently is in the textwrap module). The RFE is at http://python.org/sf/1237680. Any opinions? If I don't get positive comments, I'll reject it. Reinhold -- Mail address is perfectly valid! From guido at python.org Thu Sep 15 01:10:26 2005 From: guido at python.org (Guido van Rossum) Date: Wed, 14 Sep 2005 16:10:26 -0700 Subject: [Python-Dev] str.dedent In-Reply-To: References: Message-ID: >From the sound of it, it's probably not worth endowing every string object with this method and hardcoding its implementation forever in C code. There are so many corner cases and variations on the functionality of "dedenting" a block that it's better to keep it as Python source code. On 9/14/05, Reinhold Birkenfeld wrote: > Hi, > > some time ago, I proposed a string method "dedent" (which currently is in the > textwrap module). The RFE is at http://python.org/sf/1237680. > > Any opinions? If I don't get positive comments, I'll reject it. > > Reinhold -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Sep 15 01:53:59 2005 From: guido at python.org (Guido van Rossum) Date: Wed, 14 Sep 2005 16:53:59 -0700 Subject: [Python-Dev] [Python-checkins] python/dist/src/Lib/test regrtest.py, 1.171, 1.172 test_ioctl.py, 1.2, 1.3 In-Reply-To: References: <20050914180944.848421E4003@bag.python.org> Message-ID: On 9/14/05, Neal Norwitz wrote: > > Index: test_ioctl.py > > =================================================================== > > RCS file: /cvsroot/python/python/dist/src/Lib/test/test_ioctl.py,v > > retrieving revision 1.2 > > retrieving revision 1.3 > > diff -u -d -r1.2 -r1.3 > > --- test_ioctl.py 20 Mar 2003 04:33:16 -0000 1.2 > > +++ test_ioctl.py 14 Sep 2005 18:09:41 -0000 1.3 > > @@ -16,19 +16,23 @@ > > > > class IoctlTests(unittest.TestCase): > > def test_ioctl(self): > > - pgrp = os.getpgrp() > > + # If this process has been put into the background, TIOCGPGRP returns > > + # the session ID instead of the process group id. > > + ids = (os.getpgrp(), os.getsid(0)) > > tty = open("/dev/tty", "r") > > r = fcntl.ioctl(tty, termios.TIOCGPGRP, " ") > > - self.assertEquals(pgrp, struct.unpack("i", r)[0]) > > + rpgrp = struct.unpack("i", r)[0] > > + self.assert_(rpgrp in ids, "%s not in %s" % (rpgrp, ids)) > > With the change to use unsigned ints in pwd and grp modules, should > the struct.unpack() use "I" (capital i) instead of "i"? I asked the author of the patch (Monte Davidoff, who occasionally comes to baypiggies meetings :-) and his response is: """ No. The change to the pwd and grp modules and the change to test_ioctl are unrelated. Unfortunately, the term "group" is overloaded, which leads to confusion. The pwd and grp modules read the password database and the group database, which are attributes of a UNIX user. The test_ioctl test case is calling the TIOCGPGRP ioctl, which returns the process group id, which is an attribute of the process. This has type pid_t, which is signed. I hope this clarifies the situation. I am glad to hear that it is checked in! Monte """ -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Thu Sep 15 01:45:39 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 15 Sep 2005 11:45:39 +1200 Subject: [Python-Dev] os.path.diff(path1, path2) In-Reply-To: <20050913031157.46635.qmail@web51809.mail.yahoo.com> References: <20050913031157.46635.qmail@web51809.mail.yahoo.com> Message-ID: <4328B623.3070905@canterbury.ac.nz> Nathan Bullock wrote: > I find that I quite often want a > function that will give me a relative path from path A > to path B. I have created such a function, but it > would be nice if it was in the standard library. +1 from me. It's a fairly common thing to want to do. Greg From raymond.hettinger at verizon.net Thu Sep 15 03:29:41 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Wed, 14 Sep 2005 21:29:41 -0400 Subject: [Python-Dev] str.dedent In-Reply-To: Message-ID: <001901c5b994$f7b72340$b8fdcc97@oemcomputer> > some time ago, I proposed a string method "dedent" (which currently is in > the > textwrap module). The RFE is at http://python.org/sf/1237680. > > Any opinions? If I don't get positive comments, I'll reject it. -1 Let it continue to live in textwrap where the existing pure python code adequately serves all string-like objects. It's not worth losing the duck typing by attaching new methods to str, unicode, UserString, and everything else aspiring to be string-like. String methods should be limited to generic string manipulations. String applications should be in other namespaces. That is why we don't have str.md5(), str.crc32(), str.ziplib(), etc. Also, I don't want to encourage dedenting as a way of life --- programs using it often are likely to be doing things the hard way. Raymond From kbk at shore.net Thu Sep 15 06:50:44 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Thu, 15 Sep 2005 00:50:44 -0400 (EDT) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200509150450.j8F4oiHG004973@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 343 open ( +1) / 2927 closed ( +4) / 3270 total ( +5) Bugs : 908 open ( +0) / 5245 closed (+13) / 6153 total (+13) RFE : 189 open ( +1) / 185 closed ( +0) / 374 total ( +1) New / Reopened Patches ______________________ Allow to restrict ModuleFinder to get "direct" dependencies (2005-09-08) http://python.org/sf/1284670 opened by Scherer Michael pygettext - provide comments to help translators (2005-09-11) http://python.org/sf/1288056 opened by Toby Dickenson FreeBSD 5.3 and upward has thread-safe getaddrinfo(3) (2005-09-12) http://python.org/sf/1288833 opened by Maxim Sobolev Fix reload() error message (2005-09-13) CLOSED http://python.org/sf/1290454 opened by Collin Winter mmap with unsigned int offset and cross build for ppc (2005-09-14) http://python.org/sf/1291169 opened by Jos? Pedro Pereira Valente de M Patches Closed ______________ Fix reload() error message (2005-09-13) http://python.org/sf/1290454 closed by birkenfeld HP-UX ia64 64-bit Python executable (2005-06-21) http://python.org/sf/1225212 closed by gvanrossum AIX port from Elemental Security (2005-09-07) http://python.org/sf/1284289 closed by gvanrossum new patch for fixing skipitem() in getargs.c (2005-06-01) http://python.org/sf/1212928 closed by birkenfeld New / Reopened Bugs ___________________ traceback module can return undecodable byte strings (2005-09-08) http://python.org/sf/1284496 opened by Stuart Bishop logging module's setLoggerClass not really working (2005-09-08) http://python.org/sf/1284928 opened by Rotem Yaari urllib.quote is too slow (2005-09-08) http://python.org/sf/1285086 opened by Tres Seaver Call to cmd.exe fails (2005-09-08) CLOSED http://python.org/sf/1285325 opened by Alex Luso Digest Authentication not working in all cases (2005-09-08) http://python.org/sf/1285440 opened by Jakob Simon-Gaarde re special sequence '\w' (2005-09-09) CLOSED http://python.org/sf/1285809 opened by ChristianJ Python code.interact() and UTF-8 locale (2005-09-12) http://python.org/sf/1288615 opened by STINNER Victor timedelta multiply and divide by floating point (2005-09-12) http://python.org/sf/1289118 opened by Daniel Stutzbach distutils extension library path bug on cygwin (2005-09-12) http://python.org/sf/1289136 opened by John Whitley PyDoc crashes at os.py line 133 (2005-09-12) CLOSED http://python.org/sf/1289210 opened by Colin J. Williams cjkcodec compile error under AIX 5.2 on symbol 100_encode (2005-09-13) http://python.org/sf/1290333 opened by Deron Meranda --no-compile option has no effect (2005-09-13) CLOSED http://python.org/sf/1290382 opened by Tim Peters strptime(): can't switch locales more than once (2005-09-13) CLOSED http://python.org/sf/1290505 opened by Adam Monsen SSLObject breaks read semantics (2005-09-14) http://python.org/sf/1291446 opened by Jonathan Ellis Bugs Closed ___________ Call to cmd.exe fails (2005-09-08) http://python.org/sf/1285325 deleted by delenca exception when unpickling array.array objects (2005-09-04) http://python.org/sf/1281556 closed by rhettinger PyDoc crashes at os.py line 133 (2005-09-12) http://python.org/sf/1289210 closed by rhettinger --no-compile option has no effect (2005-09-13) http://python.org/sf/1290382 closed by tim_one RE parser too loose with {m,n} construct (2005-05-15) http://python.org/sf/1202493 closed by niemeyer document {m} regex matcher wrt empty matches (2005-01-31) http://python.org/sf/1113484 closed by niemeyer r'\10' as replacement pattern loops in compilation (2004-11-02) http://python.org/sf/1058786 closed by niemeyer re nested conditional matching (?()) doesn't work (2005-09-08) http://python.org/sf/1284341 closed by birkenfeld skipitem() in getargs.c missing some types (2004-02-09) http://python.org/sf/893549 closed by birkenfeld re special sequence '\w' (2005-09-09) http://python.org/sf/1285809 closed by birkenfeld suspected cPickle memory leak (2005-05-13) http://python.org/sf/1201461 closed by birkenfeld splitunc not documented (2005-08-27) http://python.org/sf/1274828 closed by birkenfeld os.startfile() doesn't accept Unicode filenames (2004-08-11) http://python.org/sf/1007046 closed by birkenfeld strptime(): can't switch locales more than once (2005-09-13) http://python.org/sf/1290505 closed by bcannon From reinhold-birkenfeld-nospam at wolke7.net Thu Sep 15 07:44:10 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Thu, 15 Sep 2005 07:44:10 +0200 Subject: [Python-Dev] str.dedent In-Reply-To: References: Message-ID: Okay. I consider it rejected. Reinhold Guido van Rossum wrote: >>From the sound of it, it's probably not worth endowing every string > object with this method and hardcoding its implementation forever in C > code. There are so many corner cases and variations on the > functionality of "dedenting" a block that it's better to keep it as > Python source code. > > On 9/14/05, Reinhold Birkenfeld wrote: >> Hi, >> >> some time ago, I proposed a string method "dedent" (which currently is in the >> textwrap module). The RFE is at http://python.org/sf/1237680. >> >> Any opinions? If I don't get positive comments, I'll reject it. >> >> Reinhold > -- Mail address is perfectly valid! From reinhold-birkenfeld-nospam at wolke7.net Thu Sep 15 07:56:39 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Thu, 15 Sep 2005 07:56:39 +0200 Subject: [Python-Dev] python/dist/src/Lib urllib.py, 1.165.2.1, 1.165.2.2 In-Reply-To: <20050915052828.A4FD81E4003@bag.python.org> References: <20050915052828.A4FD81E4003@bag.python.org> Message-ID: rhettinger at users.sourceforge.net wrote: > Update of /cvsroot/python/python/dist/src/Lib > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31892 > > Modified Files: > Tag: release24-maint > urllib.py > Log Message: > Sync-up with patches to the head. > Includes SF 1016880: urllib.urlretrieve silently truncates downloads > and the performance fix-ups. This last patch includes a new exception, are you sure that this can be safely backported? If so, the documentation changes must be backported, too. Reinhold -- Mail address is perfectly valid! From raymond.hettinger at verizon.net Thu Sep 15 08:20:26 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Thu, 15 Sep 2005 02:20:26 -0400 Subject: [Python-Dev] python/dist/src/Lib urllib.py, 1.165.2.1, 1.165.2.2 In-Reply-To: Message-ID: <000501c5b9bd$91e9bfe0$6916c797@oemcomputer> [Reinhold Birkenfeld] > This last patch includes a new exception, are you sure that this can be > safely backported? Not too worried about it. Better to have the exception reported than the silent failure that confused the heck out of everyone who tried to figure-out what caused the OP's problem. > If so, the documentation changes must be backported, too. Maybe. My thought is the new message is akin to an improved error message. However, adding it to the Py2.4 docs suggests that you could catch and handle the exception, but that cannot be done portably across Py2.4 versions. If you really feel the need, go ahead and add to the docs with \versionadded{2.4.2}. My preference is to leave it be. Raymond From reinhold-birkenfeld-nospam at wolke7.net Thu Sep 15 09:22:00 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Thu, 15 Sep 2005 09:22:00 +0200 Subject: [Python-Dev] python/dist/src/Lib urllib.py, 1.165.2.1, 1.165.2.2 In-Reply-To: <000501c5b9bd$91e9bfe0$6916c797@oemcomputer> References: <000501c5b9bd$91e9bfe0$6916c797@oemcomputer> Message-ID: Raymond Hettinger wrote: > [Reinhold Birkenfeld] >> This last patch includes a new exception, are you sure that this can > be >> safely backported? > > Not too worried about it. Better to have the exception reported than > the silent failure that confused the heck out of everyone who tried to > figure-out what caused the OP's problem. > > > >> If so, the documentation changes must be backported, too. > > Maybe. My thought is the new message is akin to an improved error > message. However, adding it to the Py2.4 docs suggests that you could > catch and handle the exception, but that cannot be done portably across > Py2.4 versions. If you really feel the need, go ahead and add to the > docs with \versionadded{2.4.2}. My preference is to leave it be. Makes sense. It'll be best to leave it be. Reinhold -- Mail address is perfectly valid! From falcon at intercable.ru Wed Sep 14 20:11:03 2005 From: falcon at intercable.ru (Sokolov Yura) Date: Wed, 14 Sep 2005 22:11:03 +0400 Subject: [Python-Dev] Skiping searching throw dictionaries of mro() members. In-Reply-To: <5.1.1.6.0.20050914134751.01b766e8@mail.telecommunity.com> References: <5.1.1.6.0.20050914134751.01b766e8@mail.telecommunity.com> Message-ID: <432867B7.8000105@intercable.ru> Phillip J. Eby wrote: > At 09:12 PM 9/14/2005 +0400, Sokolov Yura wrote: > >> We could cash looks by this way: >> Store with a class its version. Every time after creation when we change >> a class >> (add,remove or chage class member, including __base__, __bases__ and >> mro) , >> we increase the version number. Lets call it VERSION > > > FYI, there is already a mechanism for new-style classes that notifies > subclasses of changes to the base class; this mechanism could be used > to push cache flushes downward, rather than doing version checking > upward. > > > I didn't know it. So, if this thing works, it should be used. Another think. I do not really know, how emplemented scopes, but could this caching be applied to a scope realisation? So we "derive" current scope from his outer and cache all lookups, that goes out of this scope. Top scope is "derived" from a module and module it self "derived" from builtins. I think, there not many different deep upper lookups in current programs, but many reapeating, and not often we change outer scope, so cache will be fine. From falcon at intercable.ru Thu Sep 15 09:19:13 2005 From: falcon at intercable.ru (Sokolov Yura) Date: Thu, 15 Sep 2005 11:19:13 +0400 Subject: [Python-Dev] Variant of removing GIL. Message-ID: <43292071.8050604@intercable.ru> Excuse my English. I think I know how to remove GIL!!!! Obviously I am an idiot. First about Py_INCREF and Py_DECREF. We should not remove GIL at all. We should change it. It must be "one writer-many reader" in a following semantic: Lock has a "read-counter" and a "write-counter". Initially both are 0. When "reader" tries to acquire lock for "read" it sleeps until "write-counter" is 0. When he "reader" acquires lock, he increase "read-counter". When "reader" releases lock, he decreases "read-counter". One reader will not block other, since he not increases "write-counter". Reader will sleep, if there is any waiting writers, since they are increase "write-counter". When "writer" tries to acquire lock for "write", he increase "write-counter" and sleeps until "read-counter" happens 0. For "writers" lock for "write" - simple lock. when "writer" release lock, he decrease "write-counter". When there is no waiting writers, readers arise. Excuse me for telling obviouse things. I am really reinvent wheel in my head, since I was a bad studient. I think this kind of lock is native for linux (i saw it in a kernel source, but do not know is waiting writer locks new readers or not?). Now, every thread keep an queue of objects to decref. It can be implemented as array, cause it will be freed at once. Initially, every object acquires GIL for "read". Py_INCREF works as usually, Py_DECREF places a ref into a queue. When queue has became full or "100" instruction left ( :-) , it usefull), thread releases GIL for "read" and acquires for "write", when he acquire it, he decrefs all objects stored in a queue and clear queue. After all he acquires GIL for "read". But what could we do with changing objects (dicts,lists and another)? There should be a secondary "one-writer-many-reader" "public-write" GIL - PWGIL. SGIL ought to be more complicated, since it should work in RLOCK semantic for "write" lock. Lets call this lock ROWMR(reentreed one writer - many reader) So semantic for ROWMR can be: When a thread acquires ROWMR lock, it acquires it at a "read" level. Lets name it "write-level"=0. While threads "write-level"=0 it is a "reader". Thread can increase "write-level". When he turns "write-level" from 0 to 1, he becomes "writer". while "write-level">0, thread is writer. Thread can decrease "write-level". When "write-level" turns from 1 to 0, thread becomes "reader". With PWGIL : We can mark every _mutable_ object with a creator thread number. If mark match current thread number, object is "private" for the thread. If mark is 0 (or another imposible thread number) object is "public". If mark !=0 and !=current thread number, object is "alien". When we access _mutable_ object, we check is it "private"? If it is, we can do anything without locking. If it is not and we access for read, we check is it "public". If yes ("read" of "public"), then we can read it without locking. If no, we increase "write-level", if object is "alien", make it "public", if we need to change object, change it, decrease "write-level". Of couse, when we append object to "public" collection, we chould make it "public", "write-level" is already increased so we do not make many separate locks, but when we then will access thouse object for read, we will not lock for make it "public". I don't know, how nested scopes are implemented, but i think it should be considered as a mutable object. So there is a small overhead for a single threaded application ( only for comparing 2 numbers) and in a big part of multithreaded, since we are locking only writting on _mutable_ _public_ objects. Most part of "public" objects is not accessed to write often: they are numbers, classes and mostly-read collections. And one can optimize a program by accumulating results in a "private" collection and then flush it to "public" one. Also, there may be a statement for explicit increasing "write-level" around big update of "public" object and decreasing after it. PWGIL also must be released and reacquired with every "100" instructions left,but only if "write-level=0", it conforms to current GIL semantic. I think, it must be not released with flushing decref queue, since it can happen while we are in C code. And there must be strong think about blocking IO. Mostly awful situation (at my point of view): object O is "private" for a thread A. thread B accesses O and try to mark it "public", so it locks in attempt of increasing "write-level" thread A starts to change O (it is in "write-level 0"), and in a C code it releases PWGIL (around blocking IO, for example). thread B becomes "writer", changes object to "public", becomes "reader" and starts to read O, returning thread A continue to change O , remaining in a "write-level=0". But, I think, well written C code should not attemt to make blocking IO inside of changing non-local objects (and it does not attempt at the moment, as I guess. Am I mistaken?). Or/and, when it returns and continues to change O, it must check, is it "private" or it isn't? I think, big part of checks and manipulation with GIL&PWGIL could be hidden inside of current C API, so we should not change a tons of libraries written in C. Only libraries, which create mutable objects which use notstandart containers for storing. Maybe there should be only one united SGIL for incref-decref and "write public". Summary overhead: each Py_DECREF place reference in a thread local queue (it could be small enough - about 1000, and not dinamic - just an array with counter). every object (mutable?) store thread mark (onle 4 byte, i think) every access to an object whould check - mutable is it? only if yes, 'private' is it? and only for 'mutable public/alien' object we are locking. There would no more than 20% of perfomance overhead, i think. And +50% advantage in ordinary multithreated programm on dual processor box. (Maybe +90% on 3 processor, +110 % on 4 processor, since write block will lock all readers). From falcon at intercable.ru Thu Sep 15 09:47:38 2005 From: falcon at intercable.ru (Sokolov Yura) Date: Thu, 15 Sep 2005 11:47:38 +0400 Subject: [Python-Dev] Variant of removing GIL. Message-ID: <4329271A.7050702@intercable.ru> Corrections: Now, every thread keeps one n queue of objects to incref and second queue to decref. Both can be implemented as array, cause they will be freed at once. Initially, every thread acquires GIL for "read". Py_INCREF places a ref into a incref queue of a thread, Py_DECREF places a ref into a decref queue of a thread. When queue has became full or "100" instruction left ( :-) , it usefull), thread releases GIL for "read" and acquires for "write", when first process acquire it he: walk throw all incref queues in all threads, incref all thouse refs, and clear queues. then walk throw all decref queues in all threads, decref all thouse refs, and clear queues After all he acquires GIL for "read". Other processes could stupidly repeat it founding clear queues. Since there only one thread works as a "garbage collector", we will not loose any incref and decref. From ndbecker2 at gmail.com Thu Sep 15 14:01:51 2005 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 15 Sep 2005 08:01:51 -0400 Subject: [Python-Dev] python optimization Message-ID: I use cpython. I'm accustomed (from c++/gcc) to a style of coding that is highly readable, making the assumption that the compiler will do good things to optimize the code despite the style in which it's written. For example, I assume constants are removed from loops. In general, an entity is defined as close to the point of usage as possible. I don't know to what extent these kind of optimizations are available to cpython. For example, are constant calculations removed from loops? How about functions? Is there a significant cost to putting a function def inside a loop rather than outside? From aahz at pythoncraft.com Thu Sep 15 15:28:24 2005 From: aahz at pythoncraft.com (Aahz) Date: Thu, 15 Sep 2005 06:28:24 -0700 Subject: [Python-Dev] python optimization In-Reply-To: References: Message-ID: <20050915132824.GA10916@panix.com> On Thu, Sep 15, 2005, Neal Becker wrote: > > I use cpython. I'm accustomed (from c++/gcc) to a style of coding > that is highly readable, making the assumption that the compiler will > do good things to optimize the code despite the style in which it's > written. For example, I assume constants are removed from loops. > In general, an entity is defined as close to the point of usage as > possible. > > I don't know to what extent these kind of optimizations are available > to cpython. For example, are constant calculations removed from > loops? How about functions? Is there a significant cost to putting a > function def inside a loop rather than outside? This question is about using Python, not improving/fixing Python; please use comp.lang.python (python-list) for these kinds of questions and do not cc python-dev. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything. From trentm at ActiveState.com Thu Sep 15 17:38:40 2005 From: trentm at ActiveState.com (Trent Mick) Date: Thu, 15 Sep 2005 08:38:40 -0700 Subject: [Python-Dev] os.path.diff(path1, path2) In-Reply-To: <4328B623.3070905@canterbury.ac.nz> References: <20050913031157.46635.qmail@web51809.mail.yahoo.com> <4328B623.3070905@canterbury.ac.nz> Message-ID: <20050915153840.GA26674@ActiveState.com> [Greg Ewing wrote] > Nathan Bullock wrote: > > I find that I quite often want a > > function that will give me a relative path from path A > > to path B. I have created such a function, but it > > would be nice if it was in the standard library. > > +1 from me. It's a fairly common thing to want to do. If this *does* get added (I'm +0) then let's call it "relpath" or "relpathto" as in the various implementations out there: http://www.jorendorff.com/articles/python/path/ http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/302594 http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/208993 Trent -- Trent Mick TrentM at ActiveState.com From barry at python.org Thu Sep 15 19:05:38 2005 From: barry at python.org (Barry Warsaw) Date: Thu, 15 Sep 2005 13:05:38 -0400 Subject: [Python-Dev] python-checkins reply-to Message-ID: <1126803937.21459.11.camel@geddy.wooz.org> Raymond requested, and Guido concurred, that the python-checkins mailing list Reply-To header munging be turned off. I've now done this. Previously, follow-ups to checkins would be addressed to python-dev. Now, follow-ups will stay on the python-checkins list and will not appear here, unless the poster explicitly redirects it. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050915/2406e35f/attachment.pgp From evenprimes at gmail.com Thu Sep 15 14:56:24 2005 From: evenprimes at gmail.com (Chris Cioffi) Date: Thu, 15 Sep 2005 08:56:24 -0400 Subject: [Python-Dev] python optimization In-Reply-To: References: Message-ID: Hi Neal, I don't believe that cpython currently does any of the optimizations you refer to below. That said, it is very reasonable to adopt "a style of coding that is highly readable, making the assumption that the compiler will do good things" when coding in Python. Python is one of the most highly optimised languages in the world along the Programmer Productivity metric. Line for line, you can pack more readable, obvious, and maintainable meaning into Python than pretty much any other language. The upshot is that then you can profile the final running code and see if it really matters that the compiler is using an extra .034 microseconds. That's my $0.028 US (damn inflation!) On 15/09/05, Neal Becker wrote: > > I use cpython. I'm accustomed (from c++/gcc) to a style of coding that is > highly readable, making the assumption that the compiler will do good > things to optimize the code despite the style in which it's written. For > example, I assume constants are removed from loops. In general, an entity > is defined as close to the point of usage as possible. > > I don't know to what extent these kind of optimizations are available to > cpython. For example, are constant calculations removed from loops? How > about functions? Is there a significant cost to putting a function def > inside a loop rather than outside? > > -- > http://mail.python.org/mailman/listinfo/python-list > -- "A little government and a little luck are necessary in life, but only a fool trusts either of them." -- P. J. O'Rourke -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20050915/a79ca53b/attachment.html From bcannon at gmail.com Thu Sep 15 20:13:02 2005 From: bcannon at gmail.com (Brett Cannon) Date: Thu, 15 Sep 2005 11:13:02 -0700 Subject: [Python-Dev] python optimization In-Reply-To: References: Message-ID: On 9/15/05, Neal Becker wrote: > I use cpython. I'm accustomed (from c++/gcc) to a style of coding that is > highly readable, making the assumption that the compiler will do good > things to optimize the code despite the style in which it's written. For > example, I assume constants are removed from loops. In general, an entity > is defined as close to the point of usage as possible. > > I don't know to what extent these kind of optimizations are available to > cpython. For example, are constant calculations removed from loops? If you mean ``2+3``, then yes. If you mean ``2 + a`` where is a loop invariant, then no. > How > about functions? No optimization there. > Is there a significant cost to putting a function def > inside a loop rather than outside? > If you put it in a loop then the function must be reconstructed every time through the loop. I have never benchmarked the cost of function construction but I doubt it's cheap. The problem with all of these optimizations that you can do in more static languages is you get to assume most things do not change from underneath you. In Python, thanks to threading, access to frames, global namespaces of modules, etc., most things cannot be naively optimized without a lot of checking up front to make sure the optimization is valid every time it is run. If you want some gruesome detail, you can read my thesis (http://www.drifty.org/thesis.pdf) and the section on problems with introducing type inference into Python. -Brett From Rich.Burridge at Sun.COM Thu Sep 15 22:53:36 2005 From: Rich.Burridge at Sun.COM (Rich Burridge) Date: Thu, 15 Sep 2005 13:53:36 -0700 Subject: [Python-Dev] Compatibility between Python 2.3.x and Python 2.4.x Message-ID: <4329DF50.7040805@sun.com> Hi, I'm involved with the team that's working towards installing Python 2.4.x as part of a future release of the Solaris O/S. We currently have Python 2.3.x installed. We are trying to determine just how compatible these two release are (at both the binary and source levels). In other words, with Python 2.3.x .py, .pyc and .pyo files work correctly with Python 2.4.x ? Can somebody point me at this information please? Thanks. -- Rich Burridge Email: rich.burridge at Sun.COM Sun Microsystems Inc. (MPK14-260), Phone: +1.650.786.5188 4150 Network Circle, AIM/YAHOO: RicBurridge Santa Clara, CA 95054 Blog: http://blogs.sun.com/richb From fredrik at pythonware.com Thu Sep 15 23:36:42 2005 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 15 Sep 2005 23:36:42 +0200 Subject: [Python-Dev] Compatibility between Python 2.3.x and Python 2.4.x References: <4329DF50.7040805@sun.com> Message-ID: Rich Burridge wrote: > I'm involved with the team that's working towards installing Python 2.4.x > as part of a future release of the Solaris O/S. > > We currently have Python 2.3.x installed. We are trying to determine just > how compatible these two release are (at both the binary and source levels). > > In other words, with Python 2.3.x .py, .pyc and .pyo files work correctly > with Python 2.4.x ? no. most PY files may work (see the changes documents for things that have changed From nnorwitz at gmail.com Thu Sep 15 23:42:54 2005 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 15 Sep 2005 14:42:54 -0700 Subject: [Python-Dev] Compatibility between Python 2.3.x and Python 2.4.x In-Reply-To: <4329DF50.7040805@sun.com> References: <4329DF50.7040805@sun.com> Message-ID: On 9/15/05, Rich Burridge wrote: > > Hi, > > I'm involved with the team that's working towards installing Python 2.4.x > as part of a future release of the Solaris O/S. Cool. I would like to hear more about what you are doing. > We currently have Python 2.3.x installed. We are trying to determine just > how compatible these two release are (at both the binary and source levels). > > In other words, with Python 2.3.x .py, .pyc and .pyo files work correctly > with Python 2.4.x ? The binary versions (.pyc and .pyo) are not compatible, their magic number is different. I don't know the details, but if you want to investigate yourself. Get a CVS version, cd python/dist/src ; cvs diff -r r23 -r r24 Python/import.c That shows the magic number changing. You will need to read CVS logs to figure out why. It's possible the numbers don't really *need* to change. I see two comments in the logs are: * Bump the magic number to avoid sharing bytecode between 2.3 and 2.4. Revise the long comment that explained details of the magic number in gory detail. * With Guido's okay, bumped up the magic number so that this patch gets widely exercised before the alpha goes out. There are other patches which may have bumped the magic number for other reasons. > Can somebody point me at this information please? BTW, this list probably isn't the best place to post this question. comp.lang.python (python-list at python.org) is generally preferred for details about implementation. If you are really interested in finding out more about Python, perhaps you can come to the BayPIGGIES (Python Users Group) meeting at Google tonight (7.30pm) in Mt. View. Contact me off-list if you want more info. HTH, n From fredrik at pythonware.com Thu Sep 15 23:44:34 2005 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 15 Sep 2005 23:44:34 +0200 Subject: [Python-Dev] Compatibility between Python 2.3.x and Python 2.4.x References: <4329DF50.7040805@sun.com> Message-ID: (oops. trying again) Rich Burridge wrote: > I'm involved with the team that's working towards installing Python 2.4.x > as part of a future release of the Solaris O/S. > > We currently have Python 2.3.x installed. We are trying to determine just > how compatible these two release are (at both the binary and source levels). > > In other words, with Python 2.3.x .py, .pyc and .pyo files work correctly > with Python 2.4.x ? short answer: no longer answer: most PY files work. the exception is code that (accidentally or intentionally) depends on implementation artifacts, undocumented behaviour or bugs (see the changes documents for things that may affect your applications). PYC and PYO files contain bytecode, and are not guaranteed to be portable between different 2.X releases. to check the bytecode revision used by a given Python interpreter, you can use imp.get_magic(): Python 2.3.5 >>> import imp >>> imp.get_magic() ';\xf2\r\n' Python 2.4.1 >>> import imp >>> imp.get_magic() 'm\xf2\r\n' From raymond.hettinger at verizon.net Thu Sep 15 23:44:28 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Thu, 15 Sep 2005 17:44:28 -0400 Subject: [Python-Dev] Compatibility between Python 2.3.x and Python 2.4.x In-Reply-To: Message-ID: <003901c5ba3e$a69ef0a0$6916c797@oemcomputer> [Neal Norwitz] > That shows the magic number changing. You will need to read CVS logs > to figure out why. It's possible the numbers don't really *need* to > change. The new LIST_APPEND opcode won't run on Py2.3. Raymond From pje at telecommunity.com Fri Sep 16 00:09:36 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 15 Sep 2005 18:09:36 -0400 Subject: [Python-Dev] Compatibility between Python 2.3.x and Python 2.4.x In-Reply-To: <003901c5ba3e$a69ef0a0$6916c797@oemcomputer> References: Message-ID: <5.1.1.6.0.20050915180808.01f37500@mail.telecommunity.com> At 05:44 PM 9/15/2005 -0400, Raymond Hettinger wrote: >[Neal Norwitz] > > That shows the magic number changing. You will need to read CVS logs > > to figure out why. It's possible the numbers don't really *need* to > > change. > >The new LIST_APPEND opcode won't run on Py2.3. The OP asked about upgrading existing 2.3 .pyc/.pyo files to 2.4, not downgrading to 2.3, so this doesn't affect them, except insofar as it means the magic number is different. From Rich.Burridge at Sun.COM Fri Sep 16 00:08:33 2005 From: Rich.Burridge at Sun.COM (Rich Burridge) Date: Thu, 15 Sep 2005 15:08:33 -0700 Subject: [Python-Dev] Compatibility between Python 2.3.x and Python 2.4.x In-Reply-To: References: <4329DF50.7040805@sun.com> Message-ID: <4329F0E1.5010501@sun.com> Hi Neal, Ray, Neal wrote: > The binary versions (.pyc and .pyo) are not compatible, their magic > number is different. I don't know the details, but if you want to > investigate yourself. Get a CVS version, cd python/dist/src ; cvs > diff -r r23 -r r24 Python/import.c > > That shows the magic number changing. You will need to read CVS logs > to figure out why. It's possible the numbers don't really *need* to > change. I see two comments in the logs are: > > * Bump the magic number to avoid sharing bytecode between 2.3 and 2.4. > Revise the long comment that explained details of the magic number > in gory detail. > > * With Guido's okay, bumped up the magic number so that this patch gets > widely exercised before the alpha goes out. > > There are other patches which may have bumped the magic number for > other reasons. Thanks! >>Can somebody point me at this information please? > And I forgot to give you this link: > http://docs.python.org/whatsnew/whatsnew24.html That lists the changes. It's not clear (at least to me) whether these changes are incompatible. > BTW, this list probably isn't the best place to post this question. > comp.lang.python (python-list at python.org) is generally preferred for > details about implementation. Understood. I'll try there tomorrow. > If you are really interested in finding out more about Python, perhaps > you can come to the BayPIGGIES (Python Users Group) meeting at Google > tonight (7.30pm) in Mt. View. Contact me off-list if you want more > info. Thanks for the pointer. Unfortunately I have other plans for tonight. Raymond wrote: > The new LIST_APPEND opcode won't run on Py2.3. It was more a case of wondering whether Python 2.3.x .py, .pyo, .pyc files should "just work" with the Python 2.4.x run-time environment. We are not too concerned with 2.4.x files working with the Python 2.3.x runtime. This sort of binary incompatibly is common. From anthony at interlink.com.au Fri Sep 16 02:38:24 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri, 16 Sep 2005 10:38:24 +1000 Subject: [Python-Dev] IMPORTANT: 2.4.2c1 Wednesday, 21st September - branch will be closed for a week. Message-ID: <200509161038.26130.anthony@interlink.com.au> So we'll be cutting the 2.4.2 release candidate on Wednesday the 21st (next week). Can people please make sure they are running the test suite fully for any checkins between now and then? Also, please consider the release24-maint branch closed from 0000 UTC/GMT on the 21st - this will remain closed for all but critical brown-paper-bag bugs for a week, until we (hopefully) cut 2.4.2 final on the 28th. I'll send a reminder email about 12 hours out from the branch close. Anthony From nnorwitz at gmail.com Fri Sep 16 02:44:40 2005 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 15 Sep 2005 17:44:40 -0700 Subject: [Python-Dev] IMPORTANT: 2.4.2c1 Wednesday, 21st September - branch will be closed for a week. In-Reply-To: <200509161038.26130.anthony@interlink.com.au> References: <200509161038.26130.anthony@interlink.com.au> Message-ID: On 9/15/05, Anthony Baxter wrote: > So we'll be cutting the 2.4.2 release candidate on Wednesday the 21st > (next week). Can people please make sure they are running the test > suite fully for any checkins between now and then? Also, please > consider the release24-maint branch closed from 0000 UTC/GMT on the > 21st - this will remain closed for all but critical brown-paper-bag > bugs for a week, until we (hopefully) cut 2.4.2 final on the 28th. Has anyone run valgrind on either 2.4.x or 2.5 recently? I have not, but I'll try to if no one else has run it. n From greg.ewing at canterbury.ac.nz Fri Sep 16 04:51:58 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 16 Sep 2005 14:51:58 +1200 Subject: [Python-Dev] os.path.diff(path1, path2) In-Reply-To: <20050915153840.GA26674@ActiveState.com> References: <20050913031157.46635.qmail@web51809.mail.yahoo.com> <4328B623.3070905@canterbury.ac.nz> <20050915153840.GA26674@ActiveState.com> Message-ID: <432A334E.3070500@canterbury.ac.nz> Trent Mick wrote: > If this *does* get added (I'm +0) then let's call it "relpath" or > "relpathto" as in the various implementations out there: +1 on that, too. Preferably just "relpath". -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Fri Sep 16 05:55:41 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 16 Sep 2005 15:55:41 +1200 Subject: [Python-Dev] python optimization In-Reply-To: References: Message-ID: <432A423D.1030806@canterbury.ac.nz> Brett Cannon wrote: >>I don't know to what extent these kind of optimizations are available to >>cpython. For example, are constant calculations removed from loops? > > If you mean ``2+3``, then yes. Actually, no. Constant folding *could* be done, but it currently isn't: >>> def f(): ... return 2+3 ... >>> import dis >>> dis.dis(f) 2 0 LOAD_CONST 1 (2) 3 LOAD_CONST 2 (3) 6 BINARY_ADD 7 RETURN_VALUE 8 LOAD_CONST 0 (None) 11 RETURN_VALUE >>> -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From raymond.hettinger at verizon.net Fri Sep 16 06:26:35 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Fri, 16 Sep 2005 00:26:35 -0400 Subject: [Python-Dev] python optimization In-Reply-To: <432A423D.1030806@canterbury.ac.nz> Message-ID: <005801c5ba76$d31c9460$6916c797@oemcomputer> [Neal Becker] > >>I don't know to what extent these kind of optimizations are available to > >>cpython. For example, are constant calculations removed from loops? [Brett Cannon] > > If you mean ``2+3``, then yes. [Greg Ewing] > Actually, no. Constant folding *could* be done, but it currently isn't: > > >>> def f(): > ... return 2+3 > ... > >>> import dis > >>> dis.dis(f) > 2 0 LOAD_CONST 1 (2) > 3 LOAD_CONST 2 (3) > 6 BINARY_ADD > 7 RETURN_VALUE > 8 LOAD_CONST 0 (None) > 11 RETURN_VALUE That looks like a disassembly from the ancient and primitive Py2.3 ;-) It looks a little different in the ahead-of-its-time Py2.5 alpha: Python 2.5a0 (#46, Sep 15 2005, 00:51:34) [MSC v.1200 32 bit (Intel)] on win32 Type "copyright", "credits" or "license()" for more information. >>> def f(): return 2+3 >>> import dis >>> dis.dis(f) 2 0 LOAD_CONST 3 (5) 3 RETURN_VALUE Raymond From ndbecker2 at gmail.com Fri Sep 16 12:50:48 2005 From: ndbecker2 at gmail.com (Neal Becker) Date: Fri, 16 Sep 2005 06:50:48 -0400 Subject: [Python-Dev] python optimization References: <432A423D.1030806@canterbury.ac.nz> <005801c5ba76$d31c9460$6916c797@oemcomputer> Message-ID: One possible way to improve the situation is, that if we really believe python cannot easily support such optimizations because the code is too "dynamic", is to allow manual annotation of functions. For example, gcc has allowed such annotations using __attribute__ for quite a while. This would allow the programmer to specify that a variable is constant, or that a function is pure (having no side effects). From adurdin at gmail.com Fri Sep 16 14:25:00 2005 From: adurdin at gmail.com (Andrew Durdin) Date: Fri, 16 Sep 2005 22:25:00 +1000 Subject: [Python-Dev] C coding experiment In-Reply-To: <004b01c5aef7$96d936a0$4320c797@oemcomputer> References: <004b01c5aef7$96d936a0$4320c797@oemcomputer> Message-ID: <59e9fd3a050916052529e8b60a@mail.gmail.com> On 9/1/05, Raymond Hettinger wrote: > The goal is to determine whether the setobject.c implementation would be > improved by recoding the set_lookkey() function to optimize key > insertion order using Brent's variation of Algorithm D (See Knuth vol. > III, section 6.4, page 525). Brent's variation depends on the next probe position for a key being derivable just from the key and its current position. The use of perturbation in set_lookkey() prevents that, as we cannot say, given a key at a certain position, where the next probe location for that key would have been, without doing a complete lookup of that key (obviously too expensive). It would be interesting to see whether Brent's variation or perturbation give better expected overall performance -- the latter may well prove better in most cases, as it should reduce the number of probes needed for both successful and unsuccessful lookups, while Brent's variation only speeds up successful lookups. Well, this sort of question is what the experiment is meant to resolve, no? From solipsis at pitrou.net Fri Sep 16 14:36:55 2005 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 16 Sep 2005 14:36:55 +0200 Subject: [Python-Dev] C coding experiment In-Reply-To: <59e9fd3a050916052529e8b60a@mail.gmail.com> References: <004b01c5aef7$96d936a0$4320c797@oemcomputer> <59e9fd3a050916052529e8b60a@mail.gmail.com> Message-ID: <1126874215.27418.25.camel@p-dvsi-418-1.rd.francetelecom.fr> Hi, > Brent's variation depends on the next probe position for a key being > derivable just from the key and its current position. The use of > perturbation in set_lookkey() prevents that, as we cannot say, given a > key at a certain position, where the next probe location for that key > would have been, without doing a complete lookup of that key > (obviously too expensive). I've been experimenting on this subject with Raymond H. You will find some code here: http://pitrou.net/python/sets/ - hash1.py is an algorithmic simulation of the current set implementation; it will give you statistics about e.g. the number of table walks in various test cases (that you can customize, of course) - hash2.py is a very crude benchmark of the set implementation - brent-patch.txt is a patch against CPython CVS to add Brent's variation to the set implementation; it made hash2.py between 5% slower and 2% faster depending on the test case I believe Raymond has since started experimenting on other approaches. Regards Antoine. From ncoghlan at gmail.com Fri Sep 16 16:09:39 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 17 Sep 2005 00:09:39 +1000 Subject: [Python-Dev] python optimization In-Reply-To: References: <432A423D.1030806@canterbury.ac.nz> <005801c5ba76$d31c9460$6916c797@oemcomputer> Message-ID: <432AD223.9020902@gmail.com> Neal Becker wrote: > One possible way to improve the situation is, that if we really believe > python cannot easily support such optimizations because the code is too > "dynamic", is to allow manual annotation of functions. For example, gcc > has allowed such annotations using __attribute__ for quite a while. This > would allow the programmer to specify that a variable is constant, or that > a function is pure (having no side effects). Raymond's constant binding decorator comes pretty close to achieving that: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/277940 It kicks in when the def statement is executed, rather than when the module is compiled, but that difference is not uncommon in Python (you don't know what half the names refer to until the top-level of the module is executed). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From raymond.hettinger at verizon.net Fri Sep 16 18:58:08 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Fri, 16 Sep 2005 12:58:08 -0400 Subject: [Python-Dev] C coding experiment In-Reply-To: <59e9fd3a050916052529e8b60a@mail.gmail.com> Message-ID: <002f01c5badf$d2cdc060$212ac797@oemcomputer> > -----Original Message----- > From: Andrew Durdin [mailto:adurdin at gmail.com] > Sent: Friday, September 16, 2005 8:25 AM > To: python at rcn.com > Cc: python-dev at python.org > Subject: Re: [Python-Dev] C coding experiment > > On 9/1/05, Raymond Hettinger wrote: > > The goal is to determine whether the setobject.c implementation would be > > improved by recoding the set_lookkey() function to optimize key > > insertion order using Brent's variation of Algorithm D (See Knuth vol. > > III, section 6.4, page 525). > > Brent's variation depends on the next probe position for a key being > derivable just from the key and its current position. The use of > perturbation in set_lookkey() prevents that, as we cannot say, given a > key at a certain position, where the next probe location for that key > would have been, without doing a complete lookup of that key > (obviously too expensive). Right. For Brent's variation to work, I've had to replace perturbation with a secondary hash function with linear spacing. > It would be interesting to see whether Brent's variation or > perturbation give better expected overall performance -- the latter > may well prove better in most cases, as it should reduce the number of > probes needed for both successful and unsuccessful lookups, while > Brent's variation only speeds up successful lookups. Well, this sort > of question is what the experiment is meant to resolve, no? Right again. I'm also experimenting with a simpler approach -- whenever there are more than three probes, always swap the new key into the first position and then unconditionally re-insert the swapped-out key. Most of the time that gives an improvement and it doesn't require changing the perturbation logic -- it is equivalent to changing the key insertion order. The only implementation wrinkle is that the approach needs separate lookup and insert functions (only the latter does the swap). The simpler approach is cheap to implement as it doesn't slow-down the first three probes. OTOH, te benefits are smaller than with Brent's variation -- it only relieves the worst collisions. I'm still interested in it because it helps the two most important use cases for sets (uniquification and membership testing). Raymond From martin at v.loewis.de Fri Sep 16 19:22:50 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 16 Sep 2005 19:22:50 +0200 Subject: [Python-Dev] Compatibility between Python 2.3.x and Python 2.4.x In-Reply-To: <4329F0E1.5010501@sun.com> References: <4329DF50.7040805@sun.com> <4329F0E1.5010501@sun.com> Message-ID: <432AFF6A.1080306@v.loewis.de> Rich Burridge wrote: >>And I forgot to give you this link: >>http://docs.python.org/whatsnew/whatsnew24.html > > > That lists the changes. It's not clear (at least to me) whether > these changes are incompatible. The section http://docs.python.org/whatsnew/node15.html is meant to cover incompatible changes. It shows that there is no 100% compatibility, but that only "borderline" code is affected. Probably the most significant change is that 0xC0000000 is now a positive number, when it used to be negative on 32-bit systems. Regards, Martin From ncoghlan at iinet.net.au Fri Sep 16 19:29:14 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sat, 17 Sep 2005 03:29:14 +1000 Subject: [Python-Dev] Python 2.5a1, ast-branch and PEP 342 and 343 Message-ID: <432B00EA.2070101@iinet.net.au> I think this is mainly a question for Guido - where do we stand with respect to the ast-branch for Python 2.5? If we're targetting a March release for 2.5a1, then we probably need to land implementations for PEP 342 and PEP 343 at least a couple of months before that, and the current patches on Sourceforge are against HEAD, not ast-branch (for obvious reasons). So, not only does ast-branch need to be brought up to Python 2.4 compliance and checked for any memory leaks when compilation attempts fail (I suspect there are a few of those), but additional work is needed to implement PEP 342 and PEP 343 as well. For Python 2.4, Guido gave a target date well before the first alpha release for landing the ast-branch if it was going to be included. Is the same thing going to happen for Python 2.5 (setting the target date, that is - not the fact that the ast-branch didn't get included!) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From martin at v.loewis.de Fri Sep 16 23:10:18 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 16 Sep 2005 23:10:18 +0200 Subject: [Python-Dev] Variant of removing GIL. In-Reply-To: <43292071.8050604@intercable.ru> References: <43292071.8050604@intercable.ru> Message-ID: <432B34BA.1020202@v.loewis.de> Sokolov Yura wrote: > I think I know how to remove GIL!!!! Obviously I am an idiot. Not an idiot, just lazy :-) Please try to implement your ideas, and I predict that you will find: 1. it is a lot of work to implement 2. it requires changes to all C files, in particular to extension modules outside the Python source tree proper. 3. performing the conversion, even in a semi-mechanical way, will introduce many new bugs, in the form of race conditions because of missing locks. Optionally, you may also find that the performance of the interpreter will decrease. I haven't really tried to completely understand your proposal, but you are right, in principle, that a global lock can be replaced with more fine-grained locks. However, this is really hard to do correctly - if it were simple, it would have been done long ago. Regards, Martin From skip at pobox.com Sat Sep 17 04:07:23 2005 From: skip at pobox.com (skip@pobox.com) Date: Fri, 16 Sep 2005 21:07:23 -0500 Subject: [Python-Dev] Variant of removing GIL. In-Reply-To: <432B34BA.1020202@v.loewis.de> References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> Message-ID: <17195.31323.202978.650080@montanaro.dyndns.org> Martin> However, this is really hard to do correctly - if it were Martin> simple, it would have been done long ago. I don't believe difficulty is the only (or primary) barrier. I think *someone* would have tackled it since Greg Stein did back in 1.4(?) or his free-threading changes would have been incorporated into the core had they yielded speedups on multiprocessors and not hurt performance on uniprocessors. Skip From lcaamano at gmail.com Sat Sep 17 18:32:11 2005 From: lcaamano at gmail.com (Luis P Caamano) Date: Sat, 17 Sep 2005 12:32:11 -0400 Subject: [Python-Dev] Variant of removing GIL. In-Reply-To: References: Message-ID: On 9/17/05, python-dev-request at python.org wrote: > > Message: 9 > Date: Fri, 16 Sep 2005 21:07:23 -0500 > From: skip at pobox.com > Subject: Re: [Python-Dev] Variant of removing GIL. > Message-ID: <17195.31323.202978.650080 at montanaro.dyndns.org> > > > Martin> However, this is really hard to do correctly - if it were > Martin> simple, it would have been done long ago. > > I don't believe difficulty is the only (or primary) barrier. I think > *someone* would have tackled it since Greg Stein did back in 1.4(?) or his > free-threading changes would have been incorporated into the core had they > yielded speedups on multiprocessors and not hurt performance on > uniprocessors. > > Skip > It did yield speedups on multiprocessors. The uniprocessor part could've been solved just like most kernels do, one binary for UP and another for MP. That's what IBM, RedHat, Solaris, and almost all other modern kernels that support SMP machines do. In theory, if we had those changes in the CPython interpreter, we could've been running at 1.6 times the speed on dual processor machines today (according to Greg's benchmark data) and at the same speed on UP machines running the UP compiled CPython interpreter, which would not have had all the locking calls not needed on a UP machine that would hurt its performance. By now, we probably could've improved on the scalability of MP performance when running on machines with more than three processors. Mind you though, I'm not trying to oversimplify the issue. I was not using python yet at that time (I started around 1.5/1.6) and I didn't see all the info involved in the decision making process, so I'm sure there were other issues that contributed to the decision of not keeping Greg's free threading changes. My point is that not yielding speedups on multiprocessors and hurting performance on uniprocessors is not a good or valid reason to drop free-threading. -- Luis P Caamano Atlanta, GA USA From pje at telecommunity.com Sat Sep 17 18:44:09 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 17 Sep 2005 12:44:09 -0400 Subject: [Python-Dev] Variant of removing GIL. In-Reply-To: References: Message-ID: <5.1.1.6.0.20050917123928.01b7ee60@mail.telecommunity.com> At 12:32 PM 9/17/2005 -0400, Luis P Caamano wrote: >My point is that not yielding speedups on multiprocessors >and hurting performance on uniprocessors is not a good >or valid reason to drop free-threading. It is if you have only volunteers to maintain the code base, and the changes significantly increase maintenance complexity. Also, a significant number of third-party C extensions would need to be modified for compatibility, as has already been pointed out. Note also that Jython and IronPython exist, and run on VMs that address these issues, and that the PyPy project can generate code for many kinds of backends. There's nothting stopping anybody from creating a multiprocessor-friendly backend for PyPy, for example. From martin at v.loewis.de Sat Sep 17 18:57:05 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 17 Sep 2005 18:57:05 +0200 Subject: [Python-Dev] Variant of removing GIL. In-Reply-To: References: Message-ID: <432C4AE1.8040305@v.loewis.de> Luis P Caamano wrote: > Mind you though, I'm not trying to oversimplify the issue. > I was not using python yet at that time (I started around > 1.5/1.6) and I didn't see all the info involved in the decision > making process, so I'm sure there were other issues that > contributed to the decision of not keeping Greg's free > threading changes. For historical correctness, I believe there never was a decision to "not keep Greg's free threading changes". I believe Greg never actually contributed them (at least not in a publically-visible manner). This, in turn appears to be the result of the problem that nobody (including Greg) was able to tell whether the patches are actually correct (for extension modules, it appears there was agreement that the patches are *not* correct). (more correctly, it appears that some of the code made it to Python 1.5) Instead, the issue mainly died because nobody provided working code (along with a strategy on what to do with the existing extension modules). Regards, Martin From lcaamano at gmail.com Sat Sep 17 19:10:26 2005 From: lcaamano at gmail.com (Luis P Caamano) Date: Sat, 17 Sep 2005 13:10:26 -0400 Subject: [Python-Dev] Variant of removing GIL. In-Reply-To: <5.1.1.6.0.20050917123928.01b7ee60@mail.telecommunity.com> References: <5.1.1.6.0.20050917123928.01b7ee60@mail.telecommunity.com> Message-ID: On 9/17/05, Phillip J. Eby wrote: > At 12:32 PM 9/17/2005 -0400, Luis P Caamano wrote: > >My point is that not yielding speedups on multiprocessors > >and hurting performance on uniprocessors is not a good > >or valid reason to drop free-threading. > > It is No, it's not because it's not true. > if you have only volunteers to maintain the code base, and the > changes significantly increase maintenance complexity. This is one very valid reason. > Also, a significant > number of third-party C extensions would need to be modified for > compatibility, as has already been pointed out. And another. > > Note also that Jython and IronPython exist, and run on VMs that address > these issues, and that the PyPy project can generate code for many kinds of > backends. There's nothting stopping anybody from creating a > multiprocessor-friendly backend for PyPy, for example. Yes, eventually when we have CPython running on more SMP or equivalent machines than UP machines, the majority will have an itch and it will get scratched. -- Luis P Caamano Atlanta, GA USA From jjl at pobox.com Sat Sep 17 19:18:34 2005 From: jjl at pobox.com (John J Lee) Date: Sat, 17 Sep 2005 17:18:34 +0000 (UTC) Subject: [Python-Dev] os.path.diff(path1, path2) In-Reply-To: <432A334E.3070500@canterbury.ac.nz> References: <20050913031157.46635.qmail@web51809.mail.yahoo.com> <4328B623.3070905@canterbury.ac.nz> <20050915153840.GA26674@ActiveState.com> <432A334E.3070500@canterbury.ac.nz> Message-ID: On Fri, 16 Sep 2005, Greg Ewing wrote: > Trent Mick wrote: > > > If this *does* get added (I'm +0) then let's call it "relpath" or > > "relpathto" as in the various implementations out there: > > +1 on that, too. Preferably just "relpath". [...] +1 on adding this function, and on "relpath" as the name. I wanted this function just a few weeks ago. John From lcaamano at gmail.com Sat Sep 17 19:51:29 2005 From: lcaamano at gmail.com (Luis P Caamano) Date: Sat, 17 Sep 2005 13:51:29 -0400 Subject: [Python-Dev] Variant of removing GIL. In-Reply-To: <432C4AE1.8040305@v.loewis.de> References: <432C4AE1.8040305@v.loewis.de> Message-ID: On 9/17/05, "Martin v. L?wis" wrote: > > > Instead, the issue mainly died because nobody provided > working code (along with a strategy on what to do with > the existing extension modules). > When I first started writing python code I had just come out of about 6 years of kernel development (1994-2000). These 6 years span the times when we were making changes to UP kernels to work correctly and efficiently on SMP machines, implementing kernel threads, and thread-safing libc and other changes needed to move from the now defunct user space DCE-threads to posix threads including kernel threads and even MxN support. So, I was very familiar with thread programming. I architected, designed, and developed a bunch of distributed servers and services using python and Pyro and it all was done using nice and cool thread programming practices and techniques. It was nice to see Python supported threads. Close to a year into this when we had a lot of functionality implemented I started running scalability tests and I was shocked and appalled to find out that our architecture did not scale well on SMP machines because we depended on threads. And that's when I discovered the GIL and twisted and other process based python techniques used to get around the GIL. It was too late for us to redo things and we've been in a holding pattern since then waiting to see what will hit us first, the need to scale a lot, which means we'd have to drop some time to implement some changes to support process based scalability, or no GIL. I have my money on process based scalability. :( I'm sure that has happened to a lot of people because nobody finds about the GIL in the beginning of their python development experience. If I started writing a new complex app from scratch, now I know exactly what to do. One big problem is that a lot of people defend the GIL with the premise that the GIL is OK because we release the GIL in C extensions and during IO. That is true, the GIL is not as bad as it would've been if the CPython interpreter extensions didn't do that but it's also true that it hurts performance on SMP machines in more than one way. When I found out about this it dawn on me that the performance could've been worse although it was still way below what we expected. I don't remember exactly how but what I remember is that when I presented the problem to Guido he just told me to put up or shut up. At first I was insulted but then I realized that it was a fair response given the complexity of the problem, previous history I didn't know about, the fairly good compromise of releasing the GIL in extensions, and the lack of a large percentage of python developers asking for better performance on SMP machines. I couldn't put up so ... I think I've gone beyond my quota :-) PS The GIL is probably my only peeve about the CPython interpreter. So don't get me wrong, I love the language and I'm always grateful for all the hard work you guys put in developing such great language and implementation. -- Luis P Caamano Atlanta, GA USA From jjl at pobox.com Sat Sep 17 19:48:21 2005 From: jjl at pobox.com (John J Lee) Date: Sat, 17 Sep 2005 17:48:21 +0000 (UTC) Subject: [Python-Dev] GIL, Python 3, and MP vs. UP (was Re: Variant of removing GIL.) In-Reply-To: <432B34BA.1020202@v.loewis.de> References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> Message-ID: On Fri, 16 Sep 2005, "Martin v. L?wis" wrote: > Sokolov Yura wrote: > > I think I know how to remove GIL!!!! Obviously I am an idiot. > > Not an idiot, just lazy :-) Please try to implement your ideas, > and I predict that you will find: > 1. it is a lot of work to implement > 2. it requires changes to all C files, in particular to extension > modules outside the Python source tree proper. > 3. performing the conversion, even in a semi-mechanical way, will > introduce many new bugs, in the form of race conditions because > of missing locks. > > Optionally, you may also find that the performance of the > interpreter will decrease. [...] Given the points you make, and the facts that both Python 3 and real problems with continuing to scale down semiconductor chip feature sizes are on the horizon, it seems that now would be an excellent time to start work on this, with the goal of introducing it at the same time as Python 3. a. Python 3.0 will break lots of code anyway, so the extension module issue becomes far less significant. b. In x years time (x < 10?) it seems likely multiprocessor (MP) users will be in the majority. (As a result, the uniprocessor (UP) slowdown becomes less important in practice, and also Python has the opportunity of avoiding the risk of being sidelined by a real or perceived lack of MP performance.) c. Since time is needed to iron out bugs (and perhaps also to reimplememt some pieces of code "from scratch"), very early in the life of Python 3 seems like the least-worst time to begin work on such a change. I realize that not all algorithms (nor all computational problems) scale well to MP hardware. Is it feasible to usefully compile both MP and a UP binaries from one Python source code base? (I'm also quite aware that the GIL does not prevent all means of achieving efficient use of multiprocessors. I'm just concious that different parellisation problems are presumably best expressed using different tools, and that Python 3 and increased prevalance of MP systems might tip the balance in favour of removing the GIL.) Of course, it still takes a (anti-)hero to step forward and do the work... John From bcannon at gmail.com Sun Sep 18 03:20:08 2005 From: bcannon at gmail.com (Brett Cannon) Date: Sat, 17 Sep 2005 18:20:08 -0700 Subject: [Python-Dev] removing nested tuple function parameters Message-ID: Is anyone truly attached to nested tuple function parameters; ``def fxn((a,b)): print a,b``? At one of the PyCon sprints Guido seemed okay with just having them removed when Jeremy asked about ditching them thanks to the pain they caused in the AST branch. I personally don't see them being overly useful thanks to assignment unpacking. Plus I don't think they are used very much (gut feeling, though, and not based on any grepping). Would anyone really throw a huge fit if they went away? I am willing to write a PEP for their removal in 2.6 with a deprecation in 2.5 if people are up for it. Otherwise I say they should definitely go in Python 3. -Brett From andrew-pythondev at puzzling.org Sun Sep 18 04:11:44 2005 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Sun, 18 Sep 2005 12:11:44 +1000 Subject: [Python-Dev] removing nested tuple function parameters In-Reply-To: References: Message-ID: <20050918021144.GA7789@trogdor.home.puzzling.org> On Sat, Sep 17, 2005 at 06:20:08PM -0700, Brett Cannon wrote: > Is anyone truly attached to nested tuple function parameters; ``def > fxn((a,b)): print a,b``? At one of the PyCon sprints Guido seemed > okay with just having them removed when Jeremy asked about ditching > them thanks to the pain they caused in the AST branch. I personally > don't see them being overly useful thanks to assignment unpacking. > Plus I don't think they are used very much (gut feeling, though, and > not based on any grepping). > > Would anyone really throw a huge fit if they went away? I am willing > to write a PEP for their removal in 2.6 with a deprecation in 2.5 if > people are up for it. Otherwise I say they should definitely go in > Python 3. Please keep them! Twisted code uses them in places for Deferred callbacks that need to deal with multiple return values. For instance, the twisted.cred.portal.Portal's login method returns Deferred that will be called with a tuple of (interface that the avatar implements, the avatar, a 0-arg logout callable), which naturally leads to functions like: def _cbAuthenticated(self, (iface, avatar, logout)): ... Based on a quick grep, there's well over 80 uses of tuple-unpacking in function declarations in Twisted's code base. There's almost certainly many more outside of Twisted; I know I have some code at work that uses this. So the short answer is: yes, we are attached to it, we will miss it. No guarantees about throwing fits, though . -Andrew. From steven.bethard at gmail.com Sun Sep 18 04:15:09 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Sat, 17 Sep 2005 20:15:09 -0600 Subject: [Python-Dev] removing nested tuple function parameters In-Reply-To: References: Message-ID: Brett Cannon wrote: > Is anyone truly attached to nested tuple function parameters; ``def > fxn((a,b)): print a,b``? I find 54 instances in my Python installation. >grep -r "def.*([^=]*([^)]*,[^)]*).*):" * aifc.py: def setparams(self, (nchannels, sampwidth, framerate, nframes, comptype, compname)): bdb.py: def user_exception(self, frame, (exc_type, exc_value, exc_traceback)): binhex.py: def __init__(self, (name, finfo, dlen, rlen), ofp): cgitb.py:def html((etype, evalue, etb), context=5): cgitb.py:def text((etype, evalue, etb), context=5): compiler/ast.py: def __init__(self, (left, right), lineno=None): compiler/ast.py: def __init__(self, (left, right), lineno=None): compiler/ast.py: def __init__(self, (left, right), lineno=None): compiler/ast.py: def __init__(self, (left, right), lineno=None): compiler/ast.py: def __init__(self, (left, right), lineno=None): compiler/ast.py: def __init__(self, (left, right), lineno=None): compiler/ast.py: def __init__(self, (left, right), lineno=None): compiler/ast.py: def __init__(self, (left, right), lineno=None): compiler/ast.py: def __init__(self, (left, right), lineno=None): formatter.py: def push_font(self, (size, i, b, tt)): idlelib/Debugger.py: def show_frame(self, (frame, lineno)): imputil.py: def _process_result(self, (ispkg, code, values), fqname): inspect.py: def tokeneater(self, type, token, (srow, scol), (erow, ecol), line): lib-old/fmt.py: def text(self, (h, v), str): lib-old/fmt.py: def text(self, (h, v), text): modulefinder.py: def load_module(self, fqname, fp, pathname, (suffix, mode, type)): pdb.py: def user_exception(self, frame, (exc_type, exc_value, exc_traceback)): plat-irix5/flp.py:def create_full_form(inst, (fdata, odatalist)): plat-irix5/flp.py:def merge_full_form(inst, form, (fdata, odatalist)): plat-irix6/flp.py:def create_full_form(inst, (fdata, odatalist)): plat-irix6/flp.py:def merge_full_form(inst, form, (fdata, odatalist)): plat-mac/findertools.py:def _setlocation(object_alias, (x, y)): plat-mac/findertools.py:def _setwindowsize(folder_alias, (w, h)): plat-mac/findertools.py:def _setwindowposition(folder_alias, (x, y)): pydoc.py: def fixup((name, kind, cls, value)): pydoc.py: def __init__(self, filename, (exc, value, tb)): pydoc.py: def modpkglink(self, (name, path, ispackage, shadowed)): pydoc.py: def submodules(self, (dir, package)): pydoc.py: def isnewpackage(self, (dir, package)): sunau.py: def setparams(self, (nchannels, sampwidth, framerate, nframes, comptype, compname)): test/inspect_fodder.py:def spam(a, b, c, d=3, (e, (f,))=(4, (5,)), *g, **h): test/test_compile.py: def comp_args((a, b)): test/test_compile.py: def comp_args((a, b)=(3, 4)): test/test_compile.py: def comp_args(a, (b, c)): Binary file test/test_compile.pyc matches test/test_grammar.py:def f4(two, (compound, (argument, list))): pass test/test_grammar.py:def f5((compound, first), two): pass test/test_grammar.py:def v3(a, (b, c), *rest): return a, b, c, rest test/test_math.py:def testfrexp(name, (mant, exp), (emant, eexp)): test/test_math.py:def testmodf(name, (v1, v2), (e1, e2)): Binary file test/test_parser.pyc matches test/test_scope.py:def makeAddPair((a, b)): test/test_scope.py: def addPair((c, d)): test/test_threadsignals.py:def registerSignals((for_usr1, for_usr2, for_alrm)): threading.py: def _acquire_restore(self, (count, owner)): tokenize.py:def printtoken(type, token, (srow, scol), (erow, ecol), line): # for testing urlparse.py:def urlunparse((scheme, netloc, url, params, query, fragment)): urlparse.py:def urlunsplit((scheme, netloc, url, query, fragment)): wave.py: def setparams(self, (nchannels, sampwidth, framerate, nframes, comptype, compname)): Not to mention the dozen or so in my own codebase. STeVe -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From greg.ewing at canterbury.ac.nz Sun Sep 18 12:28:20 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 18 Sep 2005 22:28:20 +1200 Subject: [Python-Dev] removing nested tuple function parameters In-Reply-To: References: Message-ID: <432D4144.3090309@canterbury.ac.nz> Brett Cannon wrote: > Would anyone really throw a huge fit if they went away? I am willing > to write a PEP for their removal in 2.6 with a deprecation in 2.5 if > people are up for it. -1. I don't think this could realistically be done before 3.0, because it would break a lot of existing code for no good reason. I probably wouldn't mind all that much if they went away in 3.0, although they can be handy. Greg From skip at pobox.com Sun Sep 18 16:40:20 2005 From: skip at pobox.com (skip@pobox.com) Date: Sun, 18 Sep 2005 09:40:20 -0500 Subject: [Python-Dev] test__locale on Mac OS X Message-ID: <17197.31828.309708.686982@montanaro.dyndns.org> test__locale still fails for me on Mac OS X 10.3.9. This is on both the 2.4 branch and HEAD. Does it succeed for anyone on 10.4? If not, perhaps we should list that as an expected failure on that platform? Skip From fw at deneb.enyo.de Sun Sep 18 21:15:47 2005 From: fw at deneb.enyo.de (Florian Weimer) Date: Sun, 18 Sep 2005 21:15:47 +0200 Subject: [Python-Dev] removing nested tuple function parameters In-Reply-To: (Brett Cannon's message of "Sat, 17 Sep 2005 18:20:08 -0700") References: Message-ID: <87k6he6vbw.fsf@mid.deneb.enyo.de> * Brett Cannon: > Is anyone truly attached to nested tuple function parameters; ``def > fxn((a,b)): print a,b``? At one of the PyCon sprints Guido seemed > okay with just having them removed when Jeremy asked about ditching > them thanks to the pain they caused in the AST branch. Will def fxn((a,b,),): print a,b continue to work? (I'm not sure if the problems are syntax or representation of the parse tree; I don't know what's going on on that AST branch.) From bcannon at gmail.com Sun Sep 18 23:37:43 2005 From: bcannon at gmail.com (Brett Cannon) Date: Sun, 18 Sep 2005 14:37:43 -0700 Subject: [Python-Dev] test__locale on Mac OS X In-Reply-To: <17197.31828.309708.686982@montanaro.dyndns.org> References: <17197.31828.309708.686982@montanaro.dyndns.org> Message-ID: [forgot to CC python-dev] On 9/18/05, skip at pobox.com wrote: > test__locale still fails for me on Mac OS X 10.3.9. This is on both the 2.4 > branch and HEAD. Does it succeed for anyone on 10.4? If not, perhaps we > should list that as an expected failure on that platform? > It passes on 10.4 for me. -Brett From ncoghlan at gmail.com Mon Sep 19 00:22:42 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 19 Sep 2005 08:22:42 +1000 Subject: [Python-Dev] removing nested tuple function parameters In-Reply-To: References: Message-ID: <432DE8B2.4020701@gmail.com> Brett Cannon wrote: > Is anyone truly attached to nested tuple function parameters; ``def > fxn((a,b)): print a,b``? At one of the PyCon sprints Guido seemed > okay with just having them removed when Jeremy asked about ditching > them thanks to the pain they caused in the AST branch. I personally > don't see them being overly useful thanks to assignment unpacking. > Plus I don't think they are used very much (gut feeling, though, and > not based on any grepping). The fixes needed to make them work properly didn't seem all that ugly to me. Also, if they weren't used in the stdlib, they wouldn't have caused us any grief on the AST branch ;) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From raymond.hettinger at verizon.net Mon Sep 19 02:10:20 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sun, 18 Sep 2005 20:10:20 -0400 Subject: [Python-Dev] removing nested tuple function parameters In-Reply-To: <432DE8B2.4020701@gmail.com> Message-ID: <002001c5bcae$8639e640$99b1958d@oemcomputer> [Brett] > Is anyone truly attached to nested tuple function parameters; ``def > fxn((a,b)): print a,b``? I am. > ditching them thanks to the pain they caused in the AST branch. Changing the grammar for the convenience of a particular AST implementation carries zero weight with me -- that is the tail wagging the dog. Besides, I had thought that one of the goals of AST was to make it easier to experiment with language. Are you finding that it has a hard time even with the existing grammar? AFAICT, nested tuple arguments presented no problem for Jython or PyPy. > Plus I don't think they are used very much (gut feeling, though, and > not based on any grepping). python-dev grammar change proposals should probably be held to a higher standard than "gut feeling, just toss it" whims. [Nick] > The fixes needed to make them work properly didn't seem all that ugly to > me. That pretty much settles it the "do it for the convenience of AST" argument. [Andrew Bennetts] > Please keep them! Twisted code uses them in places for Deferred > callbacks that need to deal with multiple return values. And that settles the question of whether people are using them in real code. Raymond From guido at python.org Mon Sep 19 02:31:52 2005 From: guido at python.org (Guido van Rossum) Date: Sun, 18 Sep 2005 17:31:52 -0700 Subject: [Python-Dev] removing nested tuple function parameters In-Reply-To: <002001c5bcae$8639e640$99b1958d@oemcomputer> References: <432DE8B2.4020701@gmail.com> <002001c5bcae$8639e640$99b1958d@oemcomputer> Message-ID: > [Brett] > > Is anyone truly attached to nested tuple function parameters; ``def > > fxn((a,b)): print a,b``? [Raymond] > I am. I agree that we shouldn't mess with them in 2.x. Yet I think they are a candidate for being dropped from Py3K. While every feature is used by *someone* (as the feedback to Brett's query clearly shows) this one has several things against it. For every user who is fond of them there are probably ten who have never even heard of it. It's purely syntactic sugar (the only place where it's not trivial to replace is in a lambda). I've encountered quite a few people who had a hard time reading code that uses it. I personally prefer reading code that doesn't use this feature; for one thing, when this is used, you can't refer to a parameter by name. I'm not going to do the survey myself, but you could go through a representative code sample and see how it looks after doing the natural transformation. (In fact, this would be one 2.x -> 3.0 transformation that could easily be automated if you can handle making up less-than-optimal names for the arguments.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pinard at iro.umontreal.ca Mon Sep 19 02:37:40 2005 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Sun, 18 Sep 2005 20:37:40 -0400 Subject: [Python-Dev] removing nested tuple function parameters In-Reply-To: <002001c5bcae$8639e640$99b1958d@oemcomputer> References: <432DE8B2.4020701@gmail.com> <002001c5bcae$8639e640$99b1958d@oemcomputer> Message-ID: <20050919003740.GA5918@alcyon.progiciels-bpi.ca> [Raymond Hettinger] > [Brett] > > > Is anyone truly attached to nested tuple function parameters; ``def > > fxn((a,b)): print a,b``? > I am. I like the possibility and use it once in a while, but not often, at least not enough to beg for its preservation, or cry after its loss. Having to go without it, a mere tuple unpacking line near the beginning of the function would do. The only practical reason to like this feature is sparing the need of finding an otherwise useless name for the formal argument. Another reason, but not practical at all, is that the concept conveys some elegance and originality (each programming language should ideally have a few of these) and is enforced in other places in Python, like in the `for' statement -- where I find implied unpacking very, very useful. Yet, if I was given the choice between nested tuple function arguments, and faster argument processing, the latter would win instantly. > > ditching them thanks to the pain they caused in the AST branch. > Changing the grammar for the convenience of a particular AST > implementation carries zero weight with me -- that is the tail wagging > the dog. A very funny image! :-) -- Fran?ois Pinard http://pinard.progiciels-bpi.ca From bcannon at gmail.com Mon Sep 19 02:41:51 2005 From: bcannon at gmail.com (Brett Cannon) Date: Sun, 18 Sep 2005 17:41:51 -0700 Subject: [Python-Dev] removing nested tuple function parameters In-Reply-To: <002001c5bcae$8639e640$99b1958d@oemcomputer> References: <432DE8B2.4020701@gmail.com> <002001c5bcae$8639e640$99b1958d@oemcomputer> Message-ID: On 9/18/05, Raymond Hettinger wrote: > [Brett] > > Is anyone truly attached to nested tuple function parameters; ``def > > fxn((a,b)): print a,b``? > > I am. > > > > > ditching them thanks to the pain they caused in the AST branch. > > Changing the grammar for the convenience of a particular AST > implementation carries zero weight with me -- that is the tail wagging > the dog. > > Besides, I had thought that one of the goals of AST was to make it > easier to experiment with language. Are you finding that it has a hard > time even with the existing grammar? AFAICT, nested tuple arguments > presented no problem for Jython or PyPy. > I obviously didn't make the statement clear enough. The suport is already in the AST branch, it was just a statement Jeremy made once while starting to work on the implementaiton for the AST branch that I happen to still remember. They are already supported. The real trigger for this is introspection issues in terms of function paramters. Without changing the way the bytecode does things the only way to know the exact parameters is to inspect the bytecode. There are possible solutions than removal, but I personally find the use of nested tuple arguments hard to read and not widely known as Guido pointed out in another email. > > > > Plus I don't think they are used very much (gut feeling, though, and > > not based on any grepping). > > python-dev grammar change proposals should probably be held to a higher > standard than "gut feeling, just toss it" whims. > Which is why I asked first before doing a full-blown push for this. -Brett From guido at python.org Mon Sep 19 02:50:13 2005 From: guido at python.org (Guido van Rossum) Date: Sun, 18 Sep 2005 17:50:13 -0700 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP (was Re: Variant of removing GIL.) In-Reply-To: References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> Message-ID: On 9/17/05, John J Lee wrote: > Given the points you make, and the facts that both Python 3 and real > problems with continuing to scale down semiconductor chip feature sizes > are on the horizon, it seems that now would be an excellent time to start > work on this, with the goal of introducing it at the same time as Python > 3. > > a. Python 3.0 will break lots of code anyway, so the extension module > issue becomes far less significant. > > b. In x years time (x < 10?) it seems likely multiprocessor (MP) users > will be in the majority. (As a result, the uniprocessor (UP) slowdown > becomes less important in practice, and also Python has the opportunity > of avoiding the risk of being sidelined by a real or perceived lack of > MP performance.) That assumes a very specific model for how all that MP power is going to be used. I personally don't think the threaded programming model as found in Java works all that well; without locks you end up with concurrent modification errors, with locks you get deadlocks and livelocks. A typical programmer has a hard enough time keeping track of a bunch of variables being modified by a single thread; add multiple threads acting simultaneously on the same variables to the mix, and it's a nightmare. If my hunch is right, I expect that instead of writing massively parallel applications, we will continue to write single-threaded applications that are tied together at the process level rather than at the thread level. > c. Since time is needed to iron out bugs (and perhaps also to reimplememt > some pieces of code "from scratch"), very early in the life of Python 3 > seems like the least-worst time to begin work on such a change. > > I realize that not all algorithms (nor all computational problems) scale > well to MP hardware. Is it feasible to usefully compile both MP and a UP > binaries from one Python source code base? That's an understatement. I expect that *most* problems (even most problems that we will be programming 10-20 years from now) get little benefit out of MP. > (I'm also quite aware that the GIL does not prevent all means of achieving > efficient use of multiprocessors. I'm just concious that different > parellisation problems are presumably best expressed using different > tools, and that Python 3 and increased prevalance of MP systems might tip > the balance in favour of removing the GIL.) > > Of course, it still takes a (anti-)hero to step forward and do the work... Be my guest. Prove me wrong. Talk is cheap; instead of arguing my points (all of which can be argued ad infinitum), come back when you've got a working GIL-free Python. Doesn't have to be CPython-based -- C# would be fine too. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From quarl at NOSPAM.quarl.org Mon Sep 19 03:50:14 2005 From: quarl at NOSPAM.quarl.org (Karl Chen) Date: Sun, 18 Sep 2005 18:50:14 -0700 Subject: [Python-Dev] list splicing Message-ID: Hi, has anybody considered adding something like this: a = [1, 2] [ 'x', *a, 'y'] as syntactic sugar for a = [1, 2] [ 'x' ] + a + [ 'y' ]. Notes: - This is a common operation - To me, the splicing form looks much better than the concatenation form - It can be implemented more efficiently than a bunch of list concatenations - This would be similar to the "apply" feature [ foo(*a) ]. The '*' operator is reminiscent of dereferencing operators in C-derived languages. Alternatively it could be '@', like perl's implicit array splicing, and Lisp's ',@' in backquoted lists. - (I know "splicing" traditionally refers to something else in Python, but other languages refer to this as splicing, and the English definition of "splice" applies better to this.) -- Karl 2005-09-18 18:26 From fredrik at pythonware.com Mon Sep 19 07:16:37 2005 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon, 19 Sep 2005 07:16:37 +0200 Subject: [Python-Dev] list splicing References: Message-ID: Karl Chen wrote: > Hi, has anybody considered adding something like this: > a = [1, 2] > [ 'x', *a, 'y'] > > as syntactic sugar for > a = [1, 2] > [ 'x' ] + a + [ 'y' ]. > > Notes: > - This is a common operation is it? From greg.ewing at canterbury.ac.nz Mon Sep 19 07:25:29 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 19 Sep 2005 17:25:29 +1200 Subject: [Python-Dev] list splicing In-Reply-To: References: Message-ID: <432E4BC9.1020100@canterbury.ac.nz> Karl Chen wrote: > Hi, has anybody considered adding something like this: > a = [1, 2] > [ 'x', *a, 'y'] > > as syntactic sugar for > a = [1, 2] > [ 'x' ] + a + [ 'y' ]. You can write that as a = [1, 2] a[1:1] = a Greg From greg.ewing at canterbury.ac.nz Mon Sep 19 07:26:02 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 19 Sep 2005 17:26:02 +1200 Subject: [Python-Dev] removing nested tuple function parameters In-Reply-To: <20050919003740.GA5918@alcyon.progiciels-bpi.ca> References: <432DE8B2.4020701@gmail.com> <002001c5bcae$8639e640$99b1958d@oemcomputer> <20050919003740.GA5918@alcyon.progiciels-bpi.ca> Message-ID: <432E4BEA.5000506@canterbury.ac.nz> Fran?ois Pinard wrote: > The only practical reason to like this feature is sparing the need of > finding an otherwise useless name for the formal argument. If the argument represents a coherent enough concept to be passed in as a tuple in the first place, it should be possible to find a meaningful name for it. Otherwise the elements should probably be passed in as separate arguments. > Yet, if I was given the choice between nested tuple function arguments, > and faster argument processing, the latter would win instantly. I believe that exactly the same bytecode results either way, so there's no speed advantage or penalty. Greg From jcarlson at uci.edu Mon Sep 19 07:38:28 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Sun, 18 Sep 2005 22:38:28 -0700 Subject: [Python-Dev] list splicing In-Reply-To: References: Message-ID: <20050918223133.AF7A.JCARLSON@uci.edu> "Fredrik Lundh" wrote: > > Karl Chen wrote: > > > Hi, has anybody considered adding something like this: > > a = [1, 2] > > [ 'x', *a, 'y'] > > > > as syntactic sugar for > > a = [1, 2] > > [ 'x' ] + a + [ 'y' ]. > > > > Notes: > > - This is a common operation > > is it? Not in the code that I read/use. While "not all 3 line functions should be a builtin", "not all <5 character typing savings should be made syntax". - Josiah From bcannon at gmail.com Mon Sep 19 08:43:46 2005 From: bcannon at gmail.com (Brett Cannon) Date: Sun, 18 Sep 2005 23:43:46 -0700 Subject: [Python-Dev] removing nested tuple function parameters In-Reply-To: <432E4BEA.5000506@canterbury.ac.nz> References: <432DE8B2.4020701@gmail.com> <002001c5bcae$8639e640$99b1958d@oemcomputer> <20050919003740.GA5918@alcyon.progiciels-bpi.ca> <432E4BEA.5000506@canterbury.ac.nz> Message-ID: On 9/18/05, Greg Ewing wrote: > Fran?ois Pinard wrote: > > > The only practical reason to like this feature is sparing the need of > > finding an otherwise useless name for the formal argument. > > If the argument represents a coherent enough concept > to be passed in as a tuple in the first place, it > should be possible to find a meaningful name for it. > Otherwise the elements should probably be passed in > as separate arguments. > > > Yet, if I was given the choice between nested tuple function arguments, > > and faster argument processing, the latter would win instantly. > > I believe that exactly the same bytecode results either > way, so there's no speed advantage or penalty. > Yep, the bytecode will be the same sans the compiler-created name for the tuple . -Brett From nnorwitz at gmail.com Mon Sep 19 09:49:33 2005 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 19 Sep 2005 00:49:33 -0700 Subject: [Python-Dev] possible memory leak on windows (valgrind report) Message-ID: I ran 2.4.x through valgrind and found two small problems on Linux that have been fixed. There may be some other issues which could benefit from more eyes (small, probably one time memory leaks). The entire run is here: http://python.org/valgrind-2.4.2.out (I need to write a lot more suppression rules for gentoo.) I think I see a memory leak in win32_startfile. Since I don't run windows I can't test it. filepath should be allocated with the et flag to PyArgs_ParseTuple(), but it wasn't freed without this patch. Does this make sense? See the attached patch. n -------------- next part -------------- A non-text attachment was scrubbed... Name: posix.diff Type: text/x-patch Size: 1018 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050919/0586b43f/posix.bin From kirby-2 at web.de Sun Sep 18 13:59:36 2005 From: kirby-2 at web.de (Matthias Andreas Benkard) Date: Sun, 18 Sep 2005 13:59:36 +0200 Subject: [Python-Dev] os.path.diff(path1, path2) In-Reply-To: <20050913031157.46635.qmail@web51809.mail.yahoo.com> References: <20050913031157.46635.qmail@web51809.mail.yahoo.com> Message-ID: <1127044776.14284.9.camel@mulkomp.mst-plus> Hi, > This function would take two paths: A and B and give > the relation between them. Here are a few of examples. > > os.path.diff("/A/C/D/", "/A/D/F/") > ==> "../../D/F" > > os.path.diff("/A/", "/A/B/C/") > ==> "B/C" > > os.path.diff("/A/B/C/", "/A/") > ==> "../.." I'm not sure whether something like this is generally non-trivial. Suppose you have the following directory structure: /home -> usr/home /usr /usr/home What does os.path.diff("/home/", "/usr/") yield? "../usr/", I would presume? But that's obviously wrong: >>> import os >>> os.stat("/home/../usr") Traceback (most recent call last): File " ", line 1, in ? OSError: [Errno 2] No such file or directory: '/home/../usr' I've not thought about this long enough to say if there's a trivial solution to this, though :) Bye, Matthias A. Benkard ________________________________________________________________________ Matthias Andreas Benkard, Anarchokommunist und Pythonprogrammierer Pers?nliche Website: http://www.mulk.de.vu/ Pers?nlicher Weblog: http://www.kompottkin.de.vu/ Flames bitte nach /dev/null schicken. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050918/a18b46b3/attachment.pgp From kirby-2 at web.de Sun Sep 18 14:48:27 2005 From: kirby-2 at web.de (Matthias Andreas Benkard) Date: Sun, 18 Sep 2005 14:48:27 +0200 Subject: [Python-Dev] os.path.diff(path1, path2) In-Reply-To: <1127044776.14284.9.camel@mulkomp.mst-plus> References: <20050913031157.46635.qmail@web51809.mail.yahoo.com> <1127044776.14284.9.camel@mulkomp.mst-plus> Message-ID: <1127047707.14284.14.camel@mulkomp.mst-plus> Hi, > /home -> usr/home Sorry, I forgot to mention what I meant by this: /home is a symlink pointing to usr/home (that is, /usr/home). Bye, Matthias ________________________________________________________________________ Matthias Andreas Benkard, Anarchokommunist und Pythonprogrammierer Pers?nliche Website: http://www.mulk.de.vu/ Pers?nlicher Weblog: http://www.kompottkin.de.vu/ Flames bitte nach /dev/null schicken. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050918/f0df6d95/attachment.pgp From falcon at intercable.ru Mon Sep 19 10:52:33 2005 From: falcon at intercable.ru (Sokolov Yura) Date: Mon, 19 Sep 2005 12:52:33 +0400 Subject: [Python-Dev] Mixins. Message-ID: <432E7C51.6080603@intercable.ru> Excuse my english. Is there any pythonic way to make real mixin into a class? Problem: Django model (and may be SQLObject). I have a common peaces of model declaration. Sometimes they are in table definition together, sometimes table have just one of them. I cannot use inheritance, cause Django doesn't handle it in right way (like mixins). Would it be usefull to define new magic class variable __mixins__ for a new style classes, which will accept list of classes and will drop all members of them into definition of class before they went to metaclass. Possible example (my wish in Django's Model): class time_from_to: time_from=meta.DateTimeField(default=datetime(1900,1,1,0,0,0)) time_to=meta.DateTimeField(default=datetime(3000,1,1,0,0,0)) class ident(meta.Model): __mixin__=[time_from_to] ident = meta.TextField() Or, as alternative: class time_from_to(mixin): time_from=meta.DateTimeField(default=datetime(1900,1,1,0,0,0)) time_to=meta.DateTimeField(default=datetime(3000,1,1,0,0,0)) class ident(time_from_to,meta.Model): ident = meta.TextField() You say: "It is problem of Django". Yes, you are right. It is common to use multiple inheritace in place of mixin. But it is handled not all time. Django does not at the moment (I hope, it would). To handle it, library writers, who implements their metaclasses, ought to write similar code again and again. I think, it would be simpler to make mixins in one true standart way. I can be mistaken. From steven.bethard at gmail.com Mon Sep 19 17:28:04 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 19 Sep 2005 09:28:04 -0600 Subject: [Python-Dev] Mixins. In-Reply-To: <432E7C51.6080603@intercable.ru> References: <432E7C51.6080603@intercable.ru> Message-ID: Sokolov Yura wrote: > Would it be usefull to define new magic class variable __mixins__ for a > new style classes, which > will accept list of classes and will drop all members of them into > definition of class before they went to metaclass. First off, this is probably more appropriate for comp.lang.python. Work it out there first, and if you still think it's a valid idea, then present it to python-dev. Secondly, no, I don't think it's useful to provide two ways (multiple inheritance and a __mixins__ attribute) to support mixins. Remember, TOOWTDI. STeVe -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From gmccaughan at synaptics-uk.com Mon Sep 19 17:43:03 2005 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Mon, 19 Sep 2005 16:43:03 +0100 Subject: [Python-Dev] list splicing In-Reply-To: <20050918223133.AF7A.JCARLSON@uci.edu> References: <20050918223133.AF7A.JCARLSON@uci.edu> Message-ID: <200509191643.05150.gmccaughan@synaptics-uk.com> On Monday 2005-09-19 06:38, Josiah Carlson wrote: > > > [ 'x', *a, 'y'] > > > > > > as syntactic sugar for > > > > > > [ 'x' ] + a + [ 'y' ]. > > > > > > Notes: > > > - This is a common operation > > > > is it? > > Not in the code that I read/use. While "not all 3 line functions should > be a builtin", "not all <5 character typing savings should be made > syntax". The problems with syntax are 1 It adds cognitive load. 2 It makes your code look like line noise. 3 It reduces options for future development. 4 It complicates the parser. In this instance, I don't think #1 applies; the rule "Putting *foo into a comma-separated list is the same as putting all the elements of foo into that list" isn't actually any harder to remember than the current rule concerning the prefix-* operator. #2 is always debatable. In this instance the proposed new form doesn't look any uglier to me (I speak for no one else) than its predecessor. #3 surely isn't true here; there isn't anything else sensible to do with prefix-* in lists, given the existing use specifically in argument lists. I don't know about #4, but I suspect it (along with the related "it requires work, and there isn't much benefit from it") is the best argument against this proposal. -- g From steven.bethard at gmail.com Mon Sep 19 17:50:01 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 19 Sep 2005 09:50:01 -0600 Subject: [Python-Dev] removing nested tuple function parameters In-Reply-To: <432E4BEA.5000506@canterbury.ac.nz> References: <432DE8B2.4020701@gmail.com> <002001c5bcae$8639e640$99b1958d@oemcomputer> <20050919003740.GA5918@alcyon.progiciels-bpi.ca> <432E4BEA.5000506@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Fran?ois Pinard wrote: > > > The only practical reason to like this feature is sparing the need of > > finding an otherwise useless name for the formal argument. > > If the argument represents a coherent enough concept > to be passed in as a tuple in the first place, it > should be possible to find a meaningful name for it. > Otherwise the elements should probably be passed in > as separate arguments. But sometimes you don't have this option. If I have a class that I'd like to support an indexing operation like: obj[x, y] then the natural __getitem__ signature will look like: def __getitem__(self, (x, y)): ... Note that I definitely can't write this as: def __getitem__(self, x, y): ... because __getitem__ is always called with a single argument, not two. Of course, I can give the parameter a name and do the tuple unpacking manually (e.g. x, y = x_y), but I'm not sure I understand yet what we gain by making __getitem__ harder to use with tuples. I guess if tuple unpacking in function parameters goes away, I think we should change the __getitem__ machinery so that: obj[x1, x2, ..., xN] is translated to: obj.__getitem__(x1, x2, ..., xN) where __getitem__ would now have to take a *args when called with tuples. STeVe -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From blais at furius.ca Mon Sep 19 18:28:50 2005 From: blais at furius.ca (Martin Blais) Date: Mon, 19 Sep 2005 12:28:50 -0400 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP (was Re: Variant of removing GIL.) In-Reply-To: References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> Message-ID: <8393fff0509190928196bba9a@mail.gmail.com> On 9/18/05, Guido van Rossum wrote: > On 9/17/05, John J Lee wrote: > > c. Since time is needed to iron out bugs (and perhaps also to reimplememt > > some pieces of code "from scratch"), very early in the life of Python 3 > > seems like the least-worst time to begin work on such a change. > > > > I realize that not all algorithms (nor all computational problems) scale > > well to MP hardware. Is it feasible to usefully compile both MP and a UP > > binaries from one Python source code base? > > That's an understatement. I expect that *most* problems (even most > problems that we will be programming 10-20 years from now) get little > benefit out of MP. Some are saying it won't be a matter of choice if we want to get the software to run faster (you know, that "MORE MORE MORE!" thing we all seem to suffer from): http://www.gotw.ca/publications/concurrency-ddj.htm The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software Herb Sutter March 2005 From guido at python.org Mon Sep 19 18:50:18 2005 From: guido at python.org (Guido van Rossum) Date: Mon, 19 Sep 2005 09:50:18 -0700 Subject: [Python-Dev] possible memory leak on windows (valgrind report) In-Reply-To: References: Message-ID: On 9/19/05, Neal Norwitz wrote: > I ran 2.4.x through valgrind and found two small problems on Linux > that have been fixed. There may be some other issues which could > benefit from more eyes (small, probably one time memory leaks). The > entire run is here: > > http://python.org/valgrind-2.4.2.out > > (I need to write a lot more suppression rules for gentoo.) > > I think I see a memory leak in win32_startfile. Since I don't run > windows I can't test it. > filepath should be allocated with the et flag to PyArgs_ParseTuple(), > but it wasn't freed without this patch. Does this make sense? See > the attached patch. That patch doesn't make sense to me -- the "s" code to PyArg_ParseTuple doesn't return newly allocated memory, it just returns a pointer into a string object that is owned by the caller (really by the call machinery I suppose). Compare other places using PyArg_ParseTuple(args, "s:..."). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nnorwitz at gmail.com Mon Sep 19 18:55:36 2005 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 19 Sep 2005 09:55:36 -0700 Subject: [Python-Dev] possible memory leak on windows (valgrind report) In-Reply-To: References: Message-ID: On 9/19/05, Guido van Rossum wrote: > > That patch doesn't make sense to me -- the "s" code to > PyArg_ParseTuple doesn't return newly allocated memory, it just > returns a pointer into a string object that is owned by the caller > (really by the call machinery I suppose). Compare other places using > PyArg_ParseTuple(args, "s:..."). "s"? The format passed to ParseTuple is "et". At least it is in the patch I'm looking at. n From guido at python.org Mon Sep 19 19:05:00 2005 From: guido at python.org (Guido van Rossum) Date: Mon, 19 Sep 2005 10:05:00 -0700 Subject: [Python-Dev] possible memory leak on windows (valgrind report) In-Reply-To: References: Message-ID: So it is. I swear I saw "s"; I must've had an out of date version. The change to "et" is less than a week old, but that's no excuse. :-( It does look like the patch is correct then (but I can't build on Windows any more either). Sorry for the confusion. On 9/19/05, Neal Norwitz wrote: > On 9/19/05, Guido van Rossum wrote: > > > > That patch doesn't make sense to me -- the "s" code to > > PyArg_ParseTuple doesn't return newly allocated memory, it just > > returns a pointer into a string object that is owned by the caller > > (really by the call machinery I suppose). Compare other places using > > PyArg_ParseTuple(args, "s:..."). > > "s"? The format passed to ParseTuple is "et". At least it is in the > patch I'm looking at. > > n > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Mon Sep 19 19:06:32 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 19 Sep 2005 13:06:32 -0400 Subject: [Python-Dev] removing nested tuple function parameters References: <432DE8B2.4020701@gmail.com><002001c5bcae$8639e640$99b1958d@oemcomputer><20050919003740.GA5918@alcyon.progiciels-bpi.ca><432E4BEA.5000506@canterbury.ac.nz> Message-ID: I consider the current situation to be a consistency feature. To a first approximation, Python function calls 'pass' objects by name-binding: param_name_list = arg_object_list Disabling structure unpacking in this assignment would make the language slightly more complex. Someone else posted the same observation in c.l.p. Another thought. By directly unpacking and not naming a sequence, one 'announces' that only the components are of interest and that nothing will be done with the sequence object itself. Terry J. Reedy From guido at python.org Mon Sep 19 19:21:28 2005 From: guido at python.org (Guido van Rossum) Date: Mon, 19 Sep 2005 10:21:28 -0700 Subject: [Python-Dev] removing nested tuple function parameters In-Reply-To: References: <432DE8B2.4020701@gmail.com> <002001c5bcae$8639e640$99b1958d@oemcomputer> <20050919003740.GA5918@alcyon.progiciels-bpi.ca> <432E4BEA.5000506@canterbury.ac.nz> Message-ID: On 9/19/05, Terry Reedy wrote: > > I consider the current situation to be a consistency feature. To a first > approximation, Python function calls 'pass' objects by name-binding: > > param_name_list = arg_object_list > > Disabling structure unpacking in this assignment would make the language > slightly more complex. Someone else posted the same observation in c.l.p. Maybe, but there are enough differences between parameter/argument lists and sequences that this consistency sounds rather foolish to me. In fact, the feature came from a similar feature in ABC, but in ABC, parameter lists *were* considered assignment targets -- the outer level of parentheses was just a special case of tuple unpacking. Not so in Python, which has keyword parameters, *varargs, **keywords, and where f(x,) is the same as f(x) -- even though (x,) is a tuple and (x) is not. Also, I bet many people will be surprised to know that this code doesn't work: add = lambda (x, y): x+y print add(1, 2) > Another thought. By directly unpacking and not naming a sequence, one > 'announces' that only the components are of interest and that nothing will > be done with the sequence object itself. Fair enough, though I'm not sure what use we can make of that information. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Mon Sep 19 19:26:56 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 19 Sep 2005 11:26:56 -0600 Subject: [Python-Dev] removing nested tuple function parameters In-Reply-To: References: <432DE8B2.4020701@gmail.com> <002001c5bcae$8639e640$99b1958d@oemcomputer> <20050919003740.GA5918@alcyon.progiciels-bpi.ca> <432E4BEA.5000506@canterbury.ac.nz> Message-ID: Guido van Rossum wrote: > Also, I bet many people will be surprised to know that this code doesn't work: > > add = lambda (x, y): x+y > print add(1, 2) What, an example using lambda syntax that's unintuitive? Never! ;-) STeVe -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From jcarlson at uci.edu Mon Sep 19 19:43:15 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 19 Sep 2005 10:43:15 -0700 Subject: [Python-Dev] list splicing In-Reply-To: <200509191643.05150.gmccaughan@synaptics-uk.com> References: <20050918223133.AF7A.JCARLSON@uci.edu> <200509191643.05150.gmccaughan@synaptics-uk.com> Message-ID: <20050919102947.AF84.JCARLSON@uci.edu> Gareth McCaughan wrote: > The problems with syntax are > > 1 It adds cognitive load. > 2 It makes your code look like line noise. > 3 It reduces options for future development. > 4 It complicates the parser. > > I don't know about #4, but I suspect it (along with the related > "it requires work, and there isn't much benefit from it") is the > best argument against this proposal. I don't think the parser would get measureably more complex, but I believe that any work to support this uncommon (from my experience) operation, rather than the dozens of other usable RFEs in the tracker, would be misspent time. - Josiah From ark at acm.org Mon Sep 19 20:11:35 2005 From: ark at acm.org (Andrew Koenig) Date: Mon, 19 Sep 2005 14:11:35 -0400 Subject: [Python-Dev] removing nested tuple function parameters In-Reply-To: Message-ID: <006701c5bd45$95e086c0$6402a8c0@arkdesktop> > I agree that we shouldn't mess with them in 2.x. Yet I think they are > a candidate for being dropped from Py3K. While every feature is used > by *someone* (as the feedback to Brett's query clearly shows) this one > has several things against it. For every user who is fond of them > there are probably ten who have never even heard of it. It's purely > syntactic sugar (the only place where it's not trivial to replace is > in a lambda). I've encountered quite a few people who had a hard time > reading code that uses it. I personally prefer reading code that > doesn't use this feature; for one thing, when this is used, you can't > refer to a parameter by name. I don't know whether this qualifies as an argument for or against the feature, but ML has a nice way of referring to such a parameter by name: fun f(x as (y, z)) = (* ... *) This defines f as a function with an argument that is a 2-element tuple. The name x is bound to the argument, and the names y and z are bound to the two components of the argument. This example is syntactic sugar for fun f x = let val (y, z) = x in (* ... *) end but it I find the sugared version easier and more natural to write and understand. If you don't know ML, the unsugared version might be easier to follow if I indent it Pythonically: fun f x = let val (y, z) = x in (* ... *) end The idea is that the stuff between "in" and "end" is executed in the scope of the bindings between "let" and "in", after which those bindings are discarded. From ark at acm.org Mon Sep 19 20:12:42 2005 From: ark at acm.org (Andrew Koenig) Date: Mon, 19 Sep 2005 14:12:42 -0400 Subject: [Python-Dev] removing nested tuple function parameters In-Reply-To: <20050919003740.GA5918@alcyon.progiciels-bpi.ca> Message-ID: <006801c5bd45$bda753a0$6402a8c0@arkdesktop> > The only practical reason to like this feature is sparing the need of > finding an otherwise useless name for the formal argument. Another > reason, but not practical at all, is that the concept conveys some > elegance and originality (each programming language should ideally have > a few of these) and is enforced in other places in Python, like in the > `for' statement -- where I find implied unpacking very, very useful. One other reason: It is possible to imagine using the feature to catch some type errors at the point of call, rather than having to get into the function itself before detecting them. From reinhold-birkenfeld-nospam at wolke7.net Mon Sep 19 20:36:19 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Mon, 19 Sep 2005 20:36:19 +0200 Subject: [Python-Dev] possible memory leak on windows (valgrind report) In-Reply-To: References: Message-ID: Neal Norwitz wrote: > I ran 2.4.x through valgrind and found two small problems on Linux > that have been fixed. There may be some other issues which could > benefit from more eyes (small, probably one time memory leaks). The > entire run is here: > > http://python.org/valgrind-2.4.2.out > > (I need to write a lot more suppression rules for gentoo.) > > I think I see a memory leak in win32_startfile. Since I don't run > windows I can't test it. > filepath should be allocated with the et flag to PyArgs_ParseTuple(), > but it wasn't freed without this patch. Does this make sense? See > the attached patch. Applied the patch, this was my fault. Reinhold -- Mail address is perfectly valid! From jimjjewett at gmail.com Mon Sep 19 20:47:18 2005 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 19 Sep 2005 14:47:18 -0400 Subject: [Python-Dev] removing nested tuple function parameters Message-ID: Andrew Koenig wrote: > It is possible to imagine using the feature to catch some > type errors at the point of call, rather than having to get into the > function itself before detecting them. There can certainly be value in type-checking parameters and return values, but in this case, I think the (mental) cost is too high. With nested tuples, it hasn't actually ever helped me, but I have been burned by the extra fragility it introduces. Nesting encourages complicated parameter lists. Nested variables can (wrongly) match if your data itself is made of tuples. If the function signature changes, it is a lot harder to notice or track down. Normally, I can defend against some of this by using keywords -- but not if the parameter list is nested. Sure, it makes def __getitem__(self, (x, y)) easier -- but I don't think that is a good thing. When working with someone else's code, I would much rather *read* def __getitem__(self, keytuple): x, y = keytuple -jJ From mwh at python.net Mon Sep 19 22:07:34 2005 From: mwh at python.net (Michael Hudson) Date: Mon, 19 Sep 2005 21:07:34 +0100 Subject: [Python-Dev] removing nested tuple function parameters In-Reply-To: <006801c5bd45$bda753a0$6402a8c0@arkdesktop> (Andrew Koenig's message of "Mon, 19 Sep 2005 14:12:42 -0400") References: <006801c5bd45$bda753a0$6402a8c0@arkdesktop> Message-ID: <2mmzm8lt2x.fsf@starship.python.net> "Andrew Koenig" writes: >> The only practical reason to like this feature is sparing the need of >> finding an otherwise useless name for the formal argument. Another >> reason, but not practical at all, is that the concept conveys some >> elegance and originality (each programming language should ideally have >> a few of these) and is enforced in other places in Python, like in the >> `for' statement -- where I find implied unpacking very, very useful. > > One other reason: It is possible to imagine using the feature to catch some > type errors at the point of call, rather than having to get into the > function itself before detecting them. Conceptually, maybe, but implementation wise, that's not how things work. Cheers, mwh -- I love the way Microsoft follows standards. In much the same manner that fish follow migrating caribou. -- Paul Tomblin -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html From mwh at python.net Mon Sep 19 22:12:05 2005 From: mwh at python.net (Michael Hudson) Date: Mon, 19 Sep 2005 21:12:05 +0100 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <8393fff0509190928196bba9a@mail.gmail.com> (Martin Blais's message of "Mon, 19 Sep 2005 12:28:50 -0400") References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <8393fff0509190928196bba9a@mail.gmail.com> Message-ID: <2mirwwlsve.fsf@starship.python.net> Martin Blais writes: > On 9/18/05, Guido van Rossum wrote: >> On 9/17/05, John J Lee wrote: >> > c. Since time is needed to iron out bugs (and perhaps also to reimplememt >> > some pieces of code "from scratch"), very early in the life of Python 3 >> > seems like the least-worst time to begin work on such a change. >> > >> > I realize that not all algorithms (nor all computational problems) scale >> > well to MP hardware. Is it feasible to usefully compile both MP and a UP >> > binaries from one Python source code base? >> >> That's an understatement. I expect that *most* problems (even most >> problems that we will be programming 10-20 years from now) get little >> benefit out of MP. > > Some are saying it won't be a matter of choice if we want to get the > software to run faster (you know, that "MORE MORE MORE!" thing we all > seem to suffer from): People have been saying this for _years_, and it hasn't happened yet. This time around it's a bit more convincing, but I reserve the right to remain a touch skeptical. > http://www.gotw.ca/publications/concurrency-ddj.htm > The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software > Herb Sutter > March 2005 I was disappointed that that article (hey, it was the only issue of ddj I've ever actually bought! :) didn't consider any concurrency models other than shared memory threading. Cheers, mwh -- . <- the point your article -> . |------------------------- a long way ------------------------| -- Christophe Rhodes, ucam.chat From tlesher at gmail.com Mon Sep 19 22:20:05 2005 From: tlesher at gmail.com (Tim Lesher) Date: Mon, 19 Sep 2005 16:20:05 -0400 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <2mirwwlsve.fsf@starship.python.net> References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <8393fff0509190928196bba9a@mail.gmail.com> <2mirwwlsve.fsf@starship.python.net> Message-ID: <9613db6005091913206dcf51ec@mail.gmail.com> On 9/19/05, Michael Hudson wrote: > I was disappointed that that article (hey, it was the only issue of > ddj I've ever actually bought! :) didn't consider any concurrency > models other than shared memory threading. The problem is that, for all its limitations, shared-memory threading is the most popular concurrency model on the most popular operating system, so future hardware platforms targeting that system will be optimizing for that case. We can either rail against the sea, or accept it. -- Tim Lesher From fw at deneb.enyo.de Mon Sep 19 22:23:36 2005 From: fw at deneb.enyo.de (Florian Weimer) Date: Mon, 19 Sep 2005 22:23:36 +0200 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <8393fff0509190928196bba9a@mail.gmail.com> (Martin Blais's message of "Mon, 19 Sep 2005 12:28:50 -0400") References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <8393fff0509190928196bba9a@mail.gmail.com> Message-ID: <87vf0wrelz.fsf@mid.deneb.enyo.de> * Martin Blais: > http://www.gotw.ca/publications/concurrency-ddj.htm > The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software > Herb Sutter > March 2005 This piece is fundamentally wrong. We all have been writing concurrent server-side software for eons. I don't know what Herb was thinking when he wrote that piece. From fw at deneb.enyo.de Mon Sep 19 22:29:06 2005 From: fw at deneb.enyo.de (Florian Weimer) Date: Mon, 19 Sep 2005 22:29:06 +0200 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: (Guido van Rossum's message of "Sun, 18 Sep 2005 17:50:13 -0700") References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> Message-ID: <87r7bkrect.fsf@mid.deneb.enyo.de> * Guido van Rossum: > That assumes a very specific model for how all that MP power is going > to be used. Indeed. > I personally don't think the threaded programming model as found in > Java works all that well; without locks you end up with concurrent > modification errors, with locks you get deadlocks and livelocks. Java is bascially forced into that model because VM startup costs are so high. To some extent, Python has similar problems, but you don't have to care about preserving class loader semantics, so you should be in a better position to cut down process creation time. > Be my guest. Prove me wrong. Talk is cheap; instead of arguing my > points (all of which can be argued ad infinitum), come back when > you've got a working GIL-free Python. Doesn't have to be CPython-based > -- C# would be fine too. By the way, has anybody ever tried to create a CPython variant which uses a (mostly) copying garbage collector (or something else except reference counting or Boehm GC)? Copying GC might help to get rid of the GIL *and* improve performance in the accept+fork model (because read-only object access does not trigger copy-on-write anymore). From mcherm at mcherm.com Mon Sep 19 22:36:59 2005 From: mcherm at mcherm.com (Michael Chermside) Date: Mon, 19 Sep 2005 13:36:59 -0700 Subject: [Python-Dev] list splicing Message-ID: <20050919133659.260zmu4vjmcgkk80@login.werra.lunarpages.com> Karl Chen writes: > Hi, has anybody considered adding something like this: > a = [1, 2] > [ 'x', *a, 'y'] > > as syntactic sugar for > a = [1, 2] > [ 'x' ] + a + [ 'y' ]. A bit later in the thread, Josiah Carlson replies: > I don't think the parser would get measureably more complex, but I > believe that any work to support this uncommon (from my experience) > operation, rather than the dozens of other usable RFEs in the tracker, > would be misspent time. I'd just like to point out that this is a FRF (Frequently Requested Feature). I'm not arguing in favor of it, just pointing out that using "star unpacking" in tuple and list literals is an idea that I'm sure I've seen proposed at least a couple of times before. This doesn't necessarily make it a good idea, nor a bad one. -- Michael Chermside From fdrake at acm.org Mon Sep 19 22:47:51 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Mon, 19 Sep 2005 16:47:51 -0400 Subject: [Python-Dev] list splicing In-Reply-To: <20050919133659.260zmu4vjmcgkk80@login.werra.lunarpages.com> References: <20050919133659.260zmu4vjmcgkk80@login.werra.lunarpages.com> Message-ID: <200509191647.51829.fdrake@acm.org> On Monday 19 September 2005 16:36, Michael Chermside wrote: > I'd just like to point out that this is a FRF (Frequently Requested > Feature). I'm not arguing in favor of it, just pointing out that > using "star unpacking" in tuple and list literals is an idea that > I'm sure I've seen proposed at least a couple of times before. Indeed, "star unpacking" has been brought up many times; I think it would be really cool myself. (Regardless of whether it's the best use of development effort; that would be up to the contributor.) I haven't seen "star packing" proposed before, as best as I can recall. That's what strikes me as different in this discussion. -Fred -- Fred L. Drake, Jr. From mwh at python.net Mon Sep 19 22:52:55 2005 From: mwh at python.net (Michael Hudson) Date: Mon, 19 Sep 2005 21:52:55 +0100 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <87r7bkrect.fsf@mid.deneb.enyo.de> (Florian Weimer's message of "Mon, 19 Sep 2005 22:29:06 +0200") References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <87r7bkrect.fsf@mid.deneb.enyo.de> Message-ID: <2mek7klqzc.fsf@starship.python.net> Florian Weimer writes: > By the way, has anybody ever tried to create a CPython variant which > uses a (mostly) copying garbage collector (or something else except > reference counting or Boehm GC)? Not to my knowledge. I've always thought that it would be pretty hard. I'd be interested in being proved wrong. PyPy will hopefully experiment with this sort of thing. > Copying GC might help to get rid of the GIL *and* improve performance > in the accept+fork model (because read-only object access does not > trigger copy-on-write anymore). How does a copying gc differ much from a non-copying non-refcounted gc here? Cheers, mwh -- Darn! I've only got 10 minutes left to get falling-down drunk! I suppose I'll have to smoke crack instead now. -- Tim Peters is checking things in on 2002-12-31 From fw at deneb.enyo.de Mon Sep 19 23:17:13 2005 From: fw at deneb.enyo.de (Florian Weimer) Date: Mon, 19 Sep 2005 23:17:13 +0200 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <2mek7klqzc.fsf@starship.python.net> (Michael Hudson's message of "Mon, 19 Sep 2005 21:52:55 +0100") References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <87r7bkrect.fsf@mid.deneb.enyo.de> <2mek7klqzc.fsf@starship.python.net> Message-ID: <87hdcgrc4m.fsf@mid.deneb.enyo.de> * Michael Hudson: > Not to my knowledge. I've always thought that it would be pretty > hard. I'd be interested in being proved wrong. The real problem is that you can ditch most extension modules. 8-( It sounds more like a fun project for the Python core, though. >> Copying GC might help to get rid of the GIL *and* improve performance >> in the accept+fork model (because read-only object access does not >> trigger copy-on-write anymore). > > How does a copying gc differ much from a non-copying non-refcounted gc > here? You could copy immutable objects to a separate set of pages and never collect them (especially if recursively refer to immutable objects only). From Scott.Daniels at Acm.Org Mon Sep 19 23:52:39 2005 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Mon, 19 Sep 2005 14:52:39 -0700 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <2mek7klqzc.fsf@starship.python.net> References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <87r7bkrect.fsf@mid.deneb.enyo.de> <2mek7klqzc.fsf@starship.python.net> Message-ID: Michael Hudson wrote: > How does a copying gc differ much from a non-copying non-refcounted gc > here? One important issue for C coded modules is that addresses may change when a GC is invoked, so no remembering addresses in your module; you must recalculate before each use. -- Scott David Daniels Scott.Daniels at Acm.Org From barry at python.org Tue Sep 20 00:23:21 2005 From: barry at python.org (Barry Warsaw) Date: Mon, 19 Sep 2005 18:23:21 -0400 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <87r7bkrect.fsf@mid.deneb.enyo.de> <2mek7klqzc.fsf@starship.python.net> Message-ID: <1127168601.31396.23.camel@geddy.wooz.org> On Mon, 2005-09-19 at 17:52, Scott David Daniels wrote: > Michael Hudson wrote: > > > How does a copying gc differ much from a non-copying non-refcounted gc > > here? > > One important issue for C coded modules is that addresses may change > when a GC is invoked, so no remembering addresses in your module; you > must recalculate before each use. And then you're left with "fun" like JNI's pining and such. That won't suck at all! :) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050919/c180cc74/attachment.pgp From maa_public at sinn.ru Tue Sep 20 01:16:27 2005 From: maa_public at sinn.ru (Alexander Myodov) Date: Tue, 20 Sep 2005 04:46:27 +0530 Subject: [Python-Dev] For/while/if statements/comprehension/generator expressions unification Message-ID: <1209206536.20050920044627@sinn.ru> Hello, Well, we have "for", "while" and "if". We also have statements, list generator expressions and list comprehensions. In comprehensions we have the opportunity to use several for-s and in a same expression (for i in range (0, 640) for j in range (0, 640)), in loops we cannot; in comprehensions, we can use if-s to filter the items; in loops, we cannot, and we have to write an inner comparison instead. My opinion is that in so smart language as Python is, it would be great to have them generalized, so that the features available in list comprehensions should be available in statements, vice versa. All three statements (for/while/if) could be joined together into a single common one, where any of these could be combined and used multiple times (but for the cases when a "else"/"elif" etc causes are required, old-school one-statements should be left); and for such common expressions, they should be equal for both loop statements and list comprehensions/generator expressions. That is, the following loops should be possible: ------- # This gives us some sugar to get rid of unnecessary indentations for x in range (0, 640) for y in range (0, 480): ------- for x in range (0, 640) if should_handle_this_column(x) \ for y in range (0, 480) if should_handle_this_row(y): ------- for nX, nY in ( f(x), f(y) for x in range (0, 640) if should_handle_this_column(x) for y in range (0, 480) if should_handle_this_row(y) ): ------- for x in range (0, 640) for y in range (0, 480) while(!timeout()): ------- for x in range (0, 640) while(!timeout_pos_in_row()) for y in range (0, 480) while(!timeout_rows()): ------- # And the latest and the hugest one: for x in range (0, 640) if should_handle_this_column(x) while(!timeout_pos_in_row()) \ for y in range (0, 480) if should_handle_this_row(y) while(!timeout_rows() : ------- # And almost the same as generator expression: for x, y in ( f(x), f(y) for x in range (0, 640) if should_handle_this_column(x) while(!timeout_pos_in_row()) for y in range (0, 480) if should_handle_this_row(y) while(!timeout_rows() ) ------- Hope I didn't miss something important... -- With best regards, Alexander mailto:maa_public at sinn.ru From raymond.hettinger at verizon.net Tue Sep 20 01:20:20 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Mon, 19 Sep 2005 19:20:20 -0400 Subject: [Python-Dev] list splicing In-Reply-To: <200509191647.51829.fdrake@acm.org> Message-ID: <002b01c5bd70$b4a49940$99b1958d@oemcomputer> [Fred L. Drake] > Indeed, "star unpacking" has been brought up many times; I think it would > be > really cool myself. It might have a chance of acceptance this time if the proponents stick with unpacking at the end: a,b,*c=sometup instead of a,*b,c=sometup. The latter has usually gotten shot down quickly, taking the former down with it. Raymond From raymond.hettinger at verizon.net Tue Sep 20 01:44:32 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Mon, 19 Sep 2005 19:44:32 -0400 Subject: [Python-Dev] "and" and "or" operators in Py3.0 Message-ID: <003201c5bd74$15e1c220$99b1958d@oemcomputer> I propose that in Py3.0, the "and" and "or" operators be simplified to always return a Boolean value instead of returning the last evaluated argument. 1) The construct can be error-prone. When an error occurs it can be invisible to the person who wrote it. I got bitten in published code that had survived testing and code review: def real(self): 'Return a vector with the real part of each input element' # do not convert integer inputs to floats return self.map(lambda z: type(z)==types.ComplexType and z.real or z) The code fails silently when z is (0+4i). It took a good while to trace down a user reported error (when Matlab results disagreed with my matrix module results) and determine that the real() method contained an error. Even when traced down, I found it hard to see the error in the code. Now that I know what to look for, it has not happened again, but I do always have to stare hard at any "and/or" group to mentally verify each case. 2) When going back and forth between languages, it is easy to forget that only Python returns something other than a boolean. 3) Even when it isn't being used, the possibility of non-boolean return value complicates the bytecode and parser. To allow for "and/or", the conditional opcodes leave the tested value on the stack. In most cases both branches go directly to a POP_TOP instruction. Since the POP_TOP shouldn't be executed twice, the body of the positive branch has to close with a jump over the other branch even when it is empty. For instance, the simplest case: if a: b compiles to: 1 0 LOAD_NAME 0 (a) 3 JUMP_IF_FALSE 8 (to 14) 6 POP_TOP 2 7 LOAD_NAME 1 (b) 10 POP_TOP 11 JUMP_FORWARD 1 (to 15) >> 14 POP_TOP >> 15 LOAD_CONST 0 (None) 18 RETURN_VALUE this could be simpler and faster: 1 0 LOAD_NAME 0 (a) 3 JUMP_IF_FALSE 8 (to 10) 2 6 LOAD_NAME 1 (b) 9 POP_TOP >> 10 LOAD_CONST 0 (None) 13 RETURN_VALUE Executive summary. Returning only Booleans reduces errors, makes the code easier to review, follows other language norms, and simplifies/speeds-up the generated code. Raymond P.S. Simplifying "and" and "or" may create a need to introduce a conditional operator but that is a discussion for another day. From guido at python.org Tue Sep 20 02:03:01 2005 From: guido at python.org (Guido van Rossum) Date: Mon, 19 Sep 2005 17:03:01 -0700 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <003201c5bd74$15e1c220$99b1958d@oemcomputer> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> Message-ID: On 9/19/05, Raymond Hettinger wrote: > I propose that in Py3.0, the "and" and "or" operators be simplified to > always return a Boolean value instead of returning the last evaluated > argument. While you're at it, maybe we should switch to && and || as well? That's another thing I always mistype when switching between languages... Also, this proposal needs to be considered together with the addition of a proper conditional operator, like x?y:z. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pinard at iro.umontreal.ca Tue Sep 20 02:27:23 2005 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Mon, 19 Sep 2005 20:27:23 -0400 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <003201c5bd74$15e1c220$99b1958d@oemcomputer> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> Message-ID: <20050920002723.GA32179@alcyon.progiciels-bpi.ca> [Raymond Hettinger] > I propose that in Py3.0, the "and" and "or" operators be simplified to > always return a Boolean value instead of returning the last evaluated > argument. > 1) The construct can be error-prone. When an error occurs it can be > invisible to the person who wrote it. I got bitten in published code > that had survived testing and code review: > def real(self): > 'Return a vector with the real part of each input element' > # do not convert integer inputs to floats > return self.map(lambda z: type(z)==types.ComplexType and z.real or z) The real problem, above, is the starve for using `lambda'. The abuse of `and' and `or' is a mere consequence of this. Remove `lambda' from Python, and most reasonable people would spontaneously write more legible code instead. I do not perceive Python as designed to prevent unreasonable people from "outclevering" themselves. > 2) When going back and forth between languages, it is easy to forget > that only Python returns something other than a boolean. C, C++, Lisp, and shells do the same as Python. Or maybe more correctly, Python borrowed short-circuiting booleans from other languages. > 3) Even when it isn't being used, the possibility of non-boolean > return value complicates the bytecode and parser. [...] P.S. > Simplifying "and" and "or" may create a need to introduce a > conditional operator but that is a discussion for another day. If removed, to be re-introduced differently, the bytecode and parser would not have gained simplicity overall. > Executive summary. Returning only Booleans reduces errors, makes > the code easier to review, follows other language norms, and > simplifies/speeds-up the generated code. The summary reduces to: "Returning only Booleans would speed-up the generated code, once in a while." Now, could we evaluate that speed up on the average code, like on the Python library say? It might not be worth the change... -- Fran?ois Pinard http://pinard.progiciels-bpi.ca From jcarlson at uci.edu Tue Sep 20 02:30:34 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 19 Sep 2005 17:30:34 -0700 Subject: [Python-Dev] For/while/if statements/comprehension/generator expressions unification In-Reply-To: <1209206536.20050920044627@sinn.ru> References: <1209206536.20050920044627@sinn.ru> Message-ID: <20050919170044.79C0.JCARLSON@uci.edu> Alexander, The essence of what you have proposed has been proposed (multiple times) before, and I seem to remember it was shot down. The below functions offer the equivalent of list comprehensions with a final post-processing step. def cross(*args): if len(args) == 1: for i in args[0]: yield i elif len(args) == 2: for i in args[0]: for j in args[1]: yield i,j else: for i in args[0]: for j in cross(*args[1:]): yield (i,) + j def cross_with_predicate(*args, **kwargs): if not 'pred' in kwargs: for i in cross(*args): yield i pred = kwargs['pred'] if len(args) > 1: for i in cross(*args): if pred(*i): yield i else: for i in cross(*args): if pred(i): yield i Feel free to use that code and/or modify it to your heart's content (be careful of attempting to simplify cross, . - Josiah Alexander Myodov wrote: > Hello, > > Well, we have "for", "while" and "if". We also have statements, > list generator expressions and list comprehensions. In > comprehensions we have the opportunity to use several for-s and in a > same expression (for i in range (0, 640) for j in range (0, 640)), > in loops we cannot; in comprehensions, we can use if-s to filter the > items; in loops, we cannot, and we have to write an inner comparison instead. > > My opinion is that in so smart language as Python is, it would be > great to have them generalized, so that the features available in > list comprehensions should be available in statements, vice versa. > All three statements (for/while/if) could be joined > together into a single common one, where any of these could be > combined and used multiple times (but for the cases when a > "else"/"elif" etc causes are required, old-school one-statements > should be left); and for such common expressions, they should be > equal for both loop statements and list comprehensions/generator > expressions. > That is, the following loops should be possible: > ------- > # This gives us some sugar to get rid of unnecessary indentations > for x in range (0, 640) for y in range (0, 480): > ------- > for x in range (0, 640) if should_handle_this_column(x) \ > for y in range (0, 480) if should_handle_this_row(y): > ------- > for nX, nY in ( > f(x), f(y) > for x in range (0, 640) if should_handle_this_column(x) > for y in range (0, 480) if should_handle_this_row(y) > ): > ------- > for x in range (0, 640) for y in range (0, 480) while(!timeout()): > ------- > for x in range (0, 640) while(!timeout_pos_in_row()) for y in range (0, 480) > while(!timeout_rows()): > ------- > # And the latest and the hugest one: > for x in range (0, 640) if should_handle_this_column(x) while(!timeout_pos_in_row()) \ > for y in range (0, 480) if should_handle_this_row(y) while(!timeout_rows() : > ------- > # And almost the same as generator expression: > for x, y in ( > f(x), f(y) > for x in range (0, 640) if should_handle_this_column(x) while(!timeout_pos_in_row()) > for y in range (0, 480) if should_handle_this_row(y) while(!timeout_rows() > ) > ------- > > Hope I didn't miss something important... > > -- > With best regards, > Alexander mailto:maa_public at sinn.ru > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jcarlson%40uci.edu From rrr at ronadam.com Tue Sep 20 02:38:25 2005 From: rrr at ronadam.com (Ron Adam) Date: Mon, 19 Sep 2005 20:38:25 -0400 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <003201c5bd74$15e1c220$99b1958d@oemcomputer> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> Message-ID: <432F5A01.1090708@ronadam.com> Raymond Hettinger wrote: > I propose that in Py3.0, the "and" and "or" operators be simplified to > always return a Boolean value instead of returning the last evaluated > argument. > > 1) The construct can be error-prone. When an error occurs it can be > invisible to the person who wrote it. I got bitten in published code > that had survived testing and code review: Yes, I wondered about that possibility myself. +1 > P.S. Simplifying "and" and "or" may create a need to introduce a > conditional operator but that is a discussion for another day. You would still be able to use multiply to covert a comparison to a value. Cheers, Ron From barry at python.org Tue Sep 20 03:08:49 2005 From: barry at python.org (Barry Warsaw) Date: Mon, 19 Sep 2005 21:08:49 -0400 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> Message-ID: <1127178529.1211.0.camel@geddy.wooz.org> On Mon, 2005-09-19 at 20:03, Guido van Rossum wrote: > While you're at it, maybe we should switch to && and || as well? > That's another thing I always mistype when switching between > languages... Please no! 'and' and 'or' is so readably beautiful. > Also, this proposal needs to be considered together with the addition > of a proper conditional operator, like x?y:z. Definitely. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050919/7cb6dcea/attachment.pgp From jonathan-lists at cleverdevil.org Tue Sep 20 02:13:05 2005 From: jonathan-lists at cleverdevil.org (Jonathan LaCour) Date: Mon, 19 Sep 2005 20:13:05 -0400 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <003201c5bd74$15e1c220$99b1958d@oemcomputer> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> Message-ID: <609EDD13-EB19-441D-92FB-844507669C08@cleverdevil.org> > P.S. Simplifying "and" and "or" may create a need to introduce a > conditional operator but that is a discussion for another day. While I don't disagree with some of your main points, I do think that your proposal would eliminate a natural and easy to understand use of the current behavior of "or" that I tend to use quite a bit. Your proposal would break a lot of code, and I can't think of a better "conditional operator" than the one thats already there. I often find myself using 'or' to conditionally select a meaningful value in the absence of a real value: person = Person.fetch(id=5) name = person.name or 'John Doe' birth_date = person.birth_date or '00-00-0000' ssn = person.social_security or 'not provided' logger.log_msg('%s born on %s with SSN %s' % (name, birth_date, ssn)) To me, its just as natural as: { 'key' : value }.get('not_there', 'default') but more general purpose. I like the current behavior, and I think it makes a whole lot of sense. Jonathan LaCour http://cleverdevil.org From andrewm at object-craft.com.au Tue Sep 20 04:28:01 2005 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Tue, 20 Sep 2005 12:28:01 +1000 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <609EDD13-EB19-441D-92FB-844507669C08@cleverdevil.org> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <609EDD13-EB19-441D-92FB-844507669C08@cleverdevil.org> Message-ID: <20050920022801.EAC116F4AEC@longblack.object-craft.com.au> >While I don't disagree with some of your main points, I do think that >your proposal would eliminate a natural and easy to understand use of >the current behavior of "or" that I tend to use quite a bit. Your >proposal would break a lot of code, and I can't think of a better >"conditional operator" than the one thats already there. > >I often find myself using 'or' to conditionally select a meaningful >value in the absence of a real value: I agree. I find I often have an object with an optional friendly name (label) and a manditory system name. So this sort of thing becomes common: '%s blah blah' % (foo.label or foo.name) The if-else-expression alternative works, but isn't quite as readable: '%s blah blah' % (foo.label ? foo.label : foo.name) -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From greg.ewing at canterbury.ac.nz Tue Sep 20 04:30:43 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 20 Sep 2005 14:30:43 +1200 Subject: [Python-Dev] os.path.diff(path1, path2) In-Reply-To: <1127044776.14284.9.camel@mulkomp.mst-plus> References: <20050913031157.46635.qmail@web51809.mail.yahoo.com> <1127044776.14284.9.camel@mulkomp.mst-plus> Message-ID: <432F7453.1050105@canterbury.ac.nz> Matthias Andreas Benkard wrote: > /home -> usr/home > /usr > /usr/home > > What does os.path.diff("/home/", "/usr/") yield? "../usr/", I would > presume? But that's obviously wrong: IMO, the relpath method should just work textually on the pathnames. It's up to the user to ensure it makes sense to do so, e.g. by resolving symlinks beforehand if necessary. The alternative would be for relpath to do this itself, but that would make it very unusual compared to the other path-munging functions, none of which touch the file system. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From tdelaney at avaya.com Tue Sep 20 04:36:42 2005 From: tdelaney at avaya.com (Delaney, Timothy (Tim)) Date: Tue, 20 Sep 2005 12:36:42 +1000 Subject: [Python-Dev] os.path.diff(path1, path2) Message-ID: <2773CAC687FD5F4689F526998C7E4E5F4DB633@au3010avexu1.global.avaya.com> Greg Ewing wrote: > Matthias Andreas Benkard wrote: > >> /home -> usr/home >> /usr >> /usr/home >> >> What does os.path.diff("/home/", "/usr/") yield? "../usr/", I would >> presume? But that's obviously wrong: > > IMO, the relpath method should just work textually on > the pathnames. It's up to the user to ensure it makes > sense to do so, e.g. by resolving symlinks beforehand > if necessary. In addition, I would presume that relpath would just return the absolute path if passed an absolute path as the second parameter. So the above would simply return "/usr/" IMO. Tim Delaney From fdrake at acm.org Tue Sep 20 04:43:18 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Mon, 19 Sep 2005 22:43:18 -0400 Subject: [Python-Dev] list splicing In-Reply-To: <002b01c5bd70$b4a49940$99b1958d@oemcomputer> References: <002b01c5bd70$b4a49940$99b1958d@oemcomputer> Message-ID: <200509192243.18380.fdrake@acm.org> On Monday 19 September 2005 19:20, Raymond Hettinger wrote: > It might have a chance of acceptance this time if the proponents stick > with unpacking at the end: a,b,*c=sometup instead of a,*b,c=sometup. > The latter has usually gotten shot down quickly, taking the former down > with it. True. I'd be happy with it either way, since most of the time a, b, *rest = stuff_to_unpack is all I really want. -Fred -- Fred L. Drake, Jr. From greg.ewing at canterbury.ac.nz Tue Sep 20 04:55:02 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 20 Sep 2005 14:55:02 +1200 Subject: [Python-Dev] removing nested tuple function parameters In-Reply-To: References: <432DE8B2.4020701@gmail.com> <002001c5bcae$8639e640$99b1958d@oemcomputer> <20050919003740.GA5918@alcyon.progiciels-bpi.ca> <432E4BEA.5000506@canterbury.ac.nz> Message-ID: <432F7A06.10607@canterbury.ac.nz> Steven Bethard wrote: > I guess if tuple unpacking in function parameters goes away, I think > we should change the __getitem__ machinery so that: > > obj[x1, x2, ..., xN] > > is translated to: > > obj.__getitem__(x1, x2, ..., xN) > > where __getitem__ would now have to take a *args when called with tuples. That might be a reasonable idea to consider for Py3k in any case. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Tue Sep 20 05:07:06 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 20 Sep 2005 15:07:06 +1200 Subject: [Python-Dev] removing nested tuple function parameters In-Reply-To: References: <432DE8B2.4020701@gmail.com> <002001c5bcae$8639e640$99b1958d@oemcomputer> <20050919003740.GA5918@alcyon.progiciels-bpi.ca> <432E4BEA.5000506@canterbury.ac.nz> Message-ID: <432F7CDA.5040105@canterbury.ac.nz> Terry Reedy wrote: > I consider the current situation to be a consistency feature. To a first > approximation, Python function calls 'pass' objects by name-binding: > > param_name_list = arg_object_list > > Disabling structure unpacking in this assignment would make the language > slightly more complex. But it's a rather selective kind of consistency. To be truly consistent in this sense, arbitrary lvalues would have to be allowed in the parameter list. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Tue Sep 20 05:32:30 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 20 Sep 2005 15:32:30 +1200 Subject: [Python-Dev] removing nested tuple function parameters In-Reply-To: <006801c5bd45$bda753a0$6402a8c0@arkdesktop> References: <006801c5bd45$bda753a0$6402a8c0@arkdesktop> Message-ID: <432F82CE.5090308@canterbury.ac.nz> Andrew Koenig wrote: > One other reason: It is possible to imagine using the feature to catch some > type errors at the point of call, rather than having to get into the > function itself before detecting them. Not a big deal - you just need to look one line further up in the traceback to find where the call was. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From aahz at pythoncraft.com Tue Sep 20 05:57:38 2005 From: aahz at pythoncraft.com (Aahz) Date: Mon, 19 Sep 2005 20:57:38 -0700 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <609EDD13-EB19-441D-92FB-844507669C08@cleverdevil.org> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <609EDD13-EB19-441D-92FB-844507669C08@cleverdevil.org> Message-ID: <20050920035738.GA24966@panix.com> On Mon, Sep 19, 2005, Jonathan LaCour wrote: > Raymond Hettinger: >> >> P.S. Simplifying "and" and "or" may create a need to introduce a >> conditional operator but that is a discussion for another day. > > While I don't disagree with some of your main points, I do think that > your proposal would eliminate a natural and easy to understand use of > the current behavior of "or" that I tend to use quite a bit. Your > proposal would break a lot of code, and I can't think of a better > "conditional operator" than the one thats already there. > > I often find myself using 'or' to conditionally select a meaningful > value in the absence of a real value: > > person = Person.fetch(id=5) > name = person.name or 'John Doe' > birth_date = person.birth_date or '00-00-0000' > ssn = person.social_security or 'not provided' While I'm in philosophical agreement with Raymond, it's also true that my current company's code is littered with constructs like yours. So I have to say that it would break too much code. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything. From greg.ewing at canterbury.ac.nz Tue Sep 20 06:07:47 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 20 Sep 2005 16:07:47 +1200 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <003201c5bd74$15e1c220$99b1958d@oemcomputer> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> Message-ID: <432F8B13.9060500@canterbury.ac.nz> Raymond Hettinger wrote: > I propose that in Py3.0, the "and" and "or" operators be simplified to > always return a Boolean value instead of returning the last evaluated > argument. But then I would no longer be able to write foo = something or default_value which is one of my favourite Pythonisms! > 3) Even when it isn't being used, the possibility of non-boolean return > value complicates the bytecode and parser. To allow for "and/or", the > conditional opcodes leave the tested value on the stack. In most cases > both branches go directly to a POP_TOP instruction. Since the POP_TOP > shouldn't be executed twice, the body of the positive branch has to > close with a jump over the other branch even when it is empty. The solution to that is for the normal conditional opcodes to pop the stack, and to introduce separate bytecodes for implementing 'and' and 'or'. No need to change the language because of this. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Tue Sep 20 06:07:49 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 20 Sep 2005 16:07:49 +1200 Subject: [Python-Dev] os.path.diff(path1, path2) In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F4DB633@au3010avexu1.global.avaya.com> References: <2773CAC687FD5F4689F526998C7E4E5F4DB633@au3010avexu1.global.avaya.com> Message-ID: <432F8B15.9090903@canterbury.ac.nz> Delaney, Timothy (Tim) wrote: > > In addition, I would presume that relpath would just return the absolute > path if passed an absolute path as the second parameter. I don't think so. Wouldn't you expect the result of relpath("/usr/local/foo/bax/grump.c", "/usr/local/flump/grump.c") to be "../../flump/grump.c" rather than "/usr/local/flump/grump.c"? > > So the above would simply return "/usr/" IMO. > > Tim Delaney > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/greg.ewing%40canterbury.ac.nz -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From bcannon at gmail.com Tue Sep 20 06:11:37 2005 From: bcannon at gmail.com (Brett Cannon) Date: Mon, 19 Sep 2005 21:11:37 -0700 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <1127178529.1211.0.camel@geddy.wooz.org> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <1127178529.1211.0.camel@geddy.wooz.org> Message-ID: On 9/19/05, Barry Warsaw wrote: > On Mon, 2005-09-19 at 20:03, Guido van Rossum wrote: > > > While you're at it, maybe we should switch to && and || as well? > > That's another thing I always mistype when switching between > > languages... > > Please no! 'and' and 'or' is so readably beautiful. > I completely agree. > > Also, this proposal needs to be considered together with the addition > > of a proper conditional operator, like x?y:z. > > Definitely. Yep. -Brett From bcannon at gmail.com Tue Sep 20 06:19:16 2005 From: bcannon at gmail.com (Brett Cannon) Date: Mon, 19 Sep 2005 21:19:16 -0700 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <003201c5bd74$15e1c220$99b1958d@oemcomputer> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> Message-ID: On 9/19/05, Raymond Hettinger wrote: > I propose that in Py3.0, the "and" and "or" operators be simplified to > always return a Boolean value instead of returning the last evaluated > argument. > > 1) The construct can be error-prone. When an error occurs it can be > invisible to the person who wrote it. I got bitten in published code > that had survived testing and code review: > > def real(self): > 'Return a vector with the real part of each input element' > # do not convert integer inputs to floats > return self.map(lambda z: type(z)==types.ComplexType and z.real or > z) > > The code fails silently when z is (0+4i). It took a good while to trace > down a user reported error (when Matlab results disagreed with my matrix > module results) and determine that the real() method contained an error. > Even when traced down, I found it hard to see the error in the code. > Now that I know what to look for, it has not happened again, but I do > always have to stare hard at any "and/or" group to mentally verify each > case. > I agree. While I have used the short-circuiting and the return value myself, it has often been for just fun one-liners or when it is extemely simple and I was just too lazy to write out an 'if' statement. A conditional operator is much more proper for this (and, as Guido has said in another email, should be considered as well). > > 2) When going back and forth between languages, it is easy to forget > that only Python returns something other than a boolean. > > > 3) Even when it isn't being used, the possibility of non-boolean return > value complicates the bytecode and parser. To allow for "and/or", the > conditional opcodes leave the tested value on the stack. In most cases > both branches go directly to a POP_TOP instruction. Since the POP_TOP > shouldn't be executed twice, the body of the positive branch has to > close with a jump over the other branch even when it is empty. For > instance, the simplest case: > > if a: > b > > compiles to: > > 1 0 LOAD_NAME 0 (a) > 3 JUMP_IF_FALSE 8 (to 14) > 6 POP_TOP > > 2 7 LOAD_NAME 1 (b) > 10 POP_TOP > 11 JUMP_FORWARD 1 (to 15) > >> 14 POP_TOP > >> 15 LOAD_CONST 0 (None) > 18 RETURN_VALUE > > this could be simpler and faster: > > 1 0 LOAD_NAME 0 (a) > 3 JUMP_IF_FALSE 8 (to 10) > 2 6 LOAD_NAME 1 (b) > 9 POP_TOP > >> 10 LOAD_CONST 0 (None) > 13 RETURN_VALUE > > > Executive summary. Returning only Booleans reduces errors, makes the > code easier to review, follows other language norms, and > simplifies/speeds-up the generated code. Glad the simplification of the bytecode is not the only part of your proposal. =) -Brett From raymond.hettinger at verizon.net Tue Sep 20 06:57:17 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Tue, 20 Sep 2005 00:57:17 -0400 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <432F8B13.9060500@canterbury.ac.nz> Message-ID: <007101c5bd9f$c6ce8480$99b1958d@oemcomputer> > > 3) Even when it isn't being used, the possibility of non-boolean return > > value complicates the bytecode and parser. To allow for "and/or", the > > conditional opcodes leave the tested value on the stack. In most cases > > both branches go directly to a POP_TOP instruction. Since the POP_TOP > > shouldn't be executed twice, the body of the positive branch has to > > close with a jump over the other branch even when it is empty. > > The solution to that is for the normal conditional opcodes > to pop the stack, and to introduce separate bytecodes for > implementing 'and' and 'or'. You're right. The bytecode code be fixed-up right now with no change to the language :-) That leaves error reduction and clarity as the main motivations for changing 'and' and 'or' to act like '&&' and '||' and for introducing a conditional operator to handle everyone's favorite use cases. Raymond From nidoizo at yahoo.com Tue Sep 20 08:00:30 2005 From: nidoizo at yahoo.com (Nicolas Fleury) Date: Tue, 20 Sep 2005 02:00:30 -0400 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <003201c5bd74$15e1c220$99b1958d@oemcomputer> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> Message-ID: Raymond Hettinger wrote: > I propose that in Py3.0, the "and" and "or" operators be simplified to > always return a Boolean value instead of returning the last evaluated > argument. Please no. I find things like: def __cmp__(self, other): return (cmp(self.a, other.a) or cmp(self.b, other.b) or cmp(self.c, other.c)) somehow elegant... But I don't know how this can be useful for "and" however. > 1) The construct can be error-prone. When an error occurs it can be > invisible to the person who wrote it. I got bitten in published code > that had survived testing and code review: > > def real(self): > 'Return a vector with the real part of each input element' > # do not convert integer inputs to floats > return self.map(lambda z: type(z)==types.ComplexType and z.real or > z) As others pointed, if lambda is to be possibly removed in py3.0, it would be better to have an example not using it. FWIW, I've never had a problem with that, but however I use the feature... Quite the opposite actually, I even though about overriding and and or to build event-driving expression, which I can't... well, forget about that;) > 2) When going back and forth between languages, it is easy to forget > that only Python returns something other than a boolean. Well, there's a lot of things I miss from Python when going back and forth between languages. You need a better argument to convince me to also miss that one in Python;) I think it's not reasonable to remove the feature without adding a ?: syntax. I personally like the status quo. Regards, Nicolas From maa_public at sinn.ru Tue Sep 20 08:26:59 2005 From: maa_public at sinn.ru (Alexander Myodov) Date: Tue, 20 Sep 2005 11:56:59 +0530 Subject: [Python-Dev] For/while/if statements/comprehension/generator expressions unification Message-ID: <8410256206.20050920115659@sinn.ru> Hello Josiah, JC> Alexander, JC> The essence of what you have proposed has been proposed (multiple times) before, JC> and I seem to remember it was shot down. To increase my understanding of Python-way, can you (or someone else) explain the reasons why such proposals were rejected? JC> The below functions offer the equivalent of list comprehensions with a JC> final post-processing step. Well, what I was suggesting is not just a cross of two lists but the syntax sugar which would make statements more consistent to the generators/comprehensions and also give some new opportunities. I think that my large proposal can be splitted to several almost-independent ones, each carrying separate features, able to be implemented independently and worth independent reconsidering: 1. Bring 'if'-s from generator/comprehension 'for' syntax to 'for' statement. That's truly inconsistent that one may write list2 = [i for i in list if cond(i)] but cannot write for i in list if cond(i): 2. Bring "several for-s in a row" ability from generators/comprehensions to the statement (and expand it to "several for-s and if-s", of course). We can write list2 = [f(i, j) for i in list1 for j in list2] but cannot write for i in list1 for j in list2: yield f(i, j) That looks inconsistent as well. Yes, for this little case we could do some kind of cross, but what if there are more loops, and/or some of them require filtering by if-s? 3. Bring 'while' into the loops, both statements and iterators. Now no need to worry about "hidden-goto" 'break', especially for the cases of nested loops, while it can be easily checked right inside the looping condition? Now we can easily do for i in largelist while !found: , isn't it nice and simple? -- ? ?????????, Alexander mailto:maa_public at sinn.ru From gisle at ActiveState.com Tue Sep 20 08:44:39 2005 From: gisle at ActiveState.com (Gisle Aas) Date: 19 Sep 2005 23:44:39 -0700 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <003201c5bd74$15e1c220$99b1958d@oemcomputer> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> Message-ID: "Raymond Hettinger" writes: > 2) When going back and forth between languages, it is easy to forget > that only Python returns something other than a boolean. Perl certainly works the same way and I've never heared anybody have problems with that, but that might be because Perl do have the expr?v1:v2 triary operator as well. Most Perl programs use constructs like "a = v1 or v2 or v3" (or even "a ||= v") to its advantage. Regards, Gisle From mwh at python.net Tue Sep 20 10:02:41 2005 From: mwh at python.net (Michael Hudson) Date: Tue, 20 Sep 2005 09:02:41 +0100 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: (Guido van Rossum's message of "Mon, 19 Sep 2005 17:03:01 -0700") References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> Message-ID: <2m7jdckvz2.fsf@starship.python.net> Guido van Rossum writes: > On 9/19/05, Raymond Hettinger wrote: >> I propose that in Py3.0, the "and" and "or" operators be simplified to >> always return a Boolean value instead of returning the last evaluated >> argument. > > While you're at it, maybe we should switch to && and || as well? > That's another thing I always mistype when switching between > languages... You're joking, surely? > Also, this proposal needs to be considered together with the addition > of a proper conditional operator, like x?y:z. I think this discussion has been had before, you know. Cheers, mwh -- > Look I don't know. Thankyou everyone for arguing me round in > circles. No need for thanks, ma'am; that's what we're here for. -- LNR & Michael M Mason, cam.misc From mwh at python.net Tue Sep 20 10:11:36 2005 From: mwh at python.net (Michael Hudson) Date: Tue, 20 Sep 2005 09:11:36 +0100 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <003201c5bd74$15e1c220$99b1958d@oemcomputer> (Raymond Hettinger's message of "Mon, 19 Sep 2005 19:44:32 -0400") References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> Message-ID: <2mzmq8jgzr.fsf@starship.python.net> "Raymond Hettinger" writes: > I propose that in Py3.0, the "and" and "or" operators be simplified to > always return a Boolean value instead of returning the last evaluated > argument. -1 > 2) When going back and forth between languages, it is easy to forget > that only Python returns something other than a boolean. As others point out, this isn't true. > 3) Even when it isn't being used, the possibility of non-boolean return > value complicates the bytecode and parser. To allow for "and/or", the > conditional opcodes leave the tested value on the stack. Huh? I thought this was for chained expressions? Things like 1 < x < 3. I _like_ the explanation of 'and' and 'or' as they are now. They are basically control flow constructs -- and have to be to get short-circuiting to work -- and adding a coercion to bool at the end seems to add complexity, not reduce it (on some levels, anyway). > P.S. Simplifying "and" and "or" may create a need to introduce a > conditional operator but that is a discussion for another day. ... which was in the past, I thought. Cheers, mwh -- The meaning of "brunch" is as yet undefined. -- Simon Booth, ucam.chat From ncoghlan at gmail.com Tue Sep 20 10:58:24 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 20 Sep 2005 18:58:24 +1000 Subject: [Python-Dev] For/while/if statements/comprehension/generator expressions unification In-Reply-To: <8410256206.20050920115659@sinn.ru> References: <8410256206.20050920115659@sinn.ru> Message-ID: <432FCF30.8010001@gmail.com> Alexander Myodov wrote: > To increase my understanding of Python-way, can you (or someone else) > explain the reasons why such proposals were rejected? I wasn't around for the discussion Josiah refers to, but I would guess it is because they don't improve the power or readability of the language, and may in fact be significantly less readable than simply nesting the corresponding statements. In particular, with the current syntax, the local variables bound by the for statement are clearly visible immediately after the 'for' keyword. If multiple instances of the keyword were allowed in a single for statement, the names being bound could be hidden after an arbitrarily complex expression. This 'hidden local' problem is one of the concerns with the current behaviour where list comprehensions make their loop variables visible in the surrounding scope (this behaviour will be fixed in Py3k, and was deliberately avoided for generator expressions). > 1. Bring 'if'-s from generator/comprehension 'for' syntax to 'for' > statement. That's truly inconsistent that one may write > > list2 = [i for i in list if cond(i)] > > but cannot write > > for i in list if cond(i): No, instead you write: for i in list: if cond(i): The order of execution of loops and conditionals is entirely unambiguous, as it is shown explicitly by the nesting. Python has never been shy about requiring vertical and horizontal whitespace at the statement level in order to make the sequence of control flow more explicit. The rules are different for list comprehensions and generator expressions, because whitespace is only used to delimit tokens inside expressions, rather than being a token in its own right as it can be at the statement level. However, the order of execution of these expressions can be disambiguated by conversion to the nested statement version of the same code. > 2. Bring "several for-s in a row" ability from > generators/comprehensions to the statement (and expand it to "several > for-s and if-s", of course). We can write > > list2 = [f(i, j) for i in list1 for j in list2] > > but cannot write > > for i in list1 for j in list2: > yield f(i, j) > That looks inconsistent as well. If the apparent inconsistency genuinely bothers you, you can always write the statement version as: for x in (f(i, j) for i list1 for j in list2): yield x Note that this preserves the fact that only 'x' is bound in the surrounding scope, and this name is prominent in the 'for' statement. > Yes, for this little case we could do some kind of cross, but what if > there are more loops, and/or some of them require filtering by if-s? Then readers of the code could probably be helped out if some of the intermediate steps were named to explain what's going on. Alternately, just use a genexp as I show above. > 3. Bring 'while' into the loops, both statements and iterators. Now no > need to worry about "hidden-goto" 'break', especially for the cases of > nested loops, while it can be easily checked right inside the looping > condition? 'break' is no more a 'hidden-goto' than any other form of non-sequential control flow (like say, conditionals or loops) is a goto. The problem with having an actual goto is that it can go *anywhere*, leading to a high chance of writing spaghetti code - this is not the case when using break, as you can't go anywhere other than the statement immediately following the loop. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From reinhold-birkenfeld-nospam at wolke7.net Tue Sep 20 11:22:29 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Tue, 20 Sep 2005 11:22:29 +0200 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <003201c5bd74$15e1c220$99b1958d@oemcomputer> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> Message-ID: Raymond Hettinger wrote: > I propose that in Py3.0, the "and" and "or" operators be simplified to > always return a Boolean value instead of returning the last evaluated > argument. No, please not. It's useful sometimes and doesn't hurt most times. > 1) The construct can be error-prone. When an error occurs it can be > invisible to the person who wrote it. I got bitten in published code > that had survived testing and code review: > > def real(self): > 'Return a vector with the real part of each input element' > # do not convert integer inputs to floats > return self.map(lambda z: type(z)==types.ComplexType and z.real or > z) I'm surprised you wrote that in the first place. The "and/or conditional" is one of the few occurences where one will carefully look for false values in the "and" part. > The code fails silently when z is (0+4i). It took a good while to trace > down a user reported error (when Matlab results disagreed with my matrix > module results) and determine that the real() method contained an error. > Even when traced down, I found it hard to see the error in the code. > Now that I know what to look for, it has not happened again, but I do > always have to stare hard at any "and/or" group to mentally verify each > case. [...] > P.S. Simplifying "and" and "or" may create a need to introduce a > conditional operator but that is a discussion for another day. Exactly. A conditional was turned down some time ago, for good reasons. Reinhold -- Mail address is perfectly valid! From ncoghlan at gmail.com Tue Sep 20 11:33:19 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 20 Sep 2005 19:33:19 +1000 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <2mzmq8jgzr.fsf@starship.python.net> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2mzmq8jgzr.fsf@starship.python.net> Message-ID: <432FD75F.5030703@gmail.com> Michael Hudson wrote: > I _like_ the explanation of 'and' and 'or' as they are now. They are > basically control flow constructs -- and have to be to get > short-circuiting to work -- and adding a coercion to bool at the end > seems to add complexity, not reduce it (on some levels, anyway). Agreed. However, Raymond and Brett seem to have some ideas about optimising the normal comparison bytecode without changing the semantics, which would be good. >>P.S. Simplifying "and" and "or" may create a need to introduce a >>conditional operator but that is a discussion for another day. > > > ... which was in the past, I thought. C'mon, PEP 308 was fun for everybody };) The ease of writing buggy code when trying to fake a conditional operator using and/or does seem to cause the question to keep coming up despite the fate of PEP 308, though. But the only way I can ever see a conditional operator happening is if Guido simply selects which of the top 4** PEP 308 solutions he likes best and says "make it so!". Cheers, Nick. ** I say top 4, because there was daylight between the votes for the first 4 choices and all of the other PEP 308 options. And many of the pros and cons of those 4 choices are so subjective that I wouldn't expect further discussion to significantly change anyone's opinion. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From raymond.hettinger at verizon.net Tue Sep 20 12:40:35 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Tue, 20 Sep 2005 06:40:35 -0400 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <2mzmq8jgzr.fsf@starship.python.net> Message-ID: <000501c5bdcf$be32b000$6522c797@oemcomputer> > > 2) When going back and forth between languages, it is easy to forget > > that only Python returns something other than a boolean. > > As others point out, this isn't true. In C, C++, C#, Java, and JavaScript, what do you get when you print the result of 3 || 10? Raymond From fredrik at pythonware.com Tue Sep 20 13:11:10 2005 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 20 Sep 2005 13:11:10 +0200 Subject: [Python-Dev] "and" and "or" operators in Py3.0 References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: Michael Hudson wrote: >> While you're at it, maybe we should switch to && and || as well? >> That's another thing I always mistype when switching between >> languages... > > You're joking, surely? for Python 3000, I'd recommend switching to "and then" and "or else" instead of the current ambiguous single-keyword versions. From gmccaughan at synaptics-uk.com Tue Sep 20 13:16:53 2005 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Tue, 20 Sep 2005 12:16:53 +0100 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <000501c5bdcf$be32b000$6522c797@oemcomputer> References: <000501c5bdcf$be32b000$6522c797@oemcomputer> Message-ID: <200509201216.54415.gmccaughan@synaptics-uk.com> On Tuesday 2005-09-20 11:40, Raymond Hettinger wrote: > > > 2) When going back and forth between languages, it is easy to forget > > > that only Python returns something other than a boolean. > > > > As others point out, this isn't true. > > In C, C++, C#, Java, and JavaScript, what do you get when you print the > result of 3 || 10? JavaScript does the same as Python, according to section 11.11 of the EMCA standard as of December 1999, which is what I have to hand. The others you mention don't, because doing so would fit very uncomfortably with their static type systems. Those are not the only languages that exist other than Python. Perl and Ruby both have Python-like behaviour in this respect. So do all the Lisp variants I can readily check (CL, elisp, Scheme, Dylan). So does Lua. I can't, offhand, think of any dynamically typed language that doesn't behave this way. Oh, except Smalltalk, which has a radically different approach to the whole business. -- g From blais at furius.ca Tue Sep 20 14:41:15 2005 From: blais at furius.ca (Martin Blais) Date: Tue, 20 Sep 2005 08:41:15 -0400 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <20050920022801.EAC116F4AEC@longblack.object-craft.com.au> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <609EDD13-EB19-441D-92FB-844507669C08@cleverdevil.org> <20050920022801.EAC116F4AEC@longblack.object-craft.com.au> Message-ID: <8393fff0509200541365bc387@mail.gmail.com> On 9/19/05, Andrew McNamara wrote: > > I agree. I find I often have an object with an optional friendly name > (label) and a manditory system name. So this sort of thing becomes common: > > '%s blah blah' % (foo.label or foo.name) > > The if-else-expression alternative works, but isn't quite as readable: > > '%s blah blah' % (foo.label ? foo.label : foo.name) Where does this work? From anthony at python.org Tue Sep 20 15:06:50 2005 From: anthony at python.org (Anthony Baxter) Date: Tue, 20 Sep 2005 23:06:50 +1000 Subject: [Python-Dev] IMPORTANT: release24-maint branch is FROZEN from 2005-09-21 00:00 UTC for 2.4.2 Message-ID: <200509202306.54251.anthony@python.org> Starting in about 11 hours time, the release24-maint branch is FROZEN for the 2.4.2c1 release. The freeze will last for around a day, and then we're in a state of mostly-frozen for another week, until 2.4.2 (final). During that week, please don't check things into the branch unless you check with me first. Let's make this a nice painless release. I'll send another message once 2.4.2 is done. Thanks, Anthony -- Anthony Baxter It's never too late to have a happy childhood. From mulk at gmx.net Tue Sep 20 15:49:02 2005 From: mulk at gmx.net (Matthias Andreas Benkard) Date: Tue, 20 Sep 2005 15:49:02 +0200 Subject: [Python-Dev] os.path.diff(path1, path2) In-Reply-To: <432F7453.1050105@canterbury.ac.nz> References: <20050913031157.46635.qmail@web51809.mail.yahoo.com> <1127044776.14284.9.camel@mulkomp.mst-plus> <432F7453.1050105@canterbury.ac.nz> Message-ID: <1127224142.14254.20.camel@mulkomp.mst-plus> Hi, > IMO, the relpath method should just work textually on > the pathnames. It's up to the user to ensure it makes > sense to do so, e.g. by resolving symlinks beforehand > if necessary. I guess so. Don't forget to mention this quirk in the docs, though :) > The alternative would be for relpath to do this itself, > but that would make it very unusual compared to the > other path-munging functions, none of which touch the file > system. Yes, and this behaviour would make it unusable for virtual file systems within the application, too. Path names needn't refer to the local file system, right? Then again, the `os' module _is_ platform-specific, as is `os.path', so one shouldn't use it for those things either way. In fact, relpath _cannot_ dereference symlinks on its own, as it has no way of knowing the absolute pathname if only relative pathnames are given as arguments (for example, relpath("a/b/c", "../d")). But this begs the question: What is relpath good for, anyway? And what are the hidden costs? One-time scripts might benefit from it, but having this function in the standard library could make application developers use it without considering the consequences, resulting in strange bugs that occur on one machine, but not on another. The more I think about it, the more I'm afraid that this might lead to severe security vulnerabilities if misused in tools run by the system administrator. Better place a big, fat warning in the docs, then. Bye, Matthias ________________________________________________________________________ Matthias Andreas Benkard, Anarchokommunist und Pythonprogrammierer Pers?nliche Website: http://www.mulk.de.vu/ Pers?nlicher Weblog: http://www.kompottkin.de.vu/ Flames bitte nach /dev/null schicken. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050920/278daca3/attachment.pgp From ldlandis at gmail.com Tue Sep 20 17:45:00 2005 From: ldlandis at gmail.com (LD 'Gus' Landis) Date: Tue, 20 Sep 2005 10:45:00 -0500 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> Message-ID: Would this change break x < y < z ? Hope not. +1 on x?y:z -1 on && || replacing and/or unless and/or kept with current semantics and && || introduced Raymond's boolean idea, but then -1 too many unrelated but different spellings On 9/19/05, Guido van Rossum wrote: > On 9/19/05, Raymond Hettinger wrote: > > I propose that in Py3.0, the "and" and "or" operators be simplified to > > always return a Boolean value instead of returning the last evaluated > > argument. > > While you're at it, maybe we should switch to && and || as well? > That's another thing I always mistype when switching between > languages... > > Also, this proposal needs to be considered together with the addition > of a proper conditional operator, like x?y:z. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ldlandis%40gmail.com > -- LD Landis - N0YRQ - from the St Paul side of Minneapolis From bjourne at gmail.com Tue Sep 20 18:02:38 2005 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Tue, 20 Sep 2005 18:02:38 +0200 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: <740c3aec05092009025c929a0d@mail.gmail.com> > >> While you're at it, maybe we should switch to && and || as well? > >> That's another thing I always mistype when switching between > >> languages... > > > > You're joking, surely? > > for Python 3000, I'd recommend switching to "and then" and "or else" instead > of the current ambiguous single-keyword versions. Wouldn't it be possible to omit "and" and "or" to just "then" and "else"? x = 3 and 7 or 44 x = 3 and then 7 or else 44 x = 3 then 7 else 44 And then have another set of and and or for non short-circuiting? -- mvh Bj?rn From guido at python.org Tue Sep 20 18:04:53 2005 From: guido at python.org (Guido van Rossum) Date: Tue, 20 Sep 2005 09:04:53 -0700 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <2m7jdckvz2.fsf@starship.python.net> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: On 9/20/05, Michael Hudson wrote: > Guido van Rossum writes: > > > On 9/19/05, Raymond Hettinger wrote: > >> I propose that in Py3.0, the "and" and "or" operators be simplified to > >> always return a Boolean value instead of returning the last evaluated > >> argument. > > > > While you're at it, maybe we should switch to && and || as well? > > That's another thing I always mistype when switching between > > languages... > > You're joking, surely? I wasn't, but I wasn't pushing for it either. Consider it withdrawn given the response. > > Also, this proposal needs to be considered together with the addition > > of a proper conditional operator, like x?y:z. > > I think this discussion has been had before, you know. Unfortunately, if we were to accept Raymond's proposal, we'd have to revisit the discussion, since his proposal removes several ways we currently avoid the need. In fact, I think Raymond's example is more properly considered an argument for adding a conditional expression than for removing the current behavior of the and/or shortcut operators; had we had a conditional expression, he wouldn't have tried to use the "x and y or z" syntax that bit him. Given this realization, I'm now -1 on Raymond's idea, and +1 on adding a conditional expression. I believe (y if x else z) was my favorite last time, wasn't it? I've changed the subject accordingly. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From skip at pobox.com Tue Sep 20 18:07:21 2005 From: skip at pobox.com (Skip Montanaro) Date: Tue, 20 Sep 2005 16:07:21 +0000 (UTC) Subject: [Python-Dev] "and" and "or" operators in Py3.0 References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <432F8B13.9060500@canterbury.ac.nz> Message-ID: Greg Ewing canterbury.ac.nz> writes: > > Raymond Hettinger wrote: > > I propose that in Py3.0, the "and" and "or" operators be simplified to > > always return a Boolean value instead of returning the last evaluated > > argument. > > But then I would no longer be able to write > > foo = something or default_value > > which is one of my favourite Pythonisms! Same here. I use this frequently. In fact, I sometimes use it in preference to a default param in {}.get(): foo = somedict.get("blarg") or expensive_default() That way the expensive default isn't calculated unless you need it. (Of course, the dict has to only store values that evaluate to non-False, which might not be possible in all situations.) Skip From skip at pobox.com Tue Sep 20 18:10:27 2005 From: skip at pobox.com (Skip Montanaro) Date: Tue, 20 Sep 2005 16:10:27 +0000 (UTC) Subject: [Python-Dev] "and" and "or" operators in Py3.0 References: <432F8B13.9060500@canterbury.ac.nz> <007101c5bd9f$c6ce8480$99b1958d@oemcomputer> Message-ID: > That leaves error reduction and clarity as the main motivations for > changing 'and' and 'or' to act like '&&' and '||' and for introducing a > conditional operator to handle everyone's favorite use cases. I predict that people who use "and" and "or" correctly today will start confusing "&&" with "&" and "||" with "|", which in many instances will pass silently. In fact, I'll wager that lots of people who are familiar with boolean and/or will have never used bitwise and/or. Skip From mwh at python.net Tue Sep 20 18:34:23 2005 From: mwh at python.net (Michael Hudson) Date: Tue, 20 Sep 2005 17:34:23 +0100 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: (Guido van Rossum's message of "Tue, 20 Sep 2005 09:04:53 -0700") References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: <2mvf0vk8a8.fsf@starship.python.net> Guido van Rossum writes: > Unfortunately, if we were to accept Raymond's proposal, we'd have to > revisit the discussion, since his proposal removes several ways we > currently avoid the need. > > In fact, I think Raymond's example is more properly considered an > argument for adding a conditional expression than for removing the > current behavior of the and/or shortcut operators; had we had a > conditional expression, he wouldn't have tried to use the "x and y or > z" syntax that bit him. Indeed. > Given this realization, I'm now -1 on Raymond's idea, and +1 on adding > a conditional expression. I believe (y if x else z) was my favorite > last time, wasn't it? I've changed the subject accordingly. It's not the version from the PEP, if that means anything at all... Cheers, mwh -- 34. The string is a stark data structure and everywhere it is passed there is much duplication of process. It is a perfect vehicle for hiding information. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html From rrr at ronadam.com Tue Sep 20 18:46:43 2005 From: rrr at ronadam.com (Ron Adam) Date: Tue, 20 Sep 2005 12:46:43 -0400 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: <43303CF3.7010200@ronadam.com> Fredrik Lundh wrote: > Michael Hudson wrote: > > >>>While you're at it, maybe we should switch to && and || as well? >>>That's another thing I always mistype when switching between >>>languages... >> >>You're joking, surely? > > > for Python 3000, I'd recommend switching to "and then" and "or else" instead > of the current ambiguous single-keyword versions. > > Keeping the current behaviors in some form would probably be good. Alternate operators might be a good way. They could also be functions. first_true(a, b, c) <--- a and b and d true_else(a, b) <--- a or b This would compliment: all_true(a, b, c) none_true(a, b, c) The '?' could just be used in place of the current 'or' and the '*' would work as "and value" operator when used with bools. val = a ? b ? c # val = a or b or c val = a and b * c # val = bool(a and b) and c val = a or b * c ? d # val = bool(a or b) and c or d And a vague idea of a suggestion... For the function versions above ... If speed or the need to avoid early evaluation is a concern, one thought would be to have a few "well selected" builtin C tuple operators that look and act like functions but are instead optimized operations. Functions that don't use any intermediate values and/or are so simple that the function call is large percent of the overall execution time could be candidates. It also might be possible to avoid early evaluation in cases where it makes since to do so. Just a thought. I have no idea how hard this would be, and since one of the goals of Python 3000 is to simplify and/or clean up the core, this might not be desirable. Cheers, Ron From guido at python.org Tue Sep 20 18:59:11 2005 From: guido at python.org (Guido van Rossum) Date: Tue, 20 Sep 2005 09:59:11 -0700 Subject: [Python-Dev] 2.4.2c1 fails test_unicode on HP-UX ia64 Message-ID: I just built and ran the latest Python 2.4.2c1 from CVS on our HP-UX Itanium 2 box. sys.maxint is 9223372036854775807 on this box. I get the following failure from test_unicode (all other tests pass): File "/var/tmp/guido/p4/Lib/test/test_unicode.py", line 553, in test_codecs_errors self.assertEqual(u'Andr\202 x'.encode('ascii','replace'), "Andr? x") AssertionError: 'Andr x' != 'Andr? x' When I try it on the command line I get the same result: Python 2.4.2c1 (#4, Sep 20 2005, 09:15:22) [C] on hp-ux11 Type "help", "copyright", "credits" or "license" for more information. >>> u'Andr\202 x'.encode('ascii','replace') 'Andr x' >>> The test passes on Linux. We have about 6 hours until code freeze... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jcarlson at uci.edu Tue Sep 20 19:20:44 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 20 Sep 2005 10:20:44 -0700 Subject: [Python-Dev] For/while/if statements/comprehension/generator expressions unification In-Reply-To: <8410256206.20050920115659@sinn.ru> References: <8410256206.20050920115659@sinn.ru> Message-ID: <20050920095417.9CAC.JCARLSON@uci.edu> Alexander Myodov wrote: > Hello Josiah, > > JC> Alexander, > JC> The essence of what you have proposed has been proposed (multiple times) before, > JC> and I seem to remember it was shot down. > > To increase my understanding of Python-way, can you (or someone else) > explain the reasons why such proposals were rejected? > > JC> The below functions offer the equivalent of list comprehensions with a > JC> final post-processing step. > > Well, what I was suggesting is not just a cross of two lists but the > syntax sugar which would make statements more consistent to the > generators/comprehensions and also give some new opportunities. > I think that my large proposal can be splitted to several > almost-independent ones, each carrying separate features, able to > be implemented independently and worth independent reconsidering: Try using the code I offered. It allows the cross of an aribitrary number of restartable iterables, in the same order as an equivalent list comprehension or generator expression. >>> list(cross([1,2], [3,4], [5,6])) [(1, 3, 5), (1, 3, 6), (1, 4, 5), (1, 4, 6), (2, 3, 5), (2, 3, 6), (2, 4, 5), (2, 4, 6)] There were a few hoops I had to jump through in cross in order to be able to hande single iterables as well as tuples embedded in the passed iterables, but they work as they should. >>> list(cross([(1,1),(2,2)], [(3,3),(4,4)], [(5,5),(6,6)])) [((1, 1), (3, 3), (5, 5)), ((1, 1), (3, 3), (6, 6)), ((1, 1), (4, 4), (5, 5)), ((1, 1), (4, 4), (6, 6)), ((2, 2), (3, 3), (5, 5)), ((2, 2), (3, 3), (6, 6)), ((2, 2), (4, 4), (5, 5)), ((2, 2), (4, 4), (6, 6))] > 1. Bring 'if'-s from generator/comprehension 'for' syntax to 'for' > statement. That's truly inconsistent that one may write > > list2 = [i for i in list if cond(i)] > > but cannot write > > for i in list if cond(i): Note: list comprehensions and generator expressions have not been in Python since the beginning. List comprehensions were added in Python 2.0 as syntactic sugar for the common case of list construction with simple predicates. x = [] for i in y: for j in z: if f(i,j): x.append((i,j)) x = [(i,j) for i in y for j in z if f(i,j)] If you are good, you can add predicates anywhere you want inside of for loops using ifilter. def f(i): return i%2==0 def g(i): return i**2%3 < 2 for i in ifilter(f, x): for j in ifilter(g, y): ... No need to have syntax for something that is simple enough to use... If you want to filter on multiple items (updated in each loop), there is always the curry decorator and/or class offered in PEP 309 (minor modification would offer a rightcurry). For all of the reasons you offer, Nick has a great point that there is no need to complicate the for loop with generator/list comprehension semantics when you can embed either of them in the right side of the for loop. - Josiah From guido at python.org Tue Sep 20 19:32:26 2005 From: guido at python.org (Guido van Rossum) Date: Tue, 20 Sep 2005 10:32:26 -0700 Subject: [Python-Dev] 2.4.2c1 fails test_unicode on HP-UX ia64 In-Reply-To: References: Message-ID: This seems to disappear when I disable -O. I guess the HP-UX optimizer is as bad as it always was. Or perhaps we have an old version installed. Sorry! On 9/20/05, Guido van Rossum wrote: > I just built and ran the latest Python 2.4.2c1 from CVS on our HP-UX > Itanium 2 box. sys.maxint is 9223372036854775807 on this box. > > I get the following failure from test_unicode (all other tests pass): > > File "/var/tmp/guido/p4/Lib/test/test_unicode.py", line 553, in > test_codecs_errors > self.assertEqual(u'Andr\202 x'.encode('ascii','replace'), "Andr? x") > AssertionError: 'Andr x' != 'Andr? x' > > When I try it on the command line I get the same result: > > Python 2.4.2c1 (#4, Sep 20 2005, 09:15:22) [C] on hp-ux11 > Type "help", "copyright", "credits" or "license" for more information. > >>> u'Andr\202 x'.encode('ascii','replace') > 'Andr x' > >>> > > The test passes on Linux. We have about 6 hours until code freeze... > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From reinhold-birkenfeld-nospam at wolke7.net Tue Sep 20 19:29:46 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Tue, 20 Sep 2005 19:29:46 +0200 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: Guido van Rossum wrote: > Given this realization, I'm now -1 on Raymond's idea, and +1 on adding > a conditional expression. I believe (y if x else z) was my favorite > last time, wasn't it? I've changed the subject accordingly. As the PEP states, I'm not sure if changing the customary order of "arguments" in conditional expressions is good. Also, I assume the thing will be short-circuiting, and it's confusing to grasp that y in (y if x else z) will possibly never be evaluated. Reinhold -- Mail address is perfectly valid! From jack at performancedrivers.com Tue Sep 20 19:56:45 2005 From: jack at performancedrivers.com (Jack Diederich) Date: Tue, 20 Sep 2005 13:56:45 -0400 Subject: [Python-Dev] For/while/if statements/comprehension/generator expressions unification In-Reply-To: <20050920095417.9CAC.JCARLSON@uci.edu> References: <8410256206.20050920115659@sinn.ru> <20050920095417.9CAC.JCARLSON@uci.edu> Message-ID: <20050920175645.GB6181@performancedrivers.com> On Tue, Sep 20, 2005 at 10:20:44AM -0700, Josiah Carlson wrote: > Try using the code I offered. It allows the cross of an aribitrary > number of restartable iterables, in the same order as an equivalent list > comprehension or generator expression. > > >>> list(cross([1,2], [3,4], [5,6])) > [(1, 3, 5), (1, 3, 6), (1, 4, 5), (1, 4, 6), (2, 3, 5), (2, 3, 6), (2, 4, > 5), (2, 4, 6)] > > There were a few hoops I had to jump through in cross in order to be > able to hande single iterables as well as tuples embedded in the passed > iterables, but they work as they should. > > >>> list(cross([(1,1),(2,2)], [(3,3),(4,4)], [(5,5),(6,6)])) > [((1, 1), (3, 3), (5, 5)), ((1, 1), (3, 3), (6, 6)), ((1, 1), (4, 4), (5, > 5)), ((1, 1), (4, 4), (6, 6)), ((2, 2), (3, 3), (5, 5)), ((2, 2), (3, 3), > (6, 6)), ((2, 2), (4, 4), (5, 5)), ((2, 2), (4, 4), (6, 6))] This comes up on c.l.p every month or two. Folks offer their own solutions optimized for speed, memory, or golfed for char length. I'll throw in my same two bits as always, sprat:~# python Python 2.4.1 (#2, Mar 30 2005, 21:51:10) [GCC 3.3.5 (Debian 1:3.3.5-8ubuntu2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import probstat >>> c = probstat.Cartesian([[1,2], [3,4], [5,6]]) >>> list(c) [[1, 3, 5], [2, 3, 5], [1, 4, 5], [2, 4, 5], [1, 3, 6], [2, 3, 6], [1, 4, 6], [2, 4, 6]] >>> c = probstat.Cartesian([[(1,1),(2,2)], [(3,3),(4,4)], [(5,5),(6,6)]]) >>> list(c) [[(1, 1), (3, 3), (5, 5)], [(2, 2), (3, 3), (5, 5)], [(1, 1), (4, 4), (5, 5)], [(2, 2), (4, 4), (5, 5)], [(1, 1), (3, 3), (6, 6)], [(2, 2), (3, 3), (6, 6)], [(1, 1), (4, 4), (6, 6)], [(2, 2), (4, 4), (6, 6)]] The signature is slightly different (list of lists) but otherwise does what you want. Unchanged since 2002! http://probstat.sourceforge.net/ -jackdied From jack at performancedrivers.com Tue Sep 20 20:14:23 2005 From: jack at performancedrivers.com (Jack Diederich) Date: Tue, 20 Sep 2005 14:14:23 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: <20050920181423.GC6181@performancedrivers.com> On Tue, Sep 20, 2005 at 09:04:53AM -0700, Guido van Rossum wrote: > On 9/20/05, Michael Hudson wrote: > > Guido van Rossum writes: > > > > > On 9/19/05, Raymond Hettinger wrote: > > >> I propose that in Py3.0, the "and" and "or" operators be simplified to > > >> always return a Boolean value instead of returning the last evaluated > > >> argument. > > > > > > While you're at it, maybe we should switch to && and || as well? > > > That's another thing I always mistype when switching between > > > languages... > > > > You're joking, surely? > > I wasn't, but I wasn't pushing for it either. Consider it withdrawn > given the response. > > > > Also, this proposal needs to be considered together with the addition > > > of a proper conditional operator, like x?y:z. > > > > I think this discussion has been had before, you know. > > Unfortunately, if we were to accept Raymond's proposal, we'd have to > revisit the discussion, since his proposal removes several ways we > currently avoid the need. > > In fact, I think Raymond's example is more properly considered an > argument for adding a conditional expression than for removing the > current behavior of the and/or shortcut operators; had we had a > conditional expression, he wouldn't have tried to use the "x and y or > z" syntax that bit him. > > Given this realization, I'm now -1 on Raymond's idea, and +1 on adding > a conditional expression. I believe (y if x else z) was my favorite > last time, wasn't it? I've changed the subject accordingly. I'm +1 for allowing authors to write return bool(thing or default) when they mean a function to return a bool. I had the "privilege" of working in a large base of perl code (a learning experience) and while some engineers abused functions that were documented as only returning true/false by depending on the side effects of 'and' and 'or' this was easily fixed. The functions were changed to literally return a plain true or false value and those engineers were summarily fired. I'm a dependable Hettinger fanboy but on this one I have to agree with the we're-all-adults club. Let the authors type an extra few chars if they want to make the code match the intent. -jackdied From bcannon at gmail.com Tue Sep 20 20:28:48 2005 From: bcannon at gmail.com (Brett Cannon) Date: Tue, 20 Sep 2005 11:28:48 -0700 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <2mzmq8jgzr.fsf@starship.python.net> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2mzmq8jgzr.fsf@starship.python.net> Message-ID: On 9/20/05, Michael Hudson wrote: [SNIP] > I _like_ the explanation of 'and' and 'or' as they are now. They are > basically control flow constructs -- and have to be to get > short-circuiting to work -- and adding a coercion to bool at the end > seems to add complexity, not reduce it (on some levels, anyway). > If you change the definition of 'and' and 'or' to be boolean comparison operators (as Raymond is proposing) and not as control flow constructs then is it really that complicated? I think it would actually simplify things very slightly since you just say a boolean is returned instead of the last executed expression by the operator. > > P.S. Simplifying "and" and "or" may create a need to introduce a > > conditional operator but that is a discussion for another day. > > ... which was in the past, I thought. It was, but changing 'and' and 'or' does tweak the usefulness of a conditional operator. -Brett From guido at python.org Tue Sep 20 20:42:04 2005 From: guido at python.org (Guido van Rossum) Date: Tue, 20 Sep 2005 11:42:04 -0700 Subject: [Python-Dev] 2.4.2c1: test_macfs failing on Tiger (Mac OS X 10.4.2) Message-ID: test test_macfs failed -- Traceback (most recent call last): File "/Users/gvr/p4/Lib/test/test_macfs.py", line 53, in test_dates self.assertEqual(dates, (now, now-1, now-2)) AssertionError: (1127241062, 1127241062, 1127241061) != (1127241063, 1127241062, 1127241061) Any takers? What does this mean? (It looks like an off-by-one-error in the first timestamp, but I don't know what the code is trying to do). Is the filesystem perhaps truncating times? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Sep 20 20:46:47 2005 From: guido at python.org (Guido van Rossum) Date: Tue, 20 Sep 2005 11:46:47 -0700 Subject: [Python-Dev] test_ossaudiodev hangs Message-ID: On my Linux box (Red Hat Enterprise 3), test_ossaudiodev hangs forever when I enable it: ./python -E -tt ../Lib/test/regrtest.py -u all test_ossaudiodev test_ossaudiodev Traceback (most recent call last): File "../Lib/test/regrtest.py", line 1178, in ? main() File "../Lib/test/regrtest.py", line 333, in main ok = runtest(test, generate, verbose, quiet, testdir, huntrleaks) File "../Lib/test/regrtest.py", line 473, in runtest the_package = __import__(abstest, globals(), locals(), []) File "/home/guido/projects/python-2.4/dist/src/Lib/test/test_ossaudiodev.py", line 136, in ? test() File "/home/guido/projects/python-2.4/dist/src/Lib/test/test_ossaudiodev.py", line 124, in test play_sound_file(data, rate, ssize, nchannels) File "/home/guido/projects/python-2.4/dist/src/Lib/test/test_ossaudiodev.py", line 63, in play_sound_file dsp.write(data) KeyboardInterrupt This happens both with 2.4.2c1 and with 2.5a0 from current CVS. I know in order to debug you need more info about my audio device, but since I never use audio on Linux, you'll have to talk me through providing the info. I'm gvanrossum at gmail.com on Google talk BTW. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Tue Sep 20 21:00:41 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 20 Sep 2005 15:00:41 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: "Guido van Rossum" wrote in message news:ca471dc2050920090461aec3da at mail.gmail.com... > In fact, I think Raymond's example is more properly considered an > argument for adding a conditional expression than for removing the > current behavior of the and/or shortcut operators; had we had a > conditional expression, he wouldn't have tried to use the "x and y or > z" syntax that bit him. I agree. > Given this realization, I'm now -1 on Raymond's idea, There are a lot of people who use 'or', especially, as intended. > and +1 on adding a conditional expression. I believe (y if x else z) > was my favorite last time, wasn't it? No. That was your original proposal, which you later rejected. "The original version of this PEP proposed the following syntax: if else The out-of-order arrangement was found to be too uncomfortable for many of participants in the discussion; especially when is long, it's easy to miss the conditional while skimming." Your final 'favorite' was apparently (at the top) "Proposal The proposed syntax is as follows: (if : else: ) " (+ elif parts) selected from "Summary of the Current State of the Discussion Groups are falling into one of three camps: 1. Adopt a ternary operator built using punctuation characters: ? : 2. Adopt a ternary operator built using new or existing keywords. The leading examples are: then else (if : else: ) 3. Do nothing." Given the later addition of generator expressions with mandatory parentheses , the mandatory-parentheses version of a conditional expression looks less strange to me than it did then ;-). So I could happily use it even though I may still lean toward the other option 2 version (then-else) due to its not needing ':'s or a third elseif term for chaining. *If* you want general community input, I would suggest a runoff ballot with those four choices (and a summary of pros and cons of each), or fewer if you see any as unacceptible. Terry J. Reedy > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org > > From guido at python.org Tue Sep 20 21:17:41 2005 From: guido at python.org (Guido van Rossum) Date: Tue, 20 Sep 2005 12:17:41 -0700 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: On 9/20/05, Terry Reedy wrote: > "Guido van Rossum" wrote in message > > and +1 on adding a conditional expression. I believe (y if x else z) > > was my favorite last time, wasn't it? > > No. That was your original proposal, which you later rejected. Thanks for setting me straight; upon re-reading the PEP I couldn't even remember which one I favored at the time! > Your final 'favorite' was apparently (at the top) > > "Proposal > The proposed syntax is as follows: > (if : else: ) " (+ elif parts) > > selected from > > "Summary of the Current State of the Discussion > Groups are falling into one of three camps: > 1. Adopt a ternary operator built using punctuation characters: > ? : > 2. Adopt a ternary operator built using new or existing keywords. > The leading examples are: > then else > (if : else: ) > 3. Do nothing." > > Given the later addition of generator expressions with mandatory > parentheses , the mandatory-parentheses version of a conditional expression > looks less strange to me than it did then ;-). So I could happily use it > even though I may still lean toward the other option 2 version (then-else) > due to its not needing ':'s or a third elseif term for chaining. I think I'd prefer (if then else ) i.e. no colons. None of the other expression forms (list comprehensions and generator expressions) involving statement keywords use colons. > *If* you want general community input, I would suggest a runoff ballot with > those four choices (and a summary of pros and cons of each), or fewer if > you see any as unacceptible. If there's one thing I've learned from the PEP 308 vote, it is that votes for language don't work. I prefer some discussion on Python-dev after which I pick one. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Tue Sep 20 21:38:50 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 20 Sep 2005 15:38:50 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: <5.1.1.6.0.20050920153348.034c65e0@mail.telecommunity.com> At 12:17 PM 9/20/2005 -0700, Guido van Rossum wrote: >I think I'd prefer (if then else ) i.e. no >colons. None of the other expression forms (list comprehensions and >generator expressions) involving statement keywords use colons. +1, despite the fact that we seem on a slippery slope towards becoming a kind of infix Lisp. ;) Between (yield x), (x for x in y), and now (if x then y else z), it seems that parentheses are all the rage now. Will we get (try finally ) next? <0.5 wink> > > *If* you want general community input, I would suggest a runoff ballot with > > those four choices (and a summary of pros and cons of each), or fewer if > > you see any as unacceptible. > >If there's one thing I've learned from the PEP 308 vote, it is that >votes for language don't work. I prefer some discussion on Python-dev >after which I pick one. Also +1. :) From raymond.hettinger at verizon.net Tue Sep 20 22:17:43 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Tue, 20 Sep 2005 16:17:43 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: Message-ID: <000701c5be20$5c2840e0$6522c797@oemcomputer> [Guido van Rossum] > > In fact, I think Raymond's example is more properly considered an > > argument for adding a conditional expression than for removing the > > current behavior of the and/or shortcut operators; had we had a > > conditional expression, he wouldn't have tried to use the "x and y or > > z" syntax that bit him. [Terry Reedy] > I agree. Me too. > Given the later addition of generator expressions with mandatory > parentheses , the mandatory-parentheses version of a conditional > expression > looks less strange to me than it did then ;-). Same here. Raymond From jjl at pobox.com Tue Sep 20 22:27:23 2005 From: jjl at pobox.com (John J Lee) Date: Tue, 20 Sep 2005 20:27:23 +0000 (UTC) Subject: [Python-Dev] GIL, Python 3, and MP vs. UP (was Re: Variant of removing GIL.) In-Reply-To: References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> Message-ID: On Sun, 18 Sep 2005, Guido van Rossum wrote: > On 9/17/05, John J Lee wrote: [...snip...] [guido] > If my hunch is right, I expect that instead of writing massively > parallel applications, we will continue to write single-threaded > applications that are tied together at the process level rather than > at the thread level. I tend to agree. [...] > > I realize that not all algorithms (nor all computational problems) scale > > well to MP hardware. Is it feasible to usefully compile both MP and a UP > > binaries from one Python source code base? > > That's an understatement. I expect that *most* problems (even most > problems that we will be programming 10-20 years from now) get little > benefit out of MP. Perhaps, but I suspect we'll also get better at thinking up multiprocessor algorithms when better motivated by lack of exponential uniprocessor speed increases. [...] > > Of course, it still takes a (anti-)hero to step forward and do the work... > > Be my guest. Prove me wrong. Talk is cheap; instead of arguing my > points (all of which can be argued ad infinitum), come back when > you've got a working GIL-free Python. Doesn't have to be CPython-based > -- C# would be fine too. I don't actively want a GIL-free Python. I was just making some arguments in favour of GIL-removal that I hadn't seen made on a public list before. (In particular, removal now, since now is a special time.) John From arigo at tunes.org Tue Sep 20 21:21:14 2005 From: arigo at tunes.org (Armin Rigo) Date: Tue, 20 Sep 2005 21:21:14 +0200 Subject: [Python-Dev] IMPORTANT: release24-maint branch is FROZEN from 2005-09-21 00:00 UTC for 2.4.2 In-Reply-To: <200509202306.54251.anthony@python.org> References: <200509202306.54251.anthony@python.org> Message-ID: <20050920192114.GA9776@code1.codespeak.net> Hi, A quick note, the profile.py module is broken -- crashes on some examples and real-world programs. I think I should be able to fix it by tomorrow, but not tonight. (See example checked in in the CVS trunk -- Lib/test/test_profile -- which passes, but for some reason I get completely unrealistic results on a real-world application. Can't investigate more now...) Armin From bcannon at gmail.com Tue Sep 20 22:39:56 2005 From: bcannon at gmail.com (Brett Cannon) Date: Tue, 20 Sep 2005 13:39:56 -0700 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <5.1.1.6.0.20050920153348.034c65e0@mail.telecommunity.com> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <5.1.1.6.0.20050920153348.034c65e0@mail.telecommunity.com> Message-ID: On 9/20/05, Phillip J. Eby wrote: > At 12:17 PM 9/20/2005 -0700, Guido van Rossum wrote: > >I think I'd prefer (if then else ) i.e. no > >colons. None of the other expression forms (list comprehensions and > >generator expressions) involving statement keywords use colons. > > +1, despite the fact that we seem on a slippery slope towards becoming a > kind of infix Lisp. ;) Between (yield x), (x for x in y), and now (if x > then y else z), it seems that parentheses are all the rage now. Will we get > (try finally ) next? <0.5 wink> > +1 from me as well. The use will be much more obvious to a newbie than `` ? : ``. And I have used the syntactic-heavy listcomps in Haskell and I must say that Python's version is much nicer. I like the wordy version. =) > > > > *If* you want general community input, I would suggest a runoff ballot with > > > those four choices (and a summary of pros and cons of each), or fewer if > > > you see any as unacceptible. > > > >If there's one thing I've learned from the PEP 308 vote, it is that > >votes for language don't work. I prefer some discussion on Python-dev > >after which I pick one. > > Also +1. :) > +1 from me, although I sure got lambasted by some people when I said I liked this style during the whole decorator debate. =) -Brett From jjl at pobox.com Tue Sep 20 22:36:27 2005 From: jjl at pobox.com (John J Lee) Date: Tue, 20 Sep 2005 20:36:27 +0000 (UTC) Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <87hdcgrc4m.fsf@mid.deneb.enyo.de> References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <87r7bkrect.fsf@mid.deneb.enyo.de> <2mek7klqzc.fsf@starship.python.net> <87hdcgrc4m.fsf@mid.deneb.enyo.de> Message-ID: On Mon, 19 Sep 2005, Florian Weimer wrote: > The real problem is that you can ditch most extension modules. 8-( [...] *Is* that a showstopper for Python 3.0, though? John From jjl at pobox.com Tue Sep 20 22:38:56 2005 From: jjl at pobox.com (John J Lee) Date: Tue, 20 Sep 2005 20:38:56 +0000 (UTC) Subject: [Python-Dev] GIL, Python 3, and MP vs. UP (was Re: Variant of removing GIL.) In-Reply-To: References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> Message-ID: On Tue, 20 Sep 2005, John J Lee wrote: [...] > I don't actively want a GIL-free Python. I was just making some arguments [...] Actually, FWIW, I don't know if I even *passively* want a GIL-free Python, if it encourages threaded code (though I'd like to have that option for my occasional personal use, it seems to be an attractive nuisance for many other programmers). John From bcannon at gmail.com Tue Sep 20 22:49:31 2005 From: bcannon at gmail.com (Brett Cannon) Date: Tue, 20 Sep 2005 13:49:31 -0700 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <87r7bkrect.fsf@mid.deneb.enyo.de> <2mek7klqzc.fsf@starship.python.net> <87hdcgrc4m.fsf@mid.deneb.enyo.de> Message-ID: On 9/20/05, John J Lee wrote: > On Mon, 19 Sep 2005, Florian Weimer wrote: > > > The real problem is that you can ditch most extension modules. 8-( > [...] > > *Is* that a showstopper for Python 3.0, though? Who knows. I bet Guido doesn't even know how much breakage he is going to want to push. Some people have rather strongly pointed out (usually after I have proposed something), breaking stuff without a good reason is not worth the added level of breakage for when people try to update code to Python 3.0. Completely changing how garbage collection works is not exactly a minor thing and there is the possibility it won't pan out. It would really suck for everyone to have to learn an entirely new way of handling garbage collection and have there not be a perk for doing so, especially since this kind of change will not be directly visible to the language itself. -Brett From jjl at pobox.com Tue Sep 20 22:47:02 2005 From: jjl at pobox.com (John J Lee) Date: Tue, 20 Sep 2005 20:47:02 +0000 (UTC) Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <9613db6005091913206dcf51ec@mail.gmail.com> References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <8393fff0509190928196bba9a@mail.gmail.com> <2mirwwlsve.fsf@starship.python.net> <9613db6005091913206dcf51ec@mail.gmail.com> Message-ID: On Mon, 19 Sep 2005, Tim Lesher wrote: > On 9/19/05, Michael Hudson wrote: > > I was disappointed that that article (hey, it was the only issue of > > ddj I've ever actually bought! :) didn't consider any concurrency > > models other than shared memory threading. > > The problem is that, for all its limitations, shared-memory threading > is the most popular concurrency model on the most popular operating > system, so future hardware platforms targeting that system will be > optimizing for that case. > > We can either rail against the sea, or accept it. Hmm, that's an interesting point. Aside from that point I tend to agree with Guido: threading is not the only, nor the best, concurrency model. But maybe these chips designed with threading in mind blow that argument out of the water. I don't know enough to know whether that's true or not... John From jjl at pobox.com Tue Sep 20 22:52:45 2005 From: jjl at pobox.com (John J Lee) Date: Tue, 20 Sep 2005 20:52:45 +0000 (UTC) Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <87r7bkrect.fsf@mid.deneb.enyo.de> <2mek7klqzc.fsf@starship.python.net> <87hdcgrc4m.fsf@mid.deneb.enyo.de> Message-ID: On Tue, 20 Sep 2005, Brett Cannon wrote: > On 9/20/05, John J Lee wrote: > > On Mon, 19 Sep 2005, Florian Weimer wrote: > > > > > The real problem is that you can ditch most extension modules. 8-( > > [...] > > > > *Is* that a showstopper for Python 3.0, though? > > Who knows. I bet Guido doesn't even know how much breakage he is > going to want to push. Some people have rather strongly pointed out > (usually after I have proposed something), breaking stuff without a > good reason is not worth the added level of breakage for when people > try to update code to Python 3.0. Oh, absolutely. > Completely changing how garbage > collection works is not exactly a minor thing and there is the > possibility it won't pan out. It would really suck for everyone to > have to learn an entirely new way of handling garbage collection and > have there not be a perk for doing so, especially since this kind of > change will not be directly visible to the language itself. I didn't intend to refer to garbage collection in particular, but to removing the GIL, thus breaking extension modules (perhaps in a less-drastic way than implied by the copying garbage-collection proposal). John From arigo at tunes.org Tue Sep 20 23:21:35 2005 From: arigo at tunes.org (Armin Rigo) Date: Tue, 20 Sep 2005 23:21:35 +0200 Subject: [Python-Dev] IMPORTANT: release24-maint branch is FROZEN from 2005-09-21 00:00 UTC for 2.4.2 In-Reply-To: <20050920192114.GA9776@code1.codespeak.net> References: <200509202306.54251.anthony@python.org> <20050920192114.GA9776@code1.codespeak.net> Message-ID: <20050920212135.GA11520@code1.codespeak.net> Hi, On Tue, Sep 20, 2005 at 09:21:14PM +0200, Armin Rigo wrote: > A quick note, the profile.py module is broken -- crashes on some > examples and real-world programs. I think I should be able to fix it by > tomorrow, but not tonight. It was easier than I thought, sorry for the alarm. Armin From steven.bethard at gmail.com Tue Sep 20 23:33:19 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 20 Sep 2005 15:33:19 -0600 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: Guido van Rossum wrote: > I think I'd prefer (if then else ) i.e. no > colons. None of the other expression forms (list comprehensions and > generator expressions) involving statement keywords use colons. FWIW, I find this quite intuitive. It follows the same pattern as LCs and GEs -- remove the colons and add parentheses (or brackets for LCs). So I'm +1. STeVe -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From guido at python.org Tue Sep 20 23:40:45 2005 From: guido at python.org (Guido van Rossum) Date: Tue, 20 Sep 2005 14:40:45 -0700 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: (Adding python-dev back to the CC list) On 9/20/05, Jason Orendorff wrote: > > If there's one thing I've learned from the PEP 308 vote, it is that > > votes for language don't work. I prefer some discussion on Python-dev > > after which I pick one. > > +1 > > Some visual aids: > > return (if q: q.popleft() else: None) > return (if q then q.popleft() else None) > return q ? q.popleft() : None > > Hmmm. Score one for ?:. But: Why? Just because it's shorter? > menu.append( > if gotHerring(): popHerring() > elif gotAnyFish(): popAnyFish() > else: Tofurbot()) > > menu.append(gotHerring() ? popHerring() : gotAnyFish() ? > popAnyFish() : Tofurbot()) > > Here, I like the way the verbose syntax politely spreads itself out > over multiple lines. In C, I never know where to put the line breaks. Ouch. You're bringing up another valid issue: whether to support "elif". I think if we go with (if ... then ... else ...) or (if ...: ... else: ...) we'll have to support elif as well: (if ... then ... elif ... then ... else ...) or (if ...: ... elif ...: ... else: ...) I really don't like the latter. Here's a suggestion for a way to decide between a "wordy" version or C-style "?:" -- if we abandon and/or in favor of &&/||, we should also go with ?:; if we keep and/or, we should use a keyword-based conditional as well. Since so far the feedback is overwhelmingly in favor of keeping and/or, I think that settles the case in favor of a wordy version. My preference then would be (if ... then ... elif ... then ... else ...) which gives my a nice nostalgic feeling because (except for the elif part) Algol-60 had the same thing -- the first programming language I ever learned. :) (Oh, and a way to decide between colon or no colon: we're not using colons in list comps and genexprs either.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Sep 20 23:43:59 2005 From: guido at python.org (Guido van Rossum) Date: Tue, 20 Sep 2005 14:43:59 -0700 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <8393fff0509190928196bba9a@mail.gmail.com> <2mirwwlsve.fsf@starship.python.net> <9613db6005091913206dcf51ec@mail.gmail.com> Message-ID: On 9/20/05, John J Lee wrote: > threading is not the only, nor the best, concurrency model. > But maybe these chips designed with threading in mind blow that argument > out of the water. I don't know enough to know whether that's true or > not... I don't know that any chips are designed with threading in mind. Fast threading benefits from fast context switches which benefits from small register sets. I believe the trend is towards ever large register sets. Also, multiple processors with shared memory don't scall all that well; multiple processors with explicit IPC channels scale much better. All arguments for multi-processing and against multi-threading. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Sep 20 23:49:41 2005 From: guido at python.org (Guido van Rossum) Date: Tue, 20 Sep 2005 14:49:41 -0700 Subject: [Python-Dev] bool(iter([])) changed between 2.3 and 2.4 Message-ID: I just finished debugging some code that broke after upgrading to Python 2.4 (from 2.3). Turns out the code was testing list iterators for their boolean value (to distinguish them from None). In 2.3, a list iterator (like any iterator) is always true. In 2.4, an exhausted list iterator is false; probably by virtue of having a __len__() method that returns the number of remaining items. I realize that this was a deliberate feature, and that it exists in 2.4 as well as in 2.4.1 and will in 2.4.2; yet, I'm not sure I *like* it. Was this breakage (which is not theoretical!) considered at all? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Wed Sep 21 00:01:40 2005 From: barry at python.org (Barry Warsaw) Date: Tue, 20 Sep 2005 18:01:40 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: <1127253700.31450.32.camel@geddy.wooz.org> On Tue, 2005-09-20 at 17:40, Guido van Rossum wrote: > Ouch. You're bringing up another valid issue: whether to support > "elif". I think if we go with (if ... then ... else ...) or (if ...: > ... else: ...) we'll have to support elif as well: I'm not so sure. Once you start writing such a complicated thing, I think readability will favor just breaking the code out into traditional if-blocks. I'd be happy enough with just a binary condition. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050920/492f314d/attachment.pgp From fdrake at acm.org Wed Sep 21 00:03:30 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Tue, 20 Sep 2005 18:03:30 -0400 Subject: [Python-Dev] bool(iter([])) changed between 2.3 and 2.4 In-Reply-To: References: Message-ID: <200509201803.30977.fdrake@acm.org> On Tuesday 20 September 2005 17:49, Guido van Rossum wrote: > I realize that this was a deliberate feature, and that it exists in > 2.4 as well as in 2.4.1 and will in 2.4.2; yet, I'm not sure I *like* I wasn't paying any attention at the time, so I don't know what was discussed. Some discussion here just now leads me to believe that at least two of us here (including myself) think iterators shouldn't have length at all: they're *not* containers and shouldn't act that way. -Fred -- Fred L. Drake, Jr. From bob at redivi.com Wed Sep 21 00:10:59 2005 From: bob at redivi.com (Bob Ippolito) Date: Tue, 20 Sep 2005 18:10:59 -0400 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <8393fff0509190928196bba9a@mail.gmail.com> <2mirwwlsve.fsf@starship.python.net> <9613db6005091913206dcf51ec@mail.gmail.com> Message-ID: <1CF74C6F-7521-4751-95B1-2DF71C053251@redivi.com> On Sep 20, 2005, at 5:43 PM, Guido van Rossum wrote: > On 9/20/05, John J Lee wrote: > >> threading is not the only, nor the best, concurrency model. >> But maybe these chips designed with threading in mind blow that >> argument >> out of the water. I don't know enough to know whether that's true or >> not... >> > > I don't know that any chips are designed with threading in mind. Fast > threading benefits from fast context switches which benefits from > small register sets. I believe the trend is towards ever large > register sets. Also, multiple processors with shared memory don't > scall all that well; multiple processors with explicit IPC channels > scale much better. All arguments for multi-processing and against > multi-threading. Well, in many operating systems, there isn't a whole lot of difference between threads and processes (both are kernel threads). When using threads, you can typically still use IPC, so you could scale similarly. I think the biggest argument for threading is simply that lots of existing C/C++ code wants to use threads. What we have now works OK, but you can't reliably use Python in a real-time thread (e.g. a CoreAudio callback) or (reliably) in a signal handler because it might block too long because of something else going on. Of course, a lot of other design decisions in Python would prevent you from using it in that context too, so GIL-free threading wouldn't be a panacea. Personally my biggest issue with the way the CPython VM works is that you can't reliably do multiple interpreters in a single process. If that worked well, you could start an independent interpreter per thread and with a per-interpreter GIL you'd have pretty much everything you needed... but this would horribly break the C API and may be a performance hit. My use case for this isn't so much about threads, but plug-ins. Writing multiple Python-based plug-ins for an application is always a mess, because they share too much (sys.modules, sys.path, etc.). PyObjC would benefit greatly from this feature, because you can write Python-based plug-ins for any Cocoa app that supports plug-ins, even if they're otherwise unaware of Python's existence. There are workarounds, of course, with import hooks and similar hacks. I think that mod_python would also benefit from this, and probably other such systems. -bob From tjreedy at udel.edu Wed Sep 21 00:24:27 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 20 Sep 2005 18:24:27 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: "Guido van Rossum" wrote in message news:ca471dc2050920121778df3fcd at mail.gmail.com... > On 9/20/05, Terry Reedy wrote: >> Given the later addition of generator expressions with mandatory >> parentheses , the mandatory-parentheses version of a conditional >> expression >> looks less strange to me than it did then ;-). So I could happily use >> it >> even though I may still lean toward the other option 2 version >> (then-else) >> due to its not needing ':'s or a third elseif term for chaining. > > I think I'd prefer (if then else ) i.e. no > colons. None of the other expression forms (list comprehensions and > generator expressions) involving statement keywords use colons. I presume this revision would continue to include elif clauses. If I put on a 'designing Python for everyone' hat, then the presence of the leading 'if' looks better than the slightly-too-cute (especially for Python) abbreviated version. +1 >> *If* you want general community input, I would suggest a runoff ballot >> with >> those four choices (and a summary of pros and cons of each), or fewer if >> you see any as unacceptible. > > If there's one thing I've learned from the PEP 308 vote, it is that > votes for language don't work. I prefer some discussion on Python-dev > after which I pick one. If we reject both the status quo and the symbol-tax form and agree on the above as the best wordy form, then it is a moot point anyway ;-) C.l.p. newcomers continue to periodically request "How do I write conditional expressions?". I think most will be happier with something clear and readable. Terry J. Reedy From jason.orendorff at gmail.com Wed Sep 21 00:28:46 2005 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Tue, 20 Sep 2005 18:28:46 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: On 9/20/05, Guido wrote: > On 9/20/05, Jason Orendorff wrote: > > return (if q: q.popleft() else: None) > > return (if q then q.popleft() else None) > > return q ? q.popleft() : None > > > > Hmmm. Score one for ?:. > > Why? Just because it's shorter? Just a gut response to the look. The verbose forms strike me as cluttered in this particular case. In the multiline case, it doesn't look like clutter because the if/elif/else bits line up, which fits the way Python has already trained my brain. > (Oh, and a way to decide between colon or no colon: we're not using > colons in list comps and genexprs either.) (grin) Easily fixed: print "average weight:", avg(for c in chefs: c.weight) rdict = dict(for k, v in D.iteritems(): v, k) Honestly, I think I would prefer this syntax. Examples from real code, before and after: lines = [line for line in pr.block.body if line.logical_line.strip() != ''] lines = [for line in pr.block.body: if line.logical_line.strip() != '': line] row.values = \ [line[col.start:col.end].strip() for col in columns] row.values = \ [for col in columns: line[col.start:col.end].rstrip()] return [p for p in self.listdir(pattern) if p.isdir()] return [for p in self.listdir(pattern): if p.isdir(): p] -j From ncoghlan at gmail.com Wed Sep 21 00:30:13 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 21 Sep 2005 08:30:13 +1000 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: <43308D75.7060602@gmail.com> Steven Bethard wrote: > Guido van Rossum wrote: > >>I think I'd prefer (if then else ) i.e. no >>colons. None of the other expression forms (list comprehensions and >>generator expressions) involving statement keywords use colons. > > > FWIW, I find this quite intuitive. It follows the same pattern as LCs > and GEs -- remove the colons and add parentheses (or brackets for > LCs). So I'm +1. *But*, in LC's and GE's, the body of the main clause of the statement is also pulled out and placed in front of the keyword: def gen(): for VAR in ITERABLE: if COND: yield EXPR becomes: gen = (EXPR for VAR in ITERABLE if COND) This makes sense to me, because the most important thing in the generator expression is the way each element is populated - the source iterable and the filtering condition do matter, but they aren't as important. It also makes the expression forms more declarative in nature, rather than being procedural statements embedded inside an expression. Notice also that, in this case, if ITERABLE is empty, or COND always evaluates false in boolean context, then EXPR is never executed - in other words, Python already has precedent for out of order code execution in expression evaluation. Guido's original PEP 308 proposal worked much the same way: if COND: x = EXPR1 else: x = EXPR2 became: x = (EXPR1 if COND else EXPR2) I see this as similar in spirit to the current form of LC's and GE's - the most important things are the two possible values rather than the condition for choosing between them, and this form makes them clearly visible at the start and end of the expression, rather than embedding one of them in the middle. The post-filtered form is also similarly declarative rather than procedural. This version doesn't need a separate 'elif' form - elif can be written as: x = (EXPR1 if COND1 else EXPR2 if COND2 else EXPR3) Note that the keyword count is no higher than if 'elif' was used, because the 'then' keyword isn't needed: x = (if COND1 then EXPR1 elif COND2 then EXPR2 else EXPR3) Here's Raymond's problematic example using this original PEP 308 form: def real(self): 'Return a vector with the real part of each input element' # do not convert integer inputs to floats return self.map(lambda z: (z.real if type(z)==types.ComplexType else z)) And the other non-colon, keyword based proposal in PEP 308's top four: def real(self): 'Return a vector with the real part of each input element' # do not convert integer inputs to floats return self.map(lambda z: (if type(z)==types.ComplexType then z.real else z)) Another common use case would be for mutable default arguments: x = ([] if arg is None else list(arg)) x = (if arg is None then [] else list(arg)) Basically, I'm +1 on the original PEP 308 form because it reads more naturally (and more like LC's and GE's) to me in expression contexts, and +0 on the "if/then/elif/else" form (because I would like a real conditional operator). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From tl at gocept.com Wed Sep 21 00:32:53 2005 From: tl at gocept.com (Thomas Lotze) Date: Wed, 21 Sep 2005 00:32:53 +0200 Subject: [Python-Dev] Adding a conditional expression in Py3.0 References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: Guido van Rossum wrote: > I think I'd prefer (if then else ) i.e. no colons. > None of the other expression forms (list comprehensions and generator > expressions) involving statement keywords use colons. While I like the looks of the form without colons better, I can't quite follow this argument for it. In LCs and GEs, colons wouldn't make any sense in the first place because the expression controlled by "for" and "if" is written before it: [expression(x) for x in y]. Just where would anyone put a colon there, anyway? A colon does make sense if something follows it, as is currently the case in lambda expressions. It would also be the case in an (if x: y else: z) form. I have a feeling that trying to make all expression forms which happen to use statement keywords colon-free is seeking an artificial consistency. I'm happy to be proven wrong, though. +1 on the idea of introducing a conditional expression in the first place, though. -- Thomas From tl at gocept.com Wed Sep 21 00:21:00 2005 From: tl at gocept.com (Thomas Lotze) Date: Wed, 21 Sep 2005 00:21:00 +0200 Subject: [Python-Dev] Adding a conditional expression in Py3.0 References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <1127253700.31450.32.camel@geddy.wooz.org> Message-ID: Barry Warsaw wrote: > I'm not so sure. Once you start writing such a complicated thing, I think > readability will favor just breaking the code out into traditional > if-blocks. I'd be happy enough with just a binary condition. Nothing prevents you from spreading the code over multiple lines, like so: x = (if a then b elif c then d else e) or even x = (if a then b elif c then d else e ) especially as there are going to be parentheses around the whole thing anyway. From a readability point of view, this is no different from if-statement blocks, and the matter is IMO not worth dumbing down an if-expression thingy as compared to its if-statement companion. -- Thomas From barry at python.org Wed Sep 21 00:49:27 2005 From: barry at python.org (Barry Warsaw) Date: Tue, 20 Sep 2005 18:49:27 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <1127253700.31450.32.camel@geddy.wooz.org> Message-ID: <1127256567.1217.17.camel@geddy.wooz.org> On Tue, 2005-09-20 at 18:21, Thomas Lotze wrote: > Nothing prevents you from spreading the code over multiple lines, like so: > > x = (if a then b > elif c then d > else e) > > or even > > x = (if a then > b > elif c then > d > else > e > ) > > especially as there are going to be parentheses around the whole thing > anyway. From a readability point of view, this is no different from > if-statement blocks, and the matter is IMO not worth dumbing down an > if-expression thingy as compared to its if-statement companion. I guess that's my point. To me, your latter is actually worse than if a: x = b elif c: x = d else: x = e I think the conditional stuff gets the most bang for the buck in situations like: d[foo] = if a then b else c -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050920/561d120c/attachment.pgp From jimjjewett at gmail.com Wed Sep 21 00:50:22 2005 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 20 Sep 2005 18:50:22 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 Message-ID: Guido van Rossum: > I think I'd prefer (if then else ) In isolation, (if .. then ... else) is an improvement, but I'm not sure it should be viewed in isolation. At one point, you wanted to strengthen the distinction between statements and expressions. Is that still true? x = (if y then y.val else default) x = (if y: y.val else: default) If any statement can become an expression, that could be very useful. There are times when it would be convenient to just grab the newly created function (def replacing lambda) or class, or the newly imported module. The distinction between exec and eval starts to look increasingly arbitrary as well. But if everything becomes expression-able, people *will* start to use parentheses instead of indentation. -jJ From tl at gocept.com Wed Sep 21 00:59:49 2005 From: tl at gocept.com (Thomas Lotze) Date: Wed, 21 Sep 2005 00:59:49 +0200 Subject: [Python-Dev] Adding a conditional expression in Py3.0 References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <1127253700.31450.32.camel@geddy.wooz.org> <1127256567.1217.17.camel@geddy.wooz.org> Message-ID: Barry Warsaw wrote: >> x = (if a then >> b >> elif c then >> d >> else >> e >> ) [...] > > I guess that's my point. To me, your latter is actually worse than > > if a: > x = b > elif c: > x = d > else: > x = e Can't see a difference as far as readability is concerned. But then, tastes differ. > I think the conditional stuff gets the most bang for the buck in > situations like: > > d[foo] = if a then b else c And I think similar holds for LCs and GEs. Unwinding a complex sequence of for and if clauses is certainly no fun unless one is really used to it. (Which doesn't take long, though.) But your example poses another question: Up until now, I had the impression that parentheses should be mandatory around a conditional expression. There's certainly no avoiding them in situations like (if x then 1 else 2) + 3. But what about unambiguous cases like your example line? -- Thomas From tdelaney at avaya.com Wed Sep 21 01:23:53 2005 From: tdelaney at avaya.com (Delaney, Timothy (Tim)) Date: Wed, 21 Sep 2005 09:23:53 +1000 Subject: [Python-Dev] bool(iter([])) changed between 2.3 and 2.4 Message-ID: <2773CAC687FD5F4689F526998C7E4E5F4DB63E@au3010avexu1.global.avaya.com> Fred L. Drake, Jr. wrote: > On Tuesday 20 September 2005 17:49, Guido van Rossum wrote: > > I realize that this was a deliberate feature, and that it exists in > > 2.4 as well as in 2.4.1 and will in 2.4.2; yet, I'm not sure I > *like* > > I wasn't paying any attention at the time, so I don't know what was > discussed. Some discussion here just now leads me to believe that at > least two of us here (including myself) think iterators shouldn't > have length at all: they're *not* containers and shouldn't act that > way. In any case, it's simple to get the 2.3 behaviour back - just add __nonzero__. I think that could validly be considered a bugfix. Tim Delaney From guido at python.org Wed Sep 21 01:34:09 2005 From: guido at python.org (Guido van Rossum) Date: Tue, 20 Sep 2005 16:34:09 -0700 Subject: [Python-Dev] bool(iter([])) changed between 2.3 and 2.4 In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F4DB63E@au3010avexu1.global.avaya.com> References: <2773CAC687FD5F4689F526998C7E4E5F4DB63E@au3010avexu1.global.avaya.com> Message-ID: On 9/20/05, Delaney, Timothy (Tim) wrote: > In any case, it's simple to get the 2.3 behaviour back - just add > __nonzero__. I think that could validly be considered a bugfix. Alas not -- it would still be a functionality change relative to 2.4 and 2.4.1. Also, an object whose __len__() returns 0 but whose __nonzero__() returns True would be an anomaly. The best we can do IMO is to change it back in 2.5. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jjl at pobox.com Wed Sep 21 01:37:27 2005 From: jjl at pobox.com (John J Lee) Date: Tue, 20 Sep 2005 23:37:27 +0000 (UTC) Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <1127253700.31450.32.camel@geddy.wooz.org> <1127256567.1217.17.camel@geddy.wooz.org> Message-ID: On Wed, 21 Sep 2005, Thomas Lotze wrote: > Barry Warsaw wrote: > > >> x = (if a then > >> b > >> elif c then > >> d > >> else > >> e > >> ) > [...] > > > > I guess that's my point. To me, your latter is actually worse than > > > > if a: > > x = b > > elif c: > > x = d > > else: > > x = e > > Can't see a difference as far as readability is concerned. But then, > tastes differ. [...] With the former, we have a more C-style syntax where meaning is determined purely by delimeters rather than by whitespace. Instead of braces '{' and '}', we have 'then' and 'elif'/'else'. That's a real difference. The stricter form where you don't allow 'elif' will get used in more restricted circumstances, so gives less encouragement for widespread abuse of conditional expressions by people who don't like whitespace-based syntax. John From raymond.hettinger at verizon.net Wed Sep 21 01:37:01 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Tue, 20 Sep 2005 19:37:01 -0400 Subject: [Python-Dev] bool(iter([])) changed between 2.3 and 2.4 In-Reply-To: Message-ID: <001c01c5be3c$53130dc0$6522c797@oemcomputer> [Guido] > I just finished debugging some code that broke after upgrading to > Python 2.4 (from 2.3). Turns out the code was testing list iterators > for their boolean value (to distinguish them from None). In 2.3, a > list iterator (like any iterator) is always true. In 2.4, an exhausted > list iterator is false; probably by virtue of having a __len__() > method that returns the number of remaining items. > > I realize that this was a deliberate feature, and that it exists in > 2.4 as well as in 2.4.1 and will in 2.4.2; yet, I'm not sure I *like* > it. Was this breakage (which is not theoretical!) considered at all? It was not considered. AFAICT, 2.3 code assuming the Boolean value of an iterator being true was relying on an accidental implementation detail that may not also be true in Jython, PyPy, etc. Likewise, it is not universally true for arbitrary class based iterators which may have other methods including __nonzero__ or __len__. The Boolean value of an iterator is certainly not promised by the iterator protocol as specified in the docs or the PEP. The code, bool(it), is not really clear about its intent and seems a little weird to me. The reason it wasn't considered was that it wasn't on the radar screen as even a possible use case. On a tangential note, I think in 2.2 or 2.3, we found a number of bugs related to None testing. IIRC, the outcome of that conversation was a specific recommendation to NOT determine Noneness by Boolean tests. That recommendation ended-up making it into PEP 290: http://www.python.org/peps/pep-0290.html#testing-for-none [Fred] > think iterators shouldn't have length at all: > they're *not* containers and shouldn't act that way. Some iterators can usefully report their length with the invariant: len(it) == len(list(it)). There are some use cases for having the length when available. Also, there has been plenty of interest in being able to tell, when possible, if an iterator is empty without having to call it. AFAICT, the only downside was Guido's bool(it) situation. FWIW, the origin of the idea came from reading a comp-sci paper about ways to overcome the limitations of linking operations together using only iterators (the paper's terminology talked about map/fold operations). The issue was that decoupling benefits were partially offset by the loss of useful information about the input to an operation (i.e. the supplier may know and the consumer may want to know the input size, the input type, whether the elements are unique, whether the data is sorted, its provenance, etc.) Raymond From jjl at pobox.com Wed Sep 21 01:41:16 2005 From: jjl at pobox.com (John J Lee) Date: Tue, 20 Sep 2005 23:41:16 +0000 (UTC) Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <1127253700.31450.32.camel@geddy.wooz.org> <1127256567.1217.17.camel@geddy.wooz.org> Message-ID: On Tue, 20 Sep 2005, John J Lee wrote: [...] > With the former, we have a more C-style syntax where meaning is determined > purely by delimeters rather than by whitespace. Instead of braces '{' and > '}', we have 'then' and 'elif'/'else'. That's a real difference. [...] Um, not clear, sorry: the "real difference" I meant to refer to above is that between delimiter-based and whitespace-based syntaxes (and not between braces and differently-spelled delimiters). John From tjreedy at udel.edu Wed Sep 21 01:55:26 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 20 Sep 2005 19:55:26 -0400 Subject: [Python-Dev] bool(iter([])) changed between 2.3 and 2.4 References: Message-ID: "Guido van Rossum" wrote in message news:ca471dc20509201449652f11d at mail.gmail.com... >I just finished debugging some code that broke after upgrading to > Python 2.4 (from 2.3). Turns out the code was testing list iterators > for their boolean value (to distinguish them from None). This seem unnecessarily indirect. Why not 'it != None' or now, 'it is not None'? >In 2.3, a > list iterator (like any iterator) is always true. In 2.4, an exhausted > list iterator is false; probably by virtue of having a __len__() > method that returns the number of remaining items. According to 2.4 News, dict iterators also got __len__ method, though I saw no mention of list. > I realize that this was a deliberate feature, I presume there were two reasons: internal efficiency of preallocations (list(some_it) for example) and letting people differentiate iterator with something left to return versus nothing, just as we can differentiate collections with something versus nothing. > and that it exists in > 2.4 as well as in 2.4.1 and will in 2.4.2; yet, I'm not sure I *like* > it. Was this breakage (which is not theoretical!) considered at all? Searching the gmane archive with the link given on the python site for 'iterator len', I could not find any discussion of the __len__ method addition itself. Terry J. Reedy From guido at python.org Wed Sep 21 02:07:35 2005 From: guido at python.org (Guido van Rossum) Date: Tue, 20 Sep 2005 17:07:35 -0700 Subject: [Python-Dev] bool(iter([])) changed between 2.3 and 2.4 In-Reply-To: <001c01c5be3c$53130dc0$6522c797@oemcomputer> References: <001c01c5be3c$53130dc0$6522c797@oemcomputer> Message-ID: On 9/20/05, Raymond Hettinger wrote: > [Guido] > > I just finished debugging some code that broke after upgrading to > > Python 2.4 (from 2.3). Turns out the code was testing list iterators > > for their boolean value (to distinguish them from None). In 2.3, a > > list iterator (like any iterator) is always true. In 2.4, an exhausted > > list iterator is false; probably by virtue of having a __len__() > > method that returns the number of remaining items. > > > > I realize that this was a deliberate feature, and that it exists in > > 2.4 as well as in 2.4.1 and will in 2.4.2; yet, I'm not sure I *like* > > it. Was this breakage (which is not theoretical!) considered at all? > > It was not considered. That's too bad. > AFAICT, 2.3 code assuming the Boolean value of > an iterator being true was relying on an accidental implementation > detail that may not also be true in Jython, PyPy, etc. That's bullshit, and you know it -- you're just using this to justify that you didn't think of this. Whether an object is true or not is well-defined by the language and not by an accident of the implementation. Apart from None, all objects are always true unless they define either __nonzero__() or (in its absence) __len__(). The iterators for builtin sequences were carefully designed to have the minimal API required of iterators -- i.e., next() and __iter__() and nothing more. > Likewise, it is > not universally true for arbitrary class based iterators which may have > other methods including __nonzero__ or __len__. And those are the *only* ones that affect the boolean value. > The Boolean value of an > iterator is certainly not promised by the iterator protocol as specified > in the docs or the PEP. it was implied by not specifying a __nonzero__ or __len__. > The code, bool(it), is not really clear about > its intent and seems a little weird to me. Of course that's not what the broken code actually looked like. If was something like if ...: iter1 = iter(...) else: iter1 = None if ...: iter2 = iter(...) else: iter2 = None ... if iter1 and iter2: ... Where the arguments to the iter() functions were known to be lists. > The reason it wasn't > considered was that it wasn't on the radar screen as even a possible use > case. Could you at least admit that this was an oversight and not try to pretend it was intentional breakage? > On a tangential note, I think in 2.2 or 2.3, we found a number of bugs > related to None testing. IIRC, the outcome of that conversation was a > specific recommendation to NOT determine Noneness by Boolean tests. > That recommendation ended-up making it into PEP 290: > > http://www.python.org/peps/pep-0290.html#testing-for-none And I agree with that one in general (I was bitten by this in Zope once). But it bears a lot more weight when the type of the object is unknown or partially unknown. In my case, there was no possibility that the iter() argument was anything except a list, so the type of the iterator was fully known. > [Fred] > > think iterators shouldn't have length at all: > > they're *not* containers and shouldn't act that way. > > Some iterators can usefully report their length with the invariant: > len(it) == len(list(it)). I still consider this an erroneous hypergeneralization of the concept of iterators. Iterators should be pure iterators and not also act as containers. Which other object type implements __len__ but not __getitem__? > There are some use cases for having the length when available. Also, > there has been plenty of interest in being able to tell, when possible, > if an iterator is empty without having to call it. AFAICT, the only > downside was Guido's bool(it) situation. Theer is plenty of interest in broken features all the time. IMO giving *some* iterators discoverable length (and other properties like reversability) but not all of them makes the iterator protocol more error-prone -- we're back to the situation where someone codes an algorithm for use with arbitrary iterators but only tests it with list and tuple iterators, and ends up breaking in the field. I know you can work around it, but that requires introspection which is not a great match for this kind of application. > FWIW, the origin of the idea came from reading a comp-sci paper about > ways to overcome the limitations of linking operations together using > only iterators (the paper's terminology talked about map/fold > operations). The issue was that decoupling benefits were partially > offset by the loss of useful information about the input to an operation > (i.e. the supplier may know and the consumer may want to know the input > size, the input type, whether the elements are unique, whether the data > is sorted, its provenance, etc.) Not every idea written up in a comp-sci paper is worth implementing (as acquisition in Zope 2 has amply proved). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Sep 21 02:11:35 2005 From: guido at python.org (Guido van Rossum) Date: Tue, 20 Sep 2005 17:11:35 -0700 Subject: [Python-Dev] bool(iter([])) changed between 2.3 and 2.4 In-Reply-To: References: Message-ID: On 9/20/05, Terry Reedy wrote: > I presume there were two reasons: internal efficiency of preallocations > (list(some_it) for example) This could have been implemented without making the implementation details public. > and letting people differentiate iterator with > something left to return versus nothing, just as we can differentiate > collections with something versus nothing. But this is against the very idea of the iterator protocol -- for general iterators, there may be no way to determine whether there is a next item without actually producing the item, so the proper approach is to code with that in mind. When this type of look-ahead is required, a buffering iterator should be inserted, so that the algorithm can work with all iterators rather than only with iterators over built-in containers. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Wed Sep 21 02:24:45 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 21 Sep 2005 12:24:45 +1200 Subject: [Python-Dev] For/while/if statements/comprehension/generator expressions unification In-Reply-To: <8410256206.20050920115659@sinn.ru> References: <8410256206.20050920115659@sinn.ru> Message-ID: <4330A84D.1010806@canterbury.ac.nz> Alexander Myodov wrote: > for i in largelist while !found: This doesn't cover enough use cases to be worth it, IMO. The exit condition is often buried in the middle of other statements in the loop, not right at the beginning. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Wed Sep 21 02:55:56 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 21 Sep 2005 12:55:56 +1200 Subject: [Python-Dev] os.path.diff(path1, path2) In-Reply-To: <1127224142.14254.20.camel@mulkomp.mst-plus> References: <20050913031157.46635.qmail@web51809.mail.yahoo.com> <1127044776.14284.9.camel@mulkomp.mst-plus> <432F7453.1050105@canterbury.ac.nz> <1127224142.14254.20.camel@mulkomp.mst-plus> Message-ID: <4330AF9C.3090509@canterbury.ac.nz> Matthias Andreas Benkard wrote: > But this begs the question: What is relpath good for, anyway? [Assuming you mean "invites", "prompts", etc. the question...:-)] A couple of use cases I've encountered: * Creating symbolic links. I'm traversing a directory hierarchy, and building a parallel hierarchy somewhere else that contains symbolic links into the original one. I want the links to be relative, so that e.g. I can move my home directory without all my links breaking. * In a recent project, I had occasion to store pathnames of picture files in a relational database. I wanted to store the pathnames relative to a user-chosen "Pictures" directory, so that it could be moved without having to update all the pathnames in the database. Both of these happen to involve pathnames that exist on the currrently available file system, but I can easily imagine cases where that would not be so. E.g. I might be generating pathames to go into a tar file that will be unpacked in a different place or on another system. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From pje at telecommunity.com Wed Sep 21 03:01:19 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 20 Sep 2005 21:01:19 -0400 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <1CF74C6F-7521-4751-95B1-2DF71C053251@redivi.com> References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <8393fff0509190928196bba9a@mail.gmail.com> <2mirwwlsve.fsf@starship.python.net> <9613db6005091913206dcf51ec@mail.gmail.com> Message-ID: <5.1.1.6.0.20050920183311.0366b0d0@mail.telecommunity.com> At 06:10 PM 9/20/2005 -0400, Bob Ippolito wrote: >My use case for this isn't so much about threads, but plug-ins. >Writing multiple Python-based plug-ins for an application is always a >mess, because they share too much (sys.modules, sys.path, etc.). >PyObjC would benefit greatly from this feature, because you can write >Python-based plug-ins for any Cocoa app that supports plug-ins, even >if they're otherwise unaware of Python's existence. There are >workarounds, of course, with import hooks and similar hacks. I think >that mod_python would also benefit from this, and probably other such >systems. To that list of shared things, one can easily add sys.argv, sys.stdin, sys.stdout, sys.stderr, os.environ, and on and on and on. But the ones you mention present special issues for testing, especially testing of import hooks, path management tools and so on. Decimal contexts are slightly better in that they're at least thread-local. But even if all of these things were thread-local, you'd still have problems with task switches between pseudothreads, or just maintaining logically separate contexts between e.g. different plugins. I've actually got an idea of how to solve these problems, at least for Python-level code, although for it to work right with Python's import internals it'd be necessary to use PyObject_* APIs against sys.modules instead of PyDict_* ones. Apart from that, it's implementable in pure Python, so non-CPython implementations should be able to use it too. I've been considering writing a PEP for it, since it also adds a generally-useful way of dealing with "context variables" (like the Decimal context and the sys.* variables), while being able to support switching *all* context variables simultaneously and instantaneously when changing execution contexts (like switching between coroutines). For example, it would let 1000 pseudothreads each have a unique current value for sys.stdout or the current decimal context, addressing a gap in the fit between PEP 343 and PEP 342. There's currently no reasonable way to task switch between co-routines in the body of a 'with:' statement, so things like "with redirecting_stdout()" to two places in different coroutines breaks horribly. But context managers implemented via context variables would handle such situations automatically, and with isolation of effects to the current thread. It would also allow you to have logically distinct plugins by swapping their context variables in when you make callbacks to them, so the notion of the "task" to which a set of context variables applies is not limited to coroutines. My prototype implementation of the basic idea is <200 lines of Python and very fast: ~2usec on my budget PC to switch between two arbitrarily-sized sets of context variables. The data structure is O(1) for switching sets or changing values; it's only O(N) the next time you change a value after taking a read-only snapshot of the whole context. (Taking a snapshot is O(1) due to copy-on-write; you can take as many snapshots of the same unchanged state without creating any new objects.) It also takes about 2uS to read a context variable, alas, but a C version could drop the overhead down to as little as a single dictionary lookup if the "context pointer" were kept in the thread state structure. (A fair bit of the Python implementation's overhead is just getting to the context mapping for the current thread.) The idea isn't fully PEP-ready at this point, though. I still need to flesh out my ideas regarding how context variables would be initialized in new threads, for example, or when you spawn new coroutines. From tjreedy at udel.edu Wed Sep 21 03:21:26 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 20 Sep 2005 21:21:26 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <1127253700.31450.32.camel@geddy.wooz.org> <1127256567.1217.17.camel@geddy.wooz.org> Message-ID: "John J Lee" wrote in message news:Pine.LNX.4.58.0509202331440.6289 at alice... > The stricter form where you don't allow 'elif' will get used in more > restricted circumstances, so gives less encouragement for widespread > abuse > of conditional expressions by people who don't like whitespace-based > syntax. I think 'abusiveness' is somewhat in the eye of the beholder here. In any case, without 'elif', one could still chain expressions by writing x = (if a then b else (if c then d else e)) which is even more Lispish. I personally think one () is enough and prefer x = (if a then b elif c then d else e) which is similar to how I wrote such things in C. Elif has the same justification here as with the statement form, where it is also unnecessary but saves nesting levels. Terry J. Reedy From greg.ewing at canterbury.ac.nz Wed Sep 21 03:24:29 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 21 Sep 2005 13:24:29 +1200 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: <4330B64D.7090400@canterbury.ac.nz> Terry Reedy wrote: > "The original version of this PEP proposed the following syntax: > if else > The out-of-order arrangement was found to be too uncomfortable > for many of participants in the discussion; especially when > is long, it's easy to miss the conditional while > skimming." Guido wrote that while he was in listen-to-the-masses mode. If he's switched back to follow-my-instincts mode, it may need to be re-evaluated. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Wed Sep 21 03:30:54 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 21 Sep 2005 13:30:54 +1200 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: <4330B7CE.50302@canterbury.ac.nz> Guido van Rossum wrote: > I think if we go with (if ... then ... else ...) or (if ...: > ... else: ...) we'll have to support elif as well: > > (if ... then ... elif ... then ... else ...) > or > (if ...: ... elif ...: ... else: ...) One nice thing about "x if b else y" is that it chains without needing any more keywords: x if b else y if c else z But if you require parens, it's not so nice: (x if b else (y if c else z)) -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Wed Sep 21 03:33:57 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 21 Sep 2005 13:33:57 +1200 Subject: [Python-Dev] bool(iter([])) changed between 2.3 and 2.4 In-Reply-To: References: Message-ID: <4330B885.8050606@canterbury.ac.nz> Guido van Rossum wrote: > I just finished debugging some code that broke after upgrading to > Python 2.4 (from 2.3). Turns out the code was testing list iterators > for their boolean value (to distinguish them from None). In 2.3, a > list iterator (like any iterator) is always true. In 2.4, an exhausted > list iterator is false; probably by virtue of having a __len__() > method that returns the number of remaining items. This seems like a misfeature to me. In fact I think an iterator having a len() method at all is a misfeature -- the concept doesn't make sense. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Wed Sep 21 03:39:07 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 21 Sep 2005 13:39:07 +1200 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <740c3aec05092009025c929a0d@mail.gmail.com> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <740c3aec05092009025c929a0d@mail.gmail.com> Message-ID: <4330B9BB.4070401@canterbury.ac.nz> BJ?rn Lindqvist wrote: > Wouldn't it be possible to omit "and" and "or" to just "then" and "else"? > > x = 3 and 7 or 44 > x = 3 and then 7 or else 44 > x = 3 then 7 else 44 > > And then have another set of and and or for non short-circuiting? I don't think the OP was suggesting that 'and' and 'or' be non-short-circuiting, only that they coerce their evaluated operands to bool. Short-circuiting vs. non-short-circuiting is an orthogonal issue. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Wed Sep 21 03:39:10 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 21 Sep 2005 13:39:10 +1200 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: <4330B9BE.30903@canterbury.ac.nz> Fredrik Lundh wrote: > for Python 3000, I'd recommend switching to "and then" and "or else" instead > of the current ambiguous single-keyword versions. And then we could call the language Pyffel. :-) -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From reinhold-birkenfeld-nospam at wolke7.net Wed Sep 21 09:17:23 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Wed, 21 Sep 2005 09:17:23 +0200 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: Jason Orendorff wrote: > Honestly, I think I would prefer this syntax. Examples from real > code, before and after: > > lines = [line for line in pr.block.body > if line.logical_line.strip() != ''] > lines = [for line in pr.block.body: > if line.logical_line.strip() != '': > line] > > row.values = \ > [line[col.start:col.end].strip() for col in columns] > row.values = \ > [for col in columns: line[col.start:col.end].rstrip()] > > return [p for p in self.listdir(pattern) if p.isdir()] > return [for p in self.listdir(pattern): if p.isdir(): p] -1. Too much similarity with the for/if statement. People would say "why can we put a for statement in brackets but not a try statement". Reinhold -- Mail address is perfectly valid! From mwh at python.net Wed Sep 21 11:39:29 2005 From: mwh at python.net (Michael Hudson) Date: Wed, 21 Sep 2005 10:39:29 +0100 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: (Brett Cannon's message of "Tue, 20 Sep 2005 11:28:48 -0700") References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2mzmq8jgzr.fsf@starship.python.net> Message-ID: <2mr7bikbe6.fsf@starship.python.net> Brett Cannon writes: > On 9/20/05, Michael Hudson wrote: > [SNIP] >> I _like_ the explanation of 'and' and 'or' as they are now. They are >> basically control flow constructs -- and have to be to get >> short-circuiting to work -- and adding a coercion to bool at the end >> seems to add complexity, not reduce it (on some levels, anyway). >> > > If you change the definition of 'and' and 'or' to be boolean > comparison operators (as Raymond is proposing) and not as control flow > constructs then is it really that complicated? If you eliminate the short circuiting behaviour of 'or' and 'and' the mobs will be after you with torches and pitchforks (and I'll be with them). > I think it would actually simplify things very slightly since you > just say a boolean is returned instead of the last executed > expression by the operator. You might as well have 'and' be a builtin, then -- or do I misread you? >> > P.S. Simplifying "and" and "or" may create a need to introduce a >> > conditional operator but that is a discussion for another day. >> >> ... which was in the past, I thought. > > It was, but changing 'and' and 'or' does tweak the usefulness of a > conditional operator. Another reason why it's a bad idea :) Cheers, mwh -- I've reinvented the idea of variables and types as in a programming language, something I do on every project. -- Greg Ward, September 1998 From gmccaughan at synaptics-uk.com Wed Sep 21 11:52:53 2005 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Wed, 21 Sep 2005 10:52:53 +0100 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> Message-ID: <200509211052.54368.gmccaughan@synaptics-uk.com> > I think I'd prefer (if then else ) i.e. no > colons. None of the other expression forms (list comprehensions and > generator expressions) involving statement keywords use colons. FWLIW, this was my (strong) preference back at the time of the original discussion. -- g From mwh at python.net Wed Sep 21 12:10:37 2005 From: mwh at python.net (Michael Hudson) Date: Wed, 21 Sep 2005 11:10:37 +0100 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: (Guido van Rossum's message of "Tue, 20 Sep 2005 12:17:41 -0700") References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> Message-ID: <2mmzm6k9ya.fsf@starship.python.net> Guido van Rossum writes: >> Given the later addition of generator expressions with mandatory >> parentheses , the mandatory-parentheses version of a conditional expression >> looks less strange to me than it did then ;-). So I could happily use it >> even though I may still lean toward the other option 2 version (then-else) >> due to its not needing ':'s or a third elseif term for chaining. > > I think I'd prefer (if then else ) i.e. no > colons. My problem with this syntax is that it can be hard to read: return if self.arg is None then default else self.arg looks worryingly like return NAME NAME.NAME NAME NAME NAME NAME NAME NAME.NAME to me. > None of the other expression forms (list comprehensions and > generator expressions) involving statement keywords use colons. This is also true. >> *If* you want general community input, I would suggest a runoff ballot with >> those four choices (and a summary of pros and cons of each), or fewer if >> you see any as unacceptible. > > If there's one thing I've learned from the PEP 308 vote, it is that > votes for language don't work. I prefer some discussion on Python-dev > after which I pick one. Well, this is my input (and now I'm going to try and stay out of it). Cheers, mwh -- 59. In English every word can be verbed. Would that it were so in our programming languages. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html From abo at minkirri.apana.org.au Wed Sep 21 12:33:03 2005 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Wed, 21 Sep 2005 11:33:03 +0100 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <8393fff0509190928196bba9a@mail.gmail.com> <2mirwwlsve.fsf@starship.python.net> <9613db6005091913206dcf51ec@mail.gmail.com> Message-ID: <1127298783.15329.64.camel@parabolic.corp.google.com> On Tue, 2005-09-20 at 22:43, Guido van Rossum wrote: > On 9/20/05, John J Lee wrote: [...] > I don't know that any chips are designed with threading in mind. Fast > threading benefits from fast context switches which benefits from > small register sets. I believe the trend is towards ever large > register sets. Also, multiple processors with shared memory don't > scall all that well; multiple processors with explicit IPC channels > scale much better. All arguments for multi-processing and against > multi-threading. Exactly! I believe the latest MP opteron chipsets use hypertransport busses to directly access the other processor's memory and possibly CPU cache. In theory this means shared memory will not hurt too badly, helping threading. However, memory contention bottlenecks and cache coherency will always mean shared memory hurts more, and will never scale better, than IPC. The reality is threads were invented as a low overhead way of easily implementing concurrent applications... ON A SINGLE PROCESSOR. Taking into account threading's limitations and objectives, Python's GIL is the best way to support threads. When hardware (seriously) moves to multiple processors, other concurrency models will start to shine. In the short term there will be various hacks to try and make the existing plethora of threading applications run better on multiple processors, but ultimately the overheads of shared memory will force serious multi-processing to use IPC channels. If you want serious MP, use processes, not threads. I see anti-GIL threads again and again. Get over it... the GIL rocks for threads :-) -- Donovan Baarda From mark.russell at redmoon.me.uk Wed Sep 21 13:08:53 2005 From: mark.russell at redmoon.me.uk (Mark Russell) Date: Wed, 21 Sep 2005 12:08:53 +0100 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <2mmzm6k9ya.fsf@starship.python.net> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <2mmzm6k9ya.fsf@starship.python.net> Message-ID: <1127300933.5473.74.camel@localhost> On Wed, 2005-09-21 at 11:10, Michael Hudson wrote: > My problem with this syntax is that it can be hard to read: > > return if self.arg is None then default else self.arg > > looks worryingly like > > return NAME NAME.NAME NAME NAME NAME NAME NAME NAME.NAME > > to me. I think that requriing parens helps a lot with the list-of-names problem - it nicely delimits the conditional expression for human readers: return (if self.arg is None then default else self.arg) In particular it breaks up the misleading grouping "return if". Mark Russell From ncoghlan at gmail.com Wed Sep 21 13:14:57 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 21 Sep 2005 21:14:57 +1000 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <4330B7CE.50302@canterbury.ac.nz> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <4330B7CE.50302@canterbury.ac.nz> Message-ID: <433140B1.2070406@gmail.com> Greg Ewing wrote: > Guido van Rossum wrote: > > >>I think if we go with (if ... then ... else ...) or (if ...: >>... else: ...) we'll have to support elif as well: >> >>(if ... then ... elif ... then ... else ...) >>or >>(if ...: ... elif ...: ... else: ...) > > > One nice thing about "x if b else y" is that it > chains without needing any more keywords: > > x if b else y if c else z > > But if you require parens, it's not so nice: > > (x if b else (y if c else z)) > If Guido chose this form, I would expect the chaining to work like chaining loops in a generator expression, with parentheses being required around the whole thing, rather than around each element in the chain: (x if b else y if c else z) The point being that the result of the conditional expression is exactly one of the options included in the expression, so only one set of parentheses is required. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From skip at pobox.com Wed Sep 21 16:22:09 2005 From: skip at pobox.com (skip@pobox.com) Date: Wed, 21 Sep 2005 09:22:09 -0500 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <2mr7bikbe6.fsf@starship.python.net> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2mzmq8jgzr.fsf@starship.python.net> <2mr7bikbe6.fsf@starship.python.net> Message-ID: <17201.27793.875588.293551@montanaro.dyndns.org> I suppose this is a dead horse now, but I will kick it one more time. Under the rubrics of "explicit is better than implicit" and "there should only be one wat to do it", I would rather see bool_val = bool(foo or bar) instead of having the "or" operator implicitly call bool() for me. There's clear value to the current semantics and it's so easy to get a boolean if you want it, I see no reason for a change. Skip From guido at python.org Wed Sep 21 17:26:55 2005 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Sep 2005 08:26:55 -0700 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <1127298783.15329.64.camel@parabolic.corp.google.com> References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <8393fff0509190928196bba9a@mail.gmail.com> <2mirwwlsve.fsf@starship.python.net> <9613db6005091913206dcf51ec@mail.gmail.com> <1127298783.15329.64.camel@parabolic.corp.google.com> Message-ID: On 9/21/05, Donovan Baarda wrote: > The reality is threads were invented as a low overhead way of easily > implementing concurrent applications... ON A SINGLE PROCESSOR. Taking > into account threading's limitations and objectives, Python's GIL is the > best way to support threads. QOTF candidate! (I wonder if this thread could be summarized into a PEP we can use instead of future discussions rehashing the same issues?) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rrr at ronadam.com Wed Sep 21 17:39:45 2005 From: rrr at ronadam.com (Ron Adam) Date: Wed, 21 Sep 2005 11:39:45 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <433140B1.2070406@gmail.com> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <4330B7CE.50302@canterbury.ac.nz> <433140B1.2070406@gmail.com> Message-ID: <43317EC1.908@ronadam.com> Nick Coghlan wrote: > Greg Ewing wrote: >>One nice thing about "x if b else y" is that it >>chains without needing any more keywords: >> >> x if b else y if c else z >> >>But if you require parens, it's not so nice: >> >> (x if b else (y if c else z)) > If Guido chose this form, I would expect the chaining to work like chaining > loops in a generator expression, with parentheses being required around the > whole thing, rather than around each element in the chain: > > (x if b else y if c else z) > > The point being that the result of the conditional expression is exactly one > of the options included in the expression, so only one set of parentheses is > required. Either a or b could be a nested expression so it's important that it be readable in both cases. (a if e then b) ((a1 if e1 then b1) if e then b) (a if e then (a2 if e2 then b2)) ((a1 if e1 then b1) if e then (a2 if e2 then b2)) Without parentheses... (a if e then b) (a1 if e1 then b1 if e then b) (a if e then a2 if e2 then b2) (a1 if e1 then b1 if e then a2 if e2 then b2) I think the parentheses version is clearer. To me it's not as easy to picture what will happen when the condition is in the middle of the expression. Also in the above, is e1 evaluated before e? That would mean the result of e1 (a1 or b1) is thrown away if e if false. So looking at a few alternatives ... (e ? a : b) (e ? (e1 ? a1 : b1) : b) (e ? a : (e2 ? a2 : b2)) (e ? (e1 ? a1 : b1) : (e2 ? a2 : b2)) This better represents a decision tree I think. Each comparison gives one of two possible results which may be another comparison. Using keywords instead... (if e, a else b) (if e, (if e1, a1 else b1) else b) (if e, a else (if e2, a2 else b2)) (if e, (if e1, a1 else b1) else (if e2, a2 else b2)) (if e then a else b) (if e then (if e1 then a1 else b1) else b) (if e then a else (if e2 then a2 else b2)) (if e then (if e1 then a1 else b1) else (if e2 then a2 else b2)) or... possibly... (e selects a else b) (e selects (e1 selects a1 else b1) else b) (e selects a else (e2 selects a2 else b2)) (e selects (e1 selects a1 else b1) else (e2 selects a2 else b2)) I like this one, but maybe a shorter verb would be nice. Other possible words might be "gives", "chooses" or "picks". With the (e?a:b) syntax, I tend to want to switch the '?' and ':' here so that the colon is more consistent with how Python uses it. (e: a ? b) (e: (e1: a1 ? b1) ? b) (e: a ? (e2: a2 ? b2)) (e: (e1: a1 ? b1) ? (e2: a2 ? b2)) That may be more confusing to those who are use to 'C', but clearer to those who use Python as their main programming language. The '?' becomes an 'else' which might be useful in other expressions. Without the colon ... (e selects a ? b) (e selects (e1 selects a1 ? b1) ? b) (e selects a ? (e2 selects a2 ? b2)) (e selects (e1 selects a1 ? b1) ? (e2 selects a2 ? b2)) Or if e evaluates an integer... :-) (e selects a, b, c, ...) I think this would be quite useful and would work perfectly well with boolean expressions as well. The advantage here is that a,b,c etc.. would not be pre evaluated as they are when you use a list or dictionary. (e selects a, b) (e selects (e1 selects a1, b1), b) (e selects a, (e2 selects a2, b2)) (e selects (e1 selects a1, b1), (e2 selects a2, b2)) ( e selects (e1 selects a1, b1), (e2 selects a2, b2), (e3 selects a3, b3) ) Being able to have more than two alternative may reduce the need to nest or chain them in some cases. A variation might be to have negative index's pick from the far end just as list index's do. This would be my choice although I wasn't thinking of it when I started this reply. ;-) Cheers, Ron From gherron at islandtraining.com Wed Sep 21 17:46:47 2005 From: gherron at islandtraining.com (Gary Herron) Date: Wed, 21 Sep 2005 08:46:47 -0700 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <2mmzm6k9ya.fsf@starship.python.net> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <2mmzm6k9ya.fsf@starship.python.net> Message-ID: <43318067.3090301@islandtraining.com> Michael Hudson wrote: >Guido van Rossum writes: > > > >>>Given the later addition of generator expressions with mandatory >>>parentheses , the mandatory-parentheses version of a conditional expression >>>looks less strange to me than it did then ;-). So I could happily use it >>>even though I may still lean toward the other option 2 version (then-else) >>>due to its not needing ':'s or a third elseif term for chaining. >>> >>> >>I think I'd prefer (if then else ) i.e. no >>colons. >> >> > >My problem with this syntax is that it can be hard to read: > >return if self.arg is None then default else self.arg > >looks worryingly like > >return NAME NAME.NAME NAME NAME NAME NAME NAME NAME.NAME > >to me. > > But that's exactly what any language looks like if you get abstract enough: WORD WORD WORD WORD WORD WORD WORD And in fact, one read and understands your return statement just like an English sentence -- word by word from beginning to end. This seems an argument FOR the syntax not against. Moreover, if one uses the proposed parenthesized syntax, even the slightly odd word order of "return if" is mitigated. return (if self.arg is None then default else self.arg) > > >>None of the other expression forms (list comprehensions and >>generator expressions) involving statement keywords use colons. >> >> > >This is also true. > > > >>>*If* you want general community input, I would suggest a runoff ballot with >>>those four choices (and a summary of pros and cons of each), or fewer if >>>you see any as unacceptible. >>> >>> >>If there's one thing I've learned from the PEP 308 vote, it is that >>votes for language don't work. I prefer some discussion on Python-dev >>after which I pick one. >> >> > >Well, this is my input (and now I'm going to try and stay out of it). > >Cheers, >mwh > > > From pjones at redhat.com Wed Sep 21 18:04:03 2005 From: pjones at redhat.com (Peter Jones) Date: Wed, 21 Sep 2005 12:04:03 -0400 Subject: [Python-Dev] unintentional and unsafe use of realpath() In-Reply-To: <1126725908.23071.11.camel@localhost.localdomain> References: <1126380571.21655.18.camel@localhost.localdomain> <43276823.2000603@levkowetz.com> <1126725908.23071.11.camel@localhost.localdomain> Message-ID: <1127318643.12452.1.camel@localhost.localdomain> On Wed, 2005-09-14 at 15:25 -0400, Peter Jones wrote: [ comments and a patch for sysmodule.c and some configure related files] ... and that patch has obvious problems as well. Here's a corrected one: --- Python-2.4.1/pyconfig.h.in.canonicalize 2005-09-14 11:47:04.000000000 -0400 +++ Python-2.4.1/pyconfig.h.in 2005-09-14 11:47:02.000000000 -0400 @@ -58,6 +58,9 @@ /* Define if pthread_sigmask() does not work on your system. */ #undef HAVE_BROKEN_PTHREAD_SIGMASK +/* Define to 1 if you have the `canonicalize_file_name' function. */ +#undef HAVE_CANONICALIZE_FILE_NAME + /* Define to 1 if you have the `chown' function. */ #undef HAVE_CHOWN --- Python-2.4.1/Python/sysmodule.c.canonicalize 2005-09-14 11:53:30.000000000 -0400 +++ Python-2.4.1/Python/sysmodule.c 2005-09-14 11:52:04.000000000 -0400 @@ -1184,6 +1184,11 @@ char *p = NULL; int n = 0; PyObject *a; +#ifdef HAVE_CANONICALIZE_FILE_NAME + argv0 = canonicalize_file_name(argv0); + if (argv0 == NULL) + Py_FatalError("no mem for sys.argv"); +#else /* ! HAVE_CANONICALIZE_FILE_NAME */ #ifdef HAVE_READLINK char link[MAXPATHLEN+1]; char argv0copy[2*MAXPATHLEN+1]; @@ -1256,9 +1261,13 @@ #endif /* Unix */ } #endif /* All others */ +#endif /* ! HAVE_CANONICALIZE_FILE_NAME */ a = PyString_FromStringAndSize(argv0, n); if (a == NULL) Py_FatalError("no mem for sys.path insertion"); +#ifdef HAVE_CANONICALIZE_FILE_NAME + free(argv0); +#endif /* HAVE_CANONICALIZE_FILE_NAME */ if (PyList_Insert(path, 0, a) < 0) Py_FatalError("sys.path.insert(0) failed"); Py_DECREF(a); --- Python-2.4.1/configure.in.canonicalize 2005-09-14 11:46:00.000000000 -0400 +++ Python-2.4.1/configure.in 2005-09-14 11:47:22.000000000 -0400 @@ -2096,8 +2096,8 @@ AC_MSG_RESULT(MACHDEP_OBJS) # checks for library functions -AC_CHECK_FUNCS(alarm bind_textdomain_codeset chown clock confstr ctermid \ - execv fork fpathconf ftime ftruncate \ +AC_CHECK_FUNCS(alarm bind_textdomain_codeset canonicalize_file_name chown \ + clock confstr ctermid execv fork fpathconf ftime ftruncate \ gai_strerror getgroups getlogin getloadavg getpeername getpgid getpid \ getpriority getpwent getsid getwd \ kill killpg lchown lstat mkfifo mknod mktime \ -- Peter From steven.bethard at gmail.com Wed Sep 21 18:14:07 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 21 Sep 2005 10:14:07 -0600 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <43317EC1.908@ronadam.com> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <4330B7CE.50302@canterbury.ac.nz> <433140B1.2070406@gmail.com> <43317EC1.908@ronadam.com> Message-ID: Adam wrote: > So looking at a few alternatives ... > [snip] > (e ? (e1 ? a1 : b1) : (e2 ? a2 : b2)) > [snip] > (if e, (if e1, a1 else b1) else (if e2, a2 else b2)) > [snip] > (if e then (if e1 then a1 else b1) else (if e2 then a2 else b2)) > [snip > (e selects (e1 selects a1 else b1) else (e2 selects a2 else b2)) > [snip] > (e: (e1: a1 ? b1) ? (e2: a2 ? b2)) > [snip] > (e selects (e1 selects a1 ? b1) ? (e2 selects a2 ? b2)) > [snip] > (e selects (e1 selects a1, b1), (e2 selects a2, b2)) > Please no more syntax proposals! There were enough in the PEP. It looks like most people support a conditional expression of some sort. We need to leave the syntax to Guido. We've already proved that like the decorators discussions, we can't as a community agree on a syntax. That's what we have a BDFL for. =) Steve -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From s.percivall at chello.se Wed Sep 21 18:26:11 2005 From: s.percivall at chello.se (Simon Percivall) Date: Wed, 21 Sep 2005 18:26:11 +0200 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <1127298783.15329.64.camel@parabolic.corp.google.com> References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <8393fff0509190928196bba9a@mail.gmail.com> <2mirwwlsve.fsf@starship.python.net> <9613db6005091913206dcf51ec@mail.gmail.com> <1127298783.15329.64.camel@parabolic.corp.google.com> Message-ID: <9519083A-6B06-4F3E-A8BD-47F9684E64B5@chello.se> On 21 sep 2005, at 12.33, Donovan Baarda wrote: > In the short term there will be various hacks to try and make the > existing plethora of threading applications run better on multiple > processors, but ultimately the overheads of shared memory will force > serious multi-processing to use IPC channels. If you want serious MP, > use processes, not threads. The problem here is that while Python offers some support for thread-based multiprogramming in its standard library: theaad, threading, Queue, etc., there doesn't seem to exist much support for process-based multiprogramming beyond the basics. How is the developer helped? with data sharing, communication, control over running processes, dealing out tasks to be handled, etc. The best way to make people stop complaining about the GIL and start using process-based multiprogramming is to provide solid, standardized support for process-based multiprogramming. //Simon From guido at python.org Wed Sep 21 18:32:05 2005 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Sep 2005 09:32:05 -0700 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <4330B7CE.50302@canterbury.ac.nz> <433140B1.2070406@gmail.com> <43317EC1.908@ronadam.com> Message-ID: On 9/21/05, Steven Bethard wrote: > Please no more syntax proposals! There were enough in the PEP. It > looks like most people support a conditional expression of some sort. > We need to leave the syntax to Guido. We've already proved that like > the decorators discussions, we can't as a community agree on a syntax. > That's what we have a BDFL for. =) Another QOTFcandidate! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From solipsis at pitrou.net Wed Sep 21 18:31:59 2005 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 21 Sep 2005 18:31:59 +0200 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <9519083A-6B06-4F3E-A8BD-47F9684E64B5@chello.se> References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <8393fff0509190928196bba9a@mail.gmail.com> <2mirwwlsve.fsf@starship.python.net> <9613db6005091913206dcf51ec@mail.gmail.com> <1127298783.15329.64.camel@parabolic.corp.google.com> <9519083A-6B06-4F3E-A8BD-47F9684E64B5@chello.se> Message-ID: <1127320319.21126.18.camel@p-dvsi-418-1.rd.francetelecom.fr> > The best way to make people stop complaining about the GIL and start > using > process-based multiprogramming is to provide solid, standardized support > for process-based multiprogramming. 100% agreed. I needed a portable way to know if a program was already running (and then to send it a simple command), the only simple solution I found was to listen on a local TCP socket and store the port number in a file in a known location. Not very elegant IMO. Regards Antoine. From trentm at ActiveState.com Wed Sep 21 18:36:38 2005 From: trentm at ActiveState.com (Trent Mick) Date: Wed, 21 Sep 2005 09:36:38 -0700 Subject: [Python-Dev] os.path.diff(path1, path2) In-Reply-To: <4330AF9C.3090509@canterbury.ac.nz> References: <20050913031157.46635.qmail@web51809.mail.yahoo.com> <1127044776.14284.9.camel@mulkomp.mst-plus> <432F7453.1050105@canterbury.ac.nz> <1127224142.14254.20.camel@mulkomp.mst-plus> <4330AF9C.3090509@canterbury.ac.nz> Message-ID: <20050921163638.GC4701@ActiveState.com> > > But this begs the question: What is relpath good for, anyway? > > A couple of use cases I've encountered: > Another one: - Build tools that work with paths alot can really improve their log readability if relative paths can be used to keep paths short rather than the common fallback of just making everything an absolute path. Trent -- Trent Mick TrentM at ActiveState.com From bjourne at gmail.com Wed Sep 21 18:56:14 2005 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Wed, 21 Sep 2005 18:56:14 +0200 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <2mmzm6k9ya.fsf@starship.python.net> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <2mmzm6k9ya.fsf@starship.python.net> Message-ID: <740c3aec05092109565722b18f@mail.gmail.com> > > I think I'd prefer (if then else ) i.e. no > > colons. > > My problem with this syntax is that it can be hard to read: > > return if self.arg is None then default else self.arg > > looks worryingly like > > return NAME NAME.NAME NAME NAME NAME NAME NAME NAME.NAME That can already be written in Python: return self.arg or default And maybe the syntax of the future conditional operator could be extended so that: return self.arg else default Would be equivalent to the previous line. For stuff were a conditional expression really is needed, like in: return if self.arg then "yes" else "no" One would hope that whoever writes the conditional expression doesn't make the expressions so long and complicated that it looks like NAME NAME.NAME NAME... It doesn't matter which syntax proposal that wins, it will still be very tempting to write unreadable code with it (think nested/chained conditional expressions). That (and the fact that I've never felt a real need for a conditional expression thanks to the "or" operator) is my very humble argument against it. -- mvh Bj?rn From jonathan-lists at cleverdevil.org Wed Sep 21 19:38:13 2005 From: jonathan-lists at cleverdevil.org (Jonathan LaCour) Date: Wed, 21 Sep 2005 13:38:13 -0400 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <9519083A-6B06-4F3E-A8BD-47F9684E64B5@chello.se> References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <8393fff0509190928196bba9a@mail.gmail.com> <2mirwwlsve.fsf@starship.python.net> <9613db6005091913206dcf51ec@mail.gmail.com> <1127298783.15329.64.camel@parabolic.corp.google.com> <9519083A-6B06-4F3E-A8BD-47F9684E64B5@chello.se> Message-ID: <22e91b3e90df9e2ff9f27ac0f3042939@cleverdevil.org> > The best way to make people stop complaining about the GIL and start > using process-based multiprogramming is to provide solid, standardized > support for process-based multiprogramming. +100 Huge amounts of effort would have to be invested to remove the GIL for the benefit of threads. Not only would the effort be huge, the difficulty and complexity of writing extension modules would be increased. Regardless of the arguments about SMP systems and the GIL, Python should provide as much support for process-based multi-programming as it does for threading. How about sinking that same effort into better Python support for process-based multi-programming? All the categories that Simon suggested are great candidates for the targets of this effort. Are there any existing efforts that I don't know about? Jonathan LaCour http://cleverdevil.org From jcarlson at uci.edu Wed Sep 21 19:38:49 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed, 21 Sep 2005 10:38:49 -0700 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <1127320319.21126.18.camel@p-dvsi-418-1.rd.francetelecom.fr> References: <9519083A-6B06-4F3E-A8BD-47F9684E64B5@chello.se> <1127320319.21126.18.camel@p-dvsi-418-1.rd.francetelecom.fr> Message-ID: <20050921102057.F722.JCARLSON@uci.edu> Antoine Pitrou wrote: > > > > The best way to make people stop complaining about the GIL and start > > using > > process-based multiprogramming is to provide solid, standardized support > > for process-based multiprogramming. > > 100% agreed. I needed a portable way to know if a program was already > running (and then to send it a simple command), the only simple solution > I found was to listen on a local TCP socket and store the port number in > a file in a known location. Not very elegant IMO. On *nix, use a unix domain socket (location in the filesystem which acts as a listening socket). On Windows, you can use cTypes, pywin32, etc., to create a global mutex and/or COM interface. Alternatively, you can write your own cross-platform registry for services, have it running on your machines all the time, and never worry again. Or one can pick a port to use on all systems and not bother with the the file listing the port, ignore domain sockets, COM interfaces, and work-alikes. In terms of IPC, the application may determine which is most usable. If one had a service distributed across multiple machines, sockets are necessary. If one only needed local access, shared memory (perhaps via a memory mapped file) would be significantly faster. In the case of multiple machines with multiple processors, one could opt for shared memory locally but sockets globally, which could complicate the interface, result in non-uniform performance for transfers, which may then necessate complicating the application in order to gain the highest throughput (I've done this previously using MPI, it can be a PITA). - Josiah From paragate at gmx.net Wed Sep 21 20:11:25 2005 From: paragate at gmx.net (Wolfgang Lipp) Date: Wed, 21 Sep 2005 20:11:25 +0200 Subject: [Python-Dev] os.path.diff(path1, path2) In-Reply-To: <4330AF9C.3090509@canterbury.ac.nz> References: <20050913031157.46635.qmail@web51809.mail.yahoo.com> <1127044776.14284.9.camel@mulkomp.mst-plus> <432F7453.1050105@canterbury.ac.nz> <1127224142.14254.20.camel@mulkomp.mst-plus> <4330AF9C.3090509@canterbury.ac.nz> Message-ID: On Wed, 21 Sep 2005 02:55:56 +0200, Greg Ewing wrote: > Both of these happen to involve pathnames that exist on > the currrently available file system, but I can easily > imagine cases where that would not be so. E.g. I might > be generating pathames to go into a tar file that will > be unpacked in a different place or on another system. imho, it would be a good thing for a future 'file system handling module' to build more of an abstracted tree-like graph that may or may not be mappable (and may or may not be used in a particular case to actually map) to existing objects on a particular system. for example, i find it a bit in my way a lot of times that all the locators i have os.path handle for me are written according to the os, not in a unified, abstracted way. given that there are a number of applications (xml documents, file system handling, urls, archives...) that have very similar requirements, maybe there is a useful abstraction that covers these and other cases. some nitty-gritty details could be captured by suitable specializations of the general case. _w From ronaldoussoren at mac.com Wed Sep 21 20:11:01 2005 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Wed, 21 Sep 2005 20:11:01 +0200 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <1CF74C6F-7521-4751-95B1-2DF71C053251@redivi.com> References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <8393fff0509190928196bba9a@mail.gmail.com> <2mirwwlsve.fsf@starship.python.net> <9613db6005091913206dcf51ec@mail.gmail.com> <1CF74C6F-7521-4751-95B1-2DF71C053251@redivi.com> Message-ID: On 21-sep-2005, at 0:10, Bob Ippolito wrote: > > My use case for this isn't so much about threads, but plug-ins. > Writing multiple Python-based plug-ins for an application is always a > mess, because they share too much (sys.modules, sys.path, etc.). > PyObjC would benefit greatly from this feature, because you can write > Python-based plug-ins for any Cocoa app that supports plug-ins, even > if they're otherwise unaware of Python's existence. There are > workarounds, of course, with import hooks and similar hacks. I think > that mod_python would also benefit from this, and probably other such > systems. For PyObjC having multiple interpreters in a process would cause me severe headaches. Due to the nature of PyObjC it would be way too easy to accidently access objects from one interpreter in another interpreter (in a hypothetical universe where it would be easy to use multiple interpreters). And lets not get started about the GIL, I don't think its accidental that PyGILState_Ensure only works with one interpreter. A system like Java's classloader would be helpfull, where the classloader of a class is used to load the classes used by that class. I have no idea if this can be adapted to python at all. A strict coding style seems to work for now. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2105 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050921/fc6cd7fe/smime-0001.bin From martin at v.loewis.de Wed Sep 21 20:27:12 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 21 Sep 2005 20:27:12 +0200 Subject: [Python-Dev] possible memory leak on windows (valgrind report) In-Reply-To: References: Message-ID: <4331A600.9060707@v.loewis.de> Neal Norwitz wrote: > (I need to write a lot more suppression rules for gentoo.) This could be due to your using GCC 4. Apparently, gcc 4 is willing to inline Py_ADDRESS_IN_RANGE even though it appears at the end of the file, at -O3. To suppress that, you can declare the function as __attribute__((noinline)). You will need to conditionalize this on gcc, but not only that: it appears that noinline was one of the more recent additions. I'm not sure when it was added, but apparently, it is present in 3.3 and later. Alternatively, just recompiling the file without -O3 also works. Regards, Martin From mwh at python.net Wed Sep 21 20:30:43 2005 From: mwh at python.net (Michael Hudson) Date: Wed, 21 Sep 2005 19:30:43 +0100 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <740c3aec05092109565722b18f@mail.gmail.com> ( =?iso-8859-1?q?BJ=F6rn_Lindqvist's_message_of?= "Wed, 21 Sep 2005 18:56:14 +0200") References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <2mmzm6k9ya.fsf@starship.python.net> <740c3aec05092109565722b18f@mail.gmail.com> Message-ID: <2mfyryjmss.fsf@starship.python.net> BJ?rn Lindqvist writes: >> My problem with this syntax is that it can be hard to read: >> >> return if self.arg is None then default else self.arg >> >> looks worryingly like >> >> return NAME NAME.NAME NAME NAME NAME NAME NAME NAME.NAME > > That can already be written in Python: No it can't! Would you believe I had this exact conversation two and a half years ago? :) > return self.arg or default Consider what happens if self.arg is 0, here. Cheers, mwh -- If I had wanted your website to make noise I would have licked my finger and rubbed it across the monitor. -- signature of "istartedi" on slashdot.org From bcannon at gmail.com Wed Sep 21 20:55:41 2005 From: bcannon at gmail.com (Brett Cannon) Date: Wed, 21 Sep 2005 11:55:41 -0700 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <2mr7bikbe6.fsf@starship.python.net> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2mzmq8jgzr.fsf@starship.python.net> <2mr7bikbe6.fsf@starship.python.net> Message-ID: On 9/21/05, Michael Hudson wrote: > Brett Cannon writes: > > > On 9/20/05, Michael Hudson wrote: > > [SNIP] > >> I _like_ the explanation of 'and' and 'or' as they are now. They are > >> basically control flow constructs -- and have to be to get > >> short-circuiting to work -- and adding a coercion to bool at the end > >> seems to add complexity, not reduce it (on some levels, anyway). > >> > > > > If you change the definition of 'and' and 'or' to be boolean > > comparison operators (as Raymond is proposing) and not as control flow > > constructs then is it really that complicated? > > If you eliminate the short circuiting behaviour of 'or' and 'and' the > mobs will be after you with torches and pitchforks (and I'll be with > them). > I am not suggesting that at all. I would put myself on a pike first before the mob got there. =) > > I think it would actually simplify things very slightly since you > > just say a boolean is returned instead of the last executed > > expression by the operator. > > You might as well have 'and' be a builtin, then -- or do I misread > you? > I think you might be misreading me, but since Guido seems to have made the decision that 'and' and 'or' are not changing there is no need to try to clarify. > >> > P.S. Simplifying "and" and "or" may create a need to introduce a > >> > conditional operator but that is a discussion for another day. > >> > >> ... which was in the past, I thought. > > > > It was, but changing 'and' and 'or' does tweak the usefulness of a > > conditional operator. > > Another reason why it's a bad idea :) > =) -Brett From python-dev-list at cstork.org Wed Sep 21 20:56:01 2005 From: python-dev-list at cstork.org (Christian Stork) Date: Wed, 21 Sep 2005 11:56:01 -0700 Subject: [Python-Dev] Alternative name for str.partition() In-Reply-To: <4313AD16.3070608@canterbury.ac.nz> References: <001901c5ac5d$5985bcc0$eb29c797@oemcomputer> <4312EE82.90207@gmail.com> <4313AD16.3070608@canterbury.ac.nz> Message-ID: <20050921185601.GA22265@anthony.ics.uci.edu> I know I'm coming too late to this discussion, but just for completeness sake let me mention that the OCaml standard List module uses 'partition' already in the sense that most mathematically educated people would understand it: val partition : ('a -> bool) -> 'a list -> 'a list * 'a list partition p l returns a pair of lists (l1, l2), where l1 is the list of all the elements of l that satisfy the predicate p, and l2 is the list of all the elements of l that do not satisfy p. The order of the elements in the input list is preserved. Haskell's Data.List.partion is defined the same way. So this seems to be generally agreed upon, at least for functional programming languages. This is why I have to agree with Greg: On Tue, Aug 30, 2005 at 12:49:26PM +1200, Greg Ewing wrote: > A more descriptive name than 'partition' would be 'split_at'. 'split_at' is really what's happening. (I came up with it independently of Greg, if that is any evidence). -- Chris Stork <> Support eff.org! <> http://www.ics.uci.edu/~cstork/ OpenPGP fingerprint: B08B 602C C806 C492 D069 021E 41F3 8C8D 50F9 CA2F From tjreedy at udel.edu Wed Sep 21 20:41:39 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 21 Sep 2005 14:41:39 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <4330B7CE.50302@canterbury.ac.nz> <433140B1.2070406@gmail.com><43317EC1.908@ronadam.com> Message-ID: Guido: When you invited resumed discussion, did you intend to proceed from where the revised PEP left off (with a few variations on the table), or to start over from point zero (with potentially anything and everything on the table). In particular, do we need to rehash the reasons for rejection of the backwards if-else proposal that a few seem to wish to resurrect? Many people, perhaps most, including me, read exp1 if exp2 else exp3 # as cond if etrue else efalse # in direct analogy with cond ? etrue : efalse # from C I mentally read ?: in C as if/else! It took a few readings of the proposal to even realize that it the order flipped. There there is the obvious analogy with Python's if cond: etrue else: efalse with 'if' moved over to the first : position (and the : then removed) but with 'else' still between the alternatives (and the second : also removed). Then there are conditional functions, in various languages, iff(cond, etrue, efalse), which as far as I know always have the expressions in that order. Need I continue? Or is the dead still dead? Terry J. Reedy From guido at python.org Wed Sep 21 21:04:26 2005 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Sep 2005 12:04:26 -0700 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: References: <43292071.8050604@intercable.ru> <8393fff0509190928196bba9a@mail.gmail.com> <2mirwwlsve.fsf@starship.python.net> <9613db6005091913206dcf51ec@mail.gmail.com> <1CF74C6F-7521-4751-95B1-2DF71C053251@redivi.com> Message-ID: (Interestign to see the PyObjC folks disagree. :-) On 9/21/05, Ronald Oussoren wrote: > > On 21-sep-2005, at 0:10, Bob Ippolito wrote: > > > > My use case for this isn't so much about threads, but plug-ins. > > Writing multiple Python-based plug-ins for an application is always a > > mess, because they share too much (sys.modules, sys.path, etc.). > > PyObjC would benefit greatly from this feature, because you can write > > Python-based plug-ins for any Cocoa app that supports plug-ins, even > > if they're otherwise unaware of Python's existence. There are > > workarounds, of course, with import hooks and similar hacks. I think > > that mod_python would also benefit from this, and probably other such > > systems. > > For PyObjC having multiple interpreters in a process would cause me > severe headaches. Due to the nature of PyObjC it would be way too > easy to accidently access objects from one interpreter in another > interpreter (in a hypothetical universe where it would be easy to use > multiple interpreters). And lets not get started about the GIL, I > don't think its accidental that PyGILState_Ensure only works with one > interpreter. Actually Python itself has a hard time keeping multiple interpreters truly separate. Also of course there are some shared resources maintained by the operating system: current directory, open file descriptors, signal settings, child processes, that sort of thing. If we were to completely drop this feature, we could make built-in classes modifyable. > A system like Java's classloader would be helpfull, where the > classloader of a class is used to load the classes used by that > class. I have no idea if this can be adapted to python at all. A > strict coding style seems to work for now. You can do something like this using the restricted execution support, which works by setting the __builtins__ name in a dict where you exec code, and overriding __import__ in that __builtins__ dict. (I can't explain it too well in one paragraph, just go look up the rexec.py source code.) It's not great for guaranteeing there's absolutely no escape possible from the sandbox, but it works well enough to make accidental resource sharing a non-issue (apart from the OS shared resources and the built-in types). A misfeature (for this purpose) is that certain kinds of introspection are disabled (this was of course to enable restricted execution). I'd be willing to entertain improvements that improve the insulation this provides. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Sep 21 21:05:53 2005 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Sep 2005 12:05:53 -0700 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <4330B7CE.50302@canterbury.ac.nz> <433140B1.2070406@gmail.com> <43317EC1.908@ronadam.com> Message-ID: I think a recap of past arguments is useful. Me-too votes are not. i will read everything and pick a syntax and that's it. We can do it in Python 2.5. On 9/21/05, Terry Reedy wrote: > Guido: > > When you invited resumed discussion, did you intend to proceed from where > the revised PEP left off (with a few variations on the table), or to start > over from point zero (with potentially anything and everything on the > table). In particular, do we need to rehash the reasons for rejection of > the backwards if-else proposal that a few seem to wish to resurrect? > > Many people, perhaps most, including me, read > > exp1 if exp2 else exp3 # as > cond if etrue else efalse # in direct analogy with > cond ? etrue : efalse # from C > > I mentally read ?: in C as if/else! It took a few readings of the proposal > to even realize that it the order flipped. > > There there is the obvious analogy with Python's > if cond: etrue > else: efalse > with 'if' moved over to the first : position (and the : then removed) but > with 'else' still between the alternatives (and the second : also removed). > > Then there are conditional functions, in various languages, iff(cond, > etrue, efalse), which as far as I know always have the expressions in that > order. > > Need I continue? Or is the dead still dead? > > Terry J. Reedy > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From theller at python.net Wed Sep 21 20:59:11 2005 From: theller at python.net (Thomas Heller) Date: Wed, 21 Sep 2005 20:59:11 +0200 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <8393fff0509190928196bba9a@mail.gmail.com> <2mirwwlsve.fsf@starship.python.net> <9613db6005091913206dcf51ec@mail.gmail.com> <1CF74C6F-7521-4751-95B1-2DF71C053251@redivi.com> Message-ID: Bob Ippolito writes: > Personally my biggest issue with the way the CPython VM works is that > you can't reliably do multiple interpreters in a single process. If > that worked well, you could start an independent interpreter per > thread and with a per-interpreter GIL you'd have pretty much > everything you needed... but this would horribly break the C API and > may be a performance hit. > > My use case for this isn't so much about threads, but plug-ins. > Writing multiple Python-based plug-ins for an application is always a > mess, because they share too much (sys.modules, sys.path, etc.). > PyObjC would benefit greatly from this feature, because you can write > Python-based plug-ins for any Cocoa app that supports plug-ins, even > if they're otherwise unaware of Python's existence. There are > workarounds, of course, with import hooks and similar hacks. I think > that mod_python would also benefit from this, and probably other such > systems. Just a note in case you didn't know this already, probably off-topic for python-dev, but not meant as advertisement for py2exe): the same (plug-in) problem occurs with an inprocess COM server and a Python client program, or more than one inproc COM server in any client program. The 0.6 py2exe release with it's bundle-file option allows to build COM dlls (and client EXE programs, if needed) that APPEAR to have a statically linked copy of Python##.dll, and so have several CPython VMs running in the same process. In case anybody want's to take a look or experiment with it. Thomas From jimjjewett at gmail.com Wed Sep 21 21:14:34 2005 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 21 Sep 2005 15:14:34 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 Message-ID: Ron Adam wrote: > ( e selects (e1 selects a1, b1), > (e2 selects a2, b2), > (e3 selects a3, b3) ) I think you've just reinvented the case statement, which disagrees with "if" over the order of true and false clauses. For a conditional expression, I think the choices are really down to the following, which was already way too much freedom last (http://www.python.org/peps/pep-0308.html) time: (1) Should it be done at all? + It would be useful, and avoid certain types of bugs. - It would encourage bracketing instead of indentation PEP 308 decided "not yet, anyhow" (2) How many of the keywords (if ... then ... else ... elif) should be dropped? (if test then True else False) + standard english + standard programming idiom - adds a "then" keyword (if test True else False) + better parallels the if: statement - starts to run together (if test1 Val1 elif test2 Val2 elif test3 Val3) + parallels the if: statement - encourages overly long code * but still better than nested parens PEP 308 wasn't entirely clear; the words said to keep "elif", but the examples did not. There was some disagreement on which of the other three keywords to represent explicitly. (Rather than only by position.) (3) What order should the clauses appear? (if test then True else False) (if test1 then Val1 elif test2 Val2 elif test3 Val3 else Val4) + Natural Order - do we need "then"? (True if normal else False) (Val1 if test1 else Val2 if test2 else Val3 if test3 else Val4) + Best order for normal/default conditionals + Best order in general if we weren't used to left-right processing - But we *do* expect left->right almost everywhere except assignments PEP 308 made it clear that "else" should be last. Putting the condition before the "then" was not as clearcut. (4) What sort of punctuation should augment or replace the keywords? PEP 308 suggested avoiding punctuation, but it wasn't clearcut. -jJ From ronaldoussoren at mac.com Wed Sep 21 21:16:58 2005 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Wed, 21 Sep 2005 21:16:58 +0200 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: References: <43292071.8050604@intercable.ru> <8393fff0509190928196bba9a@mail.gmail.com> <2mirwwlsve.fsf@starship.python.net> <9613db6005091913206dcf51ec@mail.gmail.com> <1CF74C6F-7521-4751-95B1-2DF71C053251@redivi.com> Message-ID: On 21-sep-2005, at 21:04, Guido van Rossum wrote: > >> A system like Java's classloader would be helpfull, where the >> classloader of a class is used to load the classes used by that >> class. I have no idea if this can be adapted to python at all. A >> strict coding style seems to work for now. >> > > You can do something like this using the restricted execution support, > which works by setting the __builtins__ name in a dict where you exec > code, and overriding __import__ in that __builtins__ dict. (I can't > explain it too well in one paragraph, just go look up the rexec.py > source code.) > > It's not great for guaranteeing there's absolutely no escape possible > from the sandbox, but it works well enough to make accidental resource > sharing a non-issue (apart from the OS shared resources and the > built-in types). A misfeature (for this purpose) is that certain kinds > of introspection are disabled (this was of course to enable restricted > execution). Replacing __builtins__ hadn't crossed my mind yet. My first cut at building plugins played games with __path__, Bob then replaced that by a version that actually works using py2app. The sandbox doesn't need to closed for plugins, it's only needed to avoid accidental naming clashes (two independent plugins that contain a util module). I don't know if restrictions on introspection would be an issue, I haven't felt the need to write real plugins yet. Well other than the 'look I've a python interpreter inside ' demo plugins. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2105 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050921/f0d2a7ae/smime.bin From rrr at ronadam.com Wed Sep 21 21:16:28 2005 From: rrr at ronadam.com (Ron Adam) Date: Wed, 21 Sep 2005 15:16:28 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <4330B7CE.50302@canterbury.ac.nz> <433140B1.2070406@gmail.com> <43317EC1.908@ronadam.com> Message-ID: <4331B18C.8050303@ronadam.com> Steven Bethard wrote: > Please no more syntax proposals! There were enough in the PEP. It > looks like most people support a conditional expression of some sort. > We need to leave the syntax to Guido. We've already proved that like > the decorators discussions, we can't as a community agree on a syntax. > That's what we have a BDFL for. =) No problem, I'll go back to watching this interesting discussion and see what happens. :-) I really should have deleted all but the last one anyway and probably should have moved it to python-list at that point since it's quite different from whats being proposed I think. Cheers, Ron From pje at telecommunity.com Wed Sep 21 22:12:19 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 21 Sep 2005 16:12:19 -0400 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: References: <43292071.8050604@intercable.ru> <8393fff0509190928196bba9a@mail.gmail.com> <2mirwwlsve.fsf@starship.python.net> <9613db6005091913206dcf51ec@mail.gmail.com> <1CF74C6F-7521-4751-95B1-2DF71C053251@redivi.com> Message-ID: <5.1.1.6.0.20050921152358.01b6fd38@mail.telecommunity.com> At 12:04 PM 9/21/2005 -0700, Guido van Rossum wrote: >Actually Python itself has a hard time keeping multiple interpreters >truly separate. Also of course there are some shared resources >maintained by the operating system: current directory, open file >descriptors, signal settings, child processes, that sort of thing. > >If we were to completely drop this feature, we could make built-in >classes modifyable. I'd personally much rather we got back the ability to change the type of an instance of a builtin to that of a Python subclass of that builtin type, or to change it back. I have more use cases for that than for actually modifying builtins. (E.g. "observable" lists/dicts, hooking module __getattr__, etc.) > > A system like Java's classloader would be helpfull, where the > > classloader of a class is used to load the classes used by that > > class. I have no idea if this can be adapted to python at all. A > > strict coding style seems to work for now. > >You can do something like this using the restricted execution support, >which works by setting the __builtins__ name in a dict where you exec >code, and overriding __import__ in that __builtins__ dict. (I can't >explain it too well in one paragraph, just go look up the rexec.py >source code.) > >It's not great for guaranteeing there's absolutely no escape possible >from the sandbox, but it works well enough to make accidental resource >sharing a non-issue (apart from the OS shared resources and the >built-in types). A misfeature (for this purpose) is that certain kinds >of introspection are disabled (this was of course to enable restricted >execution). Another misfeature is that some C-level Python code expects to obtain sys.modules, builtins, etc. via the interpreter struct. Thus, you tend to have to reimplement those things in Python to get them to respect a virtualization of sys.modules. I have to admit I've only dabbled in attempting this, just long enough to hit a stumbling block or two and then discover that they were because sys.modules is in the interpreter struct. Of course, my next thought then was to just expose the multi-interpreter API as an extension module, so that you could create interpreters from Python code. The project I'd originally planned to do this for never materialized though, so I never actually attempted it. My thought, though, was that by swapping the current interpreter of the thread state when crossing code boundaries, you could keep both the Python-level and C-level code happy. However, it might also suffice to have a way to switch in and out the interpreter configuration (sys.modules, sys.__dict__, and __builtins__ at minimum; I don't have any clear use case for changing the three codec_* vars at the moment). >I'd be willing to entertain improvements that improve the insulation >this provides. Since there's already a way to change __builtins__ in the threadstate, maybe the C API could be changed to obtain the six interpreter variables via builtins rather than the other way around. This would allow us to drop the multi-interpreter API from C (along with support for restricted mode) but still allow complete virtualization from inside Python code. The steps would be: 1. Remove restricted mode support 2. Change the tstate structure to have a 'builtins' 3. Change code that does tstate->interp lookups to instead lookup special names in the tstate's builtins At that point, you can exec code with new builtins to bootstrap a virtual Python, subject to limitations like being able to load a given extension module only once. Systems like mod_python that use the multi-interpreter API now would just need to bootstrap a new __builtins__. Sadly, this doesn't *really* cure the GIL-ensure problem, in that you still don't have a specially-distinguished __builtins__ to use when you call into Python from a C-started thread. On the other hand, I suspect that the use cases for that, and the use cases for virtualization don't overlap much, so having a distinguished place to hold the "default" (i.e. initial) builtins probably doesn't hurt virtualization much, since you can always *modify* that set of builtins if you need to. From tjreedy at udel.edu Wed Sep 21 22:15:14 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 21 Sep 2005 16:15:14 -0400 Subject: [Python-Dev] Alternative name for str.partition() References: <001901c5ac5d$5985bcc0$eb29c797@oemcomputer><4312EE82.90207@gmail.com> <4313AD16.3070608@canterbury.ac.nz> <20050921185601.GA22265@anthony.ics.uci.edu> Message-ID: "Christian Stork" wrote in message news:20050921185601.GA22265 at anthony.ics.uci.edu... > On Tue, Aug 30, 2005 at 12:49:26PM +1200, Greg Ewing wrote: >> A more descriptive name than 'partition' would be 'split_at'. > > 'split_at' is really what's happening. (I came up with it independently > of Greg, if that is any evidence). At least semi-seriously, how about condensing 'split_at' to 'splat', a variant of split (and splash), as in 'hit the string on the separator, and with a splat, split it into 3 pieces'. (See dictionary.com for various meanings.) Terry J. Reedy From nnorwitz at gmail.com Wed Sep 21 22:33:51 2005 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 21 Sep 2005 13:33:51 -0700 Subject: [Python-Dev] possible memory leak on windows (valgrind report) In-Reply-To: <4331A600.9060707@v.loewis.de> References: <4331A600.9060707@v.loewis.de> Message-ID: On 9/21/05, "Martin v. L?wis" wrote: > Neal Norwitz wrote: > > (I need to write a lot more suppression rules for gentoo.) > > This could be due to your using GCC 4. Apparently, gcc 4 > is willing to inline Py_ADDRESS_IN_RANGE even though it > appears at the end of the file, at -O3. I don't think I'm using gcc 4. > To suppress that, you can declare the function as > __attribute__((noinline)). You will need to conditionalize > this on gcc, but not only that: it appears that noinline > was one of the more recent additions. I'm not sure when > it was added, but apparently, it is present in 3.3 and > later. This is a good point. I recall when I made Py_ADDRESS_IN_RANGE the docs said that anything could be inlined. I think it would be good to use this attribute (conditionally of course). I'll try to remember to add this. I wonder if using attributes for other features would gain us much. I would really like to be able to use attributes for PyArgs_ParseTuple(), but I don't think gcc can use user defined formats. It's only printf AFAIR. Does anyone know if this isn't true and we can define our own format -> type mappings? n From raymond.hettinger at verizon.net Wed Sep 21 22:31:53 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Wed, 21 Sep 2005 16:31:53 -0400 Subject: [Python-Dev] bool(iter([])) changed between 2.3 and 2.4 In-Reply-To: Message-ID: <004f01c5beeb$816ca460$3e3ac797@oemcomputer> [Guido van Rossum] > Could you at least admit that this was an oversight and not try to > pretend it was intentional breakage? Absolutely. I completely missed this one. Raymond From ncoghlan at gmail.com Thu Sep 22 00:35:06 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 22 Sep 2005 08:35:06 +1000 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: Message-ID: <4331E01A.4010406@gmail.com> Jim Jewett wrote: > > (3) What order should the clauses appear? > (if test then True else False) > (if test1 then Val1 elif test2 Val2 elif test3 Val3 else Val4) > + Natural Order > - do we need "then"? > (True if normal else False) > (Val1 if test1 else Val2 if test2 else Val3 if test3 else Val4) > + Best order for normal/default conditionals > + Best order in general if we weren't used to left-right processing > - But we *do* expect left->right almost everywhere except assignments + out-of-order evaluation is already used in LC's and GE's + declarative style, rather than imperative To try and make that last point clearer: real = (z.real if isinstance(z, ComplexType) else z) This translates directly into in words as: "Set real to be the real component of z if z is a complex number, otherwise set it to be the same as z" vs. real = (if isinstance(z, ComplexType) then z.real else z) I can't put that into words without changing the order of the elements either by re-using the phrasing from above (with the condition between the two outcomes), or else describing the statement form instead of the expression form by bringing the condition all the way to the front: "If z is a complex number, then set real to be the real component of z, otherwise set real to be the same as z" I find it to be like with GE's and LC's - the phrasing works better with the expression at the front, because you say basically what you're doing, then you put the additional constraints on it (i.e., which iterable is used as the data source, and what filtering is applied to the elements of that iterable) I think I've said enough on this point though, so I'll try to bite my tongue until Guido makes a decision. . . Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From janssen at parc.com Thu Sep 22 01:54:02 2005 From: janssen at parc.com (Bill Janssen) Date: Wed, 21 Sep 2005 16:54:02 PDT Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: Your message of "Wed, 21 Sep 2005 07:22:09 PDT." <17201.27793.875588.293551@montanaro.dyndns.org> Message-ID: <05Sep21.165406pdt."58617"@synergy1.parc.xerox.com> I agree with Skip. Bill > I suppose this is a dead horse now, but I will kick it one more time. > > Under the rubrics of "explicit is better than implicit" and "there should > only be one wat to do it", I would rather see > > bool_val = bool(foo or bar) > > instead of having the "or" operator implicitly call bool() for me. There's > clear value to the current semantics and it's so easy to get a boolean if > you want it, I see no reason for a change. > > Skip From janssen at parc.com Thu Sep 22 01:55:40 2005 From: janssen at parc.com (Bill Janssen) Date: Wed, 21 Sep 2005 16:55:40 PDT Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: Your message of "Wed, 21 Sep 2005 09:26:11 PDT." <9519083A-6B06-4F3E-A8BD-47F9684E64B5@chello.se> Message-ID: <05Sep21.165541pdt."58617"@synergy1.parc.xerox.com> > The best way to make people stop complaining about the GIL and start > using > process-based multiprogramming is to provide solid, standardized support > for process-based multiprogramming. And the model provided by the thread abstraction is a good API for that support... Bill From greg.ewing at canterbury.ac.nz Thu Sep 22 03:33:42 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 22 Sep 2005 13:33:42 +1200 Subject: [Python-Dev] bool(iter([])) changed between 2.3 and 2.4 In-Reply-To: <001c01c5be3c$53130dc0$6522c797@oemcomputer> References: <001c01c5be3c$53130dc0$6522c797@oemcomputer> Message-ID: <433209F6.8030507@canterbury.ac.nz> Raymond Hettinger wrote: > The Boolean value of an > iterator is certainly not promised by the iterator protocol as specified > in the docs or the PEP. But if the docs don't mention anything about true or false values for some particular type, one tends to assume that all values are true, as is the default for user-defined classes. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Thu Sep 22 04:52:02 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 22 Sep 2005 14:52:02 +1200 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <43317EC1.908@ronadam.com> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <4330B7CE.50302@canterbury.ac.nz> <433140B1.2070406@gmail.com> <43317EC1.908@ronadam.com> Message-ID: <43321C52.6080101@canterbury.ac.nz> Ron Adam wrote: > (a if e then b) > ((a1 if e1 then b1) if e then b) > (a if e then (a2 if e2 then b2)) > ((a1 if e1 then b1) if e then (a2 if e2 then b2)) I think you mean 'else' rather than 'then' in all those, don't you? -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From rrr at ronadam.com Thu Sep 22 04:57:30 2005 From: rrr at ronadam.com (Ron Adam) Date: Wed, 21 Sep 2005 22:57:30 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <43321C52.6080101@canterbury.ac.nz> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <4330B7CE.50302@canterbury.ac.nz> <433140B1.2070406@gmail.com> <43317EC1.908@ronadam.com> <43321C52.6080101@canterbury.ac.nz> Message-ID: <43321D9A.2000603@ronadam.com> Greg Ewing wrote: > Ron Adam wrote: > > >>(a if e then b) >>((a1 if e1 then b1) if e then b) >>(a if e then (a2 if e2 then b2)) >>((a1 if e1 then b1) if e then (a2 if e2 then b2)) > > > I think you mean 'else' rather than 'then' in all > those, don't you? Yes of course, thanks for correcting. From greg.ewing at canterbury.ac.nz Thu Sep 22 05:09:44 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 22 Sep 2005 15:09:44 +1200 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <43318067.3090301@islandtraining.com> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <2mmzm6k9ya.fsf@starship.python.net> <43318067.3090301@islandtraining.com> Message-ID: <43322078.1060304@canterbury.ac.nz> Gary Herron wrote: > And in fact, one read and understands your return statement just like an > English sentence -- word by word from beginning to end. This seems an > argument FOR the syntax not against. Moreover, if one uses the > proposed parenthesized syntax, even the slightly odd word order of > "return if" is mitigated. The reason I like "a if b else c" is because it has the most natural word order. In English, My dog is happy if he has a bone, else sad. sounds much more natural than My dog is, if he has a bone, happy, else sad. In return statements, return self.arg if self.arg is not None else default looks quite all right to me. I think the fact that it does resemble English word order so much prevents the word-soup problem from occurring. Interestingly, it looks *more* odd to me if parens are included: return (self.arg if self.arg is not None else default) I think this is because, without the parens, I tend to read the "if" as applying to the whole phrase "return self.arg", not just to the "self.arg". The English analogy is rewriting "My dog is happy if he has a bone" as "If he has a bone, my dog is happy", which also sounds natural, whereas "My dog is, if he has a bone, happy" sounds unnatural. So I still prefer "a if b else c" to any of the alternatives, and I still think parens should not be required. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From guido at python.org Thu Sep 22 05:14:12 2005 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Sep 2005 20:14:12 -0700 Subject: [Python-Dev] bool(iter([])) changed between 2.3 and 2.4 In-Reply-To: <004f01c5beeb$816ca460$3e3ac797@oemcomputer> References: <004f01c5beeb$816ca460$3e3ac797@oemcomputer> Message-ID: On 9/21/05, Raymond Hettinger wrote: > [Guido van Rossum] > > Could you at least admit that this was an oversight and not try to > > pretend it was intentional breakage? > > Absolutely. I completely missed this one. Thanks; spoken like a man. I strongly feel that this needs to be corrected in 2.5. Iterators should have neither __len__ nor __nonzero__. I see mostly agreement that this is a misfeature. We don't really want to start writing code like this: while it: x = it.next() ...process x... when we can already write it like this: for x in it: ...process x... do we? Keeping a special API to allow a more efficient implementation of __reversed__ is fine. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Thu Sep 22 05:23:20 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 22 Sep 2005 15:23:20 +1200 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <4330B7CE.50302@canterbury.ac.nz> <433140B1.2070406@gmail.com> <43317EC1.908@ronadam.com> Message-ID: <433223A8.3010400@canterbury.ac.nz> Terry Reedy wrote: > Many people, perhaps most, including me, read > > exp1 if exp2 else exp3 # as > cond if etrue else efalse # in direct analogy with > cond ? etrue : efalse # from C I'd have thought only Forth programmers would be prone to that! It would surprise me greatly if it's really true that *most* people would read it that way. Especially given that, in real code, you're not going to be looking at abstract names like exp1, exp2, exp3, but (hopefully) something a lot more meaningful. Can you honestly say that you would read return red_value if color == 'red' else blue_value as if red_value: return color == 'red' else: return blue_value ? -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From guido at python.org Thu Sep 22 05:26:06 2005 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Sep 2005 20:26:06 -0700 Subject: [Python-Dev] Mapping Darwin 8.2.0 to Mac OS X 10.4.2 in platform.py Message-ID: The platform module has a way to map system names such as returned by uname() to marketing names. It maps SunOS to Solaris, for example. But it doesn't map Darwin to Mac OS X. I think I know how to map Darwin version numbers to OS X version numbers: from http://www.opensource.apple.com/darwinsource/ it is clear that OS X 10.a.b corresponds to Darwin (a+4).b, except for OS X versions <= 10.1. I'd be happy to write the code and add it to system_alias() in platform.py. Is this a good idea? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Sep 22 05:28:30 2005 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Sep 2005 20:28:30 -0700 Subject: [Python-Dev] Mapping Darwin 8.2.0 to Mac OS X 10.4.2 in platform.py In-Reply-To: References: Message-ID: On 9/21/05, Guido van Rossum wrote: > The platform module has a way to map system names such as returned by > uname() to marketing names. It maps SunOS to Solaris, for example. But > it doesn't map Darwin to Mac OS X. I think I know how to map Darwin > version numbers to OS X version numbers: from > http://www.opensource.apple.com/darwinsource/ it is clear that OS X > 10.a.b corresponds to Darwin (a+4).b, except for OS X versions <= > 10.1. I'd be happy to write the code and add it to system_alias() in > platform.py. Is this a good idea? I forgot. The current code recognizes 'Rhapsody' and maps it to "MacOS X Server". But I don't see any evidence that Apple still uses the code name Rhapsody. Does uname ever return 'Rhapsody'? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From bob at redivi.com Thu Sep 22 05:44:31 2005 From: bob at redivi.com (Bob Ippolito) Date: Wed, 21 Sep 2005 23:44:31 -0400 Subject: [Python-Dev] Mapping Darwin 8.2.0 to Mac OS X 10.4.2 in platform.py In-Reply-To: References: Message-ID: <9D453817-F3B0-443D-8415-D66457AC27FE@redivi.com> On Sep 21, 2005, at 11:26 PM, Guido van Rossum wrote: > The platform module has a way to map system names such as returned by > uname() to marketing names. It maps SunOS to Solaris, for example. But > it doesn't map Darwin to Mac OS X. I think I know how to map Darwin > version numbers to OS X version numbers: from > http://www.opensource.apple.com/darwinsource/ it is clear that OS X > 10.a.b corresponds to Darwin (a+4).b, except for OS X versions <= > 10.1. I'd be happy to write the code and add it to system_alias() in > platform.py. Is this a good idea? No, it's not. There are other mismatches between Mac OS X version and Darwin kernel version (e.g. 10.3.0 and 10.3.1 used the same kernel version). The only correct way to do it with public API is to use gestalt, which platform.mac_ver() does. There are other ways (reading a plist, parsing the output of /usr/bin/sw_vers, using the same SPI that /usr/bin/sw_vers uses...), but gestalt is the only public API. The one caveat with platform.mac_ver() is that it was broken for some version(s) of Python. I don't remember when that was fixed. It definitely works for Python 2.3.5 (ships with Mac OS X 10.4) and Python 2.4.1, but I'm relatively certain it was broken with the Python 2.3.0 that shipped with Mac OS X 10.3, and perhaps also the Python 2.2.0 that shipped with Mac OS X 10.2. Anyway, this information isn't in the uname. You can guess with the uname, but it requires lots of special cases and maintenance. -bob From raymond.hettinger at verizon.net Thu Sep 22 07:12:10 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Thu, 22 Sep 2005 01:12:10 -0400 Subject: [Python-Dev] bool(iter([])) changed between 2.3 and 2.4 Message-ID: <000201c5bf34$3014c7c0$3e3ac797@oemcomputer> [Guido] > I strongly feel that this needs to be corrected in 2.5. Iterators > should have neither __len__ nor __nonzero__. Right. I'll get it fixed-up. [Terry Reedy] > I presume there were two reasons: internal efficiency of > preallocations > (list(some_it) for example) [Guido] > This could have been implemented without making the > implementation details public. I see a way to do that by renaming the __len__ method to some private name for internal use. This would preserve the performance gains while still restoring the public API. Raymond From jcarlson at uci.edu Thu Sep 22 07:48:13 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed, 21 Sep 2005 22:48:13 -0700 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <05Sep21.165541pdt."58617"@synergy1.parc.xerox.com> References: <9519083A-6B06-4F3E-A8BD-47F9684E64B5@chello.se> <05Sep21.165541pdt."58617"@synergy1.parc.xerox.com> Message-ID: <20050921201822.F72F.JCARLSON@uci.edu> Bill Janssen wrote: > > > The best way to make people stop complaining about the GIL and start > > using > > process-based multiprogramming is to provide solid, standardized support > > for process-based multiprogramming. > > And the model provided by the thread abstraction is a good API for that > support... While creating a thread is generally quite easy, the threading abstraction assumes globally shared memory. Certainly there are multiprocessing systems which handle transferring of data between disparate memories automatically (Linda/PyLinda, POSH, etc.), but no doubt some users will want a more fine-grained control of data transfer (and this is saying nothing of the shared-memory model's currently horrible performance in Python). As such, there is always the option of using the tried and true MPI and PyMPI, which looks to have been recently updated. Or even XMLRPC and PickleRPC over sockets and/or mmap'd files. Then again, with how easy it is to distribute workloads using (Py)Linda, I'd be hard pressed to suggest any other module for multiprocessing into the standard library (unless it was a work-alike)...though perhaps we should wait until it has been sped up a bit, supports more data types, etc. - Josiah From blais at furius.ca Thu Sep 22 07:48:21 2005 From: blais at furius.ca (Martin Blais) Date: Thu, 22 Sep 2005 01:48:21 -0400 Subject: [Python-Dev] os.path.diff(path1, path2) In-Reply-To: <4330AF9C.3090509@canterbury.ac.nz> References: <20050913031157.46635.qmail@web51809.mail.yahoo.com> <1127044776.14284.9.camel@mulkomp.mst-plus> <432F7453.1050105@canterbury.ac.nz> <1127224142.14254.20.camel@mulkomp.mst-plus> <4330AF9C.3090509@canterbury.ac.nz> Message-ID: <8393fff05092122483e772682@mail.gmail.com> On 9/20/05, Greg Ewing wrote: > Matthias Andreas Benkard wrote: > > * In a recent project, I had occasion to store pathnames of > picture files in a relational database. I wanted to store > the pathnames relative to a user-chosen "Pictures" > directory, so that it could be moved without having to > update all the pathnames in the database. Curator does the same, for the same reasons. http://furius.ca/curator/ Also, you can burn the static HTML files to CDROM, and the links still all work. That was the main motivator for this. cheers, From martin at v.loewis.de Thu Sep 22 08:28:51 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 22 Sep 2005 08:28:51 +0200 Subject: [Python-Dev] possible memory leak on windows (valgrind report) In-Reply-To: References: <4331A600.9060707@v.loewis.de> Message-ID: <43324F23.3030809@v.loewis.de> Neal Norwitz wrote: > I wonder if using attributes for other features would gain us much. I > would really like to be able to use attributes for > PyArgs_ParseTuple(), but I don't think gcc can use user defined > formats. It's only printf AFAIR. Does anyone know if this isn't true > and we can define our own format -> type mappings? Yes and no. Yes, it cannot do user-defined formats, but no, it is not just *printf. They support gcc_diag and gcc_cxxdiag for their own internal printf-like functions (error() and warning()); they also support strftime and strfmon. Regards, Martin From ronaldoussoren at mac.com Thu Sep 22 08:30:52 2005 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu, 22 Sep 2005 08:30:52 +0200 Subject: [Python-Dev] Mapping Darwin 8.2.0 to Mac OS X 10.4.2 in platform.py In-Reply-To: References: Message-ID: On 22-sep-2005, at 5:26, Guido van Rossum wrote: > The platform module has a way to map system names such as returned by > uname() to marketing names. It maps SunOS to Solaris, for example. But > it doesn't map Darwin to Mac OS X. I think I know how to map Darwin > version numbers to OS X version numbers: from > http://www.opensource.apple.com/darwinsource/ it is clear that OS X > 10.a.b corresponds to Darwin (a+4).b, except for OS X versions <= > 10.1. I'd be happy to write the code and add it to system_alias() in > platform.py. Is this a good idea? There's no good reason to assume that the mapping from kernel version to marketing version will stay the same in the future. The savest way to get the marketing version of the currently running OSX is to run / usr/sbin/sw_vers and parse its output. There might also be a public API for getting the same information. Py2app, and specifically the bdist_mpkg component of that, contains code to parse sw_vers output. Ronald > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ > ronaldoussoren%40mac.com > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2105 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050922/476055ec/smime.bin From anthony at python.org Thu Sep 22 09:42:10 2005 From: anthony at python.org (Anthony Baxter) Date: Thu, 22 Sep 2005 17:42:10 +1000 Subject: [Python-Dev] RELEASED Python 2.4.2, release candidate 1 Message-ID: <200509221742.12706.anthony@python.org> On behalf of the Python development team and the Python community, I'm happy to announce the release of Python 2.4.2 (release candidate 1). Python 2.4.2 is a bug-fix release. See the release notes at the website (also available as Misc/NEWS in the source distribution) for details of the more than 60 bugs squished in this release. Assuming no major problems crop up, a final release of Python 2.4.2 will follow in about a week's time. For more information on Python 2.4.2, including download links for various platforms, release notes, and known issues, please see: http://www.python.org/2.4.2 Highlights of this new release include: - Bug fixes. According to the release notes, more than 60 have been fixed, including bugs that prevented Python from working properly on 64 bit HP/UX and AIX systems. Highlights of the previous major Python release (2.4) are available from the Python 2.4 page, at http://www.python.org/2.4/highlights.html Enjoy the new release, Anthony Anthony Baxter anthony at python.org Python Release Manager (on behalf of the entire python-dev team) From mal at egenix.com Thu Sep 22 09:55:14 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 22 Sep 2005 09:55:14 +0200 Subject: [Python-Dev] Mapping Darwin 8.2.0 to Mac OS X 10.4.2 in platform.py In-Reply-To: References: Message-ID: <43326362.7070001@egenix.com> Ronald Oussoren wrote: > > On 22-sep-2005, at 5:26, Guido van Rossum wrote: > >> The platform module has a way to map system names such as returned by >> uname() to marketing names. It maps SunOS to Solaris, for example. But >> it doesn't map Darwin to Mac OS X. I think I know how to map Darwin >> version numbers to OS X version numbers: from >> http://www.opensource.apple.com/darwinsource/ it is clear that OS X >> 10.a.b corresponds to Darwin (a+4).b, except for OS X versions <= >> 10.1. I'd be happy to write the code and add it to system_alias() in >> platform.py. Is this a good idea? > > > There's no good reason to assume that the mapping from kernel version > to marketing version will stay the same in the future. The savest way > to get the marketing version of the currently running OSX is to run / > usr/sbin/sw_vers and parse its output. There might also be a public API > for getting the same information. Py2app, and specifically the > bdist_mpkg component of that, contains code to parse sw_vers output. I don't have access to Macs, so there nothing much I can say about this. In general, it's always better to rely on system tools for finding the marketing name of an OS than to try to come up with a work-around. If gestalt() returns the proper name, then this should be used. If sw_vers provides a more reliable way to do this, parsing its output seems like a better idea. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 22 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From reinhold-birkenfeld-nospam at wolke7.net Thu Sep 22 10:38:24 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Thu, 22 Sep 2005 10:38:24 +0200 Subject: [Python-Dev] IMPORTANT: release24-maint branch is FROZEN from 2005-09-21 00:00 UTC for 2.4.2 In-Reply-To: <200509202306.54251.anthony@python.org> References: <200509202306.54251.anthony@python.org> Message-ID: Anthony Baxter wrote: > Starting in about 11 hours time, the release24-maint branch is FROZEN > for the 2.4.2c1 release. The freeze will last for around a day, and > then we're in a state of mostly-frozen for another week, until 2.4.2 > (final). During that week, please don't check things into the branch > unless you check with me first. Let's make this a nice painless > release. > > I'll send another message once 2.4.2 is done. Do you think the patch at https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1298449&group_id=5470 could go in before 2.4.2 final? It's a rather major issue within cStringIO.writelines(). Reinhold -- Mail address is perfectly valid! From mwh at python.net Thu Sep 22 10:56:08 2005 From: mwh at python.net (Michael Hudson) Date: Thu, 22 Sep 2005 09:56:08 +0100 Subject: [Python-Dev] IMPORTANT: release24-maint branch is FROZEN from 2005-09-21 00:00 UTC for 2.4.2 In-Reply-To: (Reinhold Birkenfeld's message of "Thu, 22 Sep 2005 10:38:24 +0200") References: <200509202306.54251.anthony@python.org> Message-ID: <2m8xxpjxav.fsf@starship.python.net> Reinhold Birkenfeld writes: > Anthony Baxter wrote: >> Starting in about 11 hours time, the release24-maint branch is FROZEN >> for the 2.4.2c1 release. The freeze will last for around a day, and >> then we're in a state of mostly-frozen for another week, until 2.4.2 >> (final). During that week, please don't check things into the branch >> unless you check with me first. Let's make this a nice painless >> release. >> >> I'll send another message once 2.4.2 is done. > > Do you think the patch at > > https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1298449&group_id=5470 > > could go in before 2.4.2 final? It's a rather major issue > within cStringIO.writelines(). Oh, I was looking at that one too :) I was going to check it in after building and testing finished... Cheers, mwh -- The gripping hand is really that there are morons everywhere, it's just that the Americon morons are funnier than average. -- Pim van Riezen, alt.sysadmin.recovery From solipsis at pitrou.net Thu Sep 22 11:35:51 2005 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 22 Sep 2005 11:35:51 +0200 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <20050921102057.F722.JCARLSON@uci.edu> References: <9519083A-6B06-4F3E-A8BD-47F9684E64B5@chello.se> <1127320319.21126.18.camel@p-dvsi-418-1.rd.francetelecom.fr> <20050921102057.F722.JCARLSON@uci.edu> Message-ID: <1127381751.1261.5.camel@p-dvsi-418-1.rd.francetelecom.fr> Hi, > On *nix, use a unix domain socket (location in the filesystem which acts > as a listening socket). On Windows, you can use cTypes, pywin32, etc., > to create a global mutex and/or COM interface. Ok, but for a very simple and crude need like mine (the application code using this IPC means is not more than 20 lines long, and it has no performance requirement), it would be useful to have an abstraction in the standard library. Regards Antoine. From ark at acm.org Thu Sep 22 14:06:36 2005 From: ark at acm.org (Andrew Koenig) Date: Thu, 22 Sep 2005 08:06:36 -0400 Subject: [Python-Dev] "and" and "or" operators in Py3.0 In-Reply-To: <000501c5bdcf$be32b000$6522c797@oemcomputer> Message-ID: <003b01c5bf6e$17f0ec20$6402a8c0@arkdesktop> > In C, C++, C#, Java, and JavaScript, what do you get when you print the > result of 3 || 10? In C and C++, you get 1. (in c++ the result is actually true, not 1, but true prints as 1 by default for backward compatibility) From anothermax at gmail.com Thu Sep 22 15:36:04 2005 From: anothermax at gmail.com (Jeremy Maxfield) Date: Thu, 22 Sep 2005 15:36:04 +0200 Subject: [Python-Dev] IMPORTANT: release24-maint branch is FROZEN from 2005-09-21 00:00 UTC for 2.4.2 In-Reply-To: <2m8xxpjxav.fsf@starship.python.net> References: <200509202306.54251.anthony@python.org> <2m8xxpjxav.fsf@starship.python.net> Message-ID: <93dc9c320509220636677d26f6@mail.gmail.com> Can the fix for [ 1163563 ] Sub threads execute in restricted mode ( http://sourceforge.net/tracker/index.php?func=detail&aid=1163563&group_id=5470&atid=105470 ) go in before 2.4.2 final? This is a real show stopper for us - we can't move to 2.4 without it. Cheers, Max On 9/22/05, Michael Hudson wrote: > > Reinhold Birkenfeld writes: > > > Anthony Baxter wrote: > >> Starting in about 11 hours time, the release24-maint branch is FROZEN > >> for the 2.4.2c1 release. The freeze will last for around a day, and > >> then we're in a state of mostly-frozen for another week, until 2.4.2 > >> (final). During that week, please don't check things into the branch > >> unless you check with me first. Let's make this a nice painless > >> release. > >> > >> I'll send another message once 2.4.2 is done. > > > > Do you think the patch at > > > > > https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1298449&group_id=5470 > > > > could go in before 2.4.2 final? It's a rather major issue > > within cStringIO.writelines(). > > Oh, I was looking at that one too :) I was going to check it in after > building and testing finished... > > Cheers, > mwh > > -- > The gripping hand is really that there are morons everywhere, it's > just that the Americon morons are funnier than average. > -- Pim van Riezen, alt.sysadmin.recovery > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/anothermax%40gmail.com > -- flickr: http://www.flickr.com/photos/anothermax/sets -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20050922/43c111d2/attachment.html From jimjjewett at gmail.com Thu Sep 22 15:41:32 2005 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 22 Sep 2005 09:41:32 -0400 Subject: [Python-Dev] bool(iter([])) changed between 2.3 and 2.4 Message-ID: Greg Ewing wrote: > But if the docs don't mention anything about true or > false values for some particular type, one tends to > assume that all values are true, as is the default > for user-defined classes. The tutorials and such stress that python doesn't typically care about a specific "True" or "False"; the normal distinction is between "empty" and "not empty". 0, None, (), [], {} all come out as false. "Is there anything left?" is a pretty analogy for iterators, particularly since the examples tend to start with list or file iterators. x = [] or iter([]) or "nope" does just what *I* expect. If you want to change it back, so be it, but it will break code that way too; please at least make big notes in the documentation. -jJ From anthony at interlink.com.au Thu Sep 22 16:33:30 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri, 23 Sep 2005 00:33:30 +1000 Subject: [Python-Dev] IMPORTANT: release24-maint branch is FROZEN from 2005-09-21 00:00 UTC for 2.4.2 In-Reply-To: <93dc9c320509220636677d26f6@mail.gmail.com> References: <200509202306.54251.anthony@python.org> <2m8xxpjxav.fsf@starship.python.net> <93dc9c320509220636677d26f6@mail.gmail.com> Message-ID: <200509230033.33316.anthony@interlink.com.au> On Thursday 22 September 2005 23:36, Jeremy Maxfield wrote: > Can the fix for [ 1163563 ] Sub threads execute in restricted mode > ( > http://sourceforge.net/tracker/index.php?func=detail&aid=1163563&gr >oup_id=5470&atid=105470 ) > go in before 2.4.2 final? > This is a real show stopper for us - we can't move to 2.4 without > it. Cheers, > Max I'm not sure - mwh, how risky a patch is this? Anthony -- Anthony Baxter It's never too late to have a happy childhood. From anthony at python.org Thu Sep 22 16:34:58 2005 From: anthony at python.org (Anthony Baxter) Date: Fri, 23 Sep 2005 00:34:58 +1000 Subject: [Python-Dev] IMPORTANT: release24-maint branch is THAWED (partly frozen ; ) In-Reply-To: <20050920192114.GA9776@code1.codespeak.net> References: <200509202306.54251.anthony@python.org> <20050920192114.GA9776@code1.codespeak.net> Message-ID: <200509230034.58932.anthony@python.org> So the 2.4.2c1 release seems to be not a brown-paper-bag release. The branch should be considered ok for those critical fixes that need to go in before 2.4.2 final - please, please, if you're not absolutely sure, ask me first. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From maa_public at sinn.ru Thu Sep 22 12:35:15 2005 From: maa_public at sinn.ru (Alexander Myodov) Date: Thu, 22 Sep 2005 16:05:15 +0530 Subject: [Python-Dev] Visibility scope for "for/while/if" statements Message-ID: <1477206818.20050922160515@sinn.ru> Hello, Don't want to be importunate annoyingly asking the things probably trivial for experienced community, but need to ask it anyway, after spending about two hours trying to find well-camouflaged error caused by it. Why the variables defined inside "for"/"while"/"if" statements (including loop variables for "for") are visible outside this scope? This litters the namespace, makes things more complex and requires even more attention from the programmer than even the uneasy C. Or makes him to watch for using each variable only once (prohibiting from using usual i/j/k variables); otherwise copypasting a piece of code can cause hardly-findable errors. But not even the variables from inside of statements are forgot in the outside namespace; what is most ridiculous is that the inner variables from a list comprehension are kept outside the comprehension! Yes, I am aware of one use case for this... er, feature. It could be useful to remember the last item of the loop after the loop is done... sometimes. But what is it for in other cases, except confusing the programmer? Or maybe can someone hint me whether I can somehow switch the behaviour on source-level to avoid keeping the variables outside the statements? Something like Perlish "import strict"? I couldn't find it myself. While global change of Python to the variables local to statements and list comprehension could definitely affect too much programs, adding it on a per-source level would keep the compatibility while making the life of Python programmers safer. Thanks in advance. Appendix 1: examples. --------cut here-------- #!/usr/bin/env python -t a1 = ['a', 'b', 'c', 'd', 'e'] # Loop variable is alive outside the loop for k in a1: pass print "k: %s" % k # Loop variable is alive even outside the list comprehension! b1 = [l for l in a1] print "l: %s" % l # If there are several loops in a comprehension, # variables from all of them are kept intact c1 = [(m, n) for m in a1 for n in b1] print "m: %s, n: %s" % (m, n) # Loop variable is kept even from nested comprehensions d1 = [o for o in [p for p in a1]] print "o: %s, p: %s" % (o, p) # And the winner is... for r in a1: print "r0: %s, " % r, e1 = [r for r in b1] # Now try to access the "r" variable from the loop! print "r1: %s" % r --------cut here-------- -- With best regards, Alexander mailto:maa_public at sinn.ru From mwh at python.net Thu Sep 22 16:46:54 2005 From: mwh at python.net (Michael Hudson) Date: Thu, 22 Sep 2005 15:46:54 +0100 Subject: [Python-Dev] IMPORTANT: release24-maint branch is FROZEN from 2005-09-21 00:00 UTC for 2.4.2 In-Reply-To: <200509230033.33316.anthony@interlink.com.au> (Anthony Baxter's message of "Fri, 23 Sep 2005 00:33:30 +1000") References: <200509202306.54251.anthony@python.org> <2m8xxpjxav.fsf@starship.python.net> <93dc9c320509220636677d26f6@mail.gmail.com> <200509230033.33316.anthony@interlink.com.au> Message-ID: <2mzmq5i2ht.fsf@starship.python.net> Anthony Baxter writes: > On Thursday 22 September 2005 23:36, Jeremy Maxfield wrote: >> Can the fix for [ 1163563 ] Sub threads execute in restricted mode >> ( >> http://sourceforge.net/tracker/index.php?func=detail&aid=1163563&gr >>oup_id=5470&atid=105470 ) >> go in before 2.4.2 final? >> This is a real show stopper for us - we can't move to 2.4 without >> it. Cheers, >> Max > > I'm not sure - mwh, how risky a patch is this? Well, unless I'm much mistaken it's in 2.4.2c1.... Cheers, mwh -- I think if we have the choice, I'd rather we didn't explicitly put flaws in the reST syntax for the sole purpose of not insulting the almighty. -- /will on the doc-sig From anothermax at gmail.com Thu Sep 22 16:56:41 2005 From: anothermax at gmail.com (Jeremy Maxfield) Date: Thu, 22 Sep 2005 16:56:41 +0200 Subject: [Python-Dev] IMPORTANT: release24-maint branch is FROZEN from 2005-09-21 00:00 UTC for 2.4.2 In-Reply-To: <200509230033.33316.anthony@interlink.com.au> References: <200509202306.54251.anthony@python.org> <2m8xxpjxav.fsf@starship.python.net> <93dc9c320509220636677d26f6@mail.gmail.com> <200509230033.33316.anthony@interlink.com.au> Message-ID: <93dc9c3205092207564cb54d8e@mail.gmail.com> I don't think it's too risky. Michael Hudson did the work and it's been checked into the python CVS for a while. (Python/pystate.c Rev 2.42 and Modules/threadmodule.c Rev 2.64 are required files). We're stuck on Python 2.3.3 because of this bug...(2.4 thread code with this bug was backported to 2.3.5). Max On 9/22/05, Anthony Baxter wrote: > > On Thursday 22 September 2005 23:36, Jeremy Maxfield wrote: > > Can the fix for [ 1163563 ] Sub threads execute in restricted mode > > ( > > http://sourceforge.net/tracker/index.php?func=detail&aid=1163563&gr > >oup_id=5470&atid=105470 ) > > go in before 2.4.2 final? > > This is a real show stopper for us - we can't move to 2.4 without > > it. Cheers, > > Max > > I'm not sure - mwh, how risky a patch is this? > > Anthony > -- > Anthony Baxter > It's never too late to have a happy childhood. > -- flickr: http://www.flickr.com/photos/anothermax/sets -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20050922/303b8afa/attachment.htm From ark at acm.org Thu Sep 22 16:57:06 2005 From: ark at acm.org (Andrew Koenig) Date: Thu, 22 Sep 2005 10:57:06 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <2mmzm6k9ya.fsf@starship.python.net> Message-ID: <008701c5bf85$e6841aa0$6402a8c0@arkdesktop> > My problem with this syntax is that it can be hard to read: > > return if self.arg is None then default else self.arg > > looks worryingly like > > return NAME NAME.NAME NAME NAME NAME NAME NAME NAME.NAME > > to me. Interesting. What about return if self.arg is None: default else: self.arg ? From misa at redhat.com Thu Sep 22 17:01:06 2005 From: misa at redhat.com (misa@redhat.com) Date: Thu, 22 Sep 2005 11:01:06 -0400 Subject: [Python-Dev] unintentional and unsafe use of realpath() In-Reply-To: <1127318643.12452.1.camel@localhost.localdomain> References: <1126380571.21655.18.camel@localhost.localdomain> <43276823.2000603@levkowetz.com> <1126725908.23071.11.camel@localhost.localdomain> <1127318643.12452.1.camel@localhost.localdomain> Message-ID: <20050922150106.GA25649@abulafia.devel.redhat.com> Filed: https://sourceforge.net/tracker/index.php?func=detail&aid=1298813&group_id=5470&atid=305470 https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=169046 Misa On Wed, Sep 21, 2005 at 12:04:03PM -0400, Peter Jones wrote: > On Wed, 2005-09-14 at 15:25 -0400, Peter Jones wrote: > [ comments and a patch for sysmodule.c and some configure related files] > > ... and that patch has obvious problems as well. > > Here's a corrected one: > > --- Python-2.4.1/pyconfig.h.in.canonicalize 2005-09-14 11:47:04.000000000 -0400 > +++ Python-2.4.1/pyconfig.h.in 2005-09-14 11:47:02.000000000 -0400 > @@ -58,6 +58,9 @@ > /* Define if pthread_sigmask() does not work on your system. */ > #undef HAVE_BROKEN_PTHREAD_SIGMASK > > +/* Define to 1 if you have the `canonicalize_file_name' function. */ > +#undef HAVE_CANONICALIZE_FILE_NAME > + > /* Define to 1 if you have the `chown' function. */ > #undef HAVE_CHOWN > > --- Python-2.4.1/Python/sysmodule.c.canonicalize 2005-09-14 11:53:30.000000000 -0400 > +++ Python-2.4.1/Python/sysmodule.c 2005-09-14 11:52:04.000000000 -0400 > @@ -1184,6 +1184,11 @@ > char *p = NULL; > int n = 0; > PyObject *a; > +#ifdef HAVE_CANONICALIZE_FILE_NAME > + argv0 = canonicalize_file_name(argv0); > + if (argv0 == NULL) > + Py_FatalError("no mem for sys.argv"); > +#else /* ! HAVE_CANONICALIZE_FILE_NAME */ > #ifdef HAVE_READLINK > char link[MAXPATHLEN+1]; > char argv0copy[2*MAXPATHLEN+1]; > @@ -1256,9 +1261,13 @@ > #endif /* Unix */ > } > #endif /* All others */ > +#endif /* ! HAVE_CANONICALIZE_FILE_NAME */ > a = PyString_FromStringAndSize(argv0, n); > if (a == NULL) > Py_FatalError("no mem for sys.path insertion"); > +#ifdef HAVE_CANONICALIZE_FILE_NAME > + free(argv0); > +#endif /* HAVE_CANONICALIZE_FILE_NAME */ > if (PyList_Insert(path, 0, a) < 0) > Py_FatalError("sys.path.insert(0) failed"); > Py_DECREF(a); > --- Python-2.4.1/configure.in.canonicalize 2005-09-14 11:46:00.000000000 -0400 > +++ Python-2.4.1/configure.in 2005-09-14 11:47:22.000000000 -0400 > @@ -2096,8 +2096,8 @@ > AC_MSG_RESULT(MACHDEP_OBJS) > > # checks for library functions > -AC_CHECK_FUNCS(alarm bind_textdomain_codeset chown clock confstr ctermid \ > - execv fork fpathconf ftime ftruncate \ > +AC_CHECK_FUNCS(alarm bind_textdomain_codeset canonicalize_file_name chown \ > + clock confstr ctermid execv fork fpathconf ftime ftruncate \ > gai_strerror getgroups getlogin getloadavg getpeername getpgid getpid \ > getpriority getpwent getsid getwd \ > kill killpg lchown lstat mkfifo mknod mktime \ > > -- > Peter > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/misa%40redhat.com From anothermax at gmail.com Thu Sep 22 17:04:17 2005 From: anothermax at gmail.com (Jeremy Maxfield) Date: Thu, 22 Sep 2005 17:04:17 +0200 Subject: [Python-Dev] IMPORTANT: release24-maint branch is FROZEN from 2005-09-21 00:00 UTC for 2.4.2 In-Reply-To: <2mzmq5i2ht.fsf@starship.python.net> References: <200509202306.54251.anthony@python.org> <2m8xxpjxav.fsf@starship.python.net> <93dc9c320509220636677d26f6@mail.gmail.com> <200509230033.33316.anthony@interlink.com.au> <2mzmq5i2ht.fsf@starship.python.net> Message-ID: <93dc9c32050922080431064e78@mail.gmail.com> Sorry I think you're 'much mistaken'... The revision of PyState.c in the rc242c1 looks like 2.38.22 (should be 2.42 ) and threadmodule.c looks like 2.59 (should be 2.64) Cheers, Max On 9/22/05, Michael Hudson wrote: > > Anthony Baxter writes: > > > On Thursday 22 September 2005 23:36, Jeremy Maxfield wrote: > >> Can the fix for [ 1163563 ] Sub threads execute in restricted mode > >> ( > >> http://sourceforge.net/tracker/index.php?func=detail&aid=1163563&gr > >>oup_id=5470&atid=105470 ) > >> go in before 2.4.2 final? > >> This is a real show stopper for us - we can't move to 2.4 without > >> it. Cheers, > >> Max > > > > I'm not sure - mwh, how risky a patch is this? > > Well, unless I'm much mistaken it's in 2.4.2c1.... > > Cheers, > mwh > > -- > I think if we have the choice, I'd rather we didn't explicitly put > flaws in the reST syntax for the sole purpose of not insulting the > almighty. -- /will on the doc-sig > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/anothermax%40gmail.com > -- flickr: http://www.flickr.com/photos/anothermax/sets -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20050922/4d1827cb/attachment.html From guido at python.org Thu Sep 22 17:23:19 2005 From: guido at python.org (Guido van Rossum) Date: Thu, 22 Sep 2005 08:23:19 -0700 Subject: [Python-Dev] bool(iter([])) changed between 2.3 and 2.4 In-Reply-To: References: Message-ID: On 9/22/05, Jim Jewett wrote: > "Is there anything left?" is a pretty analogy for iterators, But unmaintainable for iterators in general. This was considered ad nauseam when iterators were initially introduced, and it was an explicit decision *not* to provide an API to look ahead. We should *not* give people a way to start coding like this: while it: x = it.next() ...process x... when they should be writing this instead: for x in it: ...process x... How would you implement the "is there anything left" functionality if the iterator is in fact a generator? (I know, the answer is buffering. But that has problems too. It was all considered when we designed it.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Sep 22 17:31:36 2005 From: guido at python.org (Guido van Rossum) Date: Thu, 22 Sep 2005 08:31:36 -0700 Subject: [Python-Dev] Mapping Darwin 8.2.0 to Mac OS X 10.4.2 in platform.py In-Reply-To: <43326362.7070001@egenix.com> References: <43326362.7070001@egenix.com> Message-ID: Thanks all! I won't touch it. /usr/bin/sw_vers is the way to go. On 9/22/05, M.-A. Lemburg wrote: > Ronald Oussoren wrote: > > > > On 22-sep-2005, at 5:26, Guido van Rossum wrote: > > > >> The platform module has a way to map system names such as returned by > >> uname() to marketing names. It maps SunOS to Solaris, for example. But > >> it doesn't map Darwin to Mac OS X. I think I know how to map Darwin > >> version numbers to OS X version numbers: from > >> http://www.opensource.apple.com/darwinsource/ it is clear that OS X > >> 10.a.b corresponds to Darwin (a+4).b, except for OS X versions <= > >> 10.1. I'd be happy to write the code and add it to system_alias() in > >> platform.py. Is this a good idea? > > > > > > There's no good reason to assume that the mapping from kernel version > > to marketing version will stay the same in the future. The savest way > > to get the marketing version of the currently running OSX is to run / > > usr/sbin/sw_vers and parse its output. There might also be a public API > > for getting the same information. Py2app, and specifically the > > bdist_mpkg component of that, contains code to parse sw_vers output. > > I don't have access to Macs, so there nothing much I can say > about this. > > In general, it's always better to rely on system tools for > finding the marketing name of an OS than to try to come > up with a work-around. If gestalt() returns the proper name, > then this should be used. If sw_vers provides a more reliable > way to do this, parsing its output seems like a better idea. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jcarlson at uci.edu Thu Sep 22 17:34:26 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu, 22 Sep 2005 08:34:26 -0700 Subject: [Python-Dev] Visibility scope for "for/while/if" statements In-Reply-To: <1477206818.20050922160515@sinn.ru> References: <1477206818.20050922160515@sinn.ru> Message-ID: <20050922075146.F732.JCARLSON@uci.edu> Alexander Myodov wrote: > Hello, > > Don't want to be importunate annoyingly asking the things probably > trivial for experienced community, but need to ask it anyway, after > spending about two hours trying to find well-camouflaged error caused > by it. In the future you should test your assumptions before writing software with them. > Why the variables defined inside "for"/"while"/"if" statements > (including loop variables for "for") are visible outside this scope? if and while statements don't define variables, so they can't expose them later in the scope. In regards to 'for' loops, they have always done that, there is code that relies on that behavior, and changing behavior would unnecessarily break code. As for list comprehensions, they were literally meant to be a completely equivalent translation of a set of for loops. That is: x = [] for i in foo: if f(i): x.append(i) could be translated into x = [i for i in foo if f(i)] and one would get the exact same side effects, including the 'leaking' of the most recently bound i into the local scope. The leakage was not accidental. > Yes, I am aware of one use case for this... er, feature. > It could be useful to remember the last item of the loop after the loop > is done... sometimes. But what is it for in other cases, except > confusing the programmer? There is a great reason: there generally exists two namespaces in Python, locals and globals (you can get a representation of of locals by using locals() (your changes to the dictionary won't change the local scope), and get a reference to the globals dictionary by globals()...there is also __builtins__, but you shouldn't be playing with that unless you know what you are doing). These namespaces offer you access to other namespaces (class, module, etc.). In most cases (the body of a function or method), the local scope is defined as an array with offset lookups: >>> def foo(n): ... n + 1 ... >>> dis.dis(foo) 2 0 LOAD_FAST 0 (n) <- look at this opcode 3 LOAD_CONST 1 (1) 6 BINARY_ADD 7 POP_TOP 8 LOAD_CONST 0 (None) 11 RETURN_VALUE >>> This results in the bytecode to be executed being significantly faster than if it were to reference a dictionary (like globals). >>> dis.dis(compile('n+1', '_', 'single')) 1 0 LOAD_NAME 0 (n) <- compare with this 3 LOAD_CONST 0 (1) 6 BINARY_ADD 7 PRINT_EXPR 8 LOAD_CONST 1 (None) 11 RETURN_VALUE Since the LOAD_NAME opcode does a dictionary lookup, it is necessarily slower than an array + offset lookup. Further, you should clarify what you would want this mythical non-leaky for loop to do in various cases. What would happen in the following case? i = None for i in ...: ... print i ... assuming that the loop executed more than once? Would you always get 'None' printed, or would you get the content of the last variable? What about: x = None for i in ...: x = f(i) ... print x Would 'x' be kept after the loop finished executing? I would imagine in your current code you are running something like this: i = #some important value ... #many lines of code for i in ...: ... #you access the 'i' you bound a long time ago In this case, you are making a common new programmer mistake by using the same variable name for two disparate concepts. If the 'i' that was initially bound was important through the lifetime of the scope, you should have named it differently than the 'i' that was merely a loop variable. I will also point out that in general, not leaking/exposing/etc. such variables would necessarily slow down Python. Why? Because it would necessitate nesting the concept of pseudo-namespaces. Now, when a function is compiled, nearly every local name is known, and they can be made fast (in the Python sense). If one were to nest pseudo namespaces, one would necessarily have another namespace lookup for every nested for loop. More specifically, accessing 'foo' in these three different nestings would take different amounts of time. for i in xrange(10): x = foo for i in xrange(10): for j in xrange(10): x = foo for i in xrange(10): for j in xrange(10): for k in xrange(10): x = foo And in the case that you do or don't want 'x' to 'leak' into the surrounding scope, you either take the speed hit again, or break quite a bit of code and be inconsistant. > Or maybe can someone hint me whether I can somehow switch the behaviour on > source-level to avoid keeping the variables outside the statements? No. > Something like Perlish "import strict"? I couldn't find it myself. > While global change of Python to the variables local to statements and > list comprehension could definitely affect too much programs, adding > it on a per-source level would keep the compatibility while making > the life of Python programmers safer. Python semantics seem to have been following the rule of "we are all adults here". If your assumptions have caused you bugs, then you should realize that your assumptions should have been tested before they were relied upon. That is what the interactive Python console is for. Generally though, Python follows a common C semantic of variable leakage. C Code: int i; // required in some versions of C for (i=0;i<10;i++) { ... } Equivalent Python: for i in xrange(10): ... As long as those loops don't have 'break' (or goto in the case of C) in them, Python and C will have the same value buond to 'i' after the loop. Again: test your assumptions. If your untested assumptions are wrong, don't complain that the language is broken. Also: python-dev is a mailing list for the development /of/ Python. Being that your questions as of late have been in the realm of "why does or doesn't Python do this?", you should go to python-list (or the equivalent comp.lang.python newsgroup) for answers to questions regarding current Python behavior, and why Python did or didn't do something in its past. - Josiah From jcarlson at uci.edu Thu Sep 22 17:38:07 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu, 22 Sep 2005 08:38:07 -0700 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <008701c5bf85$e6841aa0$6402a8c0@arkdesktop> References: <2mmzm6k9ya.fsf@starship.python.net> <008701c5bf85$e6841aa0$6402a8c0@arkdesktop> Message-ID: <20050922083558.F735.JCARLSON@uci.edu> "Andrew Koenig" wrote: > > > > My problem with this syntax is that it can be hard to read: > > > > return if self.arg is None then default else self.arg > > > > looks worryingly like > > > > return NAME NAME.NAME NAME NAME NAME NAME NAME NAME.NAME > > > > to me. > > Interesting. What about > > return if self.arg is None: default else: self.arg That's awful. It would confuse everyone as to why LCs and GEs don't have punctuation while these do. In that sense, I am not convinced that it should have keywords AND punctuation, one or the other. - Josiah From bob at redivi.com Thu Sep 22 19:10:53 2005 From: bob at redivi.com (Bob Ippolito) Date: Thu, 22 Sep 2005 13:10:53 -0400 Subject: [Python-Dev] Mapping Darwin 8.2.0 to Mac OS X 10.4.2 in platform.py In-Reply-To: References: Message-ID: /usr/bin/sw_vers technically calls a private (at least undocumented) CoreFoundation API, it doesn't parse that plist directly :) On further inspection, it looks like parsing the plist directly is supported API these days (see the bottom of ): import plistlib dct = plistlib.Plist.fromFile('/System/Library/CoreServices/ SystemVersion.plist') print '%(ProductName)s %(ProductVersion)s' % dct -bob On Sep 22, 2005, at 1:02 PM, Wilfredo S?nchez Vega wrote: > "rhapsody" is emitted by uname on Mac OS X Server 1.x, but not on > anything we ship today. > > Bob's right, the version number from uname only tells you about > the kernel, and not whether, for example, the Cocoa API is on the > system (it wouldn't be on a standalone Darwin OS install, which > will have the same uname output). > > Just FYI, /usr/bin/sw_vers parses /System/Library/CoreServices/ > SystemVersion.plist, which is XML. If you want that info, parsing > the file may be more efficient than forking off sw_vers. > > -wsv > > > On Sep 21, 2005, at 8:28 PM, Guido van Rossum wrote: > > >> I forgot. The current code recognizes 'Rhapsody' and maps it to >> "MacOS >> X Server". But I don't see any evidence that Apple still uses the >> code >> name Rhapsody. Does uname ever return 'Rhapsody'? >> > > From maa_public at sinn.ru Thu Sep 22 19:24:30 2005 From: maa_public at sinn.ru (Alexander Myodov) Date: Thu, 22 Sep 2005 22:54:30 +0530 Subject: [Python-Dev] Visibility scope for "for/while/if" statements Message-ID: <1191795748.20050922225430@sinn.ru> Hello Josiah, >> Why the variables defined inside "for"/"while"/"if" statements >> (including loop variables for "for") are visible outside this scope? JC> if and while statements don't define variables, so they can't expose JC> them later in the scope. They don't. They just leak internal ones: i = 0 while i != 1: i += 1 j = 5 print j JC> In regards to 'for' loops, they have always done that, there is code JC> that relies on that behavior, and changing behavior would unnecessarily JC> break code. I mentioned it below: while unconditional changing the behaviour will definitely break a lot, something like "import strict" would help those ones who rate such behaviour enough error-prone to reconsider proceeding using Python at all. JC> Further, you should clarify what you would want this mythical non-leaky JC> for loop to do in various cases. What would happen in the following JC> case? JC> i = None JC> for i in ...: JC> ... JC> print i JC> ... assuming that the loop executed more than once? Would you always JC> get 'None' printed, or would you get the content of the last variable? As for me (if my humble opinion is useful for you), the best would be to 1. assume that a loop variable is always newly-declared for this loop, 2. all variables first declared inside the loop are local to this loop. Thus, your example falls to case 1: "i" variable is newly declared for this loop. Well, we don't reuse old value of i to start the iteration from a particular place, like below? i = 5 for i in [3,4,5,6,7]: print i, More general, the variables could be assumed local only to the *same or higher* indentation level. JC> What about: JC> x = None JC> for i in ...: JC> x = f(i) JC> ... JC> print x JC> Would 'x' be kept after the loop finished executing? case 2: "x" variable is declared already. And if there is no "x = None" line, it should tell "name 'x' is not defined". Unless we are prone to funny "uncertainly defined" cases, if some variable is defined inside an "in" in the loop. JC> I would imagine in your current code you are running something like JC> this: JC> i = #some important value JC> ... #many lines of code JC> for i in ...: JC> ... JC> #you access the 'i' you bound a long time ago JC> In this case, you are making a common new programmer mistake by using JC> the same variable name for two disparate concepts. Nah, here is my error-case: I made several loops, one by one, using the "i" variable for looping. Then in the latest loop I changed the "i" name to more meaningful "imsi" name in the "for" declaration and whenever I found inside the loop. As I use "i" name *for loops exclusively*, I didn't wittingly reuse the same name for different purposes. The problem was that I missed one occurance of "i" variable inside the loop code, so it gained the same value (from the completion of previous loop) throughout all the "imsi" loop. And the interpreter didn't notice me that "I am using the undefined variable" (since it is considered defined in Python), as accustomed from other languages. That's my sorrowful story. JC> If the 'i' that was JC> initially bound was important through the lifetime of the scope, you JC> should have named it differently than the 'i' that was merely a loop JC> variable. I'm pretty happy that I didn't ever made a list comprehension with "i" variable inside the loop over "i" variable. While I wasn't aware of these side-effects, it would be even harder spottable. Now I'll look more carefully at copy-pasting list comprehensions, in every case examining all of the lower-indent variables for not to clash. JC> I will also point out that in general, not leaking/exposing/etc. such JC> variables would necessarily slow down Python. Why? I agree, this is one of obvious advantages of absence of pseudo-namespaces (wasting the resources to allocation and removal of variables, etc). Though in case of presence of pseudo-namespaces, it an easily be emulated by initializing the variable first-time on the highest-level of indentation. But for the "performance-oriented/human-friendliness" factor, Python is anyway not a rival to C and similar lowlevellers. C has pseudo-namespaces, though. JC> Python semantics seem to have been following the rule of "we are all JC> adults here". I always believed that the programming language (as any computer program) should slave to the human, rather than a human should slave to the program. JC> Generally though, Python follows a common C semantic of variable leakage. JC> C Code: JC> int i; // required in some versions of C JC> for (i=0;i<10;i++) { JC> ... JC> } "for (int i = 0; i < 10; i++)" works fine nowadays. JC> Again: test your assumptions. If your untested assumptions are wrong, JC> don't complain that the language is broken. In my case, as I explained already, it was not a assumption but an uncatched mistype, which is usually catched in other languages due to the presence of "pseudo-namespaces" (and in other languages I even accustomed to enforce the locality of temporary variables by explicitly setting the logically independent blocks of code within the curly braces). I always test the doubtful pieces before use. But *I don't dare to tell that the language is broken*. The existing behaviour has its own pros, and I also don't want a lot of code to be broken by easy behaviour change. What upsets me is the absence of alternatives (as the lack of flexibility) for the new code, especially with the fact that current behaviour mismatches with many of currently popular languages and habits learned from them. JC> Also: python-dev is a mailing list for the development /of/ Python. JC> Being that your questions as of late have been in the realm of "why does JC> or doesn't Python do this?", you should go to python-list (or the JC> equivalent comp.lang.python newsgroup) for answers to questions JC> regarding current Python behavior, and why Python did or didn't do JC> something in its past. I'm sorry for wasting the time of developers. For "for/while/if" statements, I just had an idea which (I believed) could be useful for many peoples, and could be very useful for such a human-orienter language as Python is (actually, this idea is borrowed from another language, but only partly - only in the area of ability to have "for" and "while" inside the single loop expression), so I though that the best people to estimate it should be the ones who potentially could implement this (provided they like it). For this conversation, the explained just looked to me as language-level problem (especially for list comprehensions), so I had to beg a workaroung/explanation/fix to it also from the language creators. If they don't consider it a problem and don't have a workaroung - sorry again for wasting your time, then that's indeed only my problem and I'll try to cope with it myself. I really didn't want to "claim and blame". What I asked and proposed was intended to be useful not only for myself, but - I believed - for many people, especially newcomers after the other languages. -- ? ?????????, Alexander mailto:maa_public at sinn.ru From falcon at intercable.ru Thu Sep 22 08:27:10 2005 From: falcon at intercable.ru (Sokolov Yura) Date: Thu, 22 Sep 2005 10:27:10 +0400 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP Message-ID: <43324EBE.2050705@intercable.ru> Ok. While windows already prefers threads, while linux-2.6 improves thread support and speed, while process startup expensive on time and memory, while we ought to dublicate our data and/or use obscure shared memory, while it is much simpler to make threaded program with care just about locks than multiprocessing with inventing interchange protocol (which abuses both CPU - sender and reciever), MULTIPROCESSING RULES!!! And on Opterons: you will win with multiprocessing ONLY if OS will bind process and processor/memory place pair. That would be difficult. Guido van Rossum wrote: > That's an understatement. I expect that *most* problems (even most > problems that we will be programming 10-20 years from now) get little > benefit out of MP. They are allready here. Servers they are. It is so simple to write application server in Python. It is so difficult to make it scallable in CPython. Hardware is cheap, development time is expensive. To make scallable CPython-backed server one needs too much time. And he goes to Java, .NET and others (and rare to Jython, IronPython (in the future)). CPython will not be wide popular without real multithreading. -- I want my Apache, written in Python !!!!! :-) -- I want my MySQL, written in Python !!!!! ;-) It is a joke. But every joke has a peace of truth (as russian sayes). (Excuse my English) From Rich.Burridge at Sun.COM Thu Sep 22 17:19:01 2005 From: Rich.Burridge at Sun.COM (Rich Burridge) Date: Thu, 22 Sep 2005 08:19:01 -0700 Subject: [Python-Dev] vendor-packages directory Message-ID: <4332CB65.5000507@sun.com> Hi, Recently I asked about the inclusion of a "vendor-packages" directory for Python on the Python mailing list. See the thread started at: http://mail.python.org/pipermail/python-list/2005-September/300029.html for the full reasoning behind this request, and the replies I received. I then approached Guido for his take on this suggestion. He recommended I submit the patch to the Python patch manager at SourceForge, and inform the folks on python-dev mailing list. So I have. You can find the patch submission at: http://sourceforge.net/tracker/index.php?func=detail&aid=1298835&group_id=5470&atid=305470 Thanks. -- Rich Burridge Email: rich.burridge at Sun.COM Sun Microsystems Inc. (MPK14-260), Phone: +1.650.786.5188 4150 Network Circle, AIM/YAHOO: RicBurridge Santa Clara, CA 95054 Blog: http://blogs.sun.com/richb From wsanchez at apple.com Thu Sep 22 19:02:05 2005 From: wsanchez at apple.com (=?ISO-8859-1?Q?Wilfredo_S=E1nchez_Vega?=) Date: Thu, 22 Sep 2005 10:02:05 -0700 Subject: [Python-Dev] Mapping Darwin 8.2.0 to Mac OS X 10.4.2 in platform.py In-Reply-To: References: Message-ID: "rhapsody" is emitted by uname on Mac OS X Server 1.x, but not on anything we ship today. Bob's right, the version number from uname only tells you about the kernel, and not whether, for example, the Cocoa API is on the system (it wouldn't be on a standalone Darwin OS install, which will have the same uname output). Just FYI, /usr/bin/sw_vers parses /System/Library/CoreServices/ SystemVersion.plist, which is XML. If you want that info, parsing the file may be more efficient than forking off sw_vers. -wsv On Sep 21, 2005, at 8:28 PM, Guido van Rossum wrote: > I forgot. The current code recognizes 'Rhapsody' and maps it to "MacOS > X Server". But I don't see any evidence that Apple still uses the code > name Rhapsody. Does uname ever return 'Rhapsody'? -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3057 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050922/93fe8ce7/smime-0001.bin From wsanchez at apple.com Thu Sep 22 19:19:19 2005 From: wsanchez at apple.com (=?ISO-8859-1?Q?Wilfredo_S=E1nchez_Vega?=) Date: Thu, 22 Sep 2005 10:19:19 -0700 Subject: [Python-Dev] Mapping Darwin 8.2.0 to Mac OS X 10.4.2 in platform.py In-Reply-To: References: Message-ID: <46B39630-DB41-4105-9137-57D0EF640A10@apple.com> Shockingly, it even says that parsing the file is "a better way" than using gestaltSystemVersion(). It's better for python, anyway, I think, since it doesn't require access to the Carbon API set. Be sure to handle the case where the file doesn't exist: import os version_info_file = "/System/Library/CoreServices/ SystemVersion.plist" if os.path.isfile(version_info_file): import plistlib info = plistlib.Plist.fromFile(version_info_file) print '%(ProductName)s %(ProductVersion)s' % info else: uname_os, uname_version = do_the_uname_thing print '%s %s' % (uname_os, uname_version) Or similar. -wsv On Sep 22, 2005, at 10:10 AM, Bob Ippolito wrote: > /usr/bin/sw_vers technically calls a private (at least undocumented) > CoreFoundation API, it doesn't parse that plist directly :) > > On further inspection, it looks like parsing the plist directly is > supported API these days (see the bottom of developer.apple.com/documentation/Carbon/Reference/Gestalt_Manager/ > gestalt_refchap/chapter_1.4_section_181.html>): > > import plistlib > dct = plistlib.Plist.fromFile('/System/Library/CoreServices/ > SystemVersion.plist') > print '%(ProductName)s %(ProductVersion)s' % dct > > -bob > > On Sep 22, 2005, at 1:02 PM, Wilfredo S?nchez Vega wrote: > > >> "rhapsody" is emitted by uname on Mac OS X Server 1.x, but not on >> anything we ship today. >> >> Bob's right, the version number from uname only tells you about >> the kernel, and not whether, for example, the Cocoa API is on the >> system (it wouldn't be on a standalone Darwin OS install, which >> will have the same uname output). >> >> Just FYI, /usr/bin/sw_vers parses /System/Library/CoreServices/ >> SystemVersion.plist, which is XML. If you want that info, parsing >> the file may be more efficient than forking off sw_vers. >> >> -wsv >> >> >> On Sep 21, 2005, at 8:28 PM, Guido van Rossum wrote: >> >> >> >>> I forgot. The current code recognizes 'Rhapsody' and maps it to >>> "MacOS >>> X Server". But I don't see any evidence that Apple still uses the >>> code >>> name Rhapsody. Does uname ever return 'Rhapsody'? >>> >>> >> >> >> > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ > wsanchez%40wsanchez.net > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3057 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050922/5e176f6c/smime-0001.bin From wsanchez at apple.com Thu Sep 22 19:31:54 2005 From: wsanchez at apple.com (=?ISO-8859-1?Q?Wilfredo_S=E1nchez_Vega?=) Date: Thu, 22 Sep 2005 10:31:54 -0700 Subject: [Python-Dev] Mapping Darwin 8.2.0 to Mac OS X 10.4.2 in platform.py In-Reply-To: References: Message-ID: <70A682D1-7FAA-4F1F-9DDD-D4B4E14FF8A1@apple.com> IIRC, it doesn't exist on such a system; that's a Mac OS command, not a Darwin command. (The man page correctly has "Mac OS X" in the footnote, not "Darwin" or "BSD", though I don't know that you can rely on that 100%.) -wsv On Sep 22, 2005, at 10:20 AM, Guido van Rossum wrote: > On 9/22/05, Wilfredo S?nchez Vega wrote: > >> "rhapsody" is emitted by uname on Mac OS X Server 1.x, but not on >> anything we ship today. >> >> Bob's right, the version number from uname only tells you about >> the kernel, and not whether, for example, the Cocoa API is on the >> system (it wouldn't be on a standalone Darwin OS install, which will >> have the same uname output). >> > > Just curious -- what would sw_vers print on such a system? -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3057 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050922/5c242a0c/smime.bin From mwh at python.net Thu Sep 22 19:49:44 2005 From: mwh at python.net (Michael Hudson) Date: Thu, 22 Sep 2005 18:49:44 +0100 Subject: [Python-Dev] IMPORTANT: release24-maint branch is FROZEN from 2005-09-21 00:00 UTC for 2.4.2 In-Reply-To: <93dc9c32050922080431064e78@mail.gmail.com> (Jeremy Maxfield's message of "Thu, 22 Sep 2005 17:04:17 +0200") References: <200509202306.54251.anthony@python.org> <2m8xxpjxav.fsf@starship.python.net> <93dc9c320509220636677d26f6@mail.gmail.com> <200509230033.33316.anthony@interlink.com.au> <2mzmq5i2ht.fsf@starship.python.net> <93dc9c32050922080431064e78@mail.gmail.com> Message-ID: <2mvf0thu13.fsf@starship.python.net> Jeremy Maxfield writes: > Sorry I think you're 'much mistaken'... > > The revision of PyState.c in the rc242c1 looks like 2.38.22 (should be > 2.42) > and threadmodule.c looks like 2.59 (should be 2.64) Looks like you're right but SF CVS is being *astonishingly* slow right now. The fixes will be in 2.4.2. Cheers, mwh -- There are 'infinite' number of developed artifacts and one cannot develop appreciation for them all. It would be all right to not understand something, but it would be imbecilic to put judgements on things one don't understand. -- Xah, comp.lang.lisp From gmccaughan at synaptics-uk.com Thu Sep 22 19:55:13 2005 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Thu, 22 Sep 2005 18:55:13 +0100 Subject: [Python-Dev] Visibility scope for "for/while/if" statements In-Reply-To: <1191795748.20050922225430@sinn.ru> References: <1191795748.20050922225430@sinn.ru> Message-ID: <200509221855.14571.gmccaughan@synaptics-uk.com> Alexander Myodov wrote: > Thus, your example falls to case 1: "i" variable is newly declared for > this loop. Well, we don't reuse old value of i to start the iteration > from a particular place, like below? > > i = 5 > for i in [3,4,5,6,7]: > print i, > > More general, the variables could be assumed local only to the *same > or higher* indentation level. So (since you're talking about "if" as well as "for" and "while") you're suggesting that x = 0 if foo(): x = 1 else: x = 2 should always leave x == 0? Or that the same bit of code, without the first line, should always leave x undefined? > JC> Python semantics seem to have been following the rule of "we are all > JC> adults here". > > I always believed that the programming language (as any computer > program) should slave to the human, rather than a human should slave > to the program. Right. And some of us humans *don't want* the change you're proposing. For what it's worth, I think it might well have been better if "for" and comprehensions had made their loop variables block-local; especially comprehensions. But making every for/while/if introduce a new level of scoping would be horrible. Perhaps you think it's what you want, but I think if you tried it for a month then you'd change your mind. -- g From phd at mail2.phd.pp.ru Thu Sep 22 19:58:00 2005 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Thu, 22 Sep 2005 21:58:00 +0400 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <43324EBE.2050705@intercable.ru> References: <43324EBE.2050705@intercable.ru> Message-ID: <20050922175800.GA22264@phd.pp.ru> On Thu, Sep 22, 2005 at 10:27:10AM +0400, Sokolov Yura wrote: > MULTIPROCESSING RULES!!! Everything in programming is about "divide and conquer". Separate and control. Modules. Classes. Namespaces. And now that multithreading with shared memory. That's an evil idea, it causes a lot of troubles, it requires locks and semaphores, and still leads to problems. Shared memory is evil. Multithreading is harmful. > Hardware is cheap, development time is expensive. Yes. Do not spend developer's time finding bugs in locks and semaphores, fixing race conditions, etc. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From tjreedy at udel.edu Thu Sep 22 20:00:55 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 22 Sep 2005 14:00:55 -0400 Subject: [Python-Dev] Visibility scope for "for/while/if" statements References: <1477206818.20050922160515@sinn.ru> Message-ID: "Alexander Myodov" wrote in message news:1477206818.20050922160515 at sinn.ru... > Why the variables defined inside "for"/"while"/"if" statements > (including loop variables for "for") are visible outside this scope? Questions about why Python is the way it is belong on comp.lang.python, the general Python mailing list, or gmane.comp.python.general (they are all gatewayed to each other). Moreover, they are likely to have been asked, answered, and discussed previously, with the answers possibly discoverable thru Google search of the c.l.p archives. This one certainly has been. From pje at telecommunity.com Thu Sep 22 20:12:04 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 22 Sep 2005 14:12:04 -0400 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <43324EBE.2050705@intercable.ru> Message-ID: <5.1.1.6.0.20050922140950.01b5e8a0@mail.telecommunity.com> At 10:27 AM 9/22/2005 +0400, Sokolov Yura wrote: >It is so simple to write application server in Python. >It is so difficult to make it scallable in CPython. It seems you've never heard of fork(), which works just fine to scale Python processes on multiprocessor boxes. I've actually done this, and it didn't take a lot of time to implement, so I don't know what you're talking about here. From pje at telecommunity.com Thu Sep 22 20:16:48 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 22 Sep 2005 14:16:48 -0400 Subject: [Python-Dev] vendor-packages directory In-Reply-To: <4332CB65.5000507@sun.com> Message-ID: <5.1.1.6.0.20050922141219.03354008@mail.telecommunity.com> At 08:19 AM 9/22/2005 -0700, Rich Burridge wrote: >Hi, > >Recently I asked about the inclusion of a "vendor-packages" >directory for Python on the Python mailing list. > >See the thread started at: > > http://mail.python.org/pipermail/python-list/2005-September/300029.html > >for the full reasoning behind this request, and the replies >I received. I'm with Terry on this, it needs a better rationale. Why can't you just add a .pth file to the site-packages directory? It seems that would address the issue nicely. (See http://docs.python.org/lib/module-site.html for documentation of the .pth mechanism, which would let you implement a separate vendor-packages directory without modifying Python, and would still allow local overrides of your vendor packages.) From fredrik at pythonware.com Thu Sep 22 20:42:24 2005 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 22 Sep 2005 20:42:24 +0200 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP References: <43324EBE.2050705@intercable.ru> <5.1.1.6.0.20050922140950.01b5e8a0@mail.telecommunity.com> Message-ID: Phillip J. Eby wrote: >>It is so simple to write application server in Python. >>It is so difficult to make it scallable in CPython. > > It seems you've never heard of fork(), which works just fine to scale > Python processes on multiprocessor boxes. there's a version of fork hidden somewhere in CPython that solves all interprocess communication and synchronization issues? cool. From jcarlson at uci.edu Thu Sep 22 21:00:51 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu, 22 Sep 2005 12:00:51 -0700 Subject: [Python-Dev] Visibility scope for "for/while/if" statements In-Reply-To: <1191795748.20050922225430@sinn.ru> References: <1191795748.20050922225430@sinn.ru> Message-ID: <20050922112912.F73B.JCARLSON@uci.edu> Alexander Myodov wrote: [snip Alexander Myodov complaining about how Python works] > i = 0 > while i != 1: > i += 1 > j = 5 > print j Maybe you don't realize this, but C's while also 'leaks' internal variables... int i = 0, j; while (i != 1) { i++; j = 5; } printf("%i %i\n", i, j); If you haven't yet found a good use for such 'leakage', you should spend more time programming and less time talking; you would find (quite readily) that such 'leaking' is quite beneficial. > I made several loops, one by one, using the "i" variable for looping. > Then in the latest loop I changed the "i" name to more meaningful > "imsi" name in the "for" declaration and whenever I found inside the loop. > As I use "i" name *for loops exclusively*, I didn't wittingly reuse the > same name for different purposes. The problem was that I missed one > occurance of "i" variable inside the loop code, so it gained the same > value (from the completion of previous loop) throughout all the "imsi" > loop. And the interpreter didn't notice me that "I am using the > undefined variable" (since it is considered defined in Python), as > accustomed from other languages. That's my sorrowful story. So you mistyped something. I'm crying for you, really I am. > But for the "performance-oriented/human-friendliness" factor, Python > is anyway not a rival to C and similar lowlevellers. C has > pseudo-namespaces, though. C does not have pseudo-namespaces or variable encapsulation in for loops. Ah hah hah! Look ladies and gentlemen, I caught myself a troll! Python does not rival C in the performance/friendliness realm? Who are you trying to kid? There is a reason why high school teachers are teaching kids Python instead of Pascal, Java, etc., it's because it is easier to learn and use. On the performance realm, of course Python is beat out by low-level languages; it was never meant to compete with them. Python does what it can for speed when such speed does not affect the usability of the language. What you are proposing both would reduce speed and usability, which suggests that it wasn't a good idea in the first place. > JC> Python semantics seem to have been following the rule of "we are all > JC> adults here". > I always believed that the programming language (as any computer > program) should slave to the human, rather than a human should slave > to the program. Your beliefs were unfounded. If you look at every programming language, there are specific semantics and syntax for all of them. If you fail to use and/or understand them, the langauge will not be your 'slave'; it will not run correctly, if at all. > "for (int i = 0; i < 10; i++)" works fine nowadays. I'm sorry, but you are wrong. The C99 spec states that you must define the type of i before using it in the loop. Maybe you are thinking of C++, which allows such things. > JC> Also: python-dev is a mailing list for the development /of/ Python. > JC> Being that your questions as of late have been in the realm of "why does > JC> or doesn't Python do this?", you should go to python-list (or the > JC> equivalent comp.lang.python newsgroup) for answers to questions > JC> regarding current Python behavior, and why Python did or didn't do > JC> something in its past. > I'm sorry for wasting the time of developers. For "for/while/if" > statements, I just had an idea which (I believed) could be useful for > many peoples, Test your ideas on comp.lang.python first, when more than a handful of people agree with you, come back. - Josiah From martin at v.loewis.de Thu Sep 22 21:17:59 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 22 Sep 2005 21:17:59 +0200 Subject: [Python-Dev] Compressing MSI files: 2.4.2 candidate? Message-ID: <43330367.5040104@v.loewis.de> I just found that I can save somewhat more than 1MiB in the MSI file by using LZX:21 instead of the standard MSZIP when compressing the CAB file. A resulting package can be found at http://www.dcl.hpi.uni-potsdam.de/home/loewis/python-2.4.2c1.msi Can people please test whether this installs just as well? Is it ok to use this modified procedure for 2.4.2 final (2.4.2c1 still uses MSZIP). Regards, Martin From greg at electricrain.com Thu Sep 22 21:20:09 2005 From: greg at electricrain.com (Gregory P. Smith) Date: Thu, 22 Sep 2005 12:20:09 -0700 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <2mirwwlsve.fsf@starship.python.net> References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <8393fff0509190928196bba9a@mail.gmail.com> <2mirwwlsve.fsf@starship.python.net> Message-ID: <20050922192009.GG26810@zot.electricrain.com> On Mon, Sep 19, 2005 at 09:12:05PM +0100, Michael Hudson wrote: > Martin Blais writes: > > On 9/18/05, Guido van Rossum wrote: > >> On 9/17/05, John J Lee wrote: > >> > I realize that not all algorithms (nor all computational problems) scale > >> > well to MP hardware. Is it feasible to usefully compile both MP and a UP > >> > binaries from one Python source code base? > >> > >> That's an understatement. I expect that *most* problems (even most > >> problems that we will be programming 10-20 years from now) get little > >> benefit out of MP. > > > > Some are saying it won't be a matter of choice if we want to get the > > software to run faster (you know, that "MORE MORE MORE!" thing we all > > seem to suffer from): > > People have been saying this for _years_, and it hasn't happened yet. > This time around it's a bit more convincing, but I reserve the right > to remain a touch skeptical. good! :) > > http://www.gotw.ca/publications/concurrency-ddj.htm > > The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software > > Herb Sutter > > March 2005 > > I was disappointed that that article (hey, it was the only issue of > ddj I've ever actually bought! :) didn't consider any concurrency > models other than shared memory threading. Beware. Multi-core and/or multi-threaded cpus are the only thing the high end CPU manufacturers are able to produce today that they can still claim to be "faster." There is a HUGE incentive for them to create demand for their design lest it become irrelevant and they be forced to sell only low-margin commodity single core hardware. This means we'll see a ton of papers and people paid or coerced into suggesting that this is the best thing since time sliced bread. -g From Rich.Burridge at Sun.COM Thu Sep 22 21:04:14 2005 From: Rich.Burridge at Sun.COM (Rich Burridge) Date: Thu, 22 Sep 2005 12:04:14 -0700 Subject: [Python-Dev] vendor-packages directory In-Reply-To: <5.1.1.6.0.20050922141219.03354008@mail.telecommunity.com> References: <5.1.1.6.0.20050922141219.03354008@mail.telecommunity.com> Message-ID: <4333002E.30602@sun.com> Phillip J. Eby wrote: >> Recently I asked about the inclusion of a "vendor-packages" >> directory for Python on the Python mailing list. >> >> See the thread started at: >> >> >> http://mail.python.org/pipermail/python-list/2005-September/300029.html >> >> for the full reasoning behind this request, and the replies >> I received. > > > I'm with Terry on this, it needs a better rationale. Why can't you just > add a .pth file to the site-packages directory? It seems that would > address the issue nicely. > > (See http://docs.python.org/lib/module-site.html for documentation of > the .pth mechanism, which would let you implement a separate > vendor-packages directory without modifying Python, and would still > allow local overrides of your vendor packages.) I understand how .pth files work. The rationale for requesting the "vendor-packages" approach, is that Python files, as supplied by the vendor (Sun, Apple, RedHat ...) with their operating system software, should go in a totally separate base directory, to differentiate them from Python files installed specifically at one site. From pje at telecommunity.com Thu Sep 22 21:56:35 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 22 Sep 2005 15:56:35 -0400 Subject: [Python-Dev] vendor-packages directory Message-ID: <5.1.1.6.0.20050922155628.0333e088@mail.telecommunity.com> At 12:04 PM 9/22/2005 -0700, Rich Burridge wrote: >Phillip J. Eby wrote: >>>Recently I asked about the inclusion of a "vendor-packages" >>>directory for Python on the Python mailing list. >>> >>>See the thread started at: >>> >>>http://mail.python.org/pipermail/python-list/2005-September/300029.html >>> >>>for the full reasoning behind this request, and the replies >>>I received. >> >>I'm with Terry on this, it needs a better rationale. Why can't you just >>add a .pth file to the site-packages directory? It seems that would >>address the issue nicely. >>(See http://docs.python.org/lib/module-site.html for documentation of the >>.pth mechanism, which would let you implement a separate vendor-packages >>directory without modifying Python, and would still allow local overrides >>of your vendor packages.) > >I understand how .pth files work. > >The rationale for requesting the "vendor-packages" approach, is that >Python files, >as supplied by the vendor (Sun, Apple, RedHat ...) with their operating >system >software, should go in a totally separate base directory, to differentiate >them >from Python files installed specifically at one site. Right - I'm proposing you add a vendor-packages.pth file to site-pacakges, that points to a "totally separate base directory" where those files are installed, not that you install the packages themselves under site-packages. To make sure we're disagreeing about the same thing <0.5 wink>, here's what I'm suggesting: 1. Install your packages in /usr/lib/python2.4/vendor-packages (so far, so good, we agree on this part) 2. Create a 'vendor-packages.pth' file in /usr/lib/python2.4/site-packages, containing the line: /usr/lib/python2.4/vendor-packages This will ensure that your vendor-packages "just work", unless locally overridden in site-packages. So, you install one .pth file, once, as part of your installation of Python itself, and it works *now*, without waiting for a new Python release or complicating any other Python code or requiring agreement between vendors about how to do this. This is one of the things that .pth files are for, and I believe it should be the recommended way for operating systems to establish OS-specific modifications to the default Python module search path. Otherwise, everybody will be patching site.py for their specific OS, and there are already too many OS-specific things in that code, and for backward-compatibility reasons we can't get rid of them in Python 2.x. I don't think we want to add new ones. From pje at telecommunity.com Thu Sep 22 21:58:17 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 22 Sep 2005 15:58:17 -0400 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: References: <43324EBE.2050705@intercable.ru> <5.1.1.6.0.20050922140950.01b5e8a0@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050922153903.0334fc80@mail.telecommunity.com> At 08:42 PM 9/22/2005 +0200, Fredrik Lundh wrote: >Phillip J. Eby wrote: > >At 10:27 AM 9/22/2005 +0400, Sokolov Yura wrote: > >>It is so simple to write application server in Python. > >>It is so difficult to make it scallable in CPython. > > > > It seems you've never heard of fork(), which works just fine to scale > > Python processes on multiprocessor boxes. > >there's a version of fork hidden somewhere in CPython that solves all >interprocess communication and synchronization issues? cool. Yep, it's called "fork stateless application servers that keep their state in a relational database." <0.1 wink> It works for an astonishingly wide assortment of business problems, anyway. :) Don't get me wrong, I'm not opposed to adding better IPC capabilities; a Linda-like IPC facility would be nice to have, too. I was just pointing out the fallacy in the original post (whose attribution you deleted for some reason), that *scalability* is the issue. The original poster was claiming (in effect) that IPC and synchronization are trivial and that the GIL is responsible for some mythical lack of "scalability". Threads can't scale past one machine, no matter how many processors you put in it, so it's a red herring IMO. From bob at redivi.com Thu Sep 22 22:03:22 2005 From: bob at redivi.com (Bob Ippolito) Date: Thu, 22 Sep 2005 16:03:22 -0400 Subject: [Python-Dev] vendor-packages directory In-Reply-To: <5.1.1.6.0.20050922155628.0333e088@mail.telecommunity.com> References: <5.1.1.6.0.20050922155628.0333e088@mail.telecommunity.com> Message-ID: On Sep 22, 2005, at 3:56 PM, Phillip J. Eby wrote: > At 12:04 PM 9/22/2005 -0700, Rich Burridge wrote: > >> Phillip J. Eby wrote: >> >>>> Recently I asked about the inclusion of a "vendor-packages" >>>> directory for Python on the Python mailing list. >>>> >>>> See the thread started at: >>>> >>>> http://mail.python.org/pipermail/python-list/2005-September/ >>>> 300029.html >>>> >>>> for the full reasoning behind this request, and the replies >>>> I received. >>>> >>> >>> I'm with Terry on this, it needs a better rationale. Why can't >>> you just >>> add a .pth file to the site-packages directory? It seems that would >>> address the issue nicely. >>> (See http://docs.python.org/lib/module-site.html for >>> documentation of the >>> .pth mechanism, which would let you implement a separate vendor- >>> packages >>> directory without modifying Python, and would still allow local >>> overrides >>> of your vendor packages.) >>> >> >> I understand how .pth files work. >> >> The rationale for requesting the "vendor-packages" approach, is that >> Python files, >> as supplied by the vendor (Sun, Apple, RedHat ...) with their >> operating >> system >> software, should go in a totally separate base directory, to >> differentiate >> them >> from Python files installed specifically at one site. >> > > Right - I'm proposing you add a vendor-packages.pth file to site- > pacakges, > that points to a "totally separate base directory" where those > files are > installed, not that you install the packages themselves under > site-packages. To make sure we're disagreeing about the same thing > <0.5 > wink>, here's what I'm suggesting: > > 1. Install your packages in /usr/lib/python2.4/vendor-packages (so > far, so > good, we agree on this part) > > 2. Create a 'vendor-packages.pth' file in /usr/lib/python2.4/site- > packages, > containing the line: > > /usr/lib/python2.4/vendor-packages > > This will ensure that your vendor-packages "just work", unless locally > overridden in site-packages. vendor-packages.pth should look like this (evil, but practical): import site; site.addsitedir('/usr/lib/python2.4/vendor-packages') Otherwise packages like Numeric, PIL, PyObjC, etc. that take advantage of pth files will not work when installed to that location. -bob From bob at redivi.com Thu Sep 22 22:11:33 2005 From: bob at redivi.com (Bob Ippolito) Date: Thu, 22 Sep 2005 16:11:33 -0400 Subject: [Python-Dev] vendor-packages directory In-Reply-To: <4333002E.30602@sun.com> References: <5.1.1.6.0.20050922141219.03354008@mail.telecommunity.com> <4333002E.30602@sun.com> Message-ID: <3A47C098-74C6-4FF3-BE7C-BC04F7F813A0@redivi.com> On Sep 22, 2005, at 3:04 PM, Rich Burridge wrote: > Phillip J. Eby wrote: > >>> Recently I asked about the inclusion of a "vendor-packages" >>> directory for Python on the Python mailing list. >>> >>> See the thread started at: >>> >>> >>> http://mail.python.org/pipermail/python-list/2005-September/ >>> 300029.html >>> >>> for the full reasoning behind this request, and the replies >>> I received. >>> >> >> >> I'm with Terry on this, it needs a better rationale. Why can't >> you just >> add a .pth file to the site-packages directory? It seems that would >> address the issue nicely. >> >> (See http://docs.python.org/lib/module-site.html for documentation of >> the .pth mechanism, which would let you implement a separate >> vendor-packages directory without modifying Python, and would still >> allow local overrides of your vendor packages.) >> > > I understand how .pth files work. > > The rationale for requesting the "vendor-packages" approach, is > that Python files, > as supplied by the vendor (Sun, Apple, RedHat ...) with their > operating system > software, should go in a totally separate base directory, to > differentiate them > from Python files installed specifically at one site. Note that Apple already does this via a .pth file ("Extras.pth"), as of Mac OS X 10.4, which points to the following location: /System/Library/Frameworks/Python.framework/Versions/2.3/Extras/lib/ python Currently this location includes wxPython. There is an additional vendor package which they (wrongly) integrated into their patched Python build process so it ends up in the plat-mac subsection of the standard library (CoreGraphics). This is a closed-source not-very- well-done SWIG generated CoreGraphics wrapper that they use for some print filters and such. Note that Python does have additional support on Mac OS X for an additional per-user site directory out of the box (for framework style builds anyway): ~/Library/PythonX.Y/site-packages/ There's also the admin-writable /Library/PythonX.Y/site-packages/, but that's an implementation detail in that Apple made site-packages a symlink to that location. -bob From maa_public at sinn.ru Thu Sep 22 22:17:34 2005 From: maa_public at sinn.ru (Alexander Myodov) Date: Fri, 23 Sep 2005 01:47:34 +0530 Subject: [Python-Dev] Visibility scope for "for/while/if" statements Message-ID: <39396086.20050923014734@sinn.ru> Hello Josiah, >> i = 0 >> while i != 1: >> i += 1 >> j = 5 >> print j JC> Maybe you don't realize this, but C's while also 'leaks' internal JC> variables... JC> int i = 0, j; JC> while (i != 1) { JC> i++; JC> j = 5; JC> } JC> printf("%i %i\n", i, j); Yeah, it may *leak* it in your example. But the advantage is that it may *not leak* as well: for (int i = 0; i <= 5; i++) { int j = 5; } printf ("%i\n", j); // Won't be even compiled Or in our "while" case: int i = 0; while (i != 1) { i++; int j = 5; } printf("%i %i\n", i, j); JC> If you haven't yet found a good use for such 'leakage', you should spend JC> more time programming and less time talking; you would find (quite JC> readily) that such 'leaking' is quite beneficial. I see such usages and realize the possible gains from it. But for about five years of software development in a bunch of programming languages, I've found good uses of non-leaking variables *as well*. Up to the level of manually "enclosing" the temporary variables used only once: ... int i = input(); int result; { int j = f1(i); int k = f2(i, j); result = f3(i, j, k); } // Now j and k are dead ... >> accustomed from other languages. That's my sorrowful story. JC> So you mistyped something. I'm crying for you, really I am. Yeah, I did. Just forgot removing the usage / renaming of variable neved initialized (I thought) before. And my error was that I relied upon the interpreter to catch it, mistakenly assuming the locality of variables inside loops as a consistency with other common languages. Mea culpa, without any jeers. The only reason why I was wrote my email was to get the understanding of community assessment of such behaviour, and/or probable workarounds. No blame against Python - it's great even with this unusualty. >> But for the "performance-oriented/human-friendliness" factor, Python >> is anyway not a rival to C and similar lowlevellers. C has >> pseudo-namespaces, though. JC> C does not have pseudo-namespaces or variable encapsulation in for loops. Neither for (int i = 0; i <= 5; i++) { int j = 5; } printf ("%i\n", i); nor for (int i = 0; i <= 5; i++) { int j = 5; } printf ("%i\n", j); gets compiled though with "gcc -std=c99". That is, each loop introduces a new scoping level. JC> Ah hah hah! Look ladies and gentlemen, I caught myself a troll! Python JC> does not rival C in the performance/friendliness realm? Who are you JC> trying to kid? There is a reason why high school teachers are teaching JC> kids Python instead of Pascal, Java, etc., it's because it is easier to JC> learn and use. Ohh, my slip made me considered a troll... I meant only that Python does not rival C in such area as "performance by all means, even with decrease of friendliness" (but the overall performance is indeed good, I've seen even the VoIP solution written purely on on Python). And I believe noone of Python developers want it to be rival. JC> What you are proposing both would reduce speed and JC> usability, which suggests that it wasn't a good idea in the first place. Yes, it lacks performance, but I still believe that an opportunity to close the visibility of variables (*opportunity* rather than *behaviour change*!) is better in terms of "friendliness" rather than lack of it. Just because it offers the alternatives. Because C-style closed visibility can emulate the variable lack, but not vice versa. At least one voice here (of Gareth McCaughan) is not-strictly-against it. So I cannot agree that such *option* can reduce usability. >> "for (int i = 0; i < 10; i++)" works fine nowadays. JC> I'm sorry, but you are wrong. The C99 spec states that you must define JC> the type of i before using it in the loop. Maybe you are thinking of JC> C++, which allows such things. "gcc -std=c99" compiles the line "for (int i = 0; i <= 5; i++);" perfectly. Along with another sample mentioned above . To this day, I relied upon gcc in terms of standards compatibility... JC> Tes your ideas on comp.lang.python first, when more than a handful of JC> people agree with you, come back. Ok. Next time I shall be more careful. Sorry again, and thanks for your time. And thanks for more-or-less patient answers. -- With best regards, Alexander mailto:maa_public at sinn.ru From pje at telecommunity.com Thu Sep 22 22:35:45 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 22 Sep 2005 16:35:45 -0400 Subject: [Python-Dev] vendor-packages directory In-Reply-To: <43331195.10004@sun.com> References: <5.1.1.6.0.20050922155628.0333e088@mail.telecommunity.com> <5.1.1.6.0.20050922155628.0333e088@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050922163159.02a81e58@mail.telecommunity.com> At 01:18 PM 9/22/2005 -0700, Rich Burridge wrote: >Phillip J. Eby wrote: >>Right - I'm proposing you add a vendor-packages.pth file to >>site-pacakges, that points to a "totally separate base directory" where >>those files are installed, not that you install the packages themselves >>under site-packages. To make sure we're disagreeing about the same thing >><0.5 wink>, here's what I'm suggesting: >>1. Install your packages in /usr/lib/python2.4/vendor-packages (so far, >>so good, we agree on this part) >>2. Create a 'vendor-packages.pth' file in >>/usr/lib/python2.4/site-packages, containing the line: >> /usr/lib/python2.4/vendor-packages >>This will ensure that your vendor-packages "just work", unless locally >>overridden in site-packages. >>So, you install one .pth file, once, as part of your installation of >>Python itself, and it works *now*, without waiting for a new Python >>release or complicating any other Python code or requiring agreement >>between vendors about how to do this. This is one of the things that >>.pth files are for, and I believe it should be the recommended way for >>operating systems to establish OS-specific modifications to the default >>Python module search path. Otherwise, everybody will be patching site.py >>for their specific OS, and there are already too many OS-specific things >>in that code, and for backward-compatibility reasons we can't get rid of >>them in Python 2.x. I don't think we want to add new ones. > >Bob Ippolito wrote: > > vendor-packages.pth should look like this (evil, but practical): > > > > import site; site.addsitedir('/usr/lib/python2.4/vendor-packages') > > > > Otherwise packages like Numeric, PIL, PyObjC, etc. that take advantage > > of pth files will not work when installed to that location. > >Thanks! I'll work with the architectural committee in question to see if they >are happy with this approach to solving the problem. Glad we could help. Unfortunately, there isn't any sort of document at the moment for "Python Distributors" to help explain how these kinds of problems should be solved. It would probably be a good idea for us to have one. Perhaps a volunteer or two could be found on the distutils-SIG who could help put one together, since a certain number of distributions' Python gurus tend to hang out there. (By the way, assuming this solves your problem fully, don't forget to withdraw your patch on SF, optionally citing the python.org archive URL for this email.) From rblists at free.fr Thu Sep 22 22:40:25 2005 From: rblists at free.fr (richard barran) Date: Thu, 22 Sep 2005 22:40:25 +0200 Subject: [Python-Dev] os.path.diff(path1, path2) (and a first post) Message-ID: On Sat, 17 Sep 2005, John J Lee wrote: > > On Fri, 16 Sep 2005, Greg Ewing wrote: > > > Trent Mick wrote: > > > > > If this *does* get added (I'm +0) then let's call it "relpath" or > > > "relpathto" as in the various implementations out there: > > > > +1 on that, too. Preferably just "relpath". > [...] > > +1 on adding this function, and on "relpath" as the name. I wanted this > function just a few weeks ago. Hello All, This is my first post here, so first a few words about me. My name is Richard Barran. I live in Paris (but I'm English) and earn my living by coding Oracle stored procedures in PL/SQL. I discovered Python 2 years ago and now use it in my day job for a variety of purposes: quick scripts, unit testing of my "proper" PL/SQL stuff, code deployment scripts and so on. Now down to business: I'm the author of one of the "relpath" functions posted on the ActiveState website* so I've read this thread with interest. For some time I've been thinking of contributing something back to Python, but I thought the relpath function was too specialised for inclusion in the standard library. Also I wasn't clear on how to go about getting the support of a python-dev member, as required in the development process guidelines paper :-) So I have a question: do the previous mails mean that a relpath function might possibly be a usefull addition to os.path? And if the answer to the previous question is "yes", then should I submit a patch, or is someone else already working on it? Regards, Richard Barran * my version is the one at http://aspn.activestate.com/ASPN/Cookbook/ Python/Recipe/302594 From jcarlson at uci.edu Thu Sep 22 22:58:08 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu, 22 Sep 2005 13:58:08 -0700 Subject: [Python-Dev] Visibility scope for "for/while/if" statements In-Reply-To: <39396086.20050923014734@sinn.ru> References: <39396086.20050923014734@sinn.ru> Message-ID: <20050922134429.F746.JCARLSON@uci.edu> Alexander Myodov wrote: > >> "for (int i = 0; i < 10; i++)" works fine nowadays. > JC> I'm sorry, but you are wrong. The C99 spec states that you must define > JC> the type of i before using it in the loop. Maybe you are thinking of > JC> C++, which allows such things. > "gcc -std=c99" compiles the line "for (int i = 0; i <= 5; i++);" > perfectly. Along with another sample mentioned above . > To this day, I relied upon gcc in terms of standards compatibility... I misread the error message from my compiler. It /only/ works in C99 mode. Previous C standards (which CPython itself conforms to) did not allow such things. Interestingly enough, not all C++ compilers (Microsoft) hid variables created in for loops (http://www.devx.com/cplus/10MinuteSolution/28908/0/page/2). - Josiah From guido at python.org Thu Sep 22 23:16:27 2005 From: guido at python.org (Guido van Rossum) Date: Thu, 22 Sep 2005 14:16:27 -0700 Subject: [Python-Dev] Visibility scope for "for/while/if" statements In-Reply-To: <20050922134429.F746.JCARLSON@uci.edu> References: <39396086.20050923014734@sinn.ru> <20050922134429.F746.JCARLSON@uci.edu> Message-ID: Please end this thread. It belongs in c.l.py. Nothing's going to change. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From janssen at parc.com Thu Sep 22 23:15:14 2005 From: janssen at parc.com (Bill Janssen) Date: Thu, 22 Sep 2005 14:15:14 PDT Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: Your message of "Wed, 21 Sep 2005 23:27:10 PDT." <43324EBE.2050705@intercable.ru> Message-ID: <05Sep22.141518pdt."58617"@synergy1.parc.xerox.com> Sokolov Jura writes: > It is so simple to write application server in Python. > It is so difficult to make it scallable in CPython. > CPython will not be wide popular without real multithreading. He's right. Bill From tjreedy at udel.edu Fri Sep 23 00:51:43 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 22 Sep 2005 18:51:43 -0400 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP References: <43324EBE.2050705@intercable.ru> <05Sep22.141518pdt."58617"@synergy1.parc.xerox.com> Message-ID: "Bill Janssen" wrote in message news:05Sep22.141518pdt."58617"@synergy1.parc.xerox.com... > Sokolov Jura writes: >> It is so simple to write application server in Python. >> It is so difficult to make it scallable in CPython. >> CPython will not be wide popular without real multithreading. > > He's right. I think the post is at least somewhat wrong on both technical and topicalness levels. On a technical level, the second statement seems wrong as an absolute statement. Phillip Eby claims that *some* application servers are easily scaled even if others are not. I suspect this is more likely to be the truth. In any case, the statement conveys no new technical information while to me it has the form and emotional load of a flame. The third statement is an opinion about the indefinite future which is neither right nor wrong as it stands. To ever judge it so, we need an operational definition of 'wide popular' to guide measurement. By some standards, Python is already widely popular. By others, it never will be. Guido already said that he will consider a high-quality maintainable 'real threading' patch that proves his scepticism wrong and that is neutral to beneficial to most all users, so I see little point in further flogging the issue, at least on this forum. Terry J. Reedy From trentm at ActiveState.com Fri Sep 23 02:58:13 2005 From: trentm at ActiveState.com (Trent Mick) Date: Thu, 22 Sep 2005 17:58:13 -0700 Subject: [Python-Dev] os.path.diff(path1, path2) (and a first post) In-Reply-To: References: Message-ID: <20050923005813.GB25109@ActiveState.com> [richard barran wrote] > So I have a question: do the previous mails mean that a relpath > function might possibly be a usefull addition to os.path? Yes, it seems to have support. > And if the answer to the previous question is "yes", then should I > submit a patch, or is someone else already working on it? Yes, please do. Trent -- Trent Mick TrentM at ActiveState.com From bob at redivi.com Fri Sep 23 03:09:15 2005 From: bob at redivi.com (Bob Ippolito) Date: Thu, 22 Sep 2005 21:09:15 -0400 Subject: [Python-Dev] os.path.diff(path1, path2) (and a first post) In-Reply-To: <20050923005813.GB25109@ActiveState.com> References: <20050923005813.GB25109@ActiveState.com> Message-ID: On Sep 22, 2005, at 8:58 PM, Trent Mick wrote: > [richard barran wrote] > >> So I have a question: do the previous mails mean that a relpath >> function might possibly be a usefull addition to os.path? >> > > Yes, it seems to have support. I'd like to throw in another late +1 here, I've written this myself more times than I care to count. -bob From greg.ewing at canterbury.ac.nz Fri Sep 23 03:34:32 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 23 Sep 2005 13:34:32 +1200 Subject: [Python-Dev] bool(iter([])) changed between 2.3 and 2.4 In-Reply-To: References: Message-ID: <43335BA8.9010300@canterbury.ac.nz> Jim Jewett wrote: > "Is there anything left?" is a pretty analogy for iterators, > particularly since the examples tend to start with list > or file iterators. But this can't be supported by all iterators, so it would be something special to these iterators. Keeping track of the peculiarities of iterators associated with particular types is too much to fit in my brain. I'd rather they all had the same interface. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Fri Sep 23 03:53:27 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 23 Sep 2005 13:53:27 +1200 Subject: [Python-Dev] Visibility scope for "for/while/if" statements In-Reply-To: <20050922075146.F732.JCARLSON@uci.edu> References: <1477206818.20050922160515@sinn.ru> <20050922075146.F732.JCARLSON@uci.edu> Message-ID: <43336017.5070500@canterbury.ac.nz> Josiah Carlson wrote: > As for list comprehensions, they were literally meant to be a > completely equivalent translation of a set of for loops. I don't think that's quite true. I doubt whether anyone really thought about the issue when LCs were first being discussed. I didn't, but if I had, I wouldn't have considered the variable-leaking as being something that it was necessary to preserve, because the only use case for it is something you can't do with an LC anyway. The reasons for the variable-leaking being preserved are (1) it fell out of the implementation and (2) it makes the documentation slightly simpler, since LCs can be described fully and accurately in terms of translation to for-loops. Whether those are *good* reasons or not is debatable. In Py3k it's possible that this will be resolved by making for-loop variables local to the loop as well. Or maybe not. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From guido at python.org Fri Sep 23 04:38:54 2005 From: guido at python.org (Guido van Rossum) Date: Thu, 22 Sep 2005 19:38:54 -0700 Subject: [Python-Dev] Visibility scope for "for/while/if" statements In-Reply-To: <43336017.5070500@canterbury.ac.nz> References: <1477206818.20050922160515@sinn.ru> <20050922075146.F732.JCARLSON@uci.edu> <43336017.5070500@canterbury.ac.nz> Message-ID: On 9/22/05, Greg Ewing wrote: > Josiah Carlson wrote: > > > As for list comprehensions, they were literally meant to be a > > completely equivalent translation of a set of for loops. > > I don't think that's quite true. I doubt whether anyone > really thought about the issue when LCs were first being > discussed. I didn't, but if I had, I wouldn't have > considered the variable-leaking as being something that > it was necessary to preserve, because the only use case > for it is something you can't do with an LC anyway. > > The reasons for the variable-leaking being preserved are > (1) it fell out of the implementation and (2) it makes the > documentation slightly simpler, since LCs can be described > fully and accurately in terms of translation to for-loops. > > Whether those are *good* reasons or not is debatable. It was the cheapest implementation; it was known at the time that it wasn't ideal. I'd like to fix this in 3.0, so that LC's are just a special case of GE's. > In Py3k it's possible that this will be resolved by making > for-loop variables local to the loop as well. Or maybe not. Definitely not. There are lots of uses for the loop control variable after break. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From kbk at shore.net Fri Sep 23 05:21:41 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Thu, 22 Sep 2005 23:21:41 -0400 (EDT) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200509230321.j8N3LfHG018191@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 337 open ( -6) / 2941 closed (+14) / 3278 total ( +8) Bugs : 908 open ( +0) / 5262 closed (+17) / 6170 total (+17) RFE : 194 open ( +5) / 187 closed ( +2) / 381 total ( +7) New / Reopened Patches ______________________ use LIST_APPEND opcode for X.append() method calls (2005-09-16) http://python.org/sf/1292625 opened by Neal Norwitz python.sty correction - verbatim environment (2005-09-17) http://python.org/sf/1293788 opened by Wojciech Smigaj python.sty: \py at sigline correction (2005-09-17) http://python.org/sf/1293790 opened by Wojciech Smigaj cjkcodecs does not initialize type pointer (2005-09-20) http://python.org/sf/1297028 opened by Eric Huss "Fatal Python error" from cStringIO's writelines (2005-09-22) CLOSED http://python.org/sf/1298449 opened by Andrew Bennetts Unchecked return value in cStringIO (2005-09-22) CLOSED http://python.org/sf/1298499 opened by Andrew Bennetts sysmodule.c: realpath() is unsafe (2005-09-22) http://python.org/sf/1298813 opened by Mihai Ibanescu vendor-packages directory. (2005-09-22) http://python.org/sf/1298835 opened by Rich Burridge Patches Closed ______________ Simple webbrowser fix for netscape -remote (2004-12-02) http://python.org/sf/1077979 closed by birkenfeld Replacement for webbrowser.py which has problems. (2004-05-16) http://python.org/sf/954628 closed by birkenfeld Multiple webbrowser.py bug fixes / improvements (2003-04-27) http://python.org/sf/728278 closed by birkenfeld python -c readlink()s and stat()s '-c' (2005-02-09) http://python.org/sf/1119423 closed by birkenfeld Bugfix for dircheck() in Objects/fileobject.c (2004-04-30) http://python.org/sf/944928 closed by birkenfeld tkinter hello world example bug (2005-08-31) http://python.org/sf/1277677 closed by birkenfeld Trivial typo in unicodedata._getcode (2005-06-03) http://python.org/sf/1213831 closed by loewis Cache lines in StreamReader.readlines (2005-08-24) http://python.org/sf/1268314 closed by loewis locale.getdefaultencoding: LANGUAGE not correctly parsed (2005-03-20) http://python.org/sf/1166938 closed by doko locale.getdefaultencoding: precedence of LANGUAGE / LANG (2005-03-20) http://python.org/sf/1166948 closed by doko locale: 'utf' is not a known encoding (2005-03-20) http://python.org/sf/1166957 closed by doko "Fatal Python error" from cStringIO's writelines (2005-09-22) http://python.org/sf/1298449 closed by mwh Unchecked return value in cStringIO (2005-09-22) http://python.org/sf/1298499 closed by mwh New / Reopened Bugs ___________________ Installation of waste by MacPython installer (2005-09-15) CLOSED http://python.org/sf/1291662 opened by Freek Dijkstra The _ssl build process for 2.3.5 is broken (2005-09-16) http://python.org/sf/1292634 opened by Robert Cheung doctest runner cannot handle non-ascii characters (2005-09-17) http://python.org/sf/1293741 opened by GRISEL Distutils writes keywords comma-separated (2005-09-17) http://python.org/sf/1294032 opened by Martijn Pieters Error in metaclass search order (2005-09-18) http://python.org/sf/1294232 opened by Pedro Werneck email.Parser.FeedParser leak (2005-09-18) CLOSED http://python.org/sf/1294453 opened by George Giannakopoulos Problems with /usr/lib64 builds. (2005-09-19) http://python.org/sf/1294959 opened by Sean Reifschneider termios.c in qnx4.25 (2005-09-19) http://python.org/sf/1295179 opened by kbob_ru cookielib undefined local variable error (2005-09-19) CLOSED http://python.org/sf/1295432 opened by Yusuke Shinyama expat symbols should be namespaced in pyexpat (2005-09-19) http://python.org/sf/1295808 opened by Trent Mick inspect.getsource() misses single line blocks. (2005-09-19) http://python.org/sf/1295909 opened by Ron Adam MemoryError in httplib (2005-09-20) http://python.org/sf/1296004 opened by Yue Zhang Python/C API Reference Manual, 7.2.1.1 Boolean Objects (2005-09-20) CLOSED http://python.org/sf/1296321 opened by Li Daobing expat crash python (2005-09-20) http://python.org/sf/1296433 opened by Mike Rozhnov Call by object reference sometimes call by value (2005-09-20) http://python.org/sf/1296434 opened by Alan G datetime.replace could take a dict (2005-09-20) http://python.org/sf/1296581 opened by Tom Lynn Incorrect return type for search() method (2005-09-21) http://python.org/sf/1297059 opened by Noah Spurrier hashable and mutable functions (2005-09-21) CLOSED http://python.org/sf/1297986 opened by ChristianJ shlex does not support unicode (2005-09-21) http://python.org/sf/1298120 opened by Wai Yip Tung _socket module not build under cygwin (2005-09-22) http://python.org/sf/1298709 opened by Miki Tebeka MSI installer does not pass values from UI (2005-09-22) http://python.org/sf/1298962 opened by Matthew Leslie "Howto RE" link is dead (2005-09-22) http://python.org/sf/1299051 opened by Macs R We build fails on Solaris 10 for Objects/complexobject.c (2005-09-22) http://python.org/sf/1299187 opened by Kenneth Simpson Tkinter Scale returns only integer from/to values (2005-09-22) CLOSED http://python.org/sf/1299520 opened by Tom Goddard Bugs Closed ___________ Installation of waste by MacPython installer (2005-09-15) http://python.org/sf/1291662 closed by jackjansen _iscommand() in webbrowser module (2003-02-17) http://python.org/sf/687747 closed by birkenfeld webbrowser doesn't start default Gnome browser by default (2005-02-02) http://python.org/sf/1114929 closed by birkenfeld urllib2 fails its builtin test (2003-11-18) http://python.org/sf/844336 closed by birkenfeld test_pwd fails on 64bit system (Opteron) (2004-11-15) http://python.org/sf/1066546 closed by birkenfeld Calling builtin function 'eval' from C causes seg fault. (2004-01-01) http://python.org/sf/868706 closed by birkenfeld Error in representation of complex numbers(again) (2005-02-08) http://python.org/sf/1118729 closed by birkenfeld patch 1079734 broke cgi.FieldStorage w/ multipart post req. (2005-01-31) http://python.org/sf/1112856 closed by arigo setup.py imports pwd before it's built if HOME not set (2003-02-28) http://python.org/sf/694812 closed by birkenfeld email.Parser.FeedParser leak (2005-09-18) http://python.org/sf/1294453 closed by montanaro email.Generator traceback in forwarded msg (2005-07-12) http://python.org/sf/1236906 closed by bwarsaw logging.shutdown() not correct for chained handlers (2005-09-05) http://python.org/sf/1282539 closed by vsajip cookielib undefined local variable error (2005-09-19) http://python.org/sf/1295432 closed by birkenfeld Python/C API Reference Manual, 7.2.1.1 Boolean Objects (2005-09-20) http://python.org/sf/1296321 closed by birkenfeld Build issues (lib path) on linux2-x86_64 (2004-03-27) http://python.org/sf/924333 closed by birkenfeld Finding X11 fails on SuSE Opteron (2004-01-07) http://python.org/sf/872392 closed by birkenfeld Building python 2.3.3 RPM on Mandrake 9.2. fails (2004-04-22) http://python.org/sf/939699 closed by birkenfeld Tkinter Scale returns only integer from/to values (2005-09-22) http://python.org/sf/1299520 deleted by tom_goddard RFE Closed __________ add dedent() string method (2005-07-13) http://python.org/sf/1237680 closed by birkenfeld hashable and mutable functions (2005-09-21) http://python.org/sf/1297986 closed by rhettinger From vwehren at home.nl Fri Sep 23 08:30:12 2005 From: vwehren at home.nl (Vincent Wehren) Date: Fri, 23 Sep 2005 08:30:12 +0200 Subject: [Python-Dev] Compressing MSI files: 2.4.2 candidate? In-Reply-To: <43330367.5040104@v.loewis.de> Message-ID: <20050923063013.95CFB1E4006@bag.python.org> > -----Original Message----- > From: python-dev-bounces+vwehren=home.nl at python.org > [mailto:python-dev-bounces+vwehren=home.nl at python.org] On > Behalf Of "Martin v. Lowis" > Sent: Thursday, September 22, 2005 9:18 PM > To: Python-Dev > Subject: [Python-Dev] Compressing MSI files: 2.4.2 candidate? > > I just found that I can save somewhat more than 1MiB in the > MSI file by using LZX:21 instead of the standard MSZIP when > compressing the CAB file. A resulting package can be found at > > http://www.dcl.hpi.uni-potsdam.de/home/loewis/python-2.4.2c1.msi > > Can people please test whether this installs just as well? Martin, The LZX:21-compressed package worked absolutely fine for me (Windows XP Professional Build 2600.xpsp_sp2_gdr). -- Vincent Wehren > > Is it ok to use this modified procedure for 2.4.2 final > (2.4.2c1 still uses MSZIP). > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/vwehren%40home.nl From mwh at python.net Fri Sep 23 11:23:50 2005 From: mwh at python.net (Michael Hudson) Date: Fri, 23 Sep 2005 10:23:50 +0100 Subject: [Python-Dev] IMPORTANT: release24-maint branch is FROZEN from 2005-09-21 00:00 UTC for 2.4.2 In-Reply-To: <2mvf0thu13.fsf@starship.python.net> (Michael Hudson's message of "Thu, 22 Sep 2005 18:49:44 +0100") References: <200509202306.54251.anthony@python.org> <2m8xxpjxav.fsf@starship.python.net> <93dc9c320509220636677d26f6@mail.gmail.com> <200509230033.33316.anthony@interlink.com.au> <2mzmq5i2ht.fsf@starship.python.net> <93dc9c32050922080431064e78@mail.gmail.com> <2mvf0thu13.fsf@starship.python.net> Message-ID: <2mr7bgi1cp.fsf@starship.python.net> Michael Hudson writes: > Jeremy Maxfield writes: > >> Sorry I think you're 'much mistaken'... >> >> The revision of PyState.c in the rc242c1 looks like 2.38.22 (should be >> 2.42) >> and threadmodule.c looks like 2.59 (should be 2.64) > > Looks like you're right but SF CVS is being *astonishingly* slow right > now. The fixes will be in 2.4.2. OK, I've finally managed to get these changes checked in. Cheers, mwh -- . <- the point your article -> . |------------------------- a long way ------------------------| -- Christophe Rhodes, ucam.chat From anothermax at gmail.com Fri Sep 23 11:31:29 2005 From: anothermax at gmail.com (Jeremy Maxfield) Date: Fri, 23 Sep 2005 11:31:29 +0200 Subject: [Python-Dev] IMPORTANT: release24-maint branch is FROZEN from 2005-09-21 00:00 UTC for 2.4.2 In-Reply-To: <2mr7bgi1cp.fsf@starship.python.net> References: <200509202306.54251.anthony@python.org> <2m8xxpjxav.fsf@starship.python.net> <93dc9c320509220636677d26f6@mail.gmail.com> <200509230033.33316.anthony@interlink.com.au> <2mzmq5i2ht.fsf@starship.python.net> <93dc9c32050922080431064e78@mail.gmail.com> <2mvf0thu13.fsf@starship.python.net> <2mr7bgi1cp.fsf@starship.python.net> Message-ID: <93dc9c3205092302312be40764@mail.gmail.com> That's great - thanks alot. Cheers, Max On 9/23/05, Michael Hudson wrote: > > Michael Hudson writes: > > > Jeremy Maxfield writes: > > > >> Sorry I think you're 'much mistaken'... > >> > >> The revision of PyState.c in the rc242c1 looks like 2.38.22 (should be > >> 2.42) > >> and threadmodule.c looks like 2.59 (should be 2.64) > > > > Looks like you're right but SF CVS is being *astonishingly* slow right > > now. The fixes will be in 2.4.2. > > OK, I've finally managed to get these changes checked in. > > Cheers, > mwh > > -- > . <- the point your article -> . > |------------------------- a long way ------------------------| > -- Christophe Rhodes, ucam.chat > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/anothermax%40gmail.com > -- flickr: http://www.flickr.com/photos/anothermax/sets -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20050923/474d480d/attachment.htm From m at keysolutions.ru Fri Sep 23 10:51:40 2005 From: m at keysolutions.ru (Mikhail Kashkin) Date: Fri, 23 Sep 2005 12:51:40 +0400 Subject: [Python-Dev] Repository for python developers Message-ID: Moscow, Russia, September 23th, 2005 We are happy to spread good news about `Key Solutions' `_ new initiative. We have created a new public repository for python developers. The ulitimate goal of this project is to unite companies and people under the umbrella of Russian-speaking-python (Zope/Plone/Zope3/Twisted/etc) open source programmers community, so that we could jointly push forward new programs and technologies in Russia. The repository powered by `subversion `_. **Links**: - `Press release in russian `_ - `??????? ?? ??????? `_ - `Subversion `_ About Key Solutions -------------------- Key Solutions is an open source company with primary focus in development of corporative CRM Internet/intranet systems and web-communities. The company promotes and supports Zope, Plone, Asterisk platforms in Russia. To learn more about Key Solutions visit `eng.keysolutions.ru `_ and `keysolutions.ru `_ -- Mikhail Kashkin, Key Solutions (http://keysolutions.ru/) Director Zope/Asterisk/Plone - Solutions/Consulting/Support Plone ?? ??????? http://plone.org.ru/ Plone Foundation Member (http://plone.org/foundation/members/) From mwh at python.net Fri Sep 23 13:34:12 2005 From: mwh at python.net (Michael Hudson) Date: Fri, 23 Sep 2005 12:34:12 +0100 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <20050922192009.GG26810@zot.electricrain.com> (Gregory P. Smith's message of "Thu, 22 Sep 2005 12:20:09 -0700") References: <43292071.8050604@intercable.ru> <432B34BA.1020202@v.loewis.de> <8393fff0509190928196bba9a@mail.gmail.com> <2mirwwlsve.fsf@starship.python.net> <20050922192009.GG26810@zot.electricrain.com> Message-ID: <2mmzm4hvbf.fsf@starship.python.net> "Gregory P. Smith" writes: > On Mon, Sep 19, 2005 at 09:12:05PM +0100, Michael Hudson wrote: >> Martin Blais writes: >> > http://www.gotw.ca/publications/concurrency-ddj.htm >> > The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software >> > Herb Sutter >> > March 2005 >> >> I was disappointed that that article (hey, it was the only issue of >> ddj I've ever actually bought! :) didn't consider any concurrency >> models other than shared memory threading. > > Beware. Multi-core and/or multi-threaded cpus are the only thing the > high end CPU manufacturers are able to produce today that they can > still claim to be "faster." There is a HUGE incentive for them to > create demand for their design lest it become irrelevant and they be > forced to sell only low-margin commodity single core hardware. This > means we'll see a ton of papers and people paid or coerced into > suggesting that this is the best thing since time sliced bread. I'd imagine that the more interesting/easy to deal with concurrency models can and will be implemented atop shared memory threads. At least, for the reasons you give, I hope so :) Cheers, mwh -- ZAPHOD: OK, so ten out of ten for style, but minus several million for good thinking, eh? -- The Hitch-Hikers Guide to the Galaxy, Episode 2 From gmccaughan at synaptics-uk.com Fri Sep 23 14:40:54 2005 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Fri, 23 Sep 2005 13:40:54 +0100 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <43322078.1060304@canterbury.ac.nz> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <43318067.3090301@islandtraining.com> <43322078.1060304@canterbury.ac.nz> Message-ID: <200509231340.55533.gmccaughan@synaptics-uk.com> > The reason I like "a if b else c" is because it has the > most natural word order. In English, > My dog is happy if he has a bone, else sad. > sounds much more natural than > My dog is, if he has a bone, happy, else sad. Neither sounds very natural to me; conditional expressions don't occur much in English. It's much more common to hear something like My dog is happy if he has a bone; otherwise he's sad. And that would correspond to a postfix conditional operator on *statements*, not expressions; Perl and Ruby have this, but it doesn't seem very Pythonic. > Interestingly, it looks *more* odd to me if parens are > included: > > return (self.arg if self.arg is not None else default) > > I think this is because, without the parens, I tend to read > the "if" as applying to the whole phrase "return self.arg", > not just to the "self.arg". Which is one reason why I *don't* like the proposed "consequent if antecedent else alternative" syntax; it looks like a statement modifier when it isn't really one. When you're forced (by the parens) to read it correctly, you stop liking it! :-) * Conditional expressions do occur in English, though, and they do generally have the consequent at the front. But ... they also generally have the "else-marker" right at the end. "Buy some pork if it's going cheap, and some beef if not." "You can divide an angle into N equal parts with ruler and compasses if N is a power of 2 times a product of distinct Fermat primes, but not if N has any other form." "I sleep all night and work all day." I don't think a syntax that authentically mirrors the structure of conditional expressions in English is likely to be very good. x = (123 if p==q; 321 otherwise) x = (123 if p==q, else 321 if not) x = (123 if p==q; else 321 if r==s; else 999 if not) :-) -- g From gmccaughan at synaptics-uk.com Fri Sep 23 14:48:24 2005 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Fri, 23 Sep 2005 13:48:24 +0100 Subject: [Python-Dev] Visibility scope for "for/while/if" statements In-Reply-To: <20050922112912.F73B.JCARLSON@uci.edu> References: <1191795748.20050922225430@sinn.ru> <20050922112912.F73B.JCARLSON@uci.edu> Message-ID: <200509231348.25408.gmccaughan@synaptics-uk.com> On Thursday 2005-09-22 20:00, Josiah Carlson wrote: [Alexander Myodov:] > > But for the "performance-oriented/human-friendliness" factor, Python > > is anyway not a rival to C and similar lowlevellers. C has > > pseudo-namespaces, though. > > C does not have pseudo-namespaces or variable encapsulation in for loops. > > Ah hah hah! Look ladies and gentlemen, I caught myself a troll! Python > does not rival C in the performance/friendliness realm? Who are you > trying to kid? I think you've misunderstood Alex here; he's saying that Python and C don't occupy the same region of the spectrum that runs from "high performance, human-unfriendly" to "lower performance, human friendly". Which is correct, unlike some other things he's said :-). > > "for (int i = 0; i < 10; i++)" works fine nowadays. > > I'm sorry, but you are wrong. The C99 spec states that you must define > the type of i before using it in the loop. Maybe you are thinking of > C++, which allows such things. No, Alex is right on this one too. Maybe you are thinking of C89, which forbids such things. 6.8.5.3 The for statement [#1] Except for the behavior of a continue statement in the loop body, the statement for ( clause-1 ; expr-2 ; expr-3 ) statement and the sequence of statements { clause-1 ; while ( expr-2 ) { statement expr-3 ; } } are equivalent (where clause-1 can be an expression or a declaration).123) ... 123Thus, clause-1 specifies initialization for the loop, possibly declaring one or more variables for use in the loop; expr-2, the controlling expression, specifies an evaluation made before each iteration, such that execution of the loop continues until the expression compares equal to 0; expr-3 specifies an operation (such as incrementing) that is performed after each iteration. If clause-1 is a declaration, then the scope of any variable it declares is the remainder of the declaration and the entire loop, including the other two expressions. (This is from a late draft of the C99 spec; I'm fairly sure the final version is no different.) -- g From ark at acm.org Fri Sep 23 15:45:45 2005 From: ark at acm.org (Andrew Koenig) Date: Fri, 23 Sep 2005 09:45:45 -0400 Subject: [Python-Dev] Visibility scope for "for/while/if" statements In-Reply-To: <20050922134429.F746.JCARLSON@uci.edu> Message-ID: <003501c5c045$1cc64740$6402a8c0@arkdesktop> > Interestingly enough, not all C++ compilers (Microsoft) hid variables > created in for loops > (http://www.devx.com/cplus/10MinuteSolution/28908/0/page/2). That's because the C++ spec changed during standardization, when the standards committee realized the original idea was a mistake. One of the convincing cases: if (x) for (int i = 0; i != 10; ++i) { } Is I in scope after the if statement? If so, what value does it have if x is false? If not, then apparently the subject of an "if" statement is a scope...so why can't I write this? if (x) int i = 42; and have i go out of scope? From p.f.moore at gmail.com Fri Sep 23 16:30:14 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 23 Sep 2005 15:30:14 +0100 Subject: [Python-Dev] GIL, Python 3, and MP vs. UP In-Reply-To: <20050921102057.F722.JCARLSON@uci.edu> References: <9519083A-6B06-4F3E-A8BD-47F9684E64B5@chello.se> <1127320319.21126.18.camel@p-dvsi-418-1.rd.francetelecom.fr> <20050921102057.F722.JCARLSON@uci.edu> Message-ID: <79990c6b050923073026908b7c@mail.gmail.com> On 9/21/05, Josiah Carlson wrote: > > Antoine Pitrou wrote: > > > > > > > The best way to make people stop complaining about the GIL and start > > > using > > > process-based multiprogramming is to provide solid, standardized support > > > for process-based multiprogramming. > > > > 100% agreed. I needed a portable way to know if a program was already > > running (and then to send it a simple command), the only simple solution > > I found was to listen on a local TCP socket and store the port number in > > a file in a known location. Not very elegant IMO. > > On *nix, use a unix domain socket (location in the filesystem which acts > as a listening socket). On Windows, you can use cTypes, pywin32, etc., > to create a global mutex and/or COM interface. Yes, but that's the point - having a cross-platform solution available "out of the box" (where the box may be the stdlib, or it may be third party). The problems I see are: 1. Encapsulating a suitable core feature set in a nice API 2. Making it cross-platform With something covering both of these, a case could be made for including it in the stdlib, to support/encourage multiprocess style coding in place of multithreading. I use threading quite a lot, often to spawn multiple workers all doing "the same thing" on different targets. A simple thread pool plus threading.Queue is perfect for this. A package which provided a similar abstraction using worker processes would be fine for me. Of course, *I* want it for Windows, and most multiprocess stuff tends to be Unix specific (where multiprocessing is natural and fast). And I'm happy enough with threads (I'm scaling up the number of blocking calls I'm waiting on - which already release the GIL - and so I don't care too much about SMP per se). This is getting offtopic, though, and probably more suitable for c.l.p, where design issues of a multiprocessing module should be thrashed out before coming back to python-dev with a proposal for the stdlib. Paul. From p.f.moore at gmail.com Fri Sep 23 16:55:11 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 23 Sep 2005 15:55:11 +0100 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <43308D75.7060602@gmail.com> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <43308D75.7060602@gmail.com> Message-ID: <79990c6b0509230755626ecfc3@mail.gmail.com> On 9/20/05, Nick Coghlan wrote: > Basically, I'm +1 on the original PEP 308 form because it reads more naturally > (and more like LC's and GE's) to me in expression contexts, and +0 on the > "if/then/elif/else" form (because I would like a real conditional operator). I agree that (expr if cond else expr) fits far more naturally these days given the ordering of generator expressions (and list comprehensions, but they were around when PEP 308 was discussed, so I don't think they work as well as evidence :-)) Also, (expr if cond else expr) doesn't naturally admit an elif case, reducing complexity (and the tendency to overuse the construct in complex ways) a little. And it doesn't need a new keyword. I have a vague fondness for this form. I really can't express it very well, but the if-then-else version just feels a little too "boring and conventional". I'm sure people will get used to it (heck, we've got decorators and print >> :-)) but maybe pushing a (mildly) idiosyncratic form isn't worth opening that can of worms again. I'm +1 on a conditiona expression. I suspect the time is now right. I'm +1 on *one* of (expr if cond else expr) or (if cond then expr else expr) With the latter, I don't mind if elif is added or not. I'm +1000000000 on Guido just making the decision. No more votes, please! Paul. From jcarlson at uci.edu Fri Sep 23 18:11:23 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Fri, 23 Sep 2005 09:11:23 -0700 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <200509231340.55533.gmccaughan@synaptics-uk.com> References: <43322078.1060304@canterbury.ac.nz> <200509231340.55533.gmccaughan@synaptics-uk.com> Message-ID: <20050923090744.9788.JCARLSON@uci.edu> Gareth McCaughan wrote: [seems to have gone off list with a portion of the discussion] > > The reason I like "a if b else c" is because it has the > > most natural word order. In English, > > My dog is happy if he has a bone, else sad. > > sounds much more natural than > > My dog is, if he has a bone, happy, else sad. > > Neither sounds very natural to me; conditional > expressions don't occur much in English. It's > much more common to hear something like > > My dog is happy if he has a bone; otherwise he's sad. Natural english wise, this also sounds natural: If my dog has a bone he's happy, otherwise he's sad. But yeah, I'll use them regardless of the syntax, even if I have preferences. - Josiah From martin at v.loewis.de Fri Sep 23 18:43:50 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 23 Sep 2005 18:43:50 +0200 Subject: [Python-Dev] Compressing MSI files: 2.4.2 candidate? In-Reply-To: <20050923063013.95CFB1E4006@bag.python.org> References: <20050923063013.95CFB1E4006@bag.python.org> Message-ID: <433430C6.5040202@v.loewis.de> Vincent Wehren wrote: > The LZX:21-compressed package worked absolutely fine for me (Windows XP > Professional Build 2600.xpsp_sp2_gdr). Thanks for all the confirmations; I'm going to use it then for 2.4.2. Regards, Martin From tjreedy at udel.edu Sat Sep 24 03:48:39 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 23 Sep 2005 21:48:39 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <4330B7CE.50302@canterbury.ac.nz> <433140B1.2070406@gmail.com><43317EC1.908@ronadam.com> Message-ID: >> Need I continue? Or is the dead still dead? Since 'a if b else c' is not obviously dead, I will summarize my argument against it thusly: It is ambiguous to people because it is can be parsed (by people, who are not automatons) as either '(a if) b (else c)' or 'a (if b) (else c)'. The first parsing, seeing a as the conditional rather than the consequent, as in the the second, is at least as reasonable as the second given precedents in both other algorithm languages and natural languages (in English, at least, but I suspect others also). Ambiguity sometimes leads to discomfort. As important, it sometimes leads to bugs either in writing or reading. The impetus for this discussion was a real-life buggy example of the and/or construct. Its replacement should be something not similarly bug-prone, even if for a different reason. Terry J. Reedy From tjreedy at udel.edu Sat Sep 24 04:19:48 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 23 Sep 2005 22:19:48 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 References: Message-ID: "Jim Jewett" wrote in message news:fb6fbf5605092112145cfda669 at mail.gmail.com... A nice summary, to which I will add just a little. > For a conditional expression, I think the choices are really down > to the following, which was already way too much freedom last > (http://www.python.org/peps/pep-0308.html) time: > > (1) Should it be done at all? > + It would be useful, and avoid certain types of bugs. > - It would encourage bracketing instead of indentation > > PEP 308 decided "not yet, anyhow" > > (2) How many of the keywords (if ... then ... else ... elif) should be > dropped? > (if test then True else False) > + standard english > + standard programming idiom > - adds a "then" keyword > (if test True else False) > + better parallels the if: statement > - starts to run together I think the first form with 'then' better parallels the 'if :' statement because 'then' functions as a separator between test and True just as ':' or ':\n' functions as a separator between and . > (if test1 Val1 elif test2 Val2 elif test3 Val3) > + parallels the if: statement > - encourages overly long code > * but still better than nested parens > > PEP 308 wasn't entirely clear; the words said to keep "elif", but > the examples did not. There was some disagreement on which > of the other three keywords to represent explicitly. (Rather than > only by position.) > > (3) What order should the clauses appear? > (if test then True else False) > (if test1 then Val1 elif test2 Val2 elif test3 Val3 else Val4) > + Natural Order > - do we need "then"? > (True if normal else False) > (Val1 if test1 else Val2 if test2 else Val3 if test3 else Val4) > + Best order for normal/default conditionals > + Best order in general if we weren't used to left-right > processing I an far from convinced on either of these, perhaps because I am not sure about the actual claim. > - But we *do* expect left->right almost everywhere except > assignments - Ambiguous. When expressions a and b both lack boolean operators, as is common in Python given that all objects have a boolean value, a if b else c is mentally parseable as either (a if) b (else c) or a (if b) (else c). I initially though the proposal intended the first. -- Bug prone. The left->right expectation leads to the wrong parsing. Hence we may expect occasional bugs both in writing and reading this construct. > PEP 308 made it clear that "else" should be last. Putting the condition > before the "then" was not as clearcut. > > (4) What sort of punctuation should augment or replace the keywords? > > PEP 308 suggested avoiding punctuation, but it wasn't clearcut. Terry J. Reedy From ncoghlan at gmail.com Sat Sep 24 04:22:23 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 24 Sep 2005 12:22:23 +1000 Subject: [Python-Dev] Visibility scope for "for/while/if" statements In-Reply-To: <003501c5c045$1cc64740$6402a8c0@arkdesktop> References: <003501c5c045$1cc64740$6402a8c0@arkdesktop> Message-ID: <4334B85F.8060408@gmail.com> Andrew Koenig wrote: >>Interestingly enough, not all C++ compilers (Microsoft) hid variables >>created in for loops >>(http://www.devx.com/cplus/10MinuteSolution/28908/0/page/2). > > > That's because the C++ spec changed during standardization, when the > standards committee realized the original idea was a mistake. > > One of the convincing cases: > > if (x) > for (int i = 0; i != 10; ++i) { } > > Is I in scope after the if statement? If so, what value does it have if x > is false? If not, then apparently the subject of an "if" statement is a > scope...so why can't I write this? > > if (x) > int i = 42; > > and have i go out of scope? The difference is that C++ uses {} to delineate a new scope, whereas Python uses only def statements. The behaviour in C++ and C99 can indeed be quite handy, and aligns perfectly with the static type system of those languages. Importantly, creating a new scope is cheap - it's just a matter of moving the stack pointer around a bit (and maybe invoking some destructors that would have been invoked eventually anyway). Python, however, uses a dynamic name binding system and scopes are expensive because they require setting up all of the machinery to support nested visibility. That is, additional inline scoping would add significant *runtime* overhead. So a different design choice was made in Python, but that choice is still internally consistent. Using the above example: if x: for i in range(10): pass print i if x: i = 42 print i What happens on the 'print i' line in Python if 'x' is false? Easy: attempting to access 'i' will result in "UnboundLocalError" being raised at runtime, just as it would if the 'print i' line was *before* the if statement. That's because the name 'i' *is* in scope - it just hasn't been bound to anything yet. As Guido has said, this is really a discussion for c.l.p. Even before taking it there, however, I suggest reviewing the c.l.p discussion from January regarding the idea of 'statement local namespaces' [1] (and also checking the archives to see if the discussion has moved on since then, as I haven't been following c.l.p. for a good long while). Regards, Nick. [1] http://mail.python.org/pipermail/python-list/2005-January/259556.html -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From greg.ewing at canterbury.ac.nz Sat Sep 24 05:02:03 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 24 Sep 2005 15:02:03 +1200 Subject: [Python-Dev] Visibility scope for "for/while/if" statements In-Reply-To: <4334B85F.8060408@gmail.com> References: <003501c5c045$1cc64740$6402a8c0@arkdesktop> <4334B85F.8060408@gmail.com> Message-ID: <4334C1AB.7000100@canterbury.ac.nz> Nick Coghlan wrote: > Python, however, uses a dynamic name binding system and scopes are expensive > because they require setting up all of the machinery to support nested > visibility. Scopes within a function needn't be anywhere near as expensive as scopes for nested functions are. The compiler can just allocate all the locals that are going to be needed on entry to the function, and keep track of which ones are visible. The only runtime overhead would be unbinding locals as they go out of scope, if that were considered necessary or desirable. The main problem is Python's lack of declarations, which means that nested locals could only be used in places where it's clear that a new variable is being introduced, such as for loops, LCs, etc. Greg From tjreedy at udel.edu Sat Sep 24 05:21:14 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 23 Sep 2005 23:21:14 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <4330B7CE.50302@canterbury.ac.nz><433140B1.2070406@gmail.com> <43317EC1.908@ronadam.com> <433223A8.3010400@canterbury.ac.nz> Message-ID: "Greg Ewing" wrote in message news:433223A8.3010400 at canterbury.ac.nz... > Terry Reedy wrote: > >> Many people, perhaps most, including me, read >> >> exp1 if exp2 else exp3 # as >> cond if etrue else efalse # in direct analogy with >> cond ? etrue : efalse # from C > > I'd have thought only Forth programmers would be prone to that! Never wrote a single line, and may have not read one (in DDJ? Byte?) for 15-20 years. How do *you* read such C? Cond 'Qmark' ....? > It would surprise me greatly if it's really true that *most* > people would read it that way. OK. I withdraw 'possibly most' as not directly relevant. It would not take that many people to have enough people mistakenly write or read it that way for this construct to be somewhat as bug prone, in practice, as and/or. During the c.l.p debate, someone counted about 100 correct uses of 'a and b or c' in the standard library. But one real misuse edged Guido toward replacing it. So I think the replacement should be as clear as reasonably possible and clearly an improvement. > Especially given that, in real > code, you're not going to be looking at abstract names like > exp1, exp2, exp3, but (hopefully) something a lot more > meaningful. Because every object in *Python* (ignoring buggy or unusual __nonzero__ methods) has a boolean value, any expression can be a conditional expression and hence conditional expressions often lack a boolean operator to flag the expression as the likely conditional. > Can you honestly say that you would read Given that setup, how could I possible misread :-? > return red_value if color == 'red' else blue_value > as > if red_value: > return color == 'red' > else: > return blue_value > ? I can imagine that if I were to read the return naively, I might wonder whether the expressions were reversed. And certainly, not all code I have read has had such obvious names. Now, can you honestly say that you would (naively) read return foo if bar else baz and be certain you knew what it meant? Terry J. Reedy From ncoghlan at gmail.com Sat Sep 24 06:59:47 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 24 Sep 2005 14:59:47 +1000 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <4330B7CE.50302@canterbury.ac.nz><433140B1.2070406@gmail.com> <43317EC1.908@ronadam.com> <433223A8.3010400@canterbury.ac.nz> Message-ID: <4334DD43.1010207@gmail.com> Terry Reedy wrote: > During the c.l.p debate, someone counted about 100 correct uses of 'a and b > or c' in the standard library. But one real misuse edged Guido toward > replacing it. So I think the replacement should be as clear as reasonably > possible and clearly an improvement. But I think there's a difference in kind here - to *fix* Raymond's example required a fundamental change to the structure of the line, none of which looked as clean as the original. There is no way to get the and/or construct to gracefully handle the case where the desired result in the 'true' case might itself be false: either you change to using an if statement, or you use a workaround like the ugly singleton-list approach. That is, the following is fundamentally broken for pure imaginary numbers: return isinstance(z, ComplexType) and z.real or z Fixing it requires changing to either: return (isinstance(z, ComplexType) and [z.real] or [z])[0] or: if isinstance(z, ComplexType) return z.real else: return z This is not the case with an in-fix notation for conditional expressions - you can fix a broken line simply by moving the relevant expressions to the correct locations. return isinstance(z, ComplexType) if z.real else z # Broken! return z.real if isinstance(z, ComplexType) else z # Correct! I see this as being in the same category of error as writing "return foo ** baz" when you really should have written "return baz ** foo" (i.e., not the language's problem). (Random aside: it would be nice if "foo ** bar % baz" automatically invoked the three argument form of 'pow') > Now, can you honestly say that you would (naively) read > > return foo if bar else baz > > and be certain you knew what it meant? Yes. I'd expect it to read like English - "Return foo if bar is true, otherwise return baz". Whether that was was what code was *supposed* to be doing, I couldn't assess without additional context (but that is true regardless of the syntax). With the prefix notation used for C's conditional operator, I simply don't read it in the order it's written - I read "return bar ? foo : baz" as "if bar is true then return foo, otherwise return baz". That's possible with C because it uses punctuation - using an English keyword instead makes it significantly harder for me to consider interpreting the construct that way (In fact, the only way I can get my brain to accept it is by mentally replacing the "if" with "?" and the "else" with ":"). Does it help if you think of "if else" as a parameterised infix operation for choosing between the expressions on either side? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From murman at gmail.com Sat Sep 24 07:40:39 2005 From: murman at gmail.com (Michael Urman) Date: Sat, 24 Sep 2005 00:40:39 -0500 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <4334DD43.1010207@gmail.com> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <4330B7CE.50302@canterbury.ac.nz> <433140B1.2070406@gmail.com> <43317EC1.908@ronadam.com> <433223A8.3010400@canterbury.ac.nz> <4334DD43.1010207@gmail.com> Message-ID: On 9/23/05, Nick Coghlan wrote: > But I think there's a difference in kind here - to *fix* Raymond's example > required a fundamental change to the structure of the line, none of which > looked as clean as the original. There is no way to get the and/or construct > to gracefully handle the case where the desired result in the 'true' case > might itself be false: either you change to using an if statement, or you use > a workaround like the ugly singleton-list approach. > > That is, the following is fundamentally broken for pure imaginary numbers: > return isinstance(z, ComplexType) and z.real or z It's hard to say whether this fixes Raymond's example when the goal wasn't clearly stated, but I find a non ternary option lambda z: complex(z).real more readable than any of the variants proposed so far. The obvious downsides are that it can raise a ValueError, and turns integers into floats. But if the input list is all numeric, it has clean results. -- Michael Urman From tjreedy at udel.edu Sat Sep 24 07:57:47 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 24 Sep 2005 01:57:47 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <4330B7CE.50302@canterbury.ac.nz><433140B1.2070406@gmail.com> <43317EC1.908@ronadam.com> <433223A8.3010400@canterbury.ac.nz> <4334DD43.1010207@gmail.com> Message-ID: "Nick Coghlan" wrote in message news:4334DD43.1010207 at gmail.com... I am reminded of how some people seem to react to fingernails on a blackboard, while the screech is just another noise to me, except that I am in the opposite position with respect to a if b else c. > Does it help if you think of "if else" as a parameterised infix > operation > for choosing between the expressions on either side? Hmm. If I parse it as a (if b else) c and sort of ignore the words and think of it as so that the whole thing is a switch pointing either left or right? A bit strange, but perhaps it does. Terry J. Reedy From greg.ewing at canterbury.ac.nz Sat Sep 24 11:00:24 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 24 Sep 2005 21:00:24 +1200 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <4330B7CE.50302@canterbury.ac.nz> <433140B1.2070406@gmail.com> <43317EC1.908@ronadam.com> <433223A8.3010400@canterbury.ac.nz> Message-ID: <433515A8.3010906@canterbury.ac.nz> Terry Reedy wrote: > Now, can you honestly say that you would (naively) read > > return foo if bar else baz > > and be certain you knew what it meant? I can honestly say that, given I knew I was reading Python code, it would never have entered by head to read "foo if" as meaning that "foo" was the condition. Bizarrely backwards too much it would be. Greg From p.f.moore at gmail.com Sat Sep 24 12:16:51 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 24 Sep 2005 11:16:51 +0100 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <4330B7CE.50302@canterbury.ac.nz> <433140B1.2070406@gmail.com> <43317EC1.908@ronadam.com> <433223A8.3010400@canterbury.ac.nz> Message-ID: <79990c6b05092403167210a9d2@mail.gmail.com> On 9/24/05, Terry Reedy wrote: > Now, can you honestly say that you would (naively) read > > return foo if bar else baz > > and be certain you knew what it meant? FWIW, yes, I can honestly say that I would be certain. Yes, you may be able to *parse* it as (foo if) bar (esle baz) as well as foo (if bar) (else baz), but I know of no conceivable *semantic* meaning for "foo if" - English has no postfix if, and nor do any programming languages, except entirely postfix ones such as Forth. Paul. From a.edmondson at eris.qinetiq.com Fri Sep 23 09:35:06 2005 From: a.edmondson at eris.qinetiq.com (Andrew Edmondson) Date: Fri, 23 Sep 2005 08:35:06 +0100 Subject: [Python-Dev] Possible bug in urllib.urljoin Message-ID: <200509230835.07053.a.edmondson@eris.qinetiq.com> Dear all, We've found a problem using urllib.urljoin when upgrading from python 2.3 to 2.4. It no longer joins a particular corner case of URLs correctly (we think!). The code appears to follow the algorithm (from http://www.ietf.org/rfc/rfc1808.txt) for resolving urls almost exacty... I believe the problem occurs when reaching "step 5" (approx line 160) which will happen if the embedded url has no scheme, netloc or path (and is nonempty). Following the algorithm the resulting url should now be returned using the base urls scheme,netloc and path but the embedded urls params / query (if present else set to base ones) which follows in 2.3: if not path: if not params: params = bparams if not query: query = bquery return urlunparse((scheme, netloc, bpath, params, query, fragment)) However in 2.4, even if the embedded urls path is empty, unless the params and query segments are empty too, flow passes to step 6. if not (path or params or query): return urlunparse((scheme, netloc, bpath, bparams, bquery, fragment)) and thus the last segment of the base path will be removed in order to append the embedded url's path, but the path is empty! and so the resulting path is returned incorrectly. Can you tell me if this was a deliberate decision to move from following the algorithm? If so then we'll work around it. -- ############################################################################## Andrew Edmondson PGP Key: http://search.keyserver.net:11371/pks/lookup?op=get&search=0xCEE814DC PGP Fingerprint: 7B32 4D1E AC4F 29E2 9EAA 9550 1A3D BBA4 CEE8 14DC From falcon at intercable.ru Sun Sep 25 09:53:13 2005 From: falcon at intercable.ru (Sokolov Yura) Date: Sun, 25 Sep 2005 11:53:13 +0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 Message-ID: <43365769.4070405@intercable.ru> Sorry for looking in every hole. Just a suggestion. A= condition and first or second problem is in case when first in (None,0,[],""). May be invent new operator 'take'. take - returns right operator when left evals to True and stops computing condidtional expression. Then we could write: A = condition take first or second. A = x==y take w or s A = z is not None and q!=12 take [] or allowable(z,q) take [(z,q)] or "Impossible" Ok, it might looks ugly. But may be not. ------------------------------------- Excuse my english. From reinhold-birkenfeld-nospam at wolke7.net Sun Sep 25 19:11:30 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Sun, 25 Sep 2005 19:11:30 +0200 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <43365769.4070405@intercable.ru> References: <43365769.4070405@intercable.ru> Message-ID: Sokolov Yura wrote: > Sorry for looking in every hole. > Just a suggestion. > > A= condition and first or second > problem is in case when first in (None,0,[],""). > May be invent new operator 'take'. > take - returns right operator when left evals to True and stops > computing condidtional expression. > Then we could write: > > A = condition take first or second. > A = x==y take w or s > A = z is not None and q!=12 take [] or allowable(z,q) take [(z,q)] or > "Impossible" > > Ok, it might looks ugly. But may be not. One of the advantages of (if x then y else z) is that it doesn't require the introduction of a new keyword (I think the "then" could be special- cased like "as" in the import statement). Reinhold -- Mail address is perfectly valid! From jjl at pobox.com Sun Sep 25 22:34:39 2005 From: jjl at pobox.com (John J Lee) Date: Sun, 25 Sep 2005 20:34:39 +0000 (UTC) Subject: [Python-Dev] Possible bug in urllib.urljoin In-Reply-To: <200509230835.07053.a.edmondson@eris.qinetiq.com> References: <200509230835.07053.a.edmondson@eris.qinetiq.com> Message-ID: On Fri, 23 Sep 2005, Andrew Edmondson wrote: > We've found a problem using urllib.urljoin when upgrading > from python 2.3 to 2.4. It no longer joins a particular > corner case of URLs correctly (we think!). > > The code appears to follow the algorithm (from > http://www.ietf.org/rfc/rfc1808.txt) for resolving urls > almost exacty... [...] > Can you tell me if this was a deliberate decision to move > from following the algorithm? If so then we'll work around it. I don't know if it was done right, but this came in at revision 1.41 of urlparse.py -- the commit comment is actually in 1.42: | Make urlparse RFC 2396 compliant. | Closes bug #450225 (thanks Michael Stone). So I guess the answer to your question is "yes". http://python.org/sf/450225 http://www.ietf.org/rfc/rfc2396.txt John From gmccaughan at synaptics-uk.com Mon Sep 26 12:14:09 2005 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Mon, 26 Sep 2005 11:14:09 +0100 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <433223A8.3010400@canterbury.ac.nz> Message-ID: <200509261114.10016.gmccaughan@synaptics-uk.com> On Saturday 2005-09-24 04:21, Terry Reedy wrote: > Never wrote a single line, and may have not read one (in DDJ? Byte?) for > 15-20 years. How do *you* read such C? Cond 'Qmark' ....? I seldom have to read C code aloud, and the ?: operator is rare-ish; but I would suggest reading a?b:c as "a chooses b else c" or (shaving a syllable) "a gives b else c" or something like that. Or, ploddingly but unambiguously, "a query b colon c". > Now, can you honestly say that you would (naively) read > > return foo if bar else baz > > and be certain you knew what it meant? I can't imagine *actually* reading it as if "foo" were the condition. But I can imagine reading it, to begin with, as if it were a Perlish conditionalized statement: "(return foo) if bar else ... aw heck, that can't be right". Which is probably benign but slows down comprehension. -- g From paolo_veronelli at libero.it Mon Sep 26 14:22:52 2005 From: paolo_veronelli at libero.it (Paolino) Date: Mon, 26 Sep 2005 14:22:52 +0200 Subject: [Python-Dev] Trasvesal thought on syntax features In-Reply-To: <200509261114.10016.gmccaughan@synaptics-uk.com> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <433223A8.3010400@canterbury.ac.nz> <200509261114.10016.gmccaughan@synaptics-uk.com> Message-ID: <4337E81C.4000101@libero.it> I don't remember where I read (here around probably) that having a "client pluggable" syntax was not good for python because then ,and more blah blah.I believed it was a rule of the thumb.. But ,when a client programmer like me writes code he tries to give a clean, unambigous interface/syntax for a possible under user.Having a fixed set of keywords and not overridable syntactic sugar is just a limit.Once a sound set of unittest is written with documentation, every useful library would add its keywords. What I start thinking reading here is that if you let redefine, instead: 1) print statement :I will redefine it like now , no matter if python 3.0 is not selling it any more in the language 2) on A take B or C: How ever and whatever one would like to write, this expression could be definable, ending all the pathetic blah blah (hope not to offend) Thanks for jour job anyway. Paolino From anthony at interlink.com.au Mon Sep 26 14:55:13 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon, 26 Sep 2005 22:55:13 +1000 Subject: [Python-Dev] IMPORTANT: release24-maint branch is FROZEN from 2005-09-28 00:00 UTC for 2.4.2 Message-ID: <200509262255.15927.anthony@interlink.com.au> So here we go - the release24-maint branch will be frozen from midnight UTC on Wednesday the 28th (about 36 hours from now). This is to cut 2.4.2 (final). If you have any pending patches you want to see landed on the branch, mail me about them - and do it soon! There's been a bunch of fixes since 2.4.2c1, but nothing that I judge making it worthwhile doing a 2.4.2c2. I'm open to other opinions here, of course. Anthony From aahz at pythoncraft.com Mon Sep 26 14:57:59 2005 From: aahz at pythoncraft.com (Aahz) Date: Mon, 26 Sep 2005 05:57:59 -0700 Subject: [Python-Dev] IMPORTANT: release24-maint branch is FROZEN from 2005-09-28 00:00 UTC for 2.4.2 In-Reply-To: <200509262255.15927.anthony@interlink.com.au> References: <200509262255.15927.anthony@interlink.com.au> Message-ID: <20050926125758.GA12834@panix.com> On Mon, Sep 26, 2005, Anthony Baxter wrote: > > So here we go - the release24-maint branch will be frozen from > midnight UTC on Wednesday the 28th (about 36 hours from now). This is > to cut 2.4.2 (final). > > If you have any pending patches you want to see landed on the branch, > mail me about them - and do it soon! > > There's been a bunch of fixes since 2.4.2c1, but nothing that I judge > making it worthwhile doing a 2.4.2c2. I'm open to other opinions > here, of course. After my experience doing weekly code rollouts for the past year, I'm somewhere between +0 and +1 on doing another release candidate. OTOH, doing a brownbag 2.4.3 won't kill us, and few people will use a release candidate instead of waiting. Still your call. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything. From nas at arctrix.com Tue Sep 27 00:24:17 2005 From: nas at arctrix.com (Neil Schemenauer) Date: Mon, 26 Sep 2005 22:24:17 +0000 (UTC) Subject: [Python-Dev] PEP 350: Codetags References: <20050926223521.GE10940@kitchen.client.attbi.com> Message-ID: Micah Elliott wrote: > ``FIXME (XXX, DEBUG, BROKEN, REFACTOR, REFACT, RFCTR, OOPS, SMELL, NEEDSWORK, INSPECT)`` > *Fix me*: Areas of problematic or ugly code needing refactoring or > cleanup. I think the standard should not have codetags that are synonyms. This is Python and there should be only one way to do it. One problem with synonyms is that they makes it harder to search using tools like grep. Neil From gjc at inescporto.pt Tue Sep 27 15:11:13 2005 From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro) Date: Tue, 27 Sep 2005 14:11:13 +0100 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <43365769.4070405@intercable.ru> Message-ID: <1127826673.15824.6.camel@localhost> On Sun, 2005-09-25 at 19:11 +0200, Reinhold Birkenfeld wrote: > Sokolov Yura wrote: > > Sorry for looking in every hole. > > Just a suggestion. > > > > A= condition and first or second > > problem is in case when first in (None,0,[],""). > > May be invent new operator 'take'. > > take - returns right operator when left evals to True and stops > > computing condidtional expression. > > Then we could write: > > > > A = condition take first or second. > > A = x==y take w or s > > A = z is not None and q!=12 take [] or allowable(z,q) take [(z,q)] or > > "Impossible" > > > > Ok, it might looks ugly. But may be not. > > One of the advantages of (if x then y else z) is that it doesn't require > the introduction of a new keyword (I think the "then" could be special- > cased like "as" in the import statement). This wouldn't look so bad either: (if x: y else: z) More realistic example: def greet(person=None): print "Hello %s" % (if person is None: "World" else: person) Not as compact as C's ?:, but more readable and intuitive. It's just like an if-else construct, but on a single line and () around to make it look like an expression instead of a statement. > > Reinhold > -- Gustavo J. A. M. Carneiro The universe is always one step beyond logic. From falcon at intercable.ru Mon Sep 26 08:31:53 2005 From: falcon at intercable.ru (Sokolov Yura) Date: Mon, 26 Sep 2005 10:31:53 +0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 Message-ID: <433795D9.9040900@intercable.ru> New keyword is so expensive? And why special case for 'then' is better than special case for 'take'? From mde at micah.elliott.name Tue Sep 27 00:35:21 2005 From: mde at micah.elliott.name (Micah Elliott) Date: Mon, 26 Sep 2005 15:35:21 -0700 Subject: [Python-Dev] PEP 350: Codetags Message-ID: <20050926223521.GE10940@kitchen.client.attbi.com> Please read/comment/vote. This circulated as a pre-PEP proposal submitted to c.l.py on August 10, but has changed quite a bit since then. I'm reposting this since it is now "Open (under consideration)" at . Thanks! -- Micah Elliott PEP: 350 Title: Codetags Version: $Revision: 1.2 $ Last-Modified: $Date: 2005/09/26 19:56:53 $ Author: Micah Elliott Status: Draft Type: Informational Content-Type: text/x-rst Created: 27-Jun-2005 Post-History: 10-Aug-2005, 26-Sep-2005 Abstract ======== This informational PEP aims to provide guidelines for consistent use of *codetags*, which would enable the construction of standard utilities to take advantage of the codetag information, as well as making Python code more uniform across projects. Codetags also represent a very lightweight programming micro-paradigm and become useful for project management, documentation, change tracking, and project health monitoring. This is submitted as a PEP because its ideas are thought to be Pythonic, although the concepts are not unique to Python programming. Herein are the definition of codetags, the philosophy behind them, a motivation for standardized conventions, some examples, a specification, a toolset description, and possible objections to the Codetag project/paradigm. This PEP is also living as a wiki_ for people to add comments. What Are Codetags? ================== Programmers widely use ad-hoc code comment markup conventions to serve as reminders of sections of code that need closer inspection or review. Examples of markup include ``FIXME``, ``TODO``, ``XXX``, ``BUG``, but there many more in wide use in existing software. Such markup will henceforth be referred to as *codetags*. These codetags may show up in application code, unit tests, scripts, general documentation, or wherever suitable. Codetags have been under discussion and in use (hundreds of codetags in the Python 2.4 sources) in many places (e.g., c2_) for many years. See References_ for further historic and current information. Philosophy ========== If you subscribe to most of these values, then codetags will likely be useful for you. 1. As much information as possible should be contained **inside the source code** (application code or unit tests). This along with use of codetags impedes duplication. Most documentation can be generated from that source code; e.g., by using help2man, man2html, docutils, epydoc/pydoc, ctdoc, etc. 2. Information should be almost **never duplicated** -- it should be recorded in a single original format and all other locations should be automatically generated from the original, or simply be referenced. This is famously known as the Single Point Of Truth (SPOT) or Don't Repeat Yourself (DRY) rule. 3. Documentation that gets into customers' hands should be **auto-generated** from single sources into all other output formats. People want documentation in many forms. It is thus important to have a documentation system that can generate all of these. 4. The **developers are the documentation team**. They write the code and should know the code the best. There should not be a dedicated, disjoint documentation team for any non-huge project. 5. **Plain text** (with non-invasive markup) is the best format for writing anything. All other formats are to be generated from the plain text. Codetag design was influenced by the following goals: A. Comments should be short whenever possible. B. Codetag fields should be optional and of minimal length. Default values and custom fields can be set by individual code shops. C. Codetags should be minimalistic. The quicker it is to jot something down, the more likely it is to get jotted. D. The most common use of codetags will only have zero to two fields specified, and these should be the easiest to type and read. Motivation ========== * **Various productivity tools can be built around codetags.** See Tools_. * **Encourages consistency.** Historically, a subset of these codetags has been used informally in the majority of source code in existence, whether in Python or in other languages. Tags have been used in an inconsistent manner with different spellings, semantics, format, and placement. For example, some programmers might include datestamps and/or user identifiers, limit to a single line or not, spell the codetag differently than others, etc. * **Encourages adherence to SPOT/DRY principle.** E.g., generating a roadmap dynamically from codetags instead of keeping TODOs in sync with separate roadmap document. * **Easy to remember.** All codetags must be concise, intuitive, and semantically non-overlapping with others. The format must also be simple. * **Use not required/imposed.** If you don't use codetags already, there's no obligation to start, and no risk of affecting code (but see Objections_). A small subset can be adopted and the Tools_ will still be useful (a few codetags have probably already been adopted on an ad-hoc basis anyway). Also it is very easy to identify and remove (and possibly record) a codetag that is no longer deemed useful. * **Gives a global view of code.** Tools can be used to generate documentation and reports. * **A logical location for capturing CRCs/Stories/Requirements.** The XP community often does not electronically capture Stories, but codetags seem like a good place to locate them. * **Extremely lightweight process.** Creating tickets in a tracking system for every thought degrades development velocity. Even if a ticketing system is employed, codetags are useful for simply containing links to those tickets. Examples ======== This shows a simple codetag as commonly found in sources everywhere (with the addition of a trailing ``<>``):: # FIXME: Seems like this loop should be finite. <> while True: ... The following contrived example demonstrates a typical use of codetags. It uses some of the available fields to specify the assignees (a pair of programmers with initials *MDE* and *CLE*), the Date of expected completion (*Week 14*), and the Priority of the item (*2*):: # FIXME: Seems like this loop should be finite. while True: ... This codetag shows a bug with fields describing author, discovery (origination) date, due date, and priority:: # BUG: Crashes if run on Sundays. # if day == 'Sunday': ... Here is a demonstration of how not to use codetags. This has many problems: 1) Codetags cannot share a line with code; 2) Missing colon after mnemonic; 3) A codetag referring to codetags is usually useless, and worse, it is not completable; 4) No need to have a bunch of fields for a trivial codetag; 5) Fields with unknown values (``t:XXX``) should not be used:: i = i + 1 # TODO Add some more codetags. # Specification ============= This describes the format: syntax, mnemonic names, fields, and semantics, and also the separate DONE File. General Syntax -------------- Each codetag should be inside a comment, and can be any number of lines. It should not share a line with code. It should match the indentation of surrounding code. The end of the codetag is marked by a pair of angle brackets ``<>`` containing optional fields, which must not be split onto multiple lines. It is preferred to have a codetag in ``#`` comments instead of string comments. There can be multiple fields per codetag, all of which are optional. .. NOTE: It may be reasonable to allow fields to fit on multiple lines, but it complicates parsing and defeats minimalism if you use this many fields. In short, a codetag consists of a mnemonic, a colon, commentary text, an opening angle bracket, an optional list of fields, and a closing angle bracket. E.g., :: # MNEMONIC: Some (maybe multi-line) commentary. Mnemonics --------- The codetags of interest are listed below, using the following format: | ``recommended mnemonic (& synonym list)`` | *canonical name*: semantics ``TODO (MILESTONE, MLSTN, DONE, YAGNI, TBD, TOBEDONE)`` *To do*: Informal tasks/features that are pending completion. ``FIXME (XXX, DEBUG, BROKEN, REFACTOR, REFACT, RFCTR, OOPS, SMELL, NEEDSWORK, INSPECT)`` *Fix me*: Areas of problematic or ugly code needing refactoring or cleanup. ``BUG (BUGFIX)`` *Bugs*: Reported defects tracked in bug database. ``NOBUG (NOFIX, WONTFIX, DONTFIX, NEVERFIX, UNFIXABLE, CANTFIX)`` *Will Not Be Fixed*: Problems that are well-known but will never be addressed due to design problems or domain limitations. ``REQ (REQUIREMENT, STORY)`` *Requirements*: Satisfactions of specific, formal requirements. ``RFE (FEETCH, NYI, FR, FTRQ, FTR)`` *Requests For Enhancement*: Roadmap items not yet implemented. ``IDEA`` *Ideas*: Possible RFE candidates, but less formal than RFE. ``??? (QUESTION, QUEST, QSTN, WTF)`` *Questions*: Misunderstood details. ``!!! (ALERT)`` *Alerts*: In need of immediate attention. ``HACK (CLEVER, MAGIC)`` *Hacks*: Temporary code to force inflexible functionality, or simply a test change, or workaround a known problem. ``PORT (PORTABILITY, WKRD)`` *Portability*: Workarounds specific to OS, Python version, etc. ``CAVEAT (CAV, CAVT, WARNING, CAUTION)`` *Caveats*: Implementation details/gotchas that stand out as non-intuitive. ``NOTE (HELP)`` *Notes*: Sections where a code reviewer found something that needs discussion or further investigation. ``FAQ`` *Frequently Asked Questions*: Interesting areas that require external explanation. ``GLOSS (GLOSSARY)`` *Glossary*: Definitions for project glossary. ``SEE (REF, REFERENCE)`` *See*: Pointers to other code, web link, etc. ``TODOC (DOCDO, DODOC, NEEDSDOC, EXPLAIN, DOCUMENT)`` *Needs Documentation*: Areas of code that still need to be documented. ``CRED (CREDIT, THANKS)`` *Credits*: Accreditations for external provision of enlightenment. ``STAT (STATUS)`` *Status*: File-level statistical indicator of maturity of this file. ``RVD (REVIEWED, REVIEW)`` *Reviewed*: File-level indicator that review was conducted. File-level codetags might be better suited as properties in the revision control system, but might still be appropriately specified in a codetag. Some of these are temporary (e.g., ``FIXME``) while others are persistent (e.g., ``REQ``). A mnemonic was chosen over a synonym using three criteria: descriptiveness, length (shorter is better), commonly used. Choosing between ``FIXME`` and ``XXX`` is difficult. ``XXX`` seems to be more common, but much less descriptive. Furthermore, ``XXX`` is a useful placeholder in a piece of code having a value that is unknown. Thus ``FIXME`` is the preferred spelling. `Sun says`__ that ``XXX`` and ``FIXME`` are slightly different, giving ``XXX`` higher severity. However, with decades of chaos on this topic, and too many millions of developers who won't be influenced by Sun, it is easy to rightly call them synonyms. __ http://java.sun.com/docs/codeconv/html/CodeConventions.doc9.html#395 ``DONE`` is always a completed ``TODO`` item, but this should probably be indicated through the revision control system and/or a completion recording mechanism (see `DONE File`_). It may be a useful metric to count ``NOTE`` tags: a high count may indicate a design (or other) problem. But of course the majority of codetags indicate areas of code needing some attention. An ``FAQ`` is probably more appropriately documented in a wiki where users can more easily view and contribute. Fields ------ All fields are optional. The proposed standard fields are described in this section. Note that upper case field characters are intended to be replaced. The *Originator/Assignee* and *Origination Date/Week* fields are the most common and don't usually require a prefix. .. NOTE: the colon after the prefix is a new addition that became necessary when it was pointed out that a "codename" field (with no digits) such as "cTiger" would be indistinguishable from a username. .. NOTE: This section started out with just assignee and due week. It has grown into a lot of fields by request. It is still probably best to use a tracking system for any items that deserve it, and not duplicate everything in a codetag (field). This lengthy list of fields is liable to scare people (the intended minimalists) away from adopting codetags, but keep in mind that these only exist to support programmers who either 1) like to keep ``BUG`` or ``RFE`` codetags in a complete form, or 2) are using codetags as their complete and only tracking system. In other words, many of these fields will be used very rarely. They are gathered largely from industry-wide conventions, and example sources include `GCC Bugzilla`__ and `Python's SourceForge`__ tracking systems. .. ???: Maybe codetags inside packages (__init__.py files) could have special global significance. __ http://gcc.gnu.org/bugzilla/ __ http://sourceforge.net/tracker/?group_id=5470 ``AAA[,BBB]...`` List of *Originator* or *Assignee* initials (the context determines which unless both should exist). It is also okay to use usernames such as ``MicahE`` instead of initials. Initials (in upper case) are the preferred form. ``a:AAA[,BBB]...`` List of *Assignee* initials. This is necessary only in (rare) cases where a codetag has both an assignee and an originator, and they are different. Otherwise the ``a:`` prefix is omitted, and context determines the intent. E.g., ``FIXME`` usually has an *Assignee*, and ``NOTE`` usually has an *Originator*, but if a ``FIXME`` was originated (and initialed) by a reviewer, then the assignee's initials would need a ``a:`` prefix. ``YYYY[-MM[-DD]]`` or ``WW[.D]w`` The *Origination Date* indicating when the comment was added, in `ISO 8601`_ format (digits and hyphens only). Or *Origination Week*, an alternative form for specifying an *Origination Date*. A day of the week can be optionally specified. The ``w`` suffix is necessary for distinguishing from a date. ``d:YYYY[-MM[-DD]]`` or ``d:WW[.D]w`` *Due Date (d)* target completion (estimate). Or *Due Week (d)*, an alternative to specifying a *Due Date*. ``p:N`` *Priority (p)* level. Range (N) is from 0..3 with 3 being the highest. 0..3 are analogous to low, medium, high, and showstopper/critical. The *Severity* field could be factored into this single number, and doing so is recommended since having both is subject to varying interpretation. The range and order should be customizable. The existence of this field is important for any tool that itemizes codetags. Thus a (customizable) default value should be supported. ``t:NNNN`` *Tracker (t)* number corresponding to associated Ticket ID in separate tracking system. The following fields are also available but expected to be less common. ``c:AAAA`` *Category (c)* indicating some specific area affected by this item. ``s:AAAA`` *Status (s)* indicating state of item. Examples are "unexplored", "understood", "inprogress", "fixed", "done", "closed". Note that when an item is completed it is probably better to remove the codetag and record it in a `DONE File`_. ``i:N`` Development cycle *Iteration (i)*. Useful for grouping codetags into completion target groups. ``r:N`` Development cycle *Release (r)*. Useful for grouping codetags into completion target groups. .. NOTE: SourceForge does not recognize a severity and I think that *Priority* (along with separate RFE codetags) should encompass and obviate *Severity*. .. NOTE: The tools will need an ability to sort codetags in order of targeted completion. I feel that *Priority* should be a unique, lone indicator of that addressability order. Other categories such as *Severity*, *Customer Importance*, etc. are related to business logic and should not be recognized by the codetag tools. If some groups want to have such logic, then it is best factored (externally) into a single value (priority) that can determine an ordering of actionable items. To summarize, the non-prefixed fields are initials and origination date, and the prefixed fields are: assignee (a), due (d), priority (p),tracker (t), category (c), status (s), iteration (i), and release (r). It should be possible for groups to define or add their own fields, and these should have upper case prefixes to distinguish them from the standard set. Examples of custom fields are *Operating System (O)*, *Severity (S)*, *Affected Version (A)*, *Customer (C)*, etc. DONE File --------- Some codetags have an ability to be *completed* (e.g., ``FIXME``, ``TODO``, ``BUG``). It is often important to retain completed items by recording them with a completion date stamp. Such completed items are best stored in a single location, global to a project (or maybe a package). The proposed format is most easily described by an example, say ``~/src/fooproj/DONE``:: # TODO: Recurse into subdirs only on blue # moons. [2005-09-26 Oops, I underestimated this one a bit. Should have used Warsaw's First Law!] # FIXME: ... ... You can see that the codetag is copied verbatim from the original source file. The date stamp is then entered on the following line with an optional post-mortem commentary. The entry is terminated by a blank line (``\n\n``). It may sound burdensome to have to delete codetag lines every time one gets completed. But in practice it is quite easy to setup a Vim or Emacs mapping to auto-record a codetag deletion in this format (sans the commentary). Tools ===== Currently, programmers (and sometimes analysts) typically use *grep* to generate a list of items corresponding to a single codetag. However, various hypothetical productivity tools could take advantage of a consistent codetag format. Some example tools follow. .. NOTE: Codetag tools are mostly unimplemented (but I'm getting started!) Document Generator Possible docs: glossary, roadmap, manpages Codetag History Track (with revision control system interface) when a ``BUG`` tag (or any codetag) originated/resolved in a code section Code Statistics A project Health-O-Meter Codetag Lint Notify of invalid use of codetags, and aid in porting to codetags Story Manager/Browser An electronic means to replace XP notecards. In MVC terms, the codetag is the Model, and the Story Manager could be a graphical Viewer/Controller to do visual rearrangement, prioritization, and assignment, milestone management. Any Text Editor Used for changing, removing, adding, rearranging, recording codetags. There are some tools already in existence that take advantage of a smaller set of pseudo-codetags (see References_). There is also an example codetags implementation under way, known as the `Codetag Project`__. __ http://tracos.org/codetag Objections ========== :Objection: Extreme Programming argues that such codetags should not ever exist in code since the code is the documentation. :Defense: Maybe you should put the codetags in the unit test files instead. Besides, it's tough to generate documentation from uncommented source code. ---- :Objection: Too much existing code has not followed proposed guidelines. :Defense: [Simple] utilities (*ctlint*) could convert existing codes. ---- :Objection: Causes duplication with tracking system. :Defense: Not really, unless fields are abused. If an item exists in the tracker, a simple ticket number in the codetag tracker field is sufficient. Maybe a duplicated title would be acceptable. Furthermore, it's too burdensome to have a ticket filed for every item that pops into a developer's mind on-the-go. Additionally, the tracking system could possibly be obviated for simple or small projects that can reasonably fit the relevant data into a codetag. ---- :Objection: Codetags are ugly and clutter code. :Defense: That is a good point. But I'd still rather have such info in a single place (the source code) than various other documents, likely getting duplicated or forgotten about. The completed codetags can be sent off to the `DONE File`_, or to the bit bucket. ---- :Objection: Codetags (and allcomments) get out of date. :Defense: Not so much if other sources (externally visible documentation) depend on their being accurate. ---- :Objection: Codetags tend to only rarely have estimated completion dates of any sort. OK, the fields are optional, but you want to suggest fields that actually will be widely used. :Defense: If an item is inestimable don't bother with specifying a date field. Using tools to display items with order and/or color by due date and/or priority, it is easier to make estimates. Having your roadmap be a dynamic reflection of your codetags makes you much more likely to keep the codetags accurate. ---- :Objection: Named variables for the field parameters in the ``<>`` should be used instead of cryptic one-character prefixes. I.e., should rather be . :Defense: It is just too much typing/verbosity to spell out fields. I argue that ``p:3 i:2`` is as readable as ``priority=3, iteration=2`` and is much more likely to by typed and remembered (see bullet C in Philosophy_). In this case practicality beats purity. There are not many fields to keep track of so one letter prefixes are suitable. ---- :Objection: Synonyms should be deprecated since it is better to have a single way to spell something. :Defense: Many programmers prefer short mnemonic names, especially in comments. This is why short mnemonics were chosen as the primary names. However, others feel that an explicit spelling is less confusing and less prone to error. There will always be two camps on this subject. Thus synonyms (and complete, full spellings) should remain supported. ---- :Objection: It is cruel to use [for mnemonics] opaque acronyms and abbreviations which drop vowels; it's hard to figure these things out. On that basis I hate: MLSTN RFCTR RFE FEETCH, NYI, FR, FTRQ, FTR WKRD RVDBY :Defense: Mnemonics are preferred since they are pretty easy to remember and take up less space. If programmers didn't like dropping vowels we would be able to fit very little code on a line. The space is important for those who write comments that often fit on a single line. But when using a canon everywhere it is much less likely to get something to fit on a line. ---- :Objection: It takes too long to type the fields. :Defense: Then don't use (most or any of) them, especially if you're the only programmer. Terminating a codetag with ``<>`` is a small chore, and in doing so you enable the use of the proposed tools. Editor auto-completion of codetags is also useful: You can program your editor to stamp a template (e.g. ``# FIXME . ``) with just a keystroke or two. ---- :Objection: *WorkWeek* is an obscure and uncommon time unit. :Defense: That's true but it is a highly suitable unit of granularity for estimation/targeting purposes, and it is very compact. The `ISO 8601`_ is widely understood but allows you to only specify either a specific day (restrictive) or month (broad). ---- :Objection: I aesthetically dislike for the comment to be terminated with <> in the empty field case. :Defense: It is necessary to have a terminator since codetags may be followed by non-codetag comments. Or codetags could be limited to a single line, but that's prohibitive. I can't think of any single-character terminator that is appropriate and significantly better than <>. Maybe ``@`` could be a terminator, but then most codetags will have an unnecessary @. ---- :Objection: I can't use codetags when writing HTML, or less specifically, XML. Maybe ``@fields@`` would be a better than `` `` as the delimiters. :Defense: Maybe you're right, but ``<>`` looks nicer whenever applicable. XML/SGML could use ``@`` while more common programming languages stick to ``<>``. References ========== Some other tools have approached defining/exploiting codetags. See http://tracos.org/codetag/wiki/Links. .. _wiki: http://tracos.org/codetag/wiki/Pep .. _ISO 8601: http://en.wikipedia.org/wiki/ISO_8601 .. _c2: http://c2.com/cgi/wiki?FixmeComment .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: From guido at python.org Tue Sep 27 18:47:24 2005 From: guido at python.org (Guido van Rossum) Date: Tue, 27 Sep 2005 09:47:24 -0700 Subject: [Python-Dev] PEP 350: Codetags In-Reply-To: References: <20050926223521.GE10940@kitchen.client.attbi.com> Message-ID: On 9/26/05, Neil Schemenauer wrote: > Micah Elliott wrote: > > ``FIXME (XXX, DEBUG, BROKEN, REFACTOR, REFACT, RFCTR, OOPS, SMELL, NEEDSWORK, INSPECT)`` > > *Fix me*: Areas of problematic or ugly code needing refactoring or > > cleanup. > > I think the standard should not have codetags that are synonyms. > This is Python and there should be only one way to do it. One > problem with synonyms is that they makes it harder to search using > tools like grep. It has always been my choice to *only* use XXX. I hope there aren't any developers contributing to the Python core who use any others? I honestly don't see much of a point for distinguishing different types; these are for humans to read and review, and different tags just makes it harder to grep. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik at pythonware.com Tue Sep 27 18:59:26 2005 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 27 Sep 2005 18:59:26 +0200 Subject: [Python-Dev] PEP 350: Codetags References: <20050926223521.GE10940@kitchen.client.attbi.com> Message-ID: Guido van Rossum wrote: > It has always been my choice to *only* use XXX. I hope there aren't > any developers contributing to the Python core who use any others? [Python-2.4.1]$ grep FIXME */*.c */*.py | wc -l 12 [Python-2.4.1]$ grep TODO */*.c */*.py | wc -l 17 [Python-2.4.1]$ grep XXX */*.c */*.py | wc -l 525 > I honestly don't see much of a point for distinguishing different types; > these are for humans to read and review, and different tags just makes > it harder to grep. I tend to use FIXME for smelly code, and a collection of TODO:s at the top of the file for things that someone should work on some day... (which explains some, but not all, of the non-XXX:es above...) From barbieri at gmail.com Tue Sep 27 19:05:28 2005 From: barbieri at gmail.com (Gustavo Barbieri) Date: Tue, 27 Sep 2005 14:05:28 -0300 Subject: [Python-Dev] Fixing pty.spawn() Message-ID: <9ef20ef30509271005402260e6@mail.gmail.com> Hello, I've come to a problem with pty.spawn() as it states on its source code: # Bugs: No signal handling. Doesn't set slave termios and window size. As I need this feature working, I want to fix this bug. Does someone have any idea on how to fix it? (My problem: I want to launch ncurses software from my python software, then come back, choose another program and launch it, ...) -- Gustavo Sverzut Barbieri --------------------------------------- Computer Engineer 2001 - UNICAMP GPSL - Grupo Pro Software Livre Cell..: +55 (19) 9165 8010 Jabber: gsbarbieri at jabber.org ICQ#: 17249123 MSN: barbieri at gmail.com Skype: gsbarbieri GPG: 0xB640E1A2 @ wwwkeys.pgp.net From pje at telecommunity.com Tue Sep 27 19:17:16 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 27 Sep 2005 13:17:16 -0400 Subject: [Python-Dev] PEP 350: Codetags In-Reply-To: <20050926223521.GE10940@kitchen.client.attbi.com> Message-ID: <5.1.1.6.0.20050927125601.02361d18@mail.telecommunity.com> At 03:35 PM 9/26/2005 -0700, Micah Elliott wrote: >Please read/comment/vote. This circulated as a pre-PEP proposal >submitted to c.l.py on August 10, but has changed quite a bit since >then. I'm reposting this since it is now "Open (under consideration)" >at . This seems a little weird to me. On the one hand, seems like a cool idea if you aren't using Eclipse or another IDE that tracks this stuff, but still need some kind of tracking system. But, if that is the case, the notation seems a little bit overkill, especially with respect to tracking who's responsible - i.e., just you. If you have a team that can agree to use the tools, I suppose it might be useful, but then I wonder, why not use something like Trac? Finally, I think there should be something besides just a comment to distinguish these things; like starting with a symbol (e.g. # !FIXME), so that that documentation extraction tools can distinguish code tags from other documentation that just happens to start with a CAPITALIZED word. Overall, I'm kind of -0.5. It seems like a spec in search of an application. The Motivation is sorely lacking - it reads like, "hey, it's optional and you can do stuff", where the stuff you can do is deferred to a later section, and is mostly stuff that could easily be done in other ways. For example, FIT-style acceptance test documents, or Python doctest files go a long way towards documenting stories in association with tests, and they don't require you to cram things into comments. (And because they're executable tests, they get kept up-to-date.) Tracking bugfixes with code history is handled nicely by tools like Trac. There are lots of Python documentation tools already. And so on. Really, it reads to me like you came up with the features to sell the format, instead of designing the format to implement specific features. My suggestion: implement some tools, use them for a while, and come back with more focused use cases to show why only this format can work, and why the Python core developers should therefore use it. I'm not saying that you can't have an informational PEP unless it should be used in the stdlib, mind you. Just pointing out that if you can't convince the core developers it's useful, I'm thinking you'll have a hard time convincing the community at large to actually use it. You need to actually have a better mousetrap to present before you ask people to move their cheese. :) From BruceEckel-Python3234 at mailblocks.com Tue Sep 27 19:18:36 2005 From: BruceEckel-Python3234 at mailblocks.com (Bruce Eckel) Date: Tue, 27 Sep 2005 11:18:36 -0600 Subject: [Python-Dev] Active Objects in Python Message-ID: <397621172.20050927111836@MailBlocks.com> According to this list's welcome message, I should introduce myself. I'm Bruce Eckel, I write, consult and give seminars about computer programming, I use Python whenever I can get away with it, and I've spoken at Pycon a number of times. There are further URLs in my signature at the bottom. I'm joining this list because I was having a conversation with Guido about concurrency support in Python (in particular, using an actor/active-object approach) and he suggested I bring it over here. Here are the highlights of my most recent reply to him (Guido's remarks are marked by the '>' signs): > Agreed. IMO the pthreads-style solution doesn't work well no matter > *what* the programming model. Exactly. I think the problem may be that low-level threads people are one step behind (much like embedded systems programmers, which is where I began). They may be just now catching up to objects, but as far as concurrency goes their brains still think in terms of threads, so it seems natural to apply thread concepts to objects. But from an OO standpoint, pthread-style thinking is two steps backwards. You effectively throw open the innards of the object that you just spent time decoupling from the rest of your system, and the coupling is now unpredictable. It's the worst of all possible worlds. > I worked on Mobile Agents at CNRI from 1995 till 2000, but they were > very heavy-weight. Yet, I think that any solution that moves objects > between CPUs on a regular basis is going to suffer the same problem > that process switching in general gets you -- the move is so expensive > that it's hard to decide when it's justified. The examples I've seen haven't relied on mobility, so that's a question: how important is mobility really to an agent system? And could it be done in a trivial fashion in Python, by sending source code. And if it was really necessary sometimes, could it be something where the cost of mobility only occurred when moving an object? It's possible that either an argument can be made against mobility and/or some kind of trivial mobility system could be developed that would work when necessary. Possibly a Linda-like put-take system with pickles (off the top of my head). I don't know that much about mobility, but it does seem like mobile agents could be a powerful solution to the problem solved by enterprise messaging systems. > I believe you pointed me to Active Objects before, right? (Years ago, > maybe when you visited us at Zope.) I may have mentioned them in the past. > If multiple active objects can co-exist in the same *process*, but > nevertheless the language implementation prevents them from sharing > data except via channels, and in addition allows dynamic reallocation > of active objects across multiple CPUs, they just might be the ticket. > But it would require a complete implementation to prove it. Yes, defining an class as "active" would: 1) Install a worker thread and concurrent queue in each object of that class. 2) Automatically turn method calls into tasks and enqueue them 3) Prevent any other interaction other than enqueued messages At that point, the only time the GIL might be needed is to lock the queue at the point of putting ant taking objects. But there has been work done on lock-free data structures, which has been incorporated into Java 5 libraries, so it might even be possible to use a lock-free queue to do this and thus eliminate the need for the GIL at all. Since the enqueuing process serializes all requests to active objects, and since each message-task runs to completion before the next one starts, the problem of threads interfering with each other is eliminated. Also, the enqueuing process will always happen, so even if the queue blocks it will be for a very short, deterministic time. So the theory is that Active Object systems cannot deadlock (although I believe they can livelock). So yes, the best way for this to work might be some kind of enforced implementation -- but forcing you to do something is not particularly Pythonic, so I would think that it would be possible to build an active object framework along with some checking tools to warn you if you are breaking the rules. But what's not clear is whether this would require knowledge of the innards of the Python interpreter (which I don't have) or if it could be built using libraries. BTW: I think that Hoare's Communicating Sequential Processes (CSP) basically describes the idea of active objects but without objects. That is, I think active objects is CSP applied to OO. Bruce Eckel http://www.BruceEckel.com Contains electronic books: "Thinking in Java 3e" & "Thinking in C++ 2e" Web log: http://www.artima.com/weblogs/index.jsp?blogger=beckel From pje at telecommunity.com Tue Sep 27 19:40:22 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 27 Sep 2005 13:40:22 -0400 Subject: [Python-Dev] Active Objects in Python In-Reply-To: <397621172.20050927111836@MailBlocks.com> Message-ID: <5.1.1.6.0.20050927132256.021c2cf8@mail.telecommunity.com> At 11:18 AM 9/27/2005 -0600, Bruce Eckel wrote: >Yes, defining an class as "active" would: >1) Install a worker thread and concurrent queue in each object of that >class. >2) Automatically turn method calls into tasks and enqueue them >3) Prevent any other interaction other than enqueued messages #3 is the sticky bit. Enforcing such restrictions in Python is *hard*. Each active object would have to have its own sys.modules, for example, because modules and classes in Python are mutable and use dictionaries, and modifying them from two thread simultaneously would be fatal. For built-in types it's not so bad, so they could be shared... except for the fact that they have reference counts, which need lock protection as well to avoid fouling up GC. So really, this idea doesn't really help with GIL removal at all. What it *does* help with, is effective use of multiprocessor machines on platforms where fork() is available, if the API works across processes as well as threads. >So yes, the best way for this to work might be some kind of enforced >implementation -- but forcing you to do something is not particularly >Pythonic, But keeping the interpreter from dumping core due to program bugs *is* Pythonic, which is why the GIL would need to stay. :) >so I would think that it would be possible to build an >active object framework along with some checking tools to warn you if >you are breaking the rules. Well, you could pickle and unpickle the objects you send from one function to another, and for cross-process communication, you'll need to do something like that anyway, or else use that shared-memory objects thing. PySHM? I don't remember its name, but it's an extension that lets you store Python objects in shared memory and use them from multiple processes, modulo certain strict limitations. > But what's not clear is whether this would >require knowledge of the innards of the Python interpreter (which I >don't have) or if it could be built using libraries. If you're not trying to get rid of the GIL or truly "enforce" things, you can do everything you need for CSP in plain old Python. peak.events has a generator-based microthread facility that's CSP-ish, for example, although it doesn't have decorators for async methods. They're not hard to write, though; Chandler has some for methods that get called across thread barriers now. That is, methods that get called from Chandler's UI thread but are run in the Twisted reactor thread. I've occasionally also thought about implementing async C#'s "chord" features in Python for peak.events, but I haven't actually had a use case that needed that much generality yet. From benji at benjiyork.com Tue Sep 27 20:04:11 2005 From: benji at benjiyork.com (Benji York) Date: Tue, 27 Sep 2005 14:04:11 -0400 Subject: [Python-Dev] Active Objects in Python In-Reply-To: <5.1.1.6.0.20050927132256.021c2cf8@mail.telecommunity.com> References: <5.1.1.6.0.20050927132256.021c2cf8@mail.telecommunity.com> Message-ID: <4339899B.5010200@benjiyork.com> Phillip J. Eby wrote: > Well, you could pickle and unpickle the objects you send from one function > to another, and for cross-process communication, you'll need to do > something like that anyway, or else use that shared-memory objects > thing. PySHM? I don't remember its name, but it's an extension that lets > you store Python objects in shared memory and use them from multiple > processes, modulo certain strict limitations. "POSH": http://poshmodule.sourceforge.net/posh/html/ -- Benji York From jcarlson at uci.edu Tue Sep 27 20:45:15 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 27 Sep 2005 11:45:15 -0700 Subject: [Python-Dev] PEP 350: Codetags In-Reply-To: <5.1.1.6.0.20050927125601.02361d18@mail.telecommunity.com> References: <20050926223521.GE10940@kitchen.client.attbi.com> <5.1.1.6.0.20050927125601.02361d18@mail.telecommunity.com> Message-ID: <20050927114015.97BF.JCARLSON@uci.edu> "Phillip J. Eby" wrote: > At 03:35 PM 9/26/2005 -0700, Micah Elliott wrote: > >Please read/comment/vote. This circulated as a pre-PEP proposal > >submitted to c.l.py on August 10, but has changed quite a bit since > >then. I'm reposting this since it is now "Open (under consideration)" > >at . > > This seems a little weird to me. On the one hand, seems like a cool idea > if you aren't using Eclipse or another IDE that tracks this stuff, but > still need some kind of tracking system. But, if that is the case, the > notation seems a little bit overkill, especially with respect to tracking > who's responsible - i.e., just you. There are various Python editors which have had support for a similar style of tags for quite a while. Some allow # : , others allow # : , even others allow more or less. Some even count exclamation points as an indicator of severity. Personally, though I use tags in some of the code I write, and though the editor I use (and wrote) supports tags, I'm of the opinion that an unofficial spec is sufficient. See koders.com and search for 'fixme' to see some common variants. - Josiah From barbieri at gmail.com Tue Sep 27 21:47:45 2005 From: barbieri at gmail.com (Gustavo Barbieri) Date: Tue, 27 Sep 2005 16:47:45 -0300 Subject: [Python-Dev] Fixing pty.spawn() In-Reply-To: <9ef20ef30509271005402260e6@mail.gmail.com> References: <9ef20ef30509271005402260e6@mail.gmail.com> Message-ID: <9ef20ef305092712473f961015@mail.gmail.com> The attached patch calls ioctl() to set master_fd win size. I use fcntl and struct modules, I have hardcoded values from termios since this module is deleted. Is it ok? -- Gustavo Sverzut Barbieri --------------------------------------- Computer Engineer 2001 - UNICAMP GPSL - Grupo Pro Software Livre Cell..: +55 (19) 9165 8010 Jabber: gsbarbieri at jabber.org ICQ#: 17249123 MSN: barbieri at gmail.com Skype: gsbarbieri GPG: 0xB640E1A2 @ wwwkeys.pgp.net -------------- next part -------------- A non-text attachment was scrubbed... Name: pty_spawn_set_winsize.patch Type: application/octet-stream Size: 1778 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050927/f3303dad/pty_spawn_set_winsize.obj From ncoghlan at gmail.com Tue Sep 27 23:49:24 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 28 Sep 2005 07:49:24 +1000 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <1127826673.15824.6.camel@localhost> References: <43365769.4070405@intercable.ru> <1127826673.15824.6.camel@localhost> Message-ID: <4339BE64.5070402@gmail.com> Gustavo J. A. M. Carneiro wrote: > More realistic example: > > def greet(person=None): > print "Hello %s" % (if person is None: "World" else: person) > > Not as compact as C's ?:, but more readable and intuitive. It's just > like an if-else construct, but on a single line and () around to make it > look like an expression instead of a statement. It looks even more like an expression without any embedded colons ;) def greet(person=None): # Infix conditional print "Hello %s" % ("World" if person is None else person) def greet(person=None): # Negated infix conditional so that 'normal' value is first print "Hello %s" % (person if person is not None else "World") def greet(person=None): # Prefix conditional print "Hello %s" % (if person is None then "World" else person) Anyway, Guido's already given some indication that the PEP 308 infix and prefix conditional operators without colons (above) are the main options he is considering choosing from. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From ncoghlan at gmail.com Wed Sep 28 00:17:03 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 28 Sep 2005 08:17:03 +1000 Subject: [Python-Dev] Active Objects in Python In-Reply-To: <397621172.20050927111836@MailBlocks.com> References: <397621172.20050927111836@MailBlocks.com> Message-ID: <4339C4DF.8070504@gmail.com> Bruce Eckel wrote: > Since the enqueuing process serializes all requests to active objects, > and since each message-task runs to completion before the next one > starts, the problem of threads interfering with each other is > eliminated. Also, the enqueuing process will always happen, so even if > the queue blocks it will be for a very short, deterministic time. So > the theory is that Active Object systems cannot deadlock (although I > believe they can livelock). I've done this at work (in C++, not Python), and it works very well. I had my doubts when my boss first put the architecture for it in place, but a couple of years of working with this architecture (and that boss!) persuaded me otherwise. Livelock is a definite issue, particular if you design the individual active objects as finite state machines - it is fairly easy to get into a situation where Object A is waiting for a particular message from Object B, while Object B is waiting for a particular message from Object A. This chain is generally indirect, which makes it hard to spot. It is, however, still easier to figure this out than it is to figure out normal threading bugs, because you can find it just by analysing a sequence diagram showing message exchange and state transitions, rather than caring about the actual underlying code. Threading bugs, on the other hand, can turn up any time you access a shared data structure. > BTW: I think that Hoare's Communicating Sequential Processes (CSP) > basically describes the idea of active objects but without objects. > That is, I think active objects is CSP applied to OO. Or to put it another way, applying OO to CSP is a matter of hiding the addressing and sending of messages behind method calls. It becomes almost like doing in-process remote procedure calls. I think there's a definite synergy with PEP 342 here as well - one of the problems with handling Active Objects is how to allow other messages to be processed while making a blocking call, without losing your current state. One way is to hold any state that persists between messages in the object itself, but that's a seriously unnatural style of programming (it can be done, it's just a pain). PEP 342's yield expressions can probably be used to help address that problem, though: class SomeAO(ActiveObject): def processSomeMessage(self): msg = yield # Do something with the message next_msg = yield makeSomeBlockingCall(self) # Do something with the next message Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From greg.ewing at canterbury.ac.nz Wed Sep 28 03:26:11 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 28 Sep 2005 13:26:11 +1200 Subject: [Python-Dev] Active Objects in Python In-Reply-To: <4339C4DF.8070504@gmail.com> References: <397621172.20050927111836@MailBlocks.com> <4339C4DF.8070504@gmail.com> Message-ID: <4339F133.80009@canterbury.ac.nz> Nick Coghlan wrote: > PEP 342's yield expressions can probably be used to help address that problem, > though: > > class SomeAO(ActiveObject): > def processSomeMessage(self): > msg = yield > # Do something with the message > next_msg = yield makeSomeBlockingCall(self) > # Do something with the next message I don't see how that helps, since makeSomeBlockingCall() is evaluated (and therefore blocks) *before* the yield happens. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From radeex at gmail.com Wed Sep 28 03:41:27 2005 From: radeex at gmail.com (Christopher Armstrong) Date: Wed, 28 Sep 2005 11:41:27 +1000 Subject: [Python-Dev] Active Objects in Python In-Reply-To: <4339F133.80009@canterbury.ac.nz> References: <397621172.20050927111836@MailBlocks.com> <4339C4DF.8070504@gmail.com> <4339F133.80009@canterbury.ac.nz> Message-ID: <60ed19d4050927184173cfabd1@mail.gmail.com> On 9/28/05, Greg Ewing wrote: > Nick Coghlan wrote: > > > PEP 342's yield expressions can probably be used to help address that problem, > > though: > > > > class SomeAO(ActiveObject): > > def processSomeMessage(self): > > msg = yield > > # Do something with the message > > next_msg = yield makeSomeBlockingCall(self) > > # Do something with the next message > > I don't see how that helps, since makeSomeBlockingCall() > is evaluated (and therefore blocks) *before* the yield > happens. Sounds like makeSomeBlockingCall is just misnamed (probably depending who you ask). I wrote a small library recently that wraps Twisted's Deferreds and asynchronous Failure objects such that you can do stuff like try: x = yield remoteObject.getSomething() except Foo: print "Oh no!" This is just a 2.5-ification of defgen, which is at twisted.internet.defer.{deferredGenerator,waitForDeferred}. So anyway, if your actor messages always return Deferreds, then this works quite nicely. -- Twisted | Christopher Armstrong: International Man of Twistery Radix | -- http://radix.twistedmatrix.com | Release Manager, Twisted Project \\\V/// | -- http://twistedmatrix.com |o O| | w----v----w-+ From BruceEckel-Python3234 at mailblocks.com Wed Sep 28 04:38:11 2005 From: BruceEckel-Python3234 at mailblocks.com (Bruce Eckel) Date: Tue, 27 Sep 2005 20:38:11 -0600 Subject: [Python-Dev] Active Objects in Python In-Reply-To: <60ed19d4050927184173cfabd1@mail.gmail.com> References: <397621172.20050927111836@MailBlocks.com> <4339C4DF.8070504@gmail.com> <4339F133.80009@canterbury.ac.nz> <60ed19d4050927184173cfabd1@mail.gmail.com> Message-ID: <638684894.20050927203811@MailBlocks.com> Oops. I forgot to add that to the list. Yes, in the working example of Active Objects that I've written in Java J2SE5, when you send a message to an active object, you get back a Future , which I suspect would be the same as your Deferred. Tuesday, September 27, 2005, 7:41:27 PM, Christopher Armstrong wrote: > On 9/28/05, Greg Ewing wrote: >> Nick Coghlan wrote: >> >> > PEP 342's yield expressions can probably be used to help address that problem, >> > though: >> > >> > class SomeAO(ActiveObject): >> > def processSomeMessage(self): >> > msg = yield >> > # Do something with the message >> > next_msg = yield makeSomeBlockingCall(self) >> > # Do something with the next message >> >> I don't see how that helps, since makeSomeBlockingCall() >> is evaluated (and therefore blocks) *before* the yield >> happens. > Sounds like makeSomeBlockingCall is just misnamed (probably depending > who you ask). > I wrote a small library recently that wraps Twisted's Deferreds and > asynchronous Failure objects such that you can do stuff like > try: > x = yield remoteObject.getSomething() > except Foo: > print "Oh no!" > This is just a 2.5-ification of defgen, which is at > twisted.internet.defer.{deferredGenerator,waitForDeferred}. So anyway, > if your actor messages always return Deferreds, then this works quite > nicely. > -- > Twisted | Christopher Armstrong: International Man of Twistery > Radix | -- http://radix.twistedmatrix.com > | Release Manager, Twisted Project > \\\V/// | -- http://twistedmatrix.com > |o O| | > w----v----w-+ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/bruceeckel-python3234%40mailblocks.com Bruce Eckel http://www.BruceEckel.com mailto:BruceEckel-Python3234 at mailblocks.com Contains electronic books: "Thinking in Java 3e" & "Thinking in C++ 2e" Web log: http://www.artima.com/weblogs/index.jsp?blogger=beckel Subscribe to my newsletter: http://www.mindview.net/Newsletter My schedule can be found at: http://www.mindview.net/Calendar From fdrake at acm.org Wed Sep 28 07:05:50 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Wed, 28 Sep 2005 01:05:50 -0400 Subject: [Python-Dev] Python 2.4.2 documentation online Message-ID: <200509280105.50475.fdrake@acm.org> The Python 2.4.2 documentation is now online in the main documentation area (as opposed to just the /dev/ area) for both www.python.org and docs.python.org. The 2.4 and 2.4.1 documentation areas have been updated to link to the 2.4.2 documentation as the preferred documentation for all 2.4.x releases. -Fred -- Fred L. Drake, Jr. From ncoghlan at gmail.com Wed Sep 28 11:00:43 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 28 Sep 2005 19:00:43 +1000 Subject: [Python-Dev] Active Objects in Python In-Reply-To: <4339F133.80009@canterbury.ac.nz> References: <397621172.20050927111836@MailBlocks.com> <4339C4DF.8070504@gmail.com> <4339F133.80009@canterbury.ac.nz> Message-ID: <433A5BBB.6000506@gmail.com> Greg Ewing wrote: > Nick Coghlan wrote: > > >>PEP 342's yield expressions can probably be used to help address that problem, >>though: >> >> class SomeAO(ActiveObject): >> def processSomeMessage(self): >> msg = yield >> # Do something with the message >> next_msg = yield makeSomeBlockingCall(self) >> # Do something with the next message > > > I don't see how that helps, since makeSomeBlockingCall() > is evaluated (and therefore blocks) *before* the yield > happens. Chris got it right - I just named the function badly. The idea is that there would be an interaction with the Active Object's event loop to get the actual result back once it was available from the thread that actually made the blocking call. In the meantime, *this* object could continue to handle other messages. What the approach allows is to have a single physical thread with a number of coroutiney-type event handlers running inside it. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From steve at holdenweb.com Wed Sep 28 12:32:26 2005 From: steve at holdenweb.com (Steve Holden) Date: Wed, 28 Sep 2005 11:32:26 +0100 Subject: [Python-Dev] PEP 350: Codetags In-Reply-To: <200509270209.06500.hancock@anansispaceworks.com> References: <20050926223521.GE10940@kitchen.client.attbi.com> <200509270209.06500.hancock@anansispaceworks.com> Message-ID: Terry Hancock wrote: > On Monday 26 September 2005 05:35 pm, Micah Elliott wrote: > >>Please read/comment/vote. This circulated as a pre-PEP proposal >>submitted to c.l.py on August 10, but has changed quite a bit since >>then. I'm reposting this since it is now "Open (under consideration)" >>at . > > > Overall, it looks good, but my objection would be: > > >>:Objection: I aesthetically dislike for the comment to be terminated >> with <> in the empty field case. >> >>:Defense: It is necessary to have a terminator since codetags may be >> followed by non-codetag comments. Or codetags could be limited to >> a single line, but that's prohibitive. I can't think of any >> single-character terminator that is appropriate and significantly >> better than <>. Maybe ``@`` could be a terminator, but then most >> codetags will have an unnecessary @. > > > The <> terminator is evil. People will hate that. If there are no fields, > you should just be able to leave it off. This will have an additional > advantage in that many will already have compliant codetags if you leave > off this requirement. > > You worry over the need to detect the end of the block, but wouldn't '\n\n' > be a much more natural delimiter? I.e.: > > # TODO: This is a multi-line todo tag. > # You see how I've gone to the next line. > > # This, on the other hand is an unrelated comment. You can tell it's not > # related, because there is an intervening blank line. I think people > # do this naturally when writing comments (I know I do -- I'm fairly > # certain I've seen other people do it). > # > # Whereas, as you can see, a mere paragraph break can be represented by > # a blank comment line. > # > # Whitespace formatting, after all, is VERY PYTHONIC. ;-) > # Delimiters on the other hand -- well, we prefer not to mention > # the sort of languages that use those, right? ;-) > +1 > Another possibility is to recognize lines like: > > #--------------------------------------- > #*************************************** > #======================================= > > I.e. a comment mark followed by a line composed of repeating characters > as an alternative separator. These are also pretty in pretty common > use. > -0 regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.pycon.org From anthony at python.org Wed Sep 28 14:44:15 2005 From: anthony at python.org (Anthony Baxter) Date: Wed, 28 Sep 2005 22:44:15 +1000 Subject: [Python-Dev] RELEASED Python 2.4.2 (final) Message-ID: <200509282244.26130.anthony@python.org> On behalf of the Python development team and the Python community, I'm happy to announce the release of Python 2.4.2 (final). Python 2.4.2 is a bug-fix release. See the release notes at the website (also available as Misc/NEWS in the source distribution) for details of the more than 60 bugs squished in this release. For more information on Python 2.4.2, including download links for various platforms, release notes, and known issues, please see: http://www.python.org/2.4.2 Highlights of this new release include: - Bug fixes. According to the release notes, more than 60 have been fixed, including bugs that prevented Python from working properly on 64 bit HP/UX and AIX systems. Highlights of the previous major Python release (2.4) are available from the Python 2.4 page, at http://www.python.org/2.4/highlights.html Enjoy the new release, Anthony Anthony Baxter anthony at python.org Python Release Manager (on behalf of the entire python-dev team) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050928/f9414077/attachment.pgp From reinhold-birkenfeld-nospam at wolke7.net Wed Sep 28 15:12:17 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Wed, 28 Sep 2005 15:12:17 +0200 Subject: [Python-Dev] inplace operators and __setitem__ Message-ID: Hi, a general question. Consider: class A(list): def __setitem__(self, index, item): # do something with index and item return list.__setitem__(self, index, item) lst = A([1,set()]) lst[0] |= 1 lst[1] |= set([1]) Do we want lst.__setitem__ to be called in the second inplace assignment? A case where this matters is here: http://python.org/sf/1306777 Reinhold -- Mail address is perfectly valid! From hancock at anansispaceworks.com Tue Sep 27 09:09:06 2005 From: hancock at anansispaceworks.com (Terry Hancock) Date: Tue, 27 Sep 2005 02:09:06 -0500 Subject: [Python-Dev] PEP 350: Codetags In-Reply-To: <20050926223521.GE10940@kitchen.client.attbi.com> References: <20050926223521.GE10940@kitchen.client.attbi.com> Message-ID: <200509270209.06500.hancock@anansispaceworks.com> On Monday 26 September 2005 05:35 pm, Micah Elliott wrote: > Please read/comment/vote. This circulated as a pre-PEP proposal > submitted to c.l.py on August 10, but has changed quite a bit since > then. I'm reposting this since it is now "Open (under consideration)" > at . Overall, it looks good, but my objection would be: > :Objection: I aesthetically dislike for the comment to be terminated > with <> in the empty field case. > > :Defense: It is necessary to have a terminator since codetags may be > followed by non-codetag comments. Or codetags could be limited to > a single line, but that's prohibitive. I can't think of any > single-character terminator that is appropriate and significantly > better than <>. Maybe ``@`` could be a terminator, but then most > codetags will have an unnecessary @. The <> terminator is evil. People will hate that. If there are no fields, you should just be able to leave it off. This will have an additional advantage in that many will already have compliant codetags if you leave off this requirement. You worry over the need to detect the end of the block, but wouldn't '\n\n' be a much more natural delimiter? I.e.: # TODO: This is a multi-line todo tag. # You see how I've gone to the next line. # This, on the other hand is an unrelated comment. You can tell it's not # related, because there is an intervening blank line. I think people # do this naturally when writing comments (I know I do -- I'm fairly # certain I've seen other people do it). # # Whereas, as you can see, a mere paragraph break can be represented by # a blank comment line. # # Whitespace formatting, after all, is VERY PYTHONIC. ;-) # Delimiters on the other hand -- well, we prefer not to mention # the sort of languages that use those, right? ;-) Another possibility is to recognize lines like: #--------------------------------------- #*************************************** #======================================= I.e. a comment mark followed by a line composed of repeating characters as an alternative separator. These are also pretty in pretty common use. Cheers, Terry -- Terry Hancock ( hancock at anansispaceworks.com ) Anansi Spaceworks http://www.anansispaceworks.com From foom at fuhm.net Wed Sep 28 16:08:44 2005 From: foom at fuhm.net (James Y Knight) Date: Wed, 28 Sep 2005 10:08:44 -0400 Subject: [Python-Dev] inplace operators and __setitem__ In-Reply-To: References: Message-ID: <7374DFA6-B7EB-4CFB-8CEA-6FAC1819ED56@fuhm.net> On Sep 28, 2005, at 9:12 AM, Reinhold Birkenfeld wrote: > Hi, > > a general question. Consider: > > class A(list): > def __setitem__(self, index, item): > # do something with index and item > return list.__setitem__(self, index, item) > > lst = A([1,set()]) > > lst[0] |= 1 > > lst[1] |= set([1]) > > Do we want lst.__setitem__ to be called in the second inplace > assignment? Yes. Right now, you can roughly explain the behavior by stating that, after "x=a", "x |= y" is the same as "x = x | y", except that "a"'s value is undefined (it might have changed, or it might have not). > A case where this matters is here: http://python.org/sf/1306777 This confusion between modification of immutable types and modification of mutable types is why I feel that it's often best to simply avoid the inplace operators in favor of their explicit equivalents. In this case, set.update(). James From jim at zope.com Wed Sep 28 16:16:12 2005 From: jim at zope.com (Jim Fulton) Date: Wed, 28 Sep 2005 10:16:12 -0400 Subject: [Python-Dev] RFC: readproperty Message-ID: <433AA5AC.6040509@zope.com> Read descriptors (defining only __get__) and data descripttors have (defining __get__ and __set__) different semantics. In particular, read descriptors are overridden by instance data, while data descriptors are not. A common use of read descriptors is for lazily computed data: class readproperty(object): "Create a read descriptor from a function" def __init__(self, func): self.func = func def __get__(self, inst, class_): if inst is None: return self return self.func(inst) class Spam: @readproperty def eggs(self): ... expensive computation of eggs self.eggs = result return result When we ask for the eggs attribute the first time, we call the descriptor, which calls the eggs function. The function sets the eggs attribute, overriding the descriptor. The next time the eggs attribute is accessed, the saved value will be used without calling the descriptor. I do this often enough that I think it would be useful to include it in python, either as a builtin (like property) or in the library. (Or possibly by adding an option to property to generate a read descriptor.) I'd be happy to add this for 2.5. Thoughts? Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From pje at telecommunity.com Wed Sep 28 16:39:42 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 28 Sep 2005 10:39:42 -0400 Subject: [Python-Dev] inplace operators and __setitem__ In-Reply-To: Message-ID: <5.1.1.6.0.20050928102956.01f6f3a8@mail.telecommunity.com> At 03:12 PM 9/28/2005 +0200, Reinhold Birkenfeld wrote: >Hi, > >a general question. Consider: > >class A(list): > def __setitem__(self, index, item): > # do something with index and item > return list.__setitem__(self, index, item) > >lst = A([1,set()]) > >lst[0] |= 1 > >lst[1] |= set([1]) > >Do we want lst.__setitem__ to be called in the second inplace assignment? Yes. See: http://www.python.org/2.0/new-python.html#SECTION000700000000000000000 The purpose of the augmented assignment forms is to allow for the possibility that the item's __i*__ method may or may not exist, and may or may not return the same object. In the case where there is no __i*__ form, or it does not return the same object, the lvalue *must* be re-bound to the new value, or the semantics break. >A case where this matters is here: http://python.org/sf/1306777 I've closed it as invalid; the behavior is as-defined. In principle, there *could* be an optimization to avoid rebinding the lvalue in the case where the __i*__ form did return self. But using it for the purpose of allowing augmented assignment to tuple members seems dubious at best, and likely to create confusion about the mutability or lack thereof of tuples. IMO it's better to have augmented assignment to tuple members always fail, so that the code has to be a little more specific about its intent. From guido at python.org Wed Sep 28 16:47:26 2005 From: guido at python.org (Guido van Rossum) Date: Wed, 28 Sep 2005 07:47:26 -0700 Subject: [Python-Dev] RFC: readproperty In-Reply-To: <433AA5AC.6040509@zope.com> References: <433AA5AC.6040509@zope.com> Message-ID: On 9/28/05, Jim Fulton wrote: > Read descriptors (defining only __get__) and data descripttors have > (defining __get__ and __set__) different semantics. In particular, > read descriptors are overridden by instance data, while data > descriptors are not. A common use of read descriptors is for lazily > computed data: > > class readproperty(object): > "Create a read descriptor from a function" > > def __init__(self, func): > self.func = func > > def __get__(self, inst, class_): > if inst is None: > return self > > return self.func(inst) > > class Spam: > > @readproperty > def eggs(self): > ... expensive computation of eggs > > self.eggs = result > return result > > When we ask for the eggs attribute the first time, we call the > descriptor, which calls the eggs function. The function sets the eggs > attribute, overriding the descriptor. The next time the eggs attribute > is accessed, the saved value will be used without calling the > descriptor. > > I do this often enough that I think it would be useful to include it > in python, either as a builtin (like property) or in the library. (Or > possibly by adding an option to property to generate a read > descriptor.) I'd be happy to add this for 2.5. > > Thoughts? I like the idea of adding this functionality somewhere. I do note that the semantics were somewhat of an accident, required by backwards compatibility with pre-2.2 method lookup semantics. I think we need to be real careful with chosing a name -- in Jim's example, *anyone* could assign to Spam().eggs to override the value. The name "readproperty" is too close to "readonlyproperty", but read-only it ain't! "Lazy" also doesn't really describe what's going on. I believe some folks use a concept of "memo functions" which resemble this proposal except the notation is different: IIRC a memo function is always invoked as a function, but stores its result in a private instance variable, which it returns upon subsequent calls. This is a common pattern. Jim's proposal differs because the access looks like an attribute, not a method call. Still, perhaps memoproperty would be a possible name. Another way to look at the naming problem is to recognize that the provided function really computes a default value if the attribute isn't already set. So perhaps defaultproperty? (Sorry to turn this into a naming game, which is bound to produce 100s of competing proposals that are totally unintuitive except to the proposer. But naming is important.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Wed Sep 28 17:00:18 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 28 Sep 2005 11:00:18 -0400 Subject: [Python-Dev] RFC: readproperty In-Reply-To: <433AA5AC.6040509@zope.com> Message-ID: <5.1.1.6.0.20050928104334.03572a50@mail.telecommunity.com> At 10:16 AM 9/28/2005 -0400, Jim Fulton wrote: >I do this often enough that I think it would be useful to include it >in python, either as a builtin (like property) or in the library. (Or >possibly by adding an option to property to generate a read >descriptor.) I'd be happy to add this for 2.5. > >Thoughts? You mean something like defaultproperty(func), where func(ob) is called at most once when there is no dictionary entry for the attribute? I started using such properties with PEAK a few years ago, and found some corner cases that made me decide to stick with ones that have both __get__ and __set__. Mostly those cases have to do with creating class-level properties, which can end up being inherited by subclasses if you don't include __set__. Also of course you can't hook the setting of attributes without a __set__. Of course, most people aren't likely to be creating metaclass properties, so it's probably not a big issue for the stdlib. But I thought *you* might want to know about it, in case you hadn't already encountered the issue. :) The other issue I found with such properties is that I really wanted to be able to use functions that didn't rebind the attribute value directly, i.e., I wanted to be able to use lambdas for short computed property descriptions. However, to make this work you have to know what attribute name(s) the property is stored under in the class, which then leads to other interesting complications. So now I use a custom C type that knows its name and takes two functions (a filter for values set, and a function to compute the default). Unfortunately, finding out a descriptor's name is non-trivial; it'd be nice if there were a descriptor hook __bind__(cls,name) that was called by classes during cls.__new__ or assignment to a class attribute, and which you could define to return a replacement descriptor. It seems like one of the first metaclasses I end up writing in any new project is something to do this, and I believe Ian Bicking has encountered the same thing in e.g. SQLObject. From mwh at python.net Wed Sep 28 17:06:54 2005 From: mwh at python.net (Michael Hudson) Date: Wed, 28 Sep 2005 16:06:54 +0100 Subject: [Python-Dev] RFC: readproperty In-Reply-To: <5.1.1.6.0.20050928104334.03572a50@mail.telecommunity.com> (Phillip J. Eby's message of "Wed, 28 Sep 2005 11:00:18 -0400") References: <5.1.1.6.0.20050928104334.03572a50@mail.telecommunity.com> Message-ID: <2m4q85jkoh.fsf@starship.python.net> "Phillip J. Eby" writes: > Unfortunately, finding out a descriptor's name is non-trivial; it'd be nice > if there were a descriptor hook __bind__(cls,name) that was called by > classes during cls.__new__ or assignment to a class attribute, and which > you could define to return a replacement descriptor. It seems like one of > the first metaclasses I end up writing in any new project is something to > do this, and I believe Ian Bicking has encountered the same thing in e.g. > SQLObject. I've done this many times too, but I've never really felt the need to propose a change to Python for it. I guess one could modify the descriptor protocol slightly, but is it worth it? Hmm, dunno. Cheers, mwh -- First time I've gotten a programming job that required a drug test. I was worried they were going to say "you don't have enough LSD in your system to do Unix programming". -- Paul Tomblin -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html From reinhold-birkenfeld-nospam at wolke7.net Wed Sep 28 17:15:56 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Wed, 28 Sep 2005 17:15:56 +0200 Subject: [Python-Dev] inplace operators and __setitem__ In-Reply-To: <5.1.1.6.0.20050928102956.01f6f3a8@mail.telecommunity.com> References: <5.1.1.6.0.20050928102956.01f6f3a8@mail.telecommunity.com> Message-ID: Phillip J. Eby wrote: >>A case where this matters is here: http://python.org/sf/1306777 > > I've closed it as invalid; the behavior is as-defined. > > In principle, there *could* be an optimization to avoid rebinding the > lvalue in the case where the __i*__ form did return self. But using it for > the purpose of allowing augmented assignment to tuple members seems dubious > at best, and likely to create confusion about the mutability or lack > thereof of tuples. IMO it's better to have augmented assignment to tuple > members always fail, so that the code has to be a little more specific > about its intent. Okay. I assume that we must accept that s = set() t = (s,) t[0] |= set([1]) changes s in spite of raising TypeError. Reinhold -- Mail address is perfectly valid! From pierre.barbier at cirad.fr Wed Sep 28 17:40:06 2005 From: pierre.barbier at cirad.fr (Pierre Barbier de Reuille) Date: Wed, 28 Sep 2005 17:40:06 +0200 Subject: [Python-Dev] inplace operators and __setitem__ In-Reply-To: <5.1.1.6.0.20050928102956.01f6f3a8@mail.telecommunity.com> References: <5.1.1.6.0.20050928102956.01f6f3a8@mail.telecommunity.com> Message-ID: <433AB956.8020109@cirad.fr> Phillip J. Eby a ?crit : > At 03:12 PM 9/28/2005 +0200, Reinhold Birkenfeld wrote: [...] > Yes. See: > > http://www.python.org/2.0/new-python.html#SECTION000700000000000000000 > > The purpose of the augmented assignment forms is to allow for the > possibility that the item's __i*__ method may or may not exist, and may or > may not return the same object. In the case where there is no __i*__ form, > or it does not return the same object, the lvalue *must* be re-bound to the > new value, or the semantics break. > > > >>A case where this matters is here: http://python.org/sf/1306777 > > > I've closed it as invalid; the behavior is as-defined. Rather than closing this as invalid, it would be wiser to update the documentation before ! Nothing corresponds to the current behavior. I think that in this page : http://docs.python.org/ref/augassign.html The last paragraph whould be replace by : """ For targets which are attribute (or indexed) references, the initial value is retrieved with a getattr() (resp. __getitem__) and the result is assigned with a setattr() (resp. __setitem__). Notice that the two methods do not necessarily refer to the same variable. When getattr() refers to a class variable, setattr() still writes to an instance variable. For example: """ That way it will be clearly defined in the documentation. Now, one can wonder if the augmented assignment is really an improvement. Lots of errors are made because they are counter-intuitive. For example, in the standard library, I found very few uses of "+=" with a mutable object, and none would be broken if "a += b" is to be replaced by "a = a+b". At worst, there will be a performance issue that will easily be fixed by using "extend" method for lists and corresponding methods for other objects. My opinion is, redefining the augmented assignment is a problem given the assignment semantic, and perhaps we should get rid of it. Pierre -- Pierre Barbier de Reuille INRA - UMR Cirad/Inra/Cnrs/Univ.MontpellierII AMAP Botanique et Bio-informatique de l'Architecture des Plantes TA40/PSII, Boulevard de la Lironde 34398 MONTPELLIER CEDEX 5, France tel : (33) 4 67 61 65 77 fax : (33) 4 67 61 56 68 From pje at telecommunity.com Wed Sep 28 17:41:46 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 28 Sep 2005 11:41:46 -0400 Subject: [Python-Dev] inplace operators and __setitem__ In-Reply-To: References: <5.1.1.6.0.20050928102956.01f6f3a8@mail.telecommunity.com> <5.1.1.6.0.20050928102956.01f6f3a8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050928113634.01f6f800@mail.telecommunity.com> At 05:15 PM 9/28/2005 +0200, Reinhold Birkenfeld wrote: >Okay. I assume that we must accept that > >s = set() >t = (s,) >t[0] |= set([1]) > >changes s in spite of raising TypeError. There are lots of operations that can be partially completed before raising an error, so I'm not sure why this case would be special. Sets do have an update() method, however, and it's unambiguous as to being an in-place update. The code above would be clearer using it, and produce no errors: s = set() t = (s,) t[0].update([1]) From pje at telecommunity.com Wed Sep 28 17:52:28 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 28 Sep 2005 11:52:28 -0400 Subject: [Python-Dev] inplace operators and __setitem__ In-Reply-To: <433AB956.8020109@cirad.fr> References: <5.1.1.6.0.20050928102956.01f6f3a8@mail.telecommunity.com> <5.1.1.6.0.20050928102956.01f6f3a8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050928114225.0347bdd8@mail.telecommunity.com> At 05:40 PM 9/28/2005 +0200, Pierre Barbier de Reuille wrote: >Rather than closing this as invalid, it would be wiser to update the >documentation before ! Nothing corresponds to the current behavior. I got my information from here: http://www.python.org/2.0/new-python.html#SECTION000700000000000000000 >I think that in this page : >http://docs.python.org/ref/augassign.html > >The last paragraph whould be replace by : > >""" >For targets which are attribute (or indexed) references, the initial >value is retrieved with a getattr() (resp. __getitem__) and the result >is assigned with a setattr() (resp. __setitem__). Notice that the two >methods do not necessarily refer to the same variable. When getattr() >refers to a class variable, setattr() still writes to an instance >variable. For example: >""" > >That way it will be clearly defined in the documentation. Actually, the broken part is this sentence: """Also, when possible, the actual operation is performed in-place, meaning that rather than creating a new object and assigning that to the target, the old object is modified instead.""" It is subtly misleading, and would be better stated as: """Also, when possible, the actual operation is performed in-place, meaning that rather than creating a new object the old object is modified instead. In either case, however, the assignment to the target is still performed.""" >Now, one can wonder if the augmented assignment is really an >improvement. Lots of errors are made because they are counter-intuitive. Huh? >For example, in the standard library, I found very few uses of "+=" with >a mutable object, and none would be broken if "a += b" is to be >replaced by "a = a+b". At worst, there will be a performance issue that >will easily be fixed by using "extend" method for lists and >corresponding methods for other objects. The intended use case (as I understand it) for augmented assignment is to allow you to hint that an operation should be done in place *if* it can be. It means that you are not expecting a new object to be the result, but are prepared for the possibility it *might* be a new object. >My opinion is, redefining the augmented assignment is a problem given >the assignment semantic, and perhaps we should get rid of it. How is it a problem? If the assignment semantic weren't what it is, what would it be good for? You could just write an in-place method and be done with it. The whole point is that it allows client code not to care whether it's in-place or not, and to allow implementations to decide (even at runtime) whether to return a different object or not. From steven.bethard at gmail.com Wed Sep 28 18:04:42 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 28 Sep 2005 10:04:42 -0600 Subject: [Python-Dev] RFC: readproperty In-Reply-To: <433AA5AC.6040509@zope.com> References: <433AA5AC.6040509@zope.com> Message-ID: Jim Fulton wrote: > A common use of read descriptors is for lazily computed data: > > class readproperty(object): > "Create a read descriptor from a function" > > def __init__(self, func): > self.func = func > > def __get__(self, inst, class_): > if inst is None: > return self > > return self.func(inst) > > class Spam: > > @readproperty > def eggs(self): > ... expensive computation of eggs > > self.eggs = result > return result I've also needed behavior like this a few times, but I use a variant of Scott David Daniel's recipe[1]: class _LazyAttribute(object): def __init__(self, calculate_function): self._calculate = calculate_function def __get__(self, obj, _=None): if obj is None: return self try: value = self._calculate(obj) except AttributeError, e: # I don't like this, but if _calculate raises an # AttributeError and I don't catch it, the descriptor # machinery hides it and I can't debug my code raise Exception(e) setattr(obj, self._calculate.func_name, value) return value It uses the .func_name attribute to put the "self.eggs = result" into the property. I like that I don't have to do the set at the end of every function, and I'm never doing anything complicated enough that I don't want the attribute named the same as the function that I passed in. [1] http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/363602 STeVe -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From pierre.barbier at cirad.fr Wed Sep 28 18:15:35 2005 From: pierre.barbier at cirad.fr (Pierre Barbier de Reuille) Date: Wed, 28 Sep 2005 18:15:35 +0200 Subject: [Python-Dev] inplace operators and __setitem__ In-Reply-To: <5.1.1.6.0.20050928114225.0347bdd8@mail.telecommunity.com> References: <5.1.1.6.0.20050928102956.01f6f3a8@mail.telecommunity.com> <5.1.1.6.0.20050928102956.01f6f3a8@mail.telecommunity.com> <5.1.1.6.0.20050928114225.0347bdd8@mail.telecommunity.com> Message-ID: <433AC1A7.7060902@cirad.fr> Phillip J. Eby a ?crit : > At 05:40 PM 9/28/2005 +0200, Pierre Barbier de Reuille wrote: > >> Rather than closing this as invalid, it would be wiser to update the >> documentation before ! Nothing corresponds to the current behavior. > > > I got my information from here: > > http://www.python.org/2.0/new-python.html#SECTION000700000000000000000 I know ... I already read this page as you already posted it ;) > > >> I think that in this page : >> http://docs.python.org/ref/augassign.html >> >> The last paragraph whould be replace by : >> >> """ >> For targets which are attribute (or indexed) references, the initial >> value is retrieved with a getattr() (resp. __getitem__) and the result >> is assigned with a setattr() (resp. __setitem__). Notice that the two >> methods do not necessarily refer to the same variable. When getattr() >> refers to a class variable, setattr() still writes to an instance >> variable. For example: >> """ >> >> That way it will be clearly defined in the documentation. > > > Actually, the broken part is this sentence: > > """Also, when possible, the actual operation is performed in-place, > meaning that rather than creating a new object and assigning that to the > target, the old object is modified instead.""" > > It is subtly misleading, and would be better stated as: > > """Also, when possible, the actual operation is performed in-place, > meaning that rather than creating a new object the old object is > modified instead. In either case, however, the assignment to the target > is still performed.""" Indeed ! I missed that one :-S Your proposal should be integrated inside documentation (if anyone knows how to do so ...) !!! > >> Now, one can wonder if the augmented assignment is really an >> improvement. Lots of errors are made because they are counter-intuitive. > > > Huh? > Regularly, you see questions about augmented assignment on Python-tutor mailing list, I often have question in my lab because of problems ... most of the time people learn to avoid these operators in the end ! And my look in the standard library confirmed my intuition about it. > >> For example, in the standard library, I found very few uses of "+=" with >> a mutable object, and none would be broken if "a += b" is to be >> replaced by "a = a+b". At worst, there will be a performance issue that >> will easily be fixed by using "extend" method for lists and >> corresponding methods for other objects. > > > The intended use case (as I understand it) for augmented assignment is > to allow you to hint that an operation should be done in place *if* it > can be. It means that you are not expecting a new object to be the > result, but are prepared for the possibility it *might* be a new object. > > >> My opinion is, redefining the augmented assignment is a problem given >> the assignment semantic, and perhaps we should get rid of it. > > > How is it a problem? If the assignment semantic weren't what it is, > what would it be good for? You could just write an in-place method and > be done with it. The whole point is that it allows client code not to > care whether it's in-place or not, and to allow implementations to > decide (even at runtime) whether to return a different object or not. The problem is: this seems to be more a problem than a solution ! There is a huge difference between in-place or not, and I find it very difficult not to consider it. If you have a use-case for this "let the object decide about in-place operation or not" I'd be interested as I found none. Pierre PS: I'm not criticizing the assignment operator semantic which is exactly what is should be ;) -- Pierre Barbier de Reuille INRA - UMR Cirad/Inra/Cnrs/Univ.MontpellierII AMAP Botanique et Bio-informatique de l'Architecture des Plantes TA40/PSII, Boulevard de la Lironde 34398 MONTPELLIER CEDEX 5, France tel : (33) 4 67 61 65 77 fax : (33) 4 67 61 56 68 From pje at telecommunity.com Wed Sep 28 18:52:11 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 28 Sep 2005 12:52:11 -0400 Subject: [Python-Dev] inplace operators and __setitem__ In-Reply-To: <433AC1A7.7060902@cirad.fr> References: <5.1.1.6.0.20050928114225.0347bdd8@mail.telecommunity.com> <5.1.1.6.0.20050928102956.01f6f3a8@mail.telecommunity.com> <5.1.1.6.0.20050928102956.01f6f3a8@mail.telecommunity.com> <5.1.1.6.0.20050928114225.0347bdd8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050928123958.01f6ff38@mail.telecommunity.com> At 06:15 PM 9/28/2005 +0200, Pierre Barbier de Reuille wrote: >Regularly, you see questions about augmented assignment on Python-tutor >mailing list, I often have question in my lab because of problems ... >most of the time people learn to avoid these operators in the end ! And >my look in the standard library confirmed my intuition about it. Some example of the problems would help. For the specific bug report being discussed, I don't understand why someone would use augmented assignment with an immutable lvalue, since x |= y is short for x = x | y, which is clearly invalid on the face of it if x is a tuple member! >The problem is: this seems to be more a problem than a solution ! >There is a huge difference between in-place or not, and I find it very >difficult not to consider it. Consider string addition. The fact that string concatenation can be implemented with += allows a string to consider based on its refcount to return a new string or to modify itself in-place. If someone uses a = b + c, it may be assumed that they still desire a reference to b, and that therefore the operation *cannot* be done in-place. If they use a += b, then this is a *hint* that an in-place operation is desirable. So, there are two features of augmented assignment: 1. It is a shortcut for spelling out the full assignment 2. Types that override augmented assignment methods may optimize in-place operations, *without the need for client code to change*. >If you have a use-case for this "let the >object decide about in-place operation or not" I'd be interested as I >found none. The whole point of it is that I don't need to *care* whether a particular use is such or not. I simply say what the code intends, and then if somebody needs to pass in something different, or the behavior of some other part of the system changes, then I get that for free. Looking for a specific use case for that is like looking for a use case for duck typing. That is, *everything* is a use case for it, because the point isn't the initial state of the system. The point is what happens when you *change* the system. From ziga.seilnacht at gmail.com Wed Sep 28 19:28:45 2005 From: ziga.seilnacht at gmail.com (=?utf-8?b?xb1pZ2E=?= Seilnacht) Date: Wed, 28 Sep 2005 17:28:45 +0000 (UTC) Subject: [Python-Dev] RFC: readproperty References: <5.1.1.6.0.20050928104334.03572a50@mail.telecommunity.com> <2m4q85jkoh.fsf@starship.python.net> Message-ID: Michael Hudson python.net> writes: > > "Phillip J. Eby" telecommunity.com> writes: > > > Unfortunately, finding out a descriptor's name is non-trivial; it'd be nice > > if there were a descriptor hook __bind__(cls,name) that was called by > > classes during cls.__new__ or assignment to a class attribute, and which > > you could define to return a replacement descriptor. It seems like one of > > the first metaclasses I end up writing in any new project is something to > > do this, and I believe Ian Bicking has encountered the same thing in e.g. > > SQLObject. > > I've done this many times too, but I've never really felt the need to > propose a change to Python for it. I guess one could modify the > descriptor protocol slightly, but is it worth it? Hmm, dunno. > > Cheers, > mwh > You can use something like this to find a descriptor's name: class Private(property): def __init__(self, permission, fget=None, fset=None, fdel=None, doc=None): fget = fget or 'r' in permission and self.default_fget or None fset = fset or 'w' in permission and self.default_fset or None fdel = fdel or 'd' in permission and self.default_fget or None super(private, self).__init__(fget, fset, fdel, doc) def get_attribute_name(self, instance): my_name = None for cls in instance.__class__.mro(): for attribute_name, attribute_object in cls.__dict__.iteritems(): if attribute_object is self: my_name = attribute_name break if my_name is not None: class_name = cls.__name__ break attribute_name = '_%s__%s' % (class_name, my_name) self.attribute_name = attribute_name return attribute_name def default_fget(self, instance): try: attribute_name = self.attribute_name except AttributeError: attribute_name = self.get_attribute_name(instance) return getattr(instance, attribute_name) def default_fset(self, instance, value): try: attribute_name = self.attribute_name except AttributeError: attribute_name = self.get_attribute_name(instance) setattr(instance, attribute_name, value) def default_fdel(self, instance): try: attribute_name = self.attribute_name except AttributeError: attribute_name = self.get_attribute_name(instance) delattr(instance, attribute_name) From phd at mail2.phd.pp.ru Wed Sep 28 19:57:49 2005 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Wed, 28 Sep 2005 21:57:49 +0400 Subject: [Python-Dev] RFC: readproperty In-Reply-To: <433AA5AC.6040509@zope.com> References: <433AA5AC.6040509@zope.com> Message-ID: <20050928175749.GB9460@phd.pp.ru> On Wed, Sep 28, 2005 at 10:16:12AM -0400, Jim Fulton wrote: > class readproperty(object): [skip] > I do this often enough I use it since about 2000 often enough under the name CachedAttribute: http://cvs.sourceforge.net/viewcvs.py/ppa/qps/qUtils.py (The current maintainer of the code is Denis Otkidach, http://sourceforge.net/users/ods/ .) It seems many people reinvent this particular wheel. Which is understandable. propertytools.py, anyone? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From martin at v.loewis.de Wed Sep 28 21:02:00 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 28 Sep 2005 21:02:00 +0200 Subject: [Python-Dev] Active Objects in Python In-Reply-To: <397621172.20050927111836@MailBlocks.com> References: <397621172.20050927111836@MailBlocks.com> Message-ID: <433AE8A8.3010500@v.loewis.de> Bruce Eckel wrote: > Since the enqueuing process serializes all requests to active objects, > and since each message-task runs to completion before the next one > starts, the problem of threads interfering with each other is > eliminated. Also, the enqueuing process will always happen, so even if > the queue blocks it will be for a very short, deterministic time. So > the theory is that Active Object systems cannot deadlock (although I > believe they can livelock). My experience with active objects is with SDL (the Specification and Description Language). There, the primitive communication between objects is through one-way messages (called signals). Such a system can deadlock if an object doesn't produce "enough" new outgoing messages in response to incoming messages. If all communication is through RPCs, i.e. you owe your caller a response, then the only way to deadlock such a system is that all active objects terminate. The interesting question then is whether an object can terminate while processing a message, and whether this will cause a response to be sent back. > But what's not clear is whether this would > require knowledge of the innards of the Python interpreter (which I > don't have) or if it could be built using libraries. As others have pointed out, you have to find a way to deal with the standard library. In principle, each active object should have its own copy of the standard library. For some modules/objects, this is not reasonable (e.g. stdout). In OCCAM, host IO itself is (conceptually) an occam process, so sys.stdout would have to become an active object. In that approach, you would essentially have to rewrite parts of the standard library, wrapping the underlying modules appropriately. > BTW: I think that Hoare's Communicating Sequential Processes (CSP) > basically describes the idea of active objects but without objects. > That is, I think active objects is CSP applied to OO. I don't believe this is true. In CSP, communication between processes is through rendezvous, which is quite different from active objects, as there is no queue for incoming calls - instead, the receiving process must expect a specific communication partner just like the sending process; there is also no inherent request/response mechanism in CSP. In CSP, there is plenty of chance for deadlocks. If a process becomes STOP (i.e. the process that will never again interact), then any communication attempt with that process also cause the partner to become STOP. Regards, Martin From pje at telecommunity.com Wed Sep 28 21:00:55 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 28 Sep 2005 15:00:55 -0400 Subject: [Python-Dev] RFC: readproperty In-Reply-To: References: <5.1.1.6.0.20050928104334.03572a50@mail.telecommunity.com> <2m4q85jkoh.fsf@starship.python.net> Message-ID: <5.1.1.6.0.20050928143952.01f6f868@mail.telecommunity.com> At 05:28 PM 9/28/2005 +0000, ?iga Seilnacht wrote: >You can use something like this to find a descriptor's name: > The given code fails if the same property appears under more than one name or is used in more than one class. It also requires the property object to be mutable, and is subject to inter-thread race conditions in the case of modification of a class. It also doesn't work for class-level descriptors (added to the class' class), and I'd prefer it to use the logical equivalent of ob.__dict__.setdefault() rather than setattr(ob,...) so that it's free of (simple) race conditions in the case where one thread sets the attribute while another is computing the default value. While these aren't crippling limitations in a given application, I think the stdlib implementation should be a bit more robust, especially since none of these features is very hard to implement, once you know what's needed. (And I already have a robust implementation that could be cribbed from.) In any case, having a stdlib version that doesn't address those issues wouldn't be very useful for me, as it wouldn't allow me to replace my custom C descriptor and metaclass. If we added __bind__ (or something like it) to the descriptor protocol I could get rid of *at least* that metaclass, and maybe others. From ncoghlan at gmail.com Thu Sep 29 00:10:14 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 29 Sep 2005 08:10:14 +1000 Subject: [Python-Dev] RFC: readproperty In-Reply-To: <20050928175749.GB9460@phd.pp.ru> References: <433AA5AC.6040509@zope.com> <20050928175749.GB9460@phd.pp.ru> Message-ID: <433B14C5.7060701@gmail.com> Oleg Broytmann wrote: > It seems many people reinvent this particular wheel. Which is > understandable. > > propertytools.py, anyone? "import magic" ;) Cheers, Nick. P.S. Such a module may actually be a good idea, if it reduces the variation in metaclass and descriptor hacks across some of the more complex framework. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From barry at python.org Thu Sep 29 00:23:15 2005 From: barry at python.org (Barry Warsaw) Date: Wed, 28 Sep 2005 18:23:15 -0400 Subject: [Python-Dev] RFC: readproperty In-Reply-To: <433AA5AC.6040509@zope.com> References: <433AA5AC.6040509@zope.com> Message-ID: <1127946195.2270.22.camel@geddy.wooz.org> On Wed, 2005-09-28 at 10:16, Jim Fulton wrote: > When we ask for the eggs attribute the first time, we call the > descriptor, which calls the eggs function. The function sets the eggs > attribute, overriding the descriptor. The next time the eggs attribute > is accessed, the saved value will be used without calling the > descriptor. > > I do this often enough that I think it would be useful to include it > in python, either as a builtin (like property) or in the library. (Or > possibly by adding an option to property to generate a read > descriptor.) I'd be happy to add this for 2.5. I /must/ be missing something. Why not just use property as a decorator? class C: @property def eggs(self): print 'in eggs' self.eggs = 7 return self.eggs >>> c = C() >>> c.eggs in eggs 7 >>> c.eggs 7 -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050928/ae8fa9bc/attachment.pgp From guido at python.org Thu Sep 29 01:09:36 2005 From: guido at python.org (Guido van Rossum) Date: Wed, 28 Sep 2005 16:09:36 -0700 Subject: [Python-Dev] RFC: readproperty In-Reply-To: <1127946195.2270.22.camel@geddy.wooz.org> References: <433AA5AC.6040509@zope.com> <1127946195.2270.22.camel@geddy.wooz.org> Message-ID: It doesn't work that way for new-style classes. On 9/28/05, Barry Warsaw wrote: > On Wed, 2005-09-28 at 10:16, Jim Fulton wrote: > > > When we ask for the eggs attribute the first time, we call the > > descriptor, which calls the eggs function. The function sets the eggs > > attribute, overriding the descriptor. The next time the eggs attribute > > is accessed, the saved value will be used without calling the > > descriptor. > > > > I do this often enough that I think it would be useful to include it > > in python, either as a builtin (like property) or in the library. (Or > > possibly by adding an option to property to generate a read > > descriptor.) I'd be happy to add this for 2.5. > > I /must/ be missing something. Why not just use property as a > decorator? > > class C: > @property > def eggs(self): > print 'in eggs' > self.eggs = 7 > return self.eggs > > >>> c = C() > >>> c.eggs > in eggs > 7 > >>> c.eggs > 7 > > -Barry > > > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.1 (GNU/Linux) > > iQCVAwUAQzsX03EjvBPtnXfVAQK43QP/dRjW11myDXRdjXcPPuaxRQ2qtUlMyAJG > 26sedhmrF00rvKVh7U0RaGJ/Cq5iwgEbQRmXm1pbS8UKzNZxz55qGjVDXjp7Rwgr > KJpJzz/UWVqVClRJJGDdgasRO8GUfxTYh2YPrmXaTDPLh3uscIwpwq1oapT1R4OH > 6xJYLrjAs9M= > =mCyw > -----END PGP SIGNATURE----- > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Thu Sep 29 01:14:46 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 28 Sep 2005 19:14:46 -0400 Subject: [Python-Dev] RFC: readproperty In-Reply-To: <1127946195.2270.22.camel@geddy.wooz.org> References: <433AA5AC.6040509@zope.com> <433AA5AC.6040509@zope.com> Message-ID: <5.1.1.6.0.20050928191349.01fa0cb8@mail.telecommunity.com> At 06:23 PM 9/28/2005 -0400, Barry Warsaw wrote: >I /must/ be missing something. Why not just use property as a >decorator? > >class C: > @property > def eggs(self): > print 'in eggs' > self.eggs = 7 > return self.eggs > > >>> c = C() > >>> c.eggs >in eggs >7 > >>> c.eggs >7 Because it only works in classic classes due to a bug in descriptor handling: >>> class C(object): @property def eggs(self): print 'in eggs' self.eggs = 7 return self.eggs >>> c=C() >>> c.eggs in eggs Traceback (most recent call last): File " ", line 1, in -toplevel- c.eggs File " ", line 4, in eggs self.eggs = 7 AttributeError: can't set attribute From greg.ewing at canterbury.ac.nz Thu Sep 29 03:34:49 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 29 Sep 2005 13:34:49 +1200 Subject: [Python-Dev] RFC: readproperty In-Reply-To: References: <433AA5AC.6040509@zope.com> Message-ID: <433B44B9.7010607@canterbury.ac.nz> Guido van Rossum wrote: > I think we need to be real careful with chosing a name In Eiffel, the keyword "once" is used for something analogous -- a method that is called once the first time it's referenced, and the return value cached. So perhaps this could be called a "once_property". -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From pje at telecommunity.com Thu Sep 29 03:53:19 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 28 Sep 2005 21:53:19 -0400 Subject: [Python-Dev] RFC: readproperty In-Reply-To: <433B44B9.7010607@canterbury.ac.nz> References: <433AA5AC.6040509@zope.com> Message-ID: <5.1.1.6.0.20050928215149.01f80dc0@mail.telecommunity.com> At 01:34 PM 9/29/2005 +1200, Greg Ewing wrote: >Guido van Rossum wrote: > > I think we need to be real careful with chosing a name > >In Eiffel, the keyword "once" is used for something >analogous -- a method that is called once the first >time it's referenced, and the return value cached. > >So perhaps this could be called a "once_property". I was actually going to propose that before I thought of defaultproperty, and in fact my equivalent descriptor was originally called "Once", but the truth is that it gets called every time you delete it and reuse it, whereas Eiffel's once functions are called once, period, with no way to reset them. (At least, IIRC.) From barry at python.org Thu Sep 29 05:44:39 2005 From: barry at python.org (Barry Warsaw) Date: Wed, 28 Sep 2005 23:44:39 -0400 Subject: [Python-Dev] RFC: readproperty In-Reply-To: <5.1.1.6.0.20050928191349.01fa0cb8@mail.telecommunity.com> References: <433AA5AC.6040509@zope.com> <433AA5AC.6040509@zope.com> <5.1.1.6.0.20050928191349.01fa0cb8@mail.telecommunity.com> Message-ID: <1127965479.9281.41.camel@presto.wooz.org> On Wed, 2005-09-28 at 19:14, Phillip J. Eby wrote: > Because it only works in classic classes due to a bug in descriptor handling: Blah. ;) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050928/dbfb4b14/attachment.pgp From anthony at interlink.com.au Thu Sep 29 06:44:51 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu, 29 Sep 2005 14:44:51 +1000 Subject: [Python-Dev] release24-maint is UNFROZEN. 2.4.3 around March or so. Message-ID: <200509291444.53869.anthony@interlink.com.au> So no brown-paper-bag bugs have been reported that I've seen, so let's unfreeze the release24-maint branch. Note that there will be at least a 2.4.3, probably around Feb-March next year (around the time of 2.5a1, as currently vaguely planned). I might also cut a final 2.4.4 after 2.5 final is out. -- Anthony Baxter It's never too late to have a happy childhood. From pierre.barbier at cirad.fr Thu Sep 29 08:10:54 2005 From: pierre.barbier at cirad.fr (Pierre Barbier de Reuille) Date: Thu, 29 Sep 2005 08:10:54 +0200 Subject: [Python-Dev] inplace operators and __setitem__ In-Reply-To: <5.1.1.6.0.20050928123958.01f6ff38@mail.telecommunity.com> References: <5.1.1.6.0.20050928114225.0347bdd8@mail.telecommunity.com> <5.1.1.6.0.20050928102956.01f6f3a8@mail.telecommunity.com> <5.1.1.6.0.20050928102956.01f6f3a8@mail.telecommunity.com> <5.1.1.6.0.20050928114225.0347bdd8@mail.telecommunity.com> <5.1.1.6.0.20050928123958.01f6ff38@mail.telecommunity.com> Message-ID: <433B856E.20801@cirad.fr> Ok, so I took a closer look at the documentation and tried a few things to understand better what you said and I have some remark ... Phillip J. Eby a ecrit : > At 06:15 PM 9/28/2005 +0200, Pierre Barbier de Reuille wrote: > >> Regularly, you see questions about augmented assignment on Python-tutor >> mailing list, I often have question in my lab because of problems ... >> most of the time people learn to avoid these operators in the end ! And >> my look in the standard library confirmed my intuition about it. > > > Some example of the problems would help. For the specific bug report > being discussed, I don't understand why someone would use augmented > assignment with an immutable lvalue, since x |= y is short for x = x | > y, which is clearly invalid on the face of it if x is a tuple member! Well, the problem is: >>> a = ([1,2], [3,4]) >>> a[0] += [5,6] ... a[0] *is* mutable, so the author of the bug report did not feel like its l-value was immutable as he supposed a[0] was the l-value, however in this case, both "a" and "a[0]" are l-values ! Otherwise, a very common problem (encounter regularly with labmates): >>> def foo(a,b): >>> a += b >>> return a >>> a = 3 >>> b = 4 >>> c = foo(a,b) # Works fine as intended >>> d = [1,2] >>> e = [3,4] >>> f = foo(d,e) # Oops ... *d* is modified Of course, actual code is much more complex, but the problem can be reduced to that most of the time. Also, on your sentence (which I find much more accurate than the current one): """Also, when possible, the actual operation is performed in-place, meaning that rather than creating a new object the old object is modified instead. In either case, however, the assignment to the target is still performed.""" I would add some pseudo-code equivalence like, if "__iadd__" is defined: a += b <=> a = a.__iadd__(b) which I believe is true from the look at the pseudo-code generated at compile-time. Like this, it is easier to remember and much less subject to interpretation than with human-language sentences ;) > >> The problem is: this seems to be more a problem than a solution ! >> There is a huge difference between in-place or not, and I find it very >> difficult not to consider it. > > > Consider string addition. The fact that string concatenation can be > implemented with += allows a string to consider based on its refcount to > return a new string or to modify itself in-place. If someone uses a = b > + c, it may be assumed that they still desire a reference to b, and that > therefore the operation *cannot* be done in-place. If they use a += b, > then this is a *hint* that an in-place operation is desirable. However, the implementation makes no use of the operator ! First, there is no "__iadd__" in the "str" class, second documentation says that "a+=b" and "a=a+b" are both optimized. So this does not validate for a use-case ^_^ > > So, there are two features of augmented assignment: > > 1. It is a shortcut for spelling out the full assignment > 2. Types that override augmented assignment methods may optimize > in-place operations, *without the need for client code to change*. Sure, I understood ... still I do not see any need for that, while I can see a bunch of problems ! >> If you have a use-case for this "let the >> object decide about in-place operation or not" I'd be interested as I >> found none. > > > The whole point of it is that I don't need to *care* whether a > particular use is such or not. I simply say what the code intends, and > then if somebody needs to pass in something different, or the behavior > of some other part of the system changes, then I get that for free. > Looking for a specific use case for that is like looking for a use case > for duck typing. That is, *everything* is a use case for it, because > the point isn't the initial state of the system. The point is what > happens when you *change* the system. My point here is: if the syntax causes a problem and does not solve any, why would we keep it ? As for duck-typing use cases are plenty ! A huge part of my code use benefit of duck-typing and so does the Python library, so there *are* use-cases ! Pierre -- Pierre Barbier de Reuille INRA - UMR Cirad/Inra/Cnrs/Univ.MontpellierII AMAP Botanique et Bio-informatique de l'Architecture des Plantes TA40/PSII, Boulevard de la Lironde 34398 MONTPELLIER CEDEX 5, France tel : (33) 4 67 61 65 77 fax : (33) 4 67 61 56 68 From jim at zope.com Thu Sep 29 10:20:31 2005 From: jim at zope.com (Jim Fulton) Date: Thu, 29 Sep 2005 04:20:31 -0400 Subject: [Python-Dev] RFC: readproperty In-Reply-To: References: <433AA5AC.6040509@zope.com> Message-ID: <433BA3CF.1090205@zope.com> Guido van Rossum wrote: > On 9/28/05, Jim Fulton wrote: > ... > I think we need to be real careful with chosing a name -- in Jim's > example, *anyone* could assign to Spam().eggs to override the value. > The name "readproperty" is too close to "readonlyproperty", In fact, property creates read-only properties for new-style classes. (I hadn't realized, until reading this thread, that for classic classes, you could still set the attribute.) > but > read-only it ain't! "Lazy" also doesn't really describe what's going > on. I agree. > I believe some folks use a concept of "memo functions" which resemble > this proposal except the notation is different: IIRC a memo function > is always invoked as a function, but stores its result in a private > instance variable, which it returns upon subsequent calls. This is a > common pattern. Jim's proposal differs because the access looks like > an attribute, not a method call. Still, perhaps memoproperty would be > a possible name. > > Another way to look at the naming problem is to recognize that the > provided function really computes a default value if the attribute > isn't already set. So perhaps defaultproperty? Works for me. Oleg Broytmann wrote: > On Wed, Sep 28, 2005 at 10:16:12AM -0400, Jim Fulton wrote: > >> class readproperty(object): > > [skip] > >>I do this often enough > > > I use it since about 2000 often enough under the name CachedAttribute: > > http://cvs.sourceforge.net/viewcvs.py/ppa/qps/qUtils.py Steven Bethard wrote: > Jim Fulton wrote: > ... > I've also needed behavior like this a few times, but I use a variant > of Scott David Daniel's recipe[1]: > > class _LazyAttribute(object): Yup, the Zope 3 sources have something very similar: http://svn.zope.org/Zope3/trunk/src/zope/cachedescriptors/property.py?view=markup I actually think this does too much. All it saves me, compared to what I proposed is one assignment. I'd rather make that assignment explicit. Anyway, all I wanted with readproperty was a property that implemented only __get__, as opposed to property, which implements __get__, __set__, and __delete__. I'd be happy to call it readproprty or getproperty or defaulproperty or whatever. :) I'd prefer that it's semantics stay fairly simple though. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From mde at tracos.org Wed Sep 28 18:10:39 2005 From: mde at tracos.org (Micah Elliott) Date: Wed, 28 Sep 2005 09:10:39 -0700 Subject: [Python-Dev] PEP 350: Codetags In-Reply-To: <20050926223521.GE10940@kitchen.client.attbi.com> References: <20050926223521.GE10940@kitchen.client.attbi.com> Message-ID: <20050928161039.GF10940@kitchen.client.attbi.com> Thanks to all who have read and/or provided feedback. There have been some great ideas and admonitions that hadn't crossed my mind. I'll paraphrase some of the responses for the sake of brevity; I don't mean to misquote anyone. Tom> ISO 8601 includes a week notation. That's great. Thanks for pointing it out; seems like it should be the preferred week format. I might also want to allow a simple W42 (say) since it's so much shorter, and I'll consider myself generally in trouble if I wrap around on the year for due items. Terry> Shorter forms such as DO, FIX, DEF, DOC, etc. are better. I like the short proposals, so I'll add, and possibly even canonize them. My proposed canons were based on popular use and also my taste. I had originally wanted to state that only the canonical forms would be supported and use of the synonyms should be deprecated. That would have simplified things a bit (human and tool parsing). But based on people's ideas about what is canonical, that would never have flown. Instead, it seems useful to just list everything that's ever been used as long as it *fits one of the categories*. And the canon is mostly arbitrary/unnecessary; we'd never settle that peacefully anyway. The main point is that official categorization enables construction of productivity tools. Terry> IDEXXX isn't vim/emacs. Offer an x:"comment" field for a Terry> completed item. Bengt> Later a tool can strip this out to the devlog.txt or DONE Bengt> file, when the tool sees an added progress line like Bengt> # ---: woohoo, completed ;-) I wish we could rely on everyone to have/use cron. These are both great ideas. I'd like to allow/have both. Bengt> 7) A way of appending an incremental progress line to an existing code Bengt> tag line, e.g., Bengt> # FIXME: This will take a while: rework foo and bar Bengt> # ...: test_foo for new foo works! Bengt> # ...: vacation Status updates? Nice!! Great syntax too. Bengt> time, embedded in strings, scope, no DONE, same line as code... Your pennies are gold! Thanks! Another thing that came to mind recently: As with docstrings, the first sentence of a multiline codetags should be a meaningful summary. So for multiline codetags I'll suggest that the fist line be something that could show up in say a manpage or a BUGFIX file. Terry> Terminator <> is evil. Whitespace is good. Bruno> Or if the codetag is immediately followed by code it's Bruno> terminated. Yes, I'd actually forgotten that it is also not-equal! And I agree that \n\n (or code) is a good terminator. I had been in the practice of putting some TODOs together near the tops of my modules, but a white line between would probably look cleaner anyway. Phillip> there should be something besides just a comment to Phillip> distinguish these things; like starting with a symbol (e.g. Phillip> # !FIXME), so that that documentation extraction tools can Phillip> distinguish code tags from other documentation that just Phillip> happens to start with a CAPITALIZED word. That might be necessary. But if the extraction tools are aware of all official codetags, then it becomes less important. It might even be useful for lint tools to comment when seeing a line that begins with say "# FOO:" but isn't a codetag. Most such uses probably fall under one of the proposed categories anyway. pythondev> It has always been my choice to *only* use XXX. I hope there pythondev> aren't any developers contributing to the Python core who pythondev> use any others? $ csrcs=$(find ~/archive/Python-2.4.1 -name *.c) $ for tag in '!!!' '\?\?\?' XXX WARN CAUTION \ TBD FIXME TODO BUG HACK Note NOTE RFE IMPORTANT; do echo -n "$tag: "; egrep"\b$tag" $csrcs |wc -l done !!!: ~1 \?\?\?: ~12 [most of these part of XXXs] XXX: 365 WARN: ~4 CAUTION: 16 TBD: ~2 FIXME: 12 TODO: 12 BUG: 0 HACK: 0 Note: ~306 NOTE: ~9 RFE: 0 IMPORTANT: ~6 [some overlap with NOTEs] I subtracted most of the false positives but I think the model is being implicitly used to a small degree already. It's just hard to know that they're in the code. I'm impressed there are so few in 365 KLOC. I also notice some WHO: initials, as well as Hmmm:, bad:, Reference:, Obscure:, NB:, Bah:, and others. pythondev> I honestly don't see much of a point for pythondev> distinguishing different types; these are for humans to read pythondev> and review, and different tags just makes it harder to grep. Yes, they are hard to grep! And so are XXXs if multi-line. You'd have to do something like "$EDITOR `grep -l XXX $csrcs`" to really grok them. That's why a different tool is needed for inspection. Even if the codetag paradigm is despised for a given project, something (pychecker/pylint) needs to do a proper scan to address/remove/alert them. I won't argue that the interpreter should adopt codetags, but it would at least benefit from lint recognition. Phillip> You still need a tracking system. Agreed, for most projects, and I think Trac is great. But some might want to use codetags as a way to track anything that is not a full-blown bug. And how many times have you seen small projects that say, "We don't have a bug tracker yet. Please just send bugs to "? Josiah> Some even count exclamation points as an indicator of severity. Michael> I prefer TODO SMELL STINK STENCH VOMIT to indicate TODO priority. These seem useful. But I personally prefer a single TODO with a numeric priority, and I feel that the p: field is the simplest way to get that (easier to remember numbers 0..3 than contaminations or !-counts). I think the example you gave could be done with a "# FIXME: ..." and still be obvious, or even "# FIXME: ...", assuming you have mapped your bletcherosity level to numbers. This also assumes Terry's whitespace idea is used so the fields could show up at the front. Note that the PEP has separated TODO from FIXME semantics. Josiah> an unofficial spec is sufficient. See koders.com and search Josiah> for 'fixme' to see some common variants. But that's the problem -- there are already a bunch of "unofficial" specs, which don't serve much purpose as such. It's a cool site. I spent some time browsing and I do see a lot of codetags in use (many thousands in Python code alone; I'm not sure if the number represented strings or files). But I'll argue that this just reinforces the need for an *official* spec/guideline, so that the tools can do something with them. Paul> Such a PEP should not be approved unless there's Paul> already an implementation (e.g. PyChecker patch) Phillip> implement some tools, use them for a while, and come back Phillip> with more focused use cases Phillip> It seems like a spec in search of an application. The Phillip> Motivation is sorely lacking My two main motivations are avoiding duplication (for documentation) and organizing tasks. I'm presently using it on a smallish project (5 KLOC) to generate manpage sections (fed to help2man) for BUGS, GLOSSARY, and RFE. These should arguably be dumped to a BUGS/BUGFIX/ChangeLog file. I haven't yet figured out how to make Trac or SourceForge do a nice creation of such a file, though it's essential IMO to provide one with a source package. BUGS files are also non-standardized, though I've seen some pretty nice (yet wildly different) ones, and a tool could help here. The other current use (for me) is as a grep replacement. The tools (just ctdoc right now) are limited (pre-alpha) but really help me address the right tasks in the right order. See for a little comparison to grepping. I do think that the health-o-meter is also valuable (I see that pylint already does a nice job of this). I agree that proof of value is necessary. Without a spec though it will be hard to get people to know about a convention/toolset, so it's a bit of a chicken-egg problem -- I can't have a pep until the tools are in use, but the tools won't be used until programmers have means/motivation to use them, a pep. But now that I have your feedback/ideas I (and maybe the lint folks) can do better job of expanding flexible tools that can prove this paradigm useful (or useless). I will continue development on the tools and encourage anyone interested in using a standard set of codetags for documentation and tracking purposes to give them a try (and provide more feedback!) as they mature. -- Micah Elliott From mal at egenix.com Thu Sep 29 12:33:21 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 29 Sep 2005 12:33:21 +0200 Subject: [Python-Dev] Mapping Darwin 8.2.0 to Mac OS X 10.4.2 in platform.py In-Reply-To: References: Message-ID: <433BC2F1.6080602@egenix.com> Bob Ippolito wrote: > /usr/bin/sw_vers technically calls a private (at least undocumented) > CoreFoundation API, it doesn't parse that plist directly :) > > On further inspection, it looks like parsing the plist directly is > supported API these days (see the bottom of developer.apple.com/documentation/Carbon/Reference/Gestalt_Manager/ > gestalt_refchap/chapter_1.4_section_181.html>): > > import plistlib > dct = plistlib.Plist.fromFile('/System/Library/CoreServices/ > SystemVersion.plist') > print '%(ProductName)s %(ProductVersion)s' % dct Is the plistlib module always available on Mac OS X ? Could you write a patch to system_alias() which uses the above method if available for Mac OS ? > -bob > > On Sep 22, 2005, at 1:02 PM, Wilfredo S?nchez Vega wrote: > > >> "rhapsody" is emitted by uname on Mac OS X Server 1.x, but not on >>anything we ship today. >> >> Bob's right, the version number from uname only tells you about >>the kernel, and not whether, for example, the Cocoa API is on the >>system (it wouldn't be on a standalone Darwin OS install, which >>will have the same uname output). >> >> Just FYI, /usr/bin/sw_vers parses /System/Library/CoreServices/ >>SystemVersion.plist, which is XML. If you want that info, parsing >>the file may be more efficient than forking off sw_vers. >> >> -wsv >> >> >>On Sep 21, 2005, at 8:28 PM, Guido van Rossum wrote: >> >> >> >>>I forgot. The current code recognizes 'Rhapsody' and maps it to >>>"MacOS >>>X Server". But I don't see any evidence that Apple still uses the >>>code >>>name Rhapsody. Does uname ever return 'Rhapsody'? >>> >> >> > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/mal%40egenix.com -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 29 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From aahz at pythoncraft.com Thu Sep 29 13:43:21 2005 From: aahz at pythoncraft.com (Aahz) Date: Thu, 29 Sep 2005 04:43:21 -0700 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <43308D75.7060602@gmail.com> References: <003201c5bd74$15e1c220$99b1958d@oemcomputer> <2m7jdckvz2.fsf@starship.python.net> <43308D75.7060602@gmail.com> Message-ID: <20050929114321.GA11086@panix.com> On Wed, Sep 21, 2005, Nick Coghlan wrote: > Steven Bethard wrote: >> Guido van Rossum wrote: >>> >>>I think I'd prefer (if then else ) i.e. no >>>colons. None of the other expression forms (list comprehensions and >>>generator expressions) involving statement keywords use colons. >> >> FWIW, I find this quite intuitive. It follows the same pattern as LCs >> and GEs -- remove the colons and add parentheses (or brackets for >> LCs). So I'm +1. > > *But*, in LC's and GE's, the body of the main clause of the statement > is also pulled out and placed in front of the keyword: > > def gen(): > for VAR in ITERABLE: > if COND: > yield EXPR > > becomes: > > gen = (EXPR for VAR in ITERABLE if COND) > > This makes sense to me, because the most important thing in the > generator expression is the way each element is populated - the source > iterable and the filtering condition do matter, but they aren't as > important. Yes, and when the looping becomes the focal point, one should generally rewrite a listcomp as a for loop. Nevertheless, because boolean expressions have non-boolean semantics in Python (i.e. they return the objects rather than a boolean value), conditional expressions will almost certainly have the condition as the focal point because a simple boolean expression is no longer sufficient. I am therefore strongly opposed to the "a if cond else b" form. (We've already agreed that Raymond's proposal to make boolean expressions return booleans is dead, right?) I'm also opposed to elif in conditional expressions -- let's keep this a simple Pythonic rewrite of C's ternary. I'm +0 on requiring parentheses precisely because they're annoying. I'm still expecting simple boolean expressions to be the primary use case, and my hunch is that overall Python code will read better with the ternary looking cluttered. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything. From raymond.hettinger at verizon.net Thu Sep 29 14:26:34 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Thu, 29 Sep 2005 08:26:34 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <20050929114321.GA11086@panix.com> Message-ID: <000d01c5c4f1$085e7e20$d63dc797@oemcomputer> [Aahz] > I'm also opposed to elif in conditional expressions -- let's keep this a > simple Pythonic rewrite of C's ternary. > > I'm +0 on requiring parentheses precisely because they're annoying. I'm > still expecting simple boolean expressions to be the primary use case, > and my hunch is that overall Python code will read better with the > ternary looking cluttered. FWIW, I scanned the standard library for all the and/or pairings where a conditional expression was applicable. This sampling can serve as a reference for what "typical" uses would look like in real Python code as created by a number of different authors. It only takes about five minutes to try out a given syntax proposal on all the fragments listed below. That short exercise provides an excellent insight into the look and feel of each proposal in real world code. Raymond --------------------------------------------------------------------- cgitb.py: file = file and os.path.abspath(file) or '?' cgitb.py: formatter = (self.format=="html") and html or text compileal1.py: cfile = fullname + (__debug__ and 'c' or 'o') csv.py: (quotes[a] > quotes[b]) and a or b, quotes.keys()) csv.py: (delims[a] > delims[b]) and a or b, delims.keys()) csv.py: modes[char] = reduce(lambda a, b: a[1] > b[1] and a or b, DocXMLRPCServer.py: anchor = (cl and cl.__name__ or '') + '-' + name fileinput.py: isfirstline() and "*" or "", line) formatter.py: self.writer.send_paragraph((blankline and 1) or 0) gettext.py: __builtin__.__dict__['_'] = unicode and self.ugettext or self.gettext imaplib.py: l = map(lambda x:'%s: "%s"' % (x[0], x[1][0] and '" "'.join(x[1]) or ''), l) imputil.py: _suffix_char = __debug__ and 'c' or 'o' keyword.py: iptfile = args and args[0] or "Python/graminit.c" pickle.py: self.write(obj and NEWTRUE or NEWFALSE) pickle.py: self.write(obj and TRUE or FALSE) pickletools.py: pos is None and " " or pos, py_compile.py: cfile = file + (__debug__ and 'c' or 'o') pydoc.py: return result and re.sub('^ *\n', '', rstrip(result)) or '' pydoc.py: anchor = (cl and cl.__name__ or '') + '-' + name pydoc.py: lhs = name and ' %s = ' % name or '' pydoc.py: contents = doc and [doc + '\n'] or [] pydoc.py: line = (name and name + ' = ' or '') + repr pydoc.py: line = (name and self.bold(name) + ' = ' or '') + repr pydoc.py: host = (sys.platform == 'mac') and '127.0.0.1' or 'localhost' pydoc.py: font = ('helvetica', sys.platform == 'win32' and 8 or 10) robotp~1.py: return (self.allowance and "Allow" or "Disallow")+": "+self.path telnet~1.py: cmd == DO and 'DO' or 'DONT', ord(opt)) telnet~1.py: cmd == WILL and 'WILL' or 'WONT', ord(opt)) thread~1.py: n!=1 and "s" or "") token.py: inFileName = args and args[0] or "Include/token.h" tokenize.py: yield (parenlev > 0 and NL or NEWLINE, unittest.py: return doc and doc.split("\n")[0].strip() or None unittest.py: return doc and doc.split("\n")[0].strip() or None unittest.py: (run, run != 1 and "s" or "", timeTaken)) urllib.py: safe_map[c] = (c in safe) and c or ('%%%02X' % i) urllib2.py: type = file and 'I' or 'D' xdrlib.py: print pred(x) and 'succeeded' or 'failed', ':', x xmlrpclib.py: write(value and "1" or "0") From aahz at pythoncraft.com Thu Sep 29 14:32:20 2005 From: aahz at pythoncraft.com (Aahz) Date: Thu, 29 Sep 2005 05:32:20 -0700 Subject: [Python-Dev] inplace operators and __setitem__ In-Reply-To: <433B856E.20801@cirad.fr> References: <5.1.1.6.0.20050928114225.0347bdd8@mail.telecommunity.com> <5.1.1.6.0.20050928102956.01f6f3a8@mail.telecommunity.com> <5.1.1.6.0.20050928102956.01f6f3a8@mail.telecommunity.com> <5.1.1.6.0.20050928114225.0347bdd8@mail.telecommunity.com> <5.1.1.6.0.20050928123958.01f6ff38@mail.telecommunity.com> <433B856E.20801@cirad.fr> Message-ID: <20050929123220.GA25564@panix.com> On Thu, Sep 29, 2005, Pierre Barbier de Reuille wrote: > > Ok, so I took a closer look at the documentation and tried a few things > to understand better what you said and I have some remark ... I've got some counter-remarks, but python-dev is not the place to discuss them. Please move this thread to comp.lang.python. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything. From pierre.barbier at cirad.fr Thu Sep 29 15:39:32 2005 From: pierre.barbier at cirad.fr (Pierre Barbier de Reuille) Date: Thu, 29 Sep 2005 15:39:32 +0200 Subject: [Python-Dev] inplace operators and __setitem__ In-Reply-To: <20050929123220.GA25564@panix.com> References: <5.1.1.6.0.20050928114225.0347bdd8@mail.telecommunity.com> <5.1.1.6.0.20050928102956.01f6f3a8@mail.telecommunity.com> <5.1.1.6.0.20050928102956.01f6f3a8@mail.telecommunity.com> <5.1.1.6.0.20050928114225.0347bdd8@mail.telecommunity.com> <5.1.1.6.0.20050928123958.01f6ff38@mail.telecommunity.com> <433B856E.20801@cirad.fr> <20050929123220.GA25564@panix.com> Message-ID: <433BEE94.7030102@cirad.fr> Done :) I summarized my point of view and I'm waiting for comments :) Pierre Aahz a ?crit : > On Thu, Sep 29, 2005, Pierre Barbier de Reuille wrote: > >>Ok, so I took a closer look at the documentation and tried a few things >>to understand better what you said and I have some remark ... > > > I've got some counter-remarks, but python-dev is not the place to > discuss them. Please move this thread to comp.lang.python. -- Pierre Barbier de Reuille INRA - UMR Cirad/Inra/Cnrs/Univ.MontpellierII AMAP Botanique et Bio-informatique de l'Architecture des Plantes TA40/PSII, Boulevard de la Lironde 34398 MONTPELLIER CEDEX 5, France tel : (33) 4 67 61 65 77 fax : (33) 4 67 61 56 68 From BruceEckel-Python3234 at mailblocks.com Thu Sep 29 15:43:37 2005 From: BruceEckel-Python3234 at mailblocks.com (Bruce Eckel) Date: Thu, 29 Sep 2005 07:43:37 -0600 Subject: [Python-Dev] Pythonic concurrency In-Reply-To: <433AE8A8.3010500@v.loewis.de> References: <397621172.20050927111836@MailBlocks.com> <433AE8A8.3010500@v.loewis.de> Message-ID: <329633301.20050929074337@MailBlocks.com> I'd like to restart this discussion; I didn't mean to put forth active objects as "the" solution, only that it seems to be one of the better, more OO solutions that I've seen so far. What I'd really like to figure out is the "pythonic" solution for concurrency. Guido and I got as far as agreeing that it wasn't threads. Here are my own criteria for what such a solution would look like: 1) It works by default, so that novices can use it without falling into the deep well of threading. That is, a program that you write using threading is broken by default, and the tool you have to fix it is "inspection." I want something that allows me to say "this is a task. Go." and have it work without the python programmer having to study and understand several tomes on the subject. 2) Tasks can be automatically distributed among processors, so it solves the problems of (a) making python run faster (b) how to utilize multiprocessor systems. 3) Tasks are cheap enough that I can make thousands of them, to solve modeling problems (in which I also lump games). This is really a solution to a cerain type of program complexity -- if I can just assign a task to each logical modeling unit, it makes such a system much easier to program. 4) Tasks are "self-guarding," so they prevent other tasks from interfering with them. The only way tasks can communicate with each other is through some kind of formal mechanism (something queue-ish, I'd imagine). 5) Deadlock is prevented by default. I suspect livelock could still happen; I don't know if it's possible to eliminate that. 6) It's natural to make an object that is actor-ish. That is, this concurrency approach works intuitively with objects. 7) Complexity should be eliminated as much as possible. If it requires greater limitations on what you can do in exchange for a clear, simple, and safe programming model, that sounds pythonic to me. The way I see it, if we can't easily use tasks without getting into trouble, people won't use them. But if we have a model that allows people to (for example) make easy use of multiple processors, they will use that approach and the (possible) extra overhead that you pay for the simplicity will be absorbed by the extra CPUs. 8) It should not exclude the possibility of mobile tasks/active objects, ideally with something relatively straightforward such as Linda-style tuple spaces. One thing that occurs to me is that a number of items on this wish list may conflict with each other, which may require a different way of thinking about the problem. For example, it may require two approaches: for "ordinary" non-OO tasks, a functional programming approach ala Erlang, in combination with an actor approach for objects. Bruce Eckel http://www.BruceEckel.com mailto:BruceEckel-Python3234 at mailblocks.com Contains electronic books: "Thinking in Java 3e" & "Thinking in C++ 2e" Web log: http://www.artima.com/weblogs/index.jsp?blogger=beckel From mwh at python.net Thu Sep 29 16:12:23 2005 From: mwh at python.net (Michael Hudson) Date: Thu, 29 Sep 2005 15:12:23 +0100 Subject: [Python-Dev] Pythonic concurrency In-Reply-To: <329633301.20050929074337@MailBlocks.com> (Bruce Eckel's message of "Thu, 29 Sep 2005 07:43:37 -0600") References: <397621172.20050927111836@MailBlocks.com> <433AE8A8.3010500@v.loewis.de> <329633301.20050929074337@MailBlocks.com> Message-ID: <2mll1ghsjc.fsf@starship.python.net> Bruce Eckel writes: > I'd like to restart this discussion; I didn't mean to put forth active > objects as "the" solution, only that it seems to be one of the better, > more OO solutions that I've seen so far. > > What I'd really like to figure out is the "pythonic" solution for > concurrency. Guido and I got as far as agreeing that it wasn't > threads. > > Here are my own criteria for what such a solution would look like: Just because I've been mentioning it everywhere else since I read it, have you seen this paper: http://research.microsoft.com/Users/simonpj/papers/stm/ ? I don't know how applicable it would be to Python but it's well worth the time it takes to read. Cheers, mwh -- This makes it possible to pass complex object hierarchies to a C coder who thinks computer science has made no worthwhile advancements since the invention of the pointer. -- Gordon McMillan, 30 Jul 1998 From guido at python.org Thu Sep 29 16:32:13 2005 From: guido at python.org (Guido van Rossum) Date: Thu, 29 Sep 2005 07:32:13 -0700 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <000d01c5c4f1$085e7e20$d63dc797@oemcomputer> References: <20050929114321.GA11086@panix.com> <000d01c5c4f1$085e7e20$d63dc797@oemcomputer> Message-ID: On 9/29/05, Raymond Hettinger wrote: > FWIW, I scanned the standard library for all the and/or pairings where a > conditional expression was applicable. This sampling can serve as a > reference for what "typical" uses would look like in real Python code as > created by a number of different authors. > > It only takes about five minutes to try out a given syntax proposal on > all the fragments listed below. That short exercise provides an > excellent insight into the look and feel of each proposal in real world > code. I did this for my favorite proposal, and ended up with the list shown further down below. I think they all looks great! The only problem is that it's not easy to come up with a regex-based way to transform C and X or Y into X if C else Y because it's hard to determine where C starts. So I recommend that people leave existing and/or code alone but start writing if/else expressions in new code only. After all there's nothing wrong with and/or. cgitb.py: file = os.path.abspath(file) if file else '?' cgitb.py: formatter = html if self.format == "html" else text compileal1.py: cfile = fullname + ('c' if __debug__ else 'o') csv.py: a if (quotes[a] > quotes[b]) else b, quotes.keys()) csv.py: a if (delims[a] > delims[b]) else b, delims.keys()) csv.py: modes[char] = reduce(lambda a, b: a if a[1] > b[1] else b, DocXMLRPCServer.py: anchor = (cl.__name__ if cl else '') + '-' + name fileinput.py: "*" if isfirstline() else "", line) formatter.py: self.writer.send_paragraph((1) if blankline else 0) gettext.py: __builtin__.__dict__['_'] = self.ugettext if unicode else self.gettext imaplib.py: l = map(lambda x:'%s: "%s"' % (x[0], '" "'.join(x[1]) if x[1][0] else ''), l) imputil.py: _suffix_char = 'c' if __debug__ else 'o' keyword.py: iptfile = args[0] if args else "Python/graminit.c" pickle.py: self.write(NEWTRUE if obj else NEWFALSE) pickle.py: self.write(TRUE if obj else FALSE) pickletools.py: " " if pos is None else pos, py_compile.py: cfile = file + ('c' if __debug__ else 'o') pydoc.py: return re.sub('^ *\n', '', rstrip(result)) if result else '' pydoc.py: anchor = (cl.__name__ if cl else '') + '-' + name pydoc.py: lhs = ' %s = ' % name if name else '' pydoc.py: contents = [doc + '\n'] if doc else [] pydoc.py: line = (name + ' = ' if name else '') + repr pydoc.py: line = (self.bold(name) + ' = ' if name else '') + repr pydoc.py: host = '127.0.0.1' if (sys.platform == 'mac') else 'localhost' pydoc.py: font = ('helvetica', 8 if sys.platform == 'win32' else 10) robotp~1.py: return (self."Allow" if allowance else "Disallow")+": "+self.path telnet~1.py: 'DO' if cmd == DO else 'DONT', ord(opt)) telnet~1.py: 'WILL' if cmd == WILL else 'WONT', ord(opt)) thread~1.py: "s" if n!=1 else "") token.py: inFileName = args[0] if args else "Include/token.h" tokenize.py: yield (parenlev > NL if 0 else NEWLINE, unittest.py: return doc.split("\n")[0].strip() if doc else None unittest.py: return doc.split("\n")[0].strip() if doc else None unittest.py: (run, run != "s" if 1 else "", timeTaken)) urllib.py: safe_map[c] = c if (c in safe) else ('%%%02X' % i) urllib2.py: type = 'I' if file else 'D' xdrlib.py: print 'succeed' if pred(x) else 'failed', ':', x xmlrpclib.py: write("1" if value else "0") -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Sep 29 16:33:53 2005 From: guido at python.org (Guido van Rossum) Date: Thu, 29 Sep 2005 07:33:53 -0700 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <20050929114321.GA11086@panix.com> <000d01c5c4f1$085e7e20$d63dc797@oemcomputer> Message-ID: On 9/29/05, Guido van Rossum wrote: [a garbled list] Stupid gmail broke the lines. Here it is again as an attachment. -- --Guido van Rossum (home page: http://www.python.org/~guido/) -------------- next part -------------- A non-text attachment was scrubbed... Name: XifCelseY.py Type: application/x-python Size: 2850 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050929/b64232a2/XifCelseY.bin From fredrik at pythonware.com Thu Sep 29 16:49:30 2005 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 29 Sep 2005 16:49:30 +0200 Subject: [Python-Dev] Adding a conditional expression in Py3.0 References: <20050929114321.GA11086@panix.com><000d01c5c4f1$085e7e20$d63dc797@oemcomputer> Message-ID: Guido van Rossum wrote: > I think they all looks great! expression if expression? looks like you've been doing too much Perl hacking lately ;-) From solipsis at pitrou.net Thu Sep 29 17:01:30 2005 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 29 Sep 2005 17:01:30 +0200 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <20050929114321.GA11086@panix.com> <000d01c5c4f1$085e7e20$d63dc797@oemcomputer> Message-ID: <1128006090.12226.36.camel@p-dvsi-418-1.rd.francetelecom.fr> > The only problem is that it's not easy to come up with a regex-based > way to transform > > C and X or Y > > into > > X if C else Y (my 2 cents) I find this proposal very confusing. The order is not logical at all. One usually expects to find the condition on one side, and the alternatives on another side (this is how it's done in every conditional construct I know of : traditional if-then-else, lisp's cond, switch statements...). But there the condition is in the middle, which breaks the natural reading order and feels obfuscated. This is especially true if the "X" in "X if C else Y" happens to be a non-trivial expression - witness your example from unittest.py: return doc.split("\n")[0].strip() if doc else None ... because then the condition (which is the most important part of the statement) is shadowed by the complexity of the first alternative; and the two alternatives, which should logically be siblings, are separated by something which has a different role in the construct. This is exactly like a switch/case statement where the "switch" would have to be inserted in the middle of two "case"'s. Also, generally, one of the most annoying things in computer languages is when they try to invent their own unnatural conditional forms: such as Perl's inverted forms or "unless" statement. Regards Antoine. From skip at pobox.com Thu Sep 29 17:11:52 2005 From: skip at pobox.com (skip@pobox.com) Date: Thu, 29 Sep 2005 10:11:52 -0500 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <20050929114321.GA11086@panix.com> <000d01c5c4f1$085e7e20$d63dc797@oemcomputer> Message-ID: <17212.1080.305445.296283@montanaro.dyndns.org> Guido> After all there's nothing wrong with and/or. Especially if it's correct. Skip From ncoghlan at gmail.com Thu Sep 29 17:19:36 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 30 Sep 2005 01:19:36 +1000 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <000d01c5c4f1$085e7e20$d63dc797@oemcomputer> References: <000d01c5c4f1$085e7e20$d63dc797@oemcomputer> Message-ID: <433C0608.1080900@gmail.com> Raymond Hettinger wrote: > [Aahz] > >>I'm also opposed to elif in conditional expressions -- let's keep this > > a > >>simple Pythonic rewrite of C's ternary. >> >>I'm +0 on requiring parentheses precisely because they're annoying. > > I'm > >>still expecting simple boolean expressions to be the primary use case, >>and my hunch is that overall Python code will read better with the >>ternary looking cluttered. > > > FWIW, I scanned the standard library for all the and/or pairings where a > conditional expression was applicable. This sampling can serve as a > reference for what "typical" uses would look like in real Python code as > created by a number of different authors. Thanks for digging those out - I was thinking that would be a useful exercise, but hadn't taken the time to think of an easy way to find relevant lines. > It only takes about five minutes to try out a given syntax proposal on > all the fragments listed below. That short exercise provides an > excellent insight into the look and feel of each proposal in real world > code. I tried it with (if C then A else B) and (A if C else B), and found both to be significantly more readable than the current code. In particular, I wouldn't want to bet money that none of the examples are buggy, as there were a few cases where the "A" value could conceivably be 0 or '', but I couldn't tell if it was possible for the condition to also be true in those cases. Comparing the two syntaxes I tried, I found that the infix notation generally required the addition of parentheses around the "A" expression when that expression was itself a binary operation - otherwise the precedence was unclear and, even with parentheses around the whole conditional, the layout gave the impression that the conditional expression only applied to the second argument to the binary operation in "A". That said, the more verbose form still felt like it had the elements out of sequence - it was just crying out for the expression to be rewritten as a statement. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From BruceEckel-Python3234 at mailblocks.com Thu Sep 29 17:30:28 2005 From: BruceEckel-Python3234 at mailblocks.com (Bruce Eckel) Date: Thu, 29 Sep 2005 09:30:28 -0600 Subject: [Python-Dev] Pythonic concurrency In-Reply-To: <2mll1ghsjc.fsf@starship.python.net> References: <397621172.20050927111836@MailBlocks.com> <433AE8A8.3010500@v.loewis.de> <329633301.20050929074337@MailBlocks.com> <2mll1ghsjc.fsf@starship.python.net> Message-ID: <564237064.20050929093028@MailBlocks.com> This paper looks very interesting and promises some good ideas. It also looks like it will require time and effort to digest. I've only read the first few pages, but one thing that does leap out is at the beginning of section 3, they say: "... a purely-declarative language is a perfect setting for transactional memory." What's not clear to me from this is whether STM will work in a non-declarative language like Python. Thursday, September 29, 2005, 8:12:23 AM, Michael Hudson wrote: > Bruce Eckel writes: >> I'd like to restart this discussion; I didn't mean to put forth active >> objects as "the" solution, only that it seems to be one of the better, >> more OO solutions that I've seen so far. >> >> What I'd really like to figure out is the "pythonic" solution for >> concurrency. Guido and I got as far as agreeing that it wasn't >> threads. >> >> Here are my own criteria for what such a solution would look like: > Just because I've been mentioning it everywhere else since I read it, > have you seen this paper: > http://research.microsoft.com/Users/simonpj/papers/stm/ > ? I don't know how applicable it would be to Python but it's well > worth the time it takes to read. > Cheers, > mwh Bruce Eckel http://www.BruceEckel.com mailto:BruceEckel-Python3234 at mailblocks.com Contains electronic books: "Thinking in Java 3e" & "Thinking in C++ 2e" Web log: http://www.artima.com/weblogs/index.jsp?blogger=beckel Subscribe to my newsletter: http://www.mindview.net/Newsletter My schedule can be found at: http://www.mindview.net/Calendar From ncoghlan at gmail.com Thu Sep 29 17:38:43 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 30 Sep 2005 01:38:43 +1000 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <1128006090.12226.36.camel@p-dvsi-418-1.rd.francetelecom.fr> References: <20050929114321.GA11086@panix.com> <000d01c5c4f1$085e7e20$d63dc797@oemcomputer> <1128006090.12226.36.camel@p-dvsi-418-1.rd.francetelecom.fr> Message-ID: <433C0A83.8030001@gmail.com> Antoine Pitrou wrote: > This is especially true if the "X" in "X if C else Y" happens to be a > non-trivial expression - witness your example from unittest.py: > return doc.split("\n")[0].strip() if doc else None > > ... because then the condition (which is the most important part of the > statement) is shadowed by the complexity of the first alternative; and > the two alternatives, which should logically be siblings, are separated > by something which has a different role in the construct. I think the perception of what's important is relevant here - to me, the behaviour in the normal case (i.e., non-empty doc) is the most important element. The conditional, and the default value returned when 'doc' is empty are interesting, but are a corner case, rather than a fundamental element of the function's behaviour. > This is exactly like a switch/case statement where the "switch" would > have to be inserted in the middle of two "case"'s. Well, no - because it is possible to consider an if-else as a parameterised binary operation that chooses between the left and right operands (i.e., it's like an "or", only we're asking that the decision be made based on something other than the truth value of the left hand operand). That is, in the case where the left hand expression has no side effects, the following pairs of expression are essentially equivalent: a or b <-> a if a else b a and b <-> a if not a else b In the switch statement example, the switch statement is inherently an n-ary operation, so there is no comparable way of finding a spot to put the switch variable "in the middle". > Also, generally, one of the most annoying things in computer languages > is when they try to invent their own unnatural conditional forms: such > as Perl's inverted forms or "unless" statement. Even more annoying are constructs that don't gel with the rest of the language, though. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From pje at telecommunity.com Thu Sep 29 17:53:06 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 29 Sep 2005 11:53:06 -0400 Subject: [Python-Dev] 64-bit bytecode compatibility (was Re: [PEAK] ez_setup on 64-bit linux problem) In-Reply-To: <433BF0D5.4070901@virtc.com> Message-ID: <5.1.1.6.0.20050929114339.01f7ea28@mail.telecommunity.com> At 09:49 AM 9/29/2005 -0400, Viren Shah wrote: >[I sent this earlier without being a subscriber and it was sent to the >moderation queue so I'm resending it after subscribing] > >Hi, > I'm running a 64-bit Fedora Core 3 with python 2.3.4. I'm trying to > install setuptools to use with Trac, and get the following error: > > [root at Mrdumpling ~]$ python ez_setup.py >Downloading >http://cheeseshop.python.org/packages/2.3/s/setuptools/setuptools-0.6a4-py2.3.egg >Traceback (most recent call last): > File "ez_setup.py", line 206, in ? > main(sys.argv[1:]) > File "ez_setup.py", line 141, in main > from setuptools.command.easy_install import main >OverflowError: signed integer is greater than maximum > > >I get the same type of error if I try installing setuptools manually. I >figure this has to do with the 64-bit nature of the OS and python, but not >being a python person, don't know what a workaround would be. > >Any ideas? Hm. It sounds like perhaps the 64-bit Python in question isn't able to read bytecode for Python from a 32-bit Python version. You'll need to download the setuptools source archive from PyPI and install it using "python setup.py install" instead. In the meantime, I'm going to inquire on Python-Dev about whether a 64-bit Python should be able to read 32-bit bytecode, as I was under the impression Python's bytecode format was supposed to be cross-platform. See e.g.: http://mail.python.org/pipermail/python-list/2004-March/213039.html From michael.walter at gmail.com Thu Sep 29 17:54:00 2005 From: michael.walter at gmail.com (Michael Walter) Date: Thu, 29 Sep 2005 17:54:00 +0200 Subject: [Python-Dev] Pythonic concurrency In-Reply-To: <564237064.20050929093028@MailBlocks.com> References: <397621172.20050927111836@MailBlocks.com> <433AE8A8.3010500@v.loewis.de> <329633301.20050929074337@MailBlocks.com> <2mll1ghsjc.fsf@starship.python.net> <564237064.20050929093028@MailBlocks.com> Message-ID: <877e9a17050929085418a518f1@mail.gmail.com> FWIW, the Perl 6 community is also investigating STM, so it appears to be a worthwhile idea for an impure, multi-paradigm language as well. Regards, Michael On 9/29/05, Bruce Eckel wrote: > This paper looks very interesting and promises some good ideas. It > also looks like it will require time and effort to digest. > > I've only read the first few pages, but one thing that does leap out > is at the beginning of section 3, they say: > > "... a purely-declarative language is a perfect setting for > transactional memory." > > What's not clear to me from this is whether STM will work in a > non-declarative language like Python. > > Thursday, September 29, 2005, 8:12:23 AM, Michael Hudson wrote: > > > Bruce Eckel writes: > > >> I'd like to restart this discussion; I didn't mean to put forth active > >> objects as "the" solution, only that it seems to be one of the better, > >> more OO solutions that I've seen so far. > >> > >> What I'd really like to figure out is the "pythonic" solution for > >> concurrency. Guido and I got as far as agreeing that it wasn't > >> threads. > >> > >> Here are my own criteria for what such a solution would look like: > > > Just because I've been mentioning it everywhere else since I read it, > > have you seen this paper: > > > http://research.microsoft.com/Users/simonpj/papers/stm/ > > > ? I don't know how applicable it would be to Python but it's well > > worth the time it takes to read. > > > Cheers, > > mwh > > > > Bruce Eckel http://www.BruceEckel.com mailto:BruceEckel-Python3234 at mailblocks.com > Contains electronic books: "Thinking in Java 3e" & "Thinking in C++ 2e" > Web log: http://www.artima.com/weblogs/index.jsp?blogger=beckel > Subscribe to my newsletter: > http://www.mindview.net/Newsletter > My schedule can be found at: > http://www.mindview.net/Calendar > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/michael.walter%40gmail.com > From edcjones at comcast.net Thu Sep 29 18:00:08 2005 From: edcjones at comcast.net (Edward C. Jones) Date: Thu, 29 Sep 2005 12:00:08 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 Message-ID: <433C0F88.1080806@comcast.net> Guido van Rossum wrote: file = os.path.abspath(file) if file else '?' ... These are all unreadable. In C "a ? b : c" is not used very often. A quick check of the Python source found 476 occurences. -1 to conditional expressions. From steve at holdenweb.com Thu Sep 29 17:54:46 2005 From: steve at holdenweb.com (Steve Holden) Date: Thu, 29 Sep 2005 16:54:46 +0100 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <20050929114321.GA11086@panix.com> <000d01c5c4f1$085e7e20$d63dc797@oemcomputer> Message-ID: Guido van Rossum wrote: > On 9/29/05, Raymond Hettinger wrote: > >>FWIW, I scanned the standard library for all the and/or pairings where a >>conditional expression was applicable. This sampling can serve as a >>reference for what "typical" uses would look like in real Python code as >>created by a number of different authors. >> >>It only takes about five minutes to try out a given syntax proposal on >>all the fragments listed below. That short exercise provides an >>excellent insight into the look and feel of each proposal in real world >>code. > > > I did this for my favorite proposal, and ended up with the list shown > further down below. > > I think they all looks great! > The fact that so few were found in whole of the standard library does put the use case into question, though, no? Though I am sure more could be found with a more thorough scan. > The only problem is that it's not easy to come up with a regex-based > way to transform > > C and X or Y > > into > > X if C else Y > > because it's hard to determine where C starts. So I recommend that > people leave existing and/or code alone but start writing if/else > expressions in new code only. After all there's nothing wrong with > and/or. > > cgitb.py: file = os.path.abspath(file) if file else '?' > cgitb.py: formatter = html if self.format == "html" else text > compileal1.py: cfile = fullname + ('c' if __debug__ else 'o') > csv.py: a if (quotes[a] > quotes[b]) else > b, quotes.keys()) > csv.py: a if (delims[a] > delims[b]) else > b, delims.keys()) > csv.py: modes[char] = reduce(lambda a, b: a if > a[1] > b[1] else b, > DocXMLRPCServer.py: anchor = (cl.__name__ if cl else '') + '-' + name > fileinput.py: "*" if isfirstline() > else "", line) > formatter.py: self.writer.send_paragraph((1) if blankline else 0) > gettext.py: __builtin__.__dict__['_'] = self.ugettext if > unicode else self.gettext > imaplib.py: l = map(lambda x:'%s: "%s"' % (x[0], '" > "'.join(x[1]) if x[1][0] else ''), l) > imputil.py: _suffix_char = 'c' if __debug__ else 'o' > keyword.py: iptfile = args[0] if args else "Python/graminit.c" > pickle.py: self.write(NEWTRUE if obj else NEWFALSE) > pickle.py: self.write(TRUE if obj else FALSE) > pickletools.py: " " if pos is > None else pos, > py_compile.py: cfile = file + ('c' if __debug__ else 'o') > pydoc.py: return re.sub('^ *\n', '', rstrip(result)) if result else '' > pydoc.py: anchor = (cl.__name__ if cl else '') + '-' + name > pydoc.py: lhs = ' %s = ' % name if name else '' > pydoc.py: contents = [doc + '\n'] if doc else [] > pydoc.py: line = (name + ' = ' if name else '') + repr > pydoc.py: line = (self.bold(name) + ' = ' if name else '') + repr > pydoc.py: host = '127.0.0.1' if (sys.platform == 'mac') > else 'localhost' > pydoc.py: font = ('helvetica', 8 if sys.platform == 'win32' else 10) > robotp~1.py: return (self."Allow" if allowance else > "Disallow")+": "+self.path > telnet~1.py: 'DO' if cmd == DO else > 'DONT', ord(opt)) > telnet~1.py: 'WILL' if cmd == WILL else > 'WONT', ord(opt)) > thread~1.py: "s" if n!=1 else "") > token.py: inFileName = args[0] if args else "Include/token.h" > tokenize.py: yield (parenlev > NL if 0 else NEWLINE, > unittest.py: return doc.split("\n")[0].strip() if doc else None > unittest.py: return doc.split("\n")[0].strip() if doc else None > unittest.py: (run, run != "s" if 1 else > "", timeTaken)) > urllib.py: safe_map[c] = c if (c in safe) else ('%%%02X' % i) > urllib2.py: type = 'I' if file else 'D' > xdrlib.py: print 'succeed' if pred(x) else 'failed', ':', x > xmlrpclib.py: write("1" if value else "0") > Having though about it more closely, the reason I believe it might confuse newbies is because until the "else" comes along the construct can easily be read as a statement with a conditional suffix, and it's only after you read the "else" it becomes obvious that it's the expression that's conditional. I realise that this reading will become habitual once this proposal is put into the language, but I do think this will be a source of confusion to newcomers (who should anyway be steered away from all the magic of ternary expressions, list comprehensions, generator expressions and the like. I would argue for mandatory parentheses around the expression, leaving room later (probably after Guido is no longer around to be sick at the site of it) for: def f(condition): return something if condition # no else! return somethingElse not-expecting-this-to-fly-ly y'rs - steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.pycon.org From pje at telecommunity.com Thu Sep 29 18:10:09 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 29 Sep 2005 12:10:09 -0400 Subject: [Python-Dev] PEP 350: Codetags In-Reply-To: <20050928161039.GF10940@kitchen.client.attbi.com> References: <20050926223521.GE10940@kitchen.client.attbi.com> <20050926223521.GE10940@kitchen.client.attbi.com> Message-ID: <5.1.1.6.0.20050929120234.01ae7390@mail.telecommunity.com> At 09:10 AM 9/28/2005 -0700, Micah Elliott wrote: >I agree that proof of value is necessary. Without a spec though it >will be hard to get people to know about a convention/toolset, so it's >a bit of a chicken-egg problem -- I can't have a pep until the tools are >in use, but the tools won't be used until programmers have >means/motivation to use them, a pep. My point about the lack of motivation was that there was little reason shown why this should be a PEP instead of either: 1. Documentation for a specific tool, or group of tools 2. A specific project's process documentation Are you proposing that this format be used by the Python developers for Python itself? A process spec like this seems orthogonal to Python-the-language. To put it another way, this seems like writing a PEP on how to do eXtreme Programming, or perhaps a PEP on how the blogging "trackback" protocol works. Certainly you might implement those things using Python, but the spec itself seems entirely orthogonal to Python. I don't really see why it's a PEP, as opposed to just a published spec on your own website, unless you intend for say, the Python stdlib to conform to it. From pje at telecommunity.com Thu Sep 29 18:20:34 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 29 Sep 2005 12:20:34 -0400 Subject: [Python-Dev] Pythonic concurrency In-Reply-To: <564237064.20050929093028@MailBlocks.com> References: <2mll1ghsjc.fsf@starship.python.net> <397621172.20050927111836@MailBlocks.com> <433AE8A8.3010500@v.loewis.de> <329633301.20050929074337@MailBlocks.com> <2mll1ghsjc.fsf@starship.python.net> Message-ID: <5.1.1.6.0.20050929121236.0399ed88@mail.telecommunity.com> At 09:30 AM 9/29/2005 -0600, Bruce Eckel wrote: >This paper looks very interesting and promises some good ideas. It >also looks like it will require time and effort to digest. > >I've only read the first few pages, but one thing that does leap out >is at the beginning of section 3, they say: > >"... a purely-declarative language is a perfect setting for >transactional memory." > >What's not clear to me from this is whether STM will work in a >non-declarative language like Python. I spent a few weekends studying that paper earlier this year in order to see if anything could be stolen for Python; my general impression was "not easily" at the least. One notable feature of the presented concept was that when code would otherwise block, they *rolled it back* to the last nonblocking execution point. In a sense, they ran the code backwards, albeit by undoing its effects. They then suspend execution until there's a change to at least one of the variables read during the forward execution, to avoid repeated retries. It was a really fascinating idea, because it was basically a way to write nonblocking code without explicit yielding constructs. But, it depends utterly on having all changes to data being transactional, and on being able to guarantee it in order to prevent bugs that would be just as bad as the ones you get from threading. Oddly enough, this paper actually demonstrates a situation where having static type checking is in fact a solution to a non-trivial problem! It uses static type checking of monads to ensure that you can't touch untransacted things inside a transaction. From rrr at ronadam.com Thu Sep 29 18:21:18 2005 From: rrr at ronadam.com (Ron Adam) Date: Thu, 29 Sep 2005 12:21:18 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <1128006090.12226.36.camel@p-dvsi-418-1.rd.francetelecom.fr> References: <20050929114321.GA11086@panix.com> <000d01c5c4f1$085e7e20$d63dc797@oemcomputer> <1128006090.12226.36.camel@p-dvsi-418-1.rd.francetelecom.fr> Message-ID: <433C147E.4090503@ronadam.com> Antoine Pitrou wrote: >>The only problem is that it's not easy to come up with a regex-based >>way to transform >> >> C and X or Y >> >>into >> >> X if C else Y One way is to parse it manually to a list. This was just a test, but more samples can be added friarly easy. samples = [ # start, cond, x, y, end ("cgitb.py: file =", "file", "os.path.abspath(file)", "'?'", ""), ("cgitb.py: formatter =", '(self.format=="html")', "html", "text", ""), ("compileal1.py: cfile = fullname + (", "__debug__","'c'","'o'",")"), ] for s,c,x,y,e in samples: print "%s %s and %s or %s %s" % (s,c,x,y,e) print "%s %s if %s else %s %s" % (s,x,c,y,e) print "%s (if %s then %s else %s) %s" % (s,c,x,y,e) print "%s (%s ? %s : %s) %s" % (s,c,x,y,e) print > (my 2 cents) > > I find this proposal very confusing. The order is not logical at all. > One usually expects to find the condition on one side, and the > alternatives on another side (this is how it's done in every conditional > construct I know of : traditional if-then-else, lisp's cond, switch > statements...). But there the condition is in the middle, which breaks > the natural reading order and feels obfuscated. I found that my preference depends on the situation. I like (if cond then expr1 else expr2) for most things because having the condition in front tells me the purpose, And it's better represents the order the terms are evaluated in. But when checking a value and either changing it or leaving it alone, I tended to want to type it as. value = (value if cond else alternate_value) In this case the condition is an accept or reject for the alternate_value. And since the first term is a simple name instead of a complex expression, the order of evaluation doesn't bother me. Personally I prefer the function form best for the pure simplicity of it, if(cond, e1, e2), but of course that doesn't do the shortcut behavior and it pre-evaluates the arguments, so it's not an answer. Cheers, Ron From pje at telecommunity.com Thu Sep 29 18:37:15 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 29 Sep 2005 12:37:15 -0400 Subject: [Python-Dev] 64-bit bytecode compatibility (was Re: [PEAK] ez_setup on 64-bit linux problem) In-Reply-To: <433C12CB.9070600@virtc.com> References: <5.1.1.6.0.20050929114339.01f7ea28@mail.telecommunity.com> <5.1.1.6.0.20050929114339.01f7ea28@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050929122931.02f3cbe0@mail.telecommunity.com> At 12:14 PM 9/29/2005 -0400, Viren Shah wrote: > File "/root/svn-install-apps/setuptools-0.6a4/pkg_resources.py", line > 949, in _get > return self.loader.get_data(path) >OverflowError: signed integer is greater than maximum Interesting. That looks like it might be a bug in the Python zipimport module, which is what implements get_data(). Apparently it happens upon importing as well; I assumed that it was a bytecode incompatibility. Checking the revision log, I find that there's a 64-bit fix for zipimport.c in Python 2.4 that looks like it would fix this issue, but it has not been backported to any revision of Python 2.3. You're going to either have to backport the fix yourself and rebuild Python 2.3, or upgrade to Python 2.4. Sorry. :( The diff that implements the fix is at: http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Modules/zipimport.c?r1=1.16&r2=1.17 I'll update my documentation pages to reflect that setuptools requires Python 2.4 if you're using a 64-bit Python. From guido at python.org Thu Sep 29 18:35:49 2005 From: guido at python.org (Guido van Rossum) Date: Thu, 29 Sep 2005 09:35:49 -0700 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <20050929114321.GA11086@panix.com> <000d01c5c4f1$085e7e20$d63dc797@oemcomputer> Message-ID: On 9/29/05, Steve Holden wrote: > I would argue for mandatory parentheses around the expression, leaving > room later (probably after Guido is no longer around to be sick at the > site of it) for: > > def f(condition): > return something if condition # no else! > return somethingElse > > not-expecting-this-to-fly-ly y'rs - steve Let me give you what you expect. If all the "X if C else Y" syntax does is prevent that atrocity from ever being introduced, it would be worth it. :) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From BruceEckel-Python3234 at mailblocks.com Thu Sep 29 18:48:37 2005 From: BruceEckel-Python3234 at mailblocks.com (Bruce Eckel) Date: Thu, 29 Sep 2005 10:48:37 -0600 Subject: [Python-Dev] Pythonic concurrency In-Reply-To: <5.1.1.6.0.20050929121236.0399ed88@mail.telecommunity.com> References: <2mll1ghsjc.fsf@starship.python.net> <397621172.20050927111836@MailBlocks.com> <433AE8A8.3010500@v.loewis.de> <329633301.20050929074337@MailBlocks.com> <2mll1ghsjc.fsf@starship.python.net> <5.1.1.6.0.20050929121236.0399ed88@mail.telecommunity.com> Message-ID: <160502469.20050929104837@MailBlocks.com> > I spent a few weekends studying that paper earlier this year in order to > see if anything could be stolen for Python; my general impression was "not > easily" at the least. One notable feature of the presented concept was > that when code would otherwise block, they *rolled it back* to the last > nonblocking execution point. In a sense, they ran the code backwards, > albeit by undoing its effects. They then suspend execution until there's a > change to at least one of the variables read during the forward execution, > to avoid repeated retries. I haven't spent the weekends on the paper yet (but it looks like that is what it would take), but I had the impression that they were talking about the lock-free techniques such as the ones used in Java 5. Basically, you start a write operation "in the background" without locking the data structure, so reads can continue while the calculation is taking place but before the result is committed. When the result is ready, an atomic "test and write" operation is used to determine whether any other task has modified the value in the meantime, and if not to commit the new value. If another task did modify the value, then the calculation begins anew. That was my take, but I haven't studied everything about STM yet, so I'm probably missing something. The one thing about this paper is that it seems to be an orthogonal perspective to anything about concurrency that *I* have seen before. > Oddly enough, this paper actually demonstrates a situation where having > static type checking is in fact a solution to a non-trivial problem! It > uses static type checking of monads to ensure that you can't touch > untransacted things inside a transaction. Yes, because of some of my diatribes against static checking people get the impression that I think it's just a bad thing. However, I'm really trying to get across the idea that "static type checking as the solution to all problems is a bad idea," and that the cost is often much greater than the benefit. But if there really is a clear payoff then I'm certainly not averse to it. In general, I really *do* like to be told when something has gone wrong -- I think there's a huge benefit in that. But if I can learn about it at runtime rather than compile time, then that is often a reasonable solution. So with concurrency, I would like to know when I do something wrong, but if I am told at runtime that's OK with me as long as I'm told. Bruce Eckel http://www.BruceEckel.com mailto:BruceEckel-Python3234 at mailblocks.com Contains electronic books: "Thinking in Java 3e" & "Thinking in C++ 2e" Web log: http://www.artima.com/weblogs/index.jsp?blogger=beckel Subscribe to my newsletter: http://www.mindview.net/Newsletter My schedule can be found at: http://www.mindview.net/Calendar From p.f.moore at gmail.com Thu Sep 29 18:48:59 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 29 Sep 2005 17:48:59 +0100 Subject: [Python-Dev] PEP 350: Codetags In-Reply-To: <5.1.1.6.0.20050929120234.01ae7390@mail.telecommunity.com> References: <20050926223521.GE10940@kitchen.client.attbi.com> <20050928161039.GF10940@kitchen.client.attbi.com> <5.1.1.6.0.20050929120234.01ae7390@mail.telecommunity.com> Message-ID: <79990c6b05092909485f6b65c3@mail.gmail.com> On 9/29/05, Phillip J. Eby wrote: > My point about the lack of motivation was that there was little reason > shown why this should be a PEP instead of either: > > 1. Documentation for a specific tool, or group of tools > 2. A specific project's process documentation That's what I feel as well. I hadn't commented on the PEP as I had simply intended to ignore it totally in my own projects... Paul. From janssen at parc.com Thu Sep 29 18:56:17 2005 From: janssen at parc.com (Bill Janssen) Date: Thu, 29 Sep 2005 09:56:17 PDT Subject: [Python-Dev] Pythonic concurrency In-Reply-To: Your message of "Thu, 29 Sep 2005 06:43:37 PDT." <329633301.20050929074337@MailBlocks.com> Message-ID: <05Sep29.095620pdt."58617"@synergy1.parc.xerox.com> > 1) It works by default, so that novices can use it without falling > into the deep well of threading. That is, a program that you write > using threading is broken by default, and the tool you have to fix it > is "inspection." I want something that allows me to say "this is a > task. Go." and have it work without the python programmer having to > study and understand several tomes on the subject. Bruce, this seems to me like wishful thinking. Either the separate threads don't interact, in which case you are running separate programs, in which case os.system() already works well enough, or they do, in which case you have the various deadlock and livelock problems of threading. And this nonsense about threaded programs being "broken by default" -- might as well say the same about programs that use variables. If you don't know how to use threads, great, just don't use them. Spend the time reading Fred Brooks' NO SILVER BULLET, instead. It's available in essay form at http://www-inst.eecs.berkeley.edu/~maratb/readings/NoSilverBullet.html. > One thing that occurs to me is that a number of items on this wish > list may conflict with each other, which may require a different way > of thinking about the problem. That seems correct to me. Bill From pje at telecommunity.com Thu Sep 29 19:06:29 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 29 Sep 2005 13:06:29 -0400 Subject: [Python-Dev] Pythonic concurrency In-Reply-To: <160502469.20050929104837@MailBlocks.com> References: <5.1.1.6.0.20050929121236.0399ed88@mail.telecommunity.com> <2mll1ghsjc.fsf@starship.python.net> <397621172.20050927111836@MailBlocks.com> <433AE8A8.3010500@v.loewis.de> <329633301.20050929074337@MailBlocks.com> <2mll1ghsjc.fsf@starship.python.net> <5.1.1.6.0.20050929121236.0399ed88@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050929125748.01fab068@mail.telecommunity.com> At 10:48 AM 9/29/2005 -0600, Bruce Eckel wrote: >I haven't spent the weekends on the paper yet (but it looks like that >is what it would take), but I had the impression that they were >talking about the lock-free techniques such as the ones used in Java >5. Basically, you start a write operation "in the background" without >locking the data structure, so reads can continue while the >calculation is taking place but before the result is committed. When >the result is ready, an atomic "test and write" operation is used to >determine whether any other task has modified the value in the >meantime, and if not to commit the new value. If another task did >modify the value, then the calculation begins anew. > >That was my take, but I haven't studied everything about STM yet, so >I'm probably missing something. No, that's certainly the general idea. The issue for an imperative language like Python is that side-effects are the norm, rather than an exception. The Haskell implementation of the idea effectively relies on the fact that in Haskell you have to jump through monadic hoops to get side effects, so if you have a low-level transactional memory facility, you can create monadic combinators that restrict those side effects to occuring in the way you want them to. This is so utterly alien to Python's execution model - or indeed most imperative languages' execution models - that I don't see how it can be modelled in Python in a way that retains the significant benefits of the approach. Now, in PyPy, I suppose it might be possible to create a custom object-space that works that way, and I hadn't thought of that before. Maybe you should run that paper past one of the PyPy wizards and see if you can get them interested in the idea, then stand back and see what happens. :) >The one thing about this paper is that it seems to be an orthogonal >perspective to anything about concurrency that *I* have seen before. Yes - it's dramatically different, and seems like the One True Way to do imperative concurrency with side-effects. Certainly transactions work nicely for databases, anyway. :) >So with concurrency, I would like to know when I do something wrong, >but if I am told at runtime that's OK with me as long as I'm told. Yeah, but in current Python implementations - with the possible exception of PyPY - detecting the problem at all is virtually impossible. PyPy is an extensible VM, though, so in principle you could create some kind of transactional object space that could keep track of things and tell you if you mess up. From mwh at python.net Thu Sep 29 19:10:20 2005 From: mwh at python.net (Michael Hudson) Date: Thu, 29 Sep 2005 18:10:20 +0100 Subject: [Python-Dev] Pythonic concurrency In-Reply-To: <5.1.1.6.0.20050929121236.0399ed88@mail.telecommunity.com> (Phillip J. Eby's message of "Thu, 29 Sep 2005 12:20:34 -0400") References: <2mll1ghsjc.fsf@starship.python.net> <397621172.20050927111836@MailBlocks.com> <433AE8A8.3010500@v.loewis.de> <329633301.20050929074337@MailBlocks.com> <2mll1ghsjc.fsf@starship.python.net> <5.1.1.6.0.20050929121236.0399ed88@mail.telecommunity.com> Message-ID: <2md5mriyv7.fsf@starship.python.net> "Phillip J. Eby" writes: > At 09:30 AM 9/29/2005 -0600, Bruce Eckel wrote: >>This paper looks very interesting and promises some good ideas. It >>also looks like it will require time and effort to digest. >> >>I've only read the first few pages, but one thing that does leap out >>is at the beginning of section 3, they say: >> >>"... a purely-declarative language is a perfect setting for >>transactional memory." >> >>What's not clear to me from this is whether STM will work in a >>non-declarative language like Python. > > I spent a few weekends studying that paper earlier this year in order to > see if anything could be stolen for Python; my general impression was "not > easily" at the least. One notable feature of the presented concept was > that when code would otherwise block, they *rolled it back* to the last > nonblocking execution point. In a sense, they ran the code backwards, > albeit by undoing its effects. They then suspend execution until there's a > change to at least one of the variables read during the forward execution, > to avoid repeated retries. > > It was a really fascinating idea, because it was basically a way to write > nonblocking code without explicit yielding constructs. But, it depends > utterly on having all changes to data being transactional, and on being > able to guarantee it in order to prevent bugs that would be just as bad as > the ones you get from threading. Oh yes, if implemented this really has be baked thoroughly into the implementation. It's not something that can be implemented as a library (as far as I can see, anyway). > Oddly enough, this paper actually demonstrates a situation where having > static type checking is in fact a solution to a non-trivial problem! It > uses static type checking of monads to ensure that you can't touch > untransacted things inside a transaction. Well, this is an extension of the way Haskell deals with side-effects in general... but yes, it's interesting to be sure. Cheers, mwh -- People think I'm a nice guy, and the fact is that I'm a scheming, conniving bastard who doesn't care for any hurt feelings or lost hours of work if it just results in what I consider to be a better system. -- Linus Torvalds From raymond.hettinger at verizon.net Thu Sep 29 20:28:05 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Thu, 29 Sep 2005 14:28:05 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: Message-ID: <001701c5c523$88fdf880$d63dc797@oemcomputer> > > It only takes about five minutes to try out a given syntax proposal on > > all the fragments listed below. That short exercise provides an > > excellent insight into the look and feel of each proposal in real world > > code. > > I did this for my favorite proposal, and ended up with the list shown > further down below. > > I think they all looks great! They mostly look great to me too :-) The exceptions are the ones in pydoc which have an odd feel to them and, to my eyes, don't parse well. I can't put my finger on the issue -- it could be that any ternary operator syntax breaks down when used in series with other operators or when there are complex sub-expressions. > pydoc.py: return re.sub('^ *\n', '', rstrip(result)) if result else '' > pydoc.py: anchor = (cl.__name__ if cl else '') + '-' + name > pydoc.py: lhs = ' %s = ' % name if name else '' > pydoc.py: contents = [doc + '\n'] if doc else [] > pydoc.py: line = (name + ' = ' if name else '') + repr > pydoc.py: line = (self.bold(name) + ' = ' if name else '') + repr > pydoc.py: host = '127.0.0.1' if (sys.platform == 'mac') else 'localhost' > pydoc.py: font = ('helvetica', 8 if sys.platform == 'win32' else 10) There is one that looks like it could be completely mis-interpreted: (name + ' = ' if name else '') # as written above ((name + ' = ') if name else '') # one mental interpretation (name + (' = ' if name else '')) # an incorrect interpretation The grouping issue doesn't arise for syntaxes that start with a keyword: (if name then name + ' = ' else '') but even those suffer from unclear precedence when used with binary operations on the right: (if cond then a else b + c). Raymond From jim at zope.com Thu Sep 29 20:35:51 2005 From: jim at zope.com (Jim Fulton) Date: Thu, 29 Sep 2005 14:35:51 -0400 Subject: [Python-Dev] Pythonic concurrency In-Reply-To: <2mll1ghsjc.fsf@starship.python.net> References: <397621172.20050927111836@MailBlocks.com> <433AE8A8.3010500@v.loewis.de> <329633301.20050929074337@MailBlocks.com> <2mll1ghsjc.fsf@starship.python.net> Message-ID: <433C3407.7000308@zope.com> Michael Hudson wrote: > Bruce Eckel writes: > > >>I'd like to restart this discussion; I didn't mean to put forth active >>objects as "the" solution, only that it seems to be one of the better, >>more OO solutions that I've seen so far. >> >>What I'd really like to figure out is the "pythonic" solution for >>concurrency. Guido and I got as far as agreeing that it wasn't >>threads. >> >>Here are my own criteria for what such a solution would look like: > > > Just because I've been mentioning it everywhere else since I read it, > have you seen this paper: > > http://research.microsoft.com/Users/simonpj/papers/stm/ > > ? I don't know how applicable it would be to Python but it's well > worth the time it takes to read. I haven't read more than the abstract and the responses to the paper here, but I think I have the gist. (I look forward to reading it later.) I'll note that we have experience with something less rigorous than but otherwise similar to this with ZODB. We often have tens of processes working on the same data, synchronizing through a common object store. Each process executes transactions which are written without any threading code at all. Processes synchronize through transaction commits. This model has worked very well for Zope, at least as long as conflicts can be kept to a minimum. :) The ZODB approach is less rigorous as it only works when your processes operate soley on database objects, but it provides a useful example, I think, of applying this model in Python and it probably covers a reasonably large set of interesting applications. There are hundreds (possibly thousands) of programmers who have used this technique in Zope, probably without even realizing that they were doing concurrent programming. J2EE systems use a similar approach. Jim -- Jim Fulton mailto:jim at zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From robey at lag.net Thu Sep 29 20:40:11 2005 From: robey at lag.net (Robey Pointer) Date: Thu, 29 Sep 2005 11:40:11 -0700 Subject: [Python-Dev] C API doc fix Message-ID: <4092C34F-5A07-47D0-A27F-1781EBFE887A@lag.net> Yesterday I ran into a bug in the C API docs. The top of this page: http://docs.python.org/api/unicodeObjects.html says: Py_UNICODE This type represents a 16-bit unsigned storage type which is used by Python internally as basis for holding Unicode ordinals. On platforms where wchar_t is available and also has 16-bits, Py_UNICODE is a typedef alias for wchar_t to enhance native platform compatibility. On all other platforms, Py_UNICODE is a typedef alias for unsigned short. This is incorrect on some platforms: on Debian, Py_UNICODE turns out to be 32 bits. I'm not sure what the correct quote should be: Does python use wchar_t whenever it's available (16 bits or not)? I solved my problem by realizing that I was going about things entirely wrong, and that I should use the python codecs from C and not worry about what Py_UNICODE contains. However, I think we should fix the docs to avoid confusing others... or maybe it would be better to document what's in Py_UNICODE and suggest always using the codec methods? I don't have a strong opinion either way. robey From steven.bethard at gmail.com Thu Sep 29 21:06:22 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Thu, 29 Sep 2005 13:06:22 -0600 Subject: [Python-Dev] C API doc fix In-Reply-To: <4092C34F-5A07-47D0-A27F-1781EBFE887A@lag.net> References: <4092C34F-5A07-47D0-A27F-1781EBFE887A@lag.net> Message-ID: On 9/29/05, Robey Pointer wrote: > Yesterday I ran into a bug in the C API docs. The top of this page: > > http://docs.python.org/api/unicodeObjects.html > > says: > > Py_UNICODE > This type represents a 16-bit unsigned storage type which is > used by Python internally as basis for holding Unicode ordinals. On > platforms where wchar_t is available and also has 16-bits, Py_UNICODE > is a typedef alias for wchar_t to enhance native platform > compatibility. On all other platforms, Py_UNICODE is a typedef alias > for unsigned short. I believe this is the same issue that was brought up in May[1]. My impression was that people could not agree on a documentation patch. [1] http://www.python.org/dev/summary/2005-05-01_2005-05-15.html STeVe -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From shane at hathawaymix.org Thu Sep 29 21:31:19 2005 From: shane at hathawaymix.org (Shane Hathaway) Date: Thu, 29 Sep 2005 13:31:19 -0600 Subject: [Python-Dev] Pythonic concurrency In-Reply-To: <329633301.20050929074337@MailBlocks.com> References: <397621172.20050927111836@MailBlocks.com> <433AE8A8.3010500@v.loewis.de> <329633301.20050929074337@MailBlocks.com> Message-ID: <433C4107.8080203@hathawaymix.org> Bruce Eckel wrote: > I'd like to restart this discussion; I didn't mean to put forth active > objects as "the" solution, only that it seems to be one of the better, > more OO solutions that I've seen so far. > > What I'd really like to figure out is the "pythonic" solution for > concurrency. Guido and I got as far as agreeing that it wasn't > threads. I've pondered this problem. Python deals programmers a double whammy when it comes to threads: not only is threading unsafe like it is in other languages, but the GIL also prevents you from using multiple processors. Thus there's more pressure to improve concurrency in Python than there is elsewhere. I like to use fork(), but fork has its own set of surprises. In particular, in the programmer's view, forking creates a disassociated copy of every object except files. Also, there's no Pythonic way for the two processes to communicate once the child has started. It's tempting to create a library around fork() that solves the communication problem, but the copied objects are still a major source of bugs. Imagine what would happen if you forked a Zope process with an open ZODB. If both the parent and child change ZODB objects, ZODB is likely to corrupt itself, since the processes share file descriptors. Thus forking can just as dangerous as threading. Therefore, I think a better Python concurrency model would be a lot like the subprocess module, but designed for calling Python code. I can already think of several ways I would use such a module. Something like the following would solve problems I've encountered with threads, forking, and the subprocess module: import pyprocess proc = pyprocess.start('mypackage.mymodule', 'myfunc', arg1, arg2=5) while proc.running(): # do something else res = proc.result() This code doesn't specify whether the subprocess should continue to exist after the function completes (or throws an exception). I can think of two ways to deal with that: 1) Provide two APIs. The first API stops the subprocess upon function completion. The second API allows the parent to call other functions in the subprocess, but never more than one function at a time. 2) Always leave subprocesses running, but use a 'with' statement to guarantee the subprocess will be closed quickly. I prefer this option. I think my suggestion fits most of your objectives. > 1) It works by default, so that novices can use it without falling > into the deep well of threading. That is, a program that you write > using threading is broken by default, and the tool you have to fix it > is "inspection." I want something that allows me to say "this is a > task. Go." and have it work without the python programmer having to > study and understand several tomes on the subject. Done, IMHO. > 2) Tasks can be automatically distributed among processors, so it > solves the problems of (a) making python run faster (b) how to utilize > multiprocessor systems. Done. The OS automatically maps subprocesses to other processors. > 3) Tasks are cheap enough that I can make thousands of them, to solve > modeling problems (in which I also lump games). This is really a > solution to a cerain type of program complexity -- if I can just > assign a task to each logical modeling unit, it makes such a system > much easier to program. Perhaps the suggested module should have a queue-oriented API. Usage would look like this: import pyprocess queue = pyprocess.ProcessQueue(max_processes=4) task = queue.put('mypackage.mymodule', 'myfunc', arg1, arg2=5) Then, you can create as many tasks as you like; parallelism will be limited to 4 concurrent tasks. A variation of ProcessQueue might manage the concurrency limit automatically. > 4) Tasks are "self-guarding," so they prevent other tasks from > interfering with them. The only way tasks can communicate with each > other is through some kind of formal mechanism (something queue-ish, > I'd imagine). Done. Subprocesses have their own Python namespace. Subprocesses receive messages through function calls and send messages by returning from functions. > 5) Deadlock is prevented by default. I suspect livelock could still > happen; I don't know if it's possible to eliminate that. No locking is done at all. (That makes me uneasy, though; have I just moved locking problems to the application developer?) > 6) It's natural to make an object that is actor-ish. That is, this > concurrency approach works intuitively with objects. Anything pickleable is legal. > 7) Complexity should be eliminated as much as possible. If it requires > greater limitations on what you can do in exchange for a clear, > simple, and safe programming model, that sounds pythonic to me. The > way I see it, if we can't easily use tasks without getting into > trouble, people won't use them. But if we have a model that allows > people to (for example) make easy use of multiple processors, they > will use that approach and the (possible) extra overhead that you pay > for the simplicity will be absorbed by the extra CPUs. I think the solution is very simple. > 8) It should not exclude the possibility of mobile tasks/active > objects, ideally with something relatively straightforward such as > Linda-style tuple spaces. The proposed module could serve as a guide for a very similar module that sends tasks to other machines. Shane From mal at egenix.com Thu Sep 29 21:53:29 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 29 Sep 2005 21:53:29 +0200 Subject: [Python-Dev] C API doc fix In-Reply-To: References: <4092C34F-5A07-47D0-A27F-1781EBFE887A@lag.net> Message-ID: <433C4639.4000201@egenix.com> Steven Bethard wrote: > On 9/29/05, Robey Pointer wrote: > >>Yesterday I ran into a bug in the C API docs. The top of this page: >> >> http://docs.python.org/api/unicodeObjects.html >> >>says: >> >>Py_UNICODE >> This type represents a 16-bit unsigned storage type which is >>used by Python internally as basis for holding Unicode ordinals. On >>platforms where wchar_t is available and also has 16-bits, Py_UNICODE >>is a typedef alias for wchar_t to enhance native platform >>compatibility. On all other platforms, Py_UNICODE is a typedef alias >>for unsigned short. > > > I believe this is the same issue that was brought up in May[1]. My > impression was that people could not agree on a documentation patch. > > [1] http://www.python.org/dev/summary/2005-05-01_2005-05-15.html The problem was not so much getting the documentation, but the fact that Python builds as UCS4 version in case it finds a TCL version built for UCS4 - contrary to the UCS2 default that is documented. If I ever get around to working on my Python todo list, this is one of the things I'd like to restore - UCS4 should always be an *explicit* compile time option due to the consequences that go with it. Unfortunately, many Linux distros nowadays build Python with UCS4 -- introducing yet another dimension to binary Python binaries. In case you wonder, we now have these dimensions: * Python version (2.3, 2.4, ...) * OS version (Linux, Solaris, Windows, Mac OS X, ...) * Architecture (PowerPC, x86, x86_64, SunSPARC, ...) * Unicode variant (UCS2, UCS4) Finding the right binary for his or her Python is getting increasingly more complicated for the Python user (and we are seeing this every day in support requests). Something we might want to introduce in Python 2.5 is a short identifier in the Python interpreter interactive startup printout that provides easy to find values for all of the above dimensions. It already includes Python version and OS name, but is missing the other bits and pieces. Perhaps a flag that fires up Python and runs platform.py would help too. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 29 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From jonathan-lists at cleverdevil.org Thu Sep 29 22:06:34 2005 From: jonathan-lists at cleverdevil.org (Jonathan LaCour) Date: Thu, 29 Sep 2005 16:06:34 -0400 Subject: [Python-Dev] Pythonic concurrency In-Reply-To: <329633301.20050929074337@MailBlocks.com> References: <397621172.20050927111836@MailBlocks.com> <433AE8A8.3010500@v.loewis.de> <329633301.20050929074337@MailBlocks.com> Message-ID: > I'd like to restart this discussion; I didn't mean to put forth active > objects as "the" solution, only that it seems to be one of the better, > more OO solutions that I've seen so far. Thanks for doing this. I think this is an issue that is going to be more and more important as Python continues to gain momentum, and I would love to see a PEP come out of this discussion. > What I'd really like to figure out is the "pythonic" solution for > concurrency. Guido and I got as far as agreeing that it wasn't > threads. Evaluating the situation is often confusing for Python beginners because: 1. You won't often find threads being the recommended solution for concurrency in Python. There is some disagreement on this point, as the recent thread on the GIL reveals, but I think the point stands. 2. Multi-processing is often brought up as a good alternative to threading. 3. There are a decent number of built-in high level abstractions for threaded programming in Python (Queues, Thread objects, Lock objects, etc.) and plenty of documentation too. These abstractions also make it relatively straightforward to make your code work on multiple platforms. 4. The only support for multi-processing is quite low-level, and can be single platform. Forking isn't really an option on windows, and neither are named pipes. Shared memory? Forget it! Sockets could be used, but thats a bit low-level. Its really a shame. There seems to be some consensus about multi-processing, but not a whole lot of interest in making it easier out of the box. When it comes to multi-processing, batteries really _aren't_ included. Sure, you have lead dioxide and some sulphuric acid, but you have to put them together to make your battery. This isn't the end of the world, but I find it tedious, and I am sure it confuses and frustrates people new to Python. > Here are my own criteria for what such a solution would look like: > > 1) It works by default, so that novices can use it without falling > into the deep well of threading. That is, a program that you write > using threading is broken by default, and the tool you have to fix it > is "inspection." I want something that allows me to say "this is a > task. Go." and have it work without the python programmer having to > study and understand several tomes on the subject. > > 2) Tasks can be automatically distributed among processors, so it > solves the problems of (a) making python run faster (b) how to utilize > multiprocessor systems. > > 3) Tasks are cheap enough that I can make thousands of them, to solve > modeling problems (in which I also lump games). This is really a > solution to a cerain type of program complexity -- if I can just > assign a task to each logical modeling unit, it makes such a system > much easier to program. > > 4) Tasks are "self-guarding," so they prevent other tasks from > interfering with them. The only way tasks can communicate with each > other is through some kind of formal mechanism (something queue-ish, > I'd imagine). > > 5) Deadlock is prevented by default. I suspect livelock could still > happen; I don't know if it's possible to eliminate that. > > 6) It's natural to make an object that is actor-ish. That is, this > concurrency approach works intuitively with objects. > > 7) Complexity should be eliminated as much as possible. If it requires > greater limitations on what you can do in exchange for a clear, > simple, and safe programming model, that sounds pythonic to me. The > way I see it, if we can't easily use tasks without getting into > trouble, people won't use them. But if we have a model that allows > people to (for example) make easy use of multiple processors, they > will use that approach and the (possible) extra overhead that you pay > for the simplicity will be absorbed by the extra CPUs. > > 8) It should not exclude the possibility of mobile tasks/active > objects, ideally with something relatively straightforward such as > Linda-style tuple spaces. This all sounds pretty brilliant to me, although even a small subset of what you define above would be totally adequate I think. To me it breaks down simply into: 1. The ability to spawn/create "tasks" (which are really processes) easily, where tasks are isolated from each other. 2. The ability to send a message into a task. A formal queueing mechanism would be nice, but the simple ability to send messages is completely enough to roll your own queueing. Preventing deadlock is hard. Mobile tasks / active objects? Getting way ahead of ourselves, I think. Really, I just want those two things above, and I want it as a standard part of Python. > One thing that occurs to me is that a number of items on this wish > list may conflict with each other, which may require a different way > of thinking about the problem. For example, it may require two > approaches: for "ordinary" non-OO tasks, a functional programming > approach ala Erlang, in combination with an actor approach for > objects. I am not sure this is the case. I don't really think of concurrency in terms of objects or good "object-oriented" design all that much. I think of concurrency in terms of processes. Is this bad? I don't think so, really. The OS knows nothing of objects, only of threads and processes (or only of processes). If we had just the two items I mentioned above, it would be enough for people to build their own higher level abstractions on top of the solid built-in support. I haven't really done the research here, and the TSM paper that was sent earlier makes my brain hurt. It seems like its way more than is necessary to solve the problem. If we just accomplished the two things I stated above, I for one would be happy. Am I alone here? -- Jonathan -- http://cleverdevil.org From bob at redivi.com Thu Sep 29 22:10:21 2005 From: bob at redivi.com (Bob Ippolito) Date: Thu, 29 Sep 2005 16:10:21 -0400 Subject: [Python-Dev] C API doc fix In-Reply-To: <433C4639.4000201@egenix.com> References: <4092C34F-5A07-47D0-A27F-1781EBFE887A@lag.net> <433C4639.4000201@egenix.com> Message-ID: <4C928DB4-83C6-4538-8B64-6738CF0D2D7C@redivi.com> On Sep 29, 2005, at 3:53 PM, M.-A. Lemburg wrote: > Perhaps a flag that fires up Python and runs platform.py > would help too. python -mplatform -bob From fredrik at pythonware.com Thu Sep 29 22:08:37 2005 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 29 Sep 2005 22:08:37 +0200 Subject: [Python-Dev] C API doc fix References: <4092C34F-5A07-47D0-A27F-1781EBFE887A@lag.net> <433C4639.4000201@egenix.com> Message-ID: M.-A. Lemburg wrote: > * Unicode variant (UCS2, UCS4) don't forget the "Py_UNICODE is wchar_t" subvariant. From kozlovsky at mail.spbnit.ru Thu Sep 29 22:23:02 2005 From: kozlovsky at mail.spbnit.ru (Alexander J. Kozlovsky) Date: Fri, 30 Sep 2005 00:23:02 +0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <20050929114321.GA11086@panix.com> <000d01c5c4f1$085e7e20$d63dc797@oemcomputer> Message-ID: <607658704.20050930002302@mail.spbnit.ru> Guido van Rossum wrote: > I did this for my favorite proposal, and ended up with the list shown > further down below. > > I think they all looks great! I'm sorry for my bad English IMHO, if condition is nontrivial, then the syntax: expr1 if some complex condition else expr2 can be more hard to read as compared with: if some complex condition then expr1 else expr2 In the second form visual comparison of expr1 and expr2 is simpler because its adjacent placement. For example, in expression 'DO' if cmd == DO and flag == OK or x is None else 'DONT' 'DO' and 'DONT' visually divided and eyes must move there and back for find the difference between them. As opposite, in expression if cmd == DO and flag == OK or x is None then 'DO' else 'DONT' it is very easy to determine what is possible expression output. It is a bit more readable from my point of view Best regards, Alexander mailto:kozlovsky at mail.spbnit.ru From mcherm at mcherm.com Thu Sep 29 22:31:46 2005 From: mcherm at mcherm.com (Michael Chermside) Date: Thu, 29 Sep 2005 13:31:46 -0700 Subject: [Python-Dev] Adding a conditional expression in Py3.0 Message-ID: <20050929133146.9v49l9yf9q9s0kww@login.werra.lunarpages.com> [ongoing discussion of conditional expressions] I waited until I had caught up on my reading before saying anything. Now I'll express my opinion in a single posting then keep quiet unless I actually find I have something novel to contribute (difficult in a topic that's been talked to death 3 or 4 times in the past few years). * I am STRONGLY in favor of introducing a conditional expression in some form. The use of the "and-or" trick is proof enough to me that there is a need for it. I can marshall other arguments also, but they appear unnecessary. * The syntax to use is a place where we need a BDFL decision. Make the decision and we'll all agree and move on. Any decision would be better than an eternity of re-discovering old ideas again and again. * I think the two best options are trueval if cond else falseval and if cond then trueval else falseval The first has brevity in it's favor, and "cleverness" which might be an advantage or disadvantage depending on your point of view. The second has order-of-arguments in its favor. In either case, wise programmers will use parenthesees when it aids clarity (imagine "v1 if c else v2 + v3"). Whether we require parenthesees when the parser could disambiguate on its own is really up to Guido. * I prefer the second form ("if cond then trueval else falseval") because it puzzles _nobody_ at the expense of being slightly more wordy and less clever. But that's just one person's opinion. Thanks Guido, for maintaining your patience in the face of this discussion. -- Michael Chermside From mal at egenix.com Thu Sep 29 22:36:47 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 29 Sep 2005 22:36:47 +0200 Subject: [Python-Dev] C API doc fix In-Reply-To: <4C928DB4-83C6-4538-8B64-6738CF0D2D7C@redivi.com> References: <4092C34F-5A07-47D0-A27F-1781EBFE887A@lag.net> <433C4639.4000201@egenix.com> <4C928DB4-83C6-4538-8B64-6738CF0D2D7C@redivi.com> Message-ID: <433C505F.7070403@egenix.com> Bob Ippolito wrote: > On Sep 29, 2005, at 3:53 PM, M.-A. Lemburg wrote: > > >>Perhaps a flag that fires up Python and runs platform.py >>would help too. > > > python -mplatform Cool :-) Now we only need to add some more information to it (like e.g. the Unicode variant). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 29 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Thu Sep 29 22:39:59 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 29 Sep 2005 22:39:59 +0200 Subject: [Python-Dev] C API doc fix In-Reply-To: References: <4092C34F-5A07-47D0-A27F-1781EBFE887A@lag.net> <433C4639.4000201@egenix.com> Message-ID: <433C511F.5060506@egenix.com> Fredrik Lundh wrote: > M.-A. Lemburg wrote: > > >>* Unicode variant (UCS2, UCS4) > > > don't forget the "Py_UNICODE is wchar_t" subvariant. True, but that's not relevant for binary compatibility of Python package (at least not AFAIK). UCS2 vs. UCS4 matters because the two versions use and expose different C APIs and thus an extension written for UCS2 doesn't run with a Python built for UCS4 and vice-versa. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 29 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From tjreedy at udel.edu Thu Sep 29 23:15:36 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 29 Sep 2005 17:15:36 -0400 Subject: [Python-Dev] Adding a conditional expression in Py3.0 References: <20050929114321.GA11086@panix.com><000d01c5c4f1$085e7e20$d63dc797@oemcomputer> Message-ID: "Guido van Rossum" wrote in message news:ca471dc2050929073236fae78c at mail.gmail.com... > After all there's nothing wrong with and/or. This is one reason 'no addition' got a relatively high rank in the vote. Examples... > telnet~1.py: 'DO' if cmd == DO else 'DONT', versus cmd == DO and 'DO' or 'DONT' I still stronly prefer this order and even slightly prefer this form. I might even prefer a complete inversion of the order: x else y if c 'DO' else 'DONT' if cmd != DO except that that does not chain at all well. Terry J. Reedy From steve at holdenweb.com Thu Sep 29 23:32:06 2005 From: steve at holdenweb.com (Steve Holden) Date: Thu, 29 Sep 2005 22:32:06 +0100 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <20050929114321.GA11086@panix.com> <000d01c5c4f1$085e7e20$d63dc797@oemcomputer> Message-ID: Guido van Rossum wrote: > On 9/29/05, Steve Holden wrote: > >>I would argue for mandatory parentheses around the expression, leaving >>room later (probably after Guido is no longer around to be sick at the >>site of it) for: >> >>def f(condition): >> return something if condition # no else! >> return somethingElse >> >>not-expecting-this-to-fly-ly y'rs - steve > > > Let me give you what you expect. If all the "X if C else Y" syntax > does is prevent that atrocity from ever being introduced, it would be > worth it. :) > Well, fine. However, it does allow atrocities like func(f for f in lst if f > -1 if f < 0 else +1) I realise that any chosen syntax is subject to abuse, but a conditional expression in a (currently allowed) conditional context will be guaranteed obscure. Your original instinct to omit conditional expressions was right! far-too-late-ly y'rs - steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ From guido at python.org Fri Sep 30 00:24:58 2005 From: guido at python.org (Guido van Rossum) Date: Thu, 29 Sep 2005 15:24:58 -0700 Subject: [Python-Dev] David Mertz on CA state e-voting panel Message-ID: http://yro.slashdot.org/yro/05/09/28/1955256.shtml?tid=103&tid=17&tid=219 Congratulations David! (Don't have his email.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jcarlson at uci.edu Fri Sep 30 00:44:34 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu, 29 Sep 2005 15:44:34 -0700 Subject: [Python-Dev] PEP 350: Codetags In-Reply-To: <20050928161039.GF10940@kitchen.client.attbi.com> References: <20050926223521.GE10940@kitchen.client.attbi.com> <20050928161039.GF10940@kitchen.client.attbi.com> Message-ID: <20050929153237.97E1.JCARLSON@uci.edu> Micah Elliott wrote: > Josiah> an unofficial spec is sufficient. See koders.com and search > Josiah> for 'fixme' to see some common variants. > > But that's the problem -- there are already a bunch of "unofficial" > specs, which don't serve much purpose as such. It's a cool site. I > spent some time browsing and I do see a lot of codetags in use (many > thousands in Python code alone; I'm not sure if the number represented > strings or files). But I'll argue that this just reinforces the need > for an *official* spec/guideline, so that the tools can do something > with them. Defining a spec for code tags doesn't mean that people will start using them. Why? Because it is a documentation spec. From my experience, documentation specs are only adhered to by the organizations (companies, groups, etc.) which the code is produced by and for, and they generally define the code specs for their organization. Further, even if it becomes a spec, it doesn't guarantee implementation in Python editors (for which you are shooting for). Take a wander through current implementations of code tags in various editors to get a feel for what they support. I've read various posts about what code tags could support, but not what editors which implement code tags and/or its variants actually currently support; which is a better indication of what people want than an informal survey via email of python-dev, python-list, and/or the PEP submission process. - Josiah From jimjjewett at gmail.com Fri Sep 30 01:05:51 2005 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 29 Sep 2005 19:05:51 -0400 Subject: [Python-Dev] Pythonic concurrency In-Reply-To: References: Message-ID: Bruce Eckel wrote: > 3) Tasks are cheap enough that I can make > thousands of them, ... > 4) Tasks are "self-guarding," so they prevent > other tasks from interfering with them. The > only way tasks can communicate with each > other is through some kind of formal > mechanism (something queue-ish, > I'd imagine). I think these two are the hardest to reconcile. Shane Hathaway's suggestion starts from the process end. A new process isn't cheap. Keeping the other process' interpreter alive and feeding it more requests through a queue just hides the problem; you can't have more non-sequential tasks than processors without restarting the whole contention issue. Even using sequential tasks (similar to "import dummy_thread") lets task1 mess up the builtins (or other library modules) for future task2. The more guards you add, the heavier each task gets. At the other end are generators; I think what generators are missing is (A) You can't easily send them a message. This can be solved by wrapping them in an object, or (probably) by waiting until 2.5. (B) The programmer has to supply a scheduler. This could be solved by a standard library module. (C) That scheduler is non-preemptive. A single greedy generator can starve all the others. You can reduce the problem by scheduling generators on more than one thread. To really solve it would require language support. That *might* be almost as simple as a new object type that automatically yielded control every so often (like threads). (D) Generators can interfere with each other unless the programmer is careful to treat all externally visible objects as immutable. Again, it would require language support. I also have a vague feeling that fixing (C) or (D) might make the other worse. (E) Generators are not reentrant. I know you said that some restrictions are reasonable, and this might fall into that category ... but I think this becomes a problem as soon as Tasks can handle more than one type of message, or can send delayed replies to specific correspondents. (To finish your request, I need some more information...) -jJ From guido at python.org Fri Sep 30 01:18:29 2005 From: guido at python.org (Guido van Rossum) Date: Thu, 29 Sep 2005 16:18:29 -0700 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <20050929114321.GA11086@panix.com> <000d01c5c4f1$085e7e20$d63dc797@oemcomputer> Message-ID: [Guido] > > Let me give you what you expect. If all the "X if C else Y" syntax > > does is prevent that atrocity from ever being introduced, it would be > > worth it. :) [Steve] > Well, fine. However, it does allow atrocities like > > func(f for f in lst if f > -1 if f < 0 else +1) No it doesn't! Inside an 'if' (of any flavor), further ifs have to be nested. So you'd have to write func(f for f in lst if f > (-1 if f < 0 else +1)) or perhaps func(f for f in lst if (f > -1 if f < 0 else +1)) But I doubt you meant to write +1 where True could have sufficed. :) An if-else expression has lower priority than anything else except lambda; the expression lambda x: x if x >= 0 else -x is equivalent to lambda x: (x if x >= 0 else -x) > I realise that any chosen syntax is subject to abuse, but a conditional > expression in a (currently allowed) conditional context will be > guaranteed obscure. Your original instinct to omit conditional > expressions was right! Now you've pushed me over the edge. I've made up my mind now, "X if C else Y" it will be. I hope to find time to implement it in Python 2.5. Let it be as controversial as bool or @decorator, I don't care. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ejones at uwaterloo.ca Fri Sep 30 02:28:46 2005 From: ejones at uwaterloo.ca (Evan Jones) Date: Thu, 29 Sep 2005 20:28:46 -0400 Subject: [Python-Dev] [PATCH][BUG] Segmentation Fault in xml.dom.minidom.parse Message-ID: <542c7cf582799e73bfc0a189918125b8@uwaterloo.ca> The following Python script causes Python 2.3, 2.4 and the latest CVS to crash with a Segmentation Fault: import xml.dom.minidom x = u'\n Comment \xe7a va ? Tr\xe8s bien ? ' dom = xml.dom.minidom.parseString( x.encode( 'latin_1' ) ) print repr( dom.childNodes[0].localName ) The problem is that this XML document does not specify an encoding. In this case, minidom assumes that it is encoded in UTF-8. However, in fact it is encoded in Latin-1. My two line patch, in the SourceForge tracker at the URL below, causes this to raise a UnicodeDecodingError instead. http://sourceforge.net/tracker/index.php? func=detail&aid=1309009&group_id=5470&atid=305470 Any chance that someone wants to commit this tiny two line fix? This might be the kind of fix that might be elegible to be backported to Python 2.4 as well. It passes "make test" on both my Linux system and my Mac. I've also attached a patch that adds this test case to test_minidom.py. Thanks, Evan Jones -- Evan Jones http://evanjones.ca/ From greg.ewing at canterbury.ac.nz Fri Sep 30 02:40:55 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 30 Sep 2005 12:40:55 +1200 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: <433C0A83.8030001@gmail.com> References: <20050929114321.GA11086@panix.com> <000d01c5c4f1$085e7e20$d63dc797@oemcomputer> <1128006090.12226.36.camel@p-dvsi-418-1.rd.francetelecom.fr> <433C0A83.8030001@gmail.com> Message-ID: <433C8997.4020209@canterbury.ac.nz> Nick Coghlan wrote: > i.e., it's > like an "or", only we're asking that the decision be made based on something > other than the truth value of the left hand operand. Hmmm, then maybe it should be a or c if b or perhaps a or, if b, c :-) -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From guido at python.org Fri Sep 30 03:21:53 2005 From: guido at python.org (Guido van Rossum) Date: Thu, 29 Sep 2005 18:21:53 -0700 Subject: [Python-Dev] Conditional Expression Resolution Message-ID: After a long discussion I've decided to add a shortcut conditional expression to Python 2.5. The syntax will be A if C else B This first evaluates C; if it is true, A is evaluated to give the result, otherwise, B is evaluated to give the result. The priorities will be such that you can write x = A if C else B x = lambda: A if C else B x = A if C else B if D else E But you'd have to write if (A if C else B): [x for x in seq if (A if C else B)] A if (X if C else Y) else B (A if C else B) if D else E Note that all these are intentionally ugly. :) In general, 'if' and 'else' bind less tight than everything except lambda. We will adjust the syntax of what goes inside an 'if' to disallow lambda; currently if lambda: x: is accepted but quite useless (it's always true) so this will be disallowed. Flames, pleas to reconsider, etc., to /dev/null. Congratulations gracefully accepted. It's still my language! :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nnorwitz at gmail.com Fri Sep 30 03:43:34 2005 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 29 Sep 2005 18:43:34 -0700 Subject: [Python-Dev] Conditional Expression Resolution In-Reply-To: References: Message-ID: On 9/29/05, Guido van Rossum wrote: > > Flames, pleas to reconsider, etc., to /dev/null. > > Congratulations gracefully accepted. > > It's still my language! :-) I don't like those options. :-) How about this: Can someone update the PEP on conditional expressions to point to this email and update the status, etc? Thanks, n From raymond.hettinger at verizon.net Fri Sep 30 03:49:19 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Thu, 29 Sep 2005 21:49:19 -0400 Subject: [Python-Dev] Conditional Expression Resolution In-Reply-To: Message-ID: <000001c5c561$2d556d40$a622a044@oemcomputer> > How about this: > Can someone update the PEP on conditional expressions to point to > this email and update the status, etc? Already done. Raymond From steven.bethard at gmail.com Fri Sep 30 04:30:03 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Thu, 29 Sep 2005 20:30:03 -0600 Subject: [Python-Dev] Conditional Expression Resolution In-Reply-To: References: Message-ID: On 9/29/05, Guido van Rossum wrote: > After a long discussion I've decided to add a shortcut conditional > expression to Python 2.5. > > The syntax will be > > A if C else B > [snip] > > Congratulations gracefully accepted. Congratulations, and many thanks for making this decision before the threads leaked into the next summary period. ;-) STeVe -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From tdelaney at avaya.com Fri Sep 30 04:34:25 2005 From: tdelaney at avaya.com (Delaney, Timothy (Tim)) Date: Fri, 30 Sep 2005 12:34:25 +1000 Subject: [Python-Dev] Conditional Expression Resolution Message-ID: <2773CAC687FD5F4689F526998C7E4E5F4DB688@au3010avexu1.global.avaya.com> Guido van Rossum wrote: > Congratulations gracefully accepted. Whilst I'm not personally fond of the syntax, congratulations. Whilst I don't feel this is a big step in the evolution of the language, it shuts up a *very* frequently asked question (and opens up a new one ;) Tim Delaney From tismer at stackless.com Fri Sep 30 05:49:42 2005 From: tismer at stackless.com (Christian Tismer) Date: Fri, 30 Sep 2005 05:49:42 +0200 Subject: [Python-Dev] Conditional Expression Resolution In-Reply-To: References: Message-ID: <433CB5D6.40400@stackless.com> Guido van Rossum wrote: ... > Flames, pleas to reconsider, etc., to /dev/null. > > Congratulations gracefully accepted. Congratulations for a clear decision! Personally, I would have preferred a "no, never in my life" alternative response slightly more, but a clear say is better than endless discussions about something as relevant as syntax. My canned reply, tailored to fit into the set of accepted answers reduces to "Thank you so much :-) ". > It's still my language! :-) Self-inconfidence alert! You would never need to say this if you not felt like a bot. Please shut down and get adjusted. Guidos don't say that. :-) -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From kbk at shore.net Fri Sep 30 06:34:16 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Fri, 30 Sep 2005 00:34:16 -0400 (EDT) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200509300434.j8U4YGHG005552@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 337 open ( +0) / 2947 closed ( +6) / 3284 total ( +6) Bugs : 912 open ( +4) / 5278 closed (+16) / 6190 total (+20) RFE : 195 open ( +1) / 187 closed ( +0) / 382 total ( +1) New / Reopened Patches ______________________ fix for distutils "upload" command (2005-09-23) http://python.org/sf/1299675 opened by Richard Jones xdrlib.py: pack_fstring() did not use null bytes for padding (2005-09-23) CLOSED http://python.org/sf/1300515 opened by Don Quijote desktop module (providing startfile as open, all platforms) (2005-09-23) http://python.org/sf/1301512 opened by Paul Boddie Update documentation for __builtins__ (2005-09-24) http://python.org/sf/1303595 opened by Collin Winter Strobe Timer for guis (2005-09-26) http://python.org/sf/1304895 opened by Brett Rosen pyexpat.c: Two line fix for decoding crash (2005-09-29) http://python.org/sf/1309009 opened by Evan Jones Patches Closed ______________ cjkcodecs does not initialize type pointer (2005-09-20) http://python.org/sf/1297028 closed by nnorwitz xdrlib.py: pack_fstring() did not use null bytes for padding (2005-09-23) http://python.org/sf/1300515 closed by birkenfeld vendor-packages directory. (2005-09-22) http://python.org/sf/1298835 closed by richburridge PyOS_InputHook inconsistency on Windows (2004-10-19) http://python.org/sf/1049855 closed by loewis FreeBSD 5.3 and upward has thread-safe getaddrinfo(3) (2005-09-13) http://python.org/sf/1288833 closed by perky use LIST_APPEND opcode for X.append() method calls (2005-09-16) http://python.org/sf/1292625 closed by nnorwitz New / Reopened Bugs ___________________ A command history for the idle interactive shell (2005-09-24) http://python.org/sf/1302267 opened by Bj?rn Lindqvist 2.4.1 windows MSI has no _socket (2005-09-24) http://python.org/sf/1302793 opened by IamPaul SSL-ed sockets don't close correct? (2004-06-24) http://python.org/sf/978833 reopened by kxroberto Bypassing __dict__ readonlyness (2005-09-24) http://python.org/sf/1303614 opened by Armin Rigo traceback on trying to load a file (2005-09-24) http://python.org/sf/1303673 opened by nanotube Extended slice bug (or feature?) (2005-09-25) CLOSED http://python.org/sf/1303928 reopened by mwh Extended slice bug (or feature?) (2005-09-25) CLOSED http://python.org/sf/1303928 opened by David M. Beazley 2.3.5 configure / make infinite loop (2005-09-25) http://python.org/sf/1304179 opened by Michael Schwartz windows unicode write is messing up the EOL. (2005-09-27) CLOSED http://python.org/sf/1305706 opened by Istvan Visegradi Syntax error in Python Library Reference, 6.1.4. Files ... (2005-09-27) CLOSED http://python.org/sf/1306043 opened by Alexander Schliep Python 2.4.2c1 fails to build for 64-bit Solaris 9/10 (2005-09-27) CLOSED http://python.org/sf/1306211 opened by John Stone Add 64-bit Solaris 9 build instructions to README (2005-09-27) http://python.org/sf/1306248 opened by John Stone Python 2.4.2c1 fails to build on 64-bit Solaris 10 (2005-09-27) http://python.org/sf/1306253 opened by John Stone PyString_AsStringAndSize() return value documented wroing (2005-09-28) CLOSED http://python.org/sf/1306449 opened by Gregory Bond compile() converts "filename" parameter to StringType (2005-09-28) http://python.org/sf/1306484 opened by V?gv?lgyi Attila Augmented assigment to mutable objects in tuples fail (2005-09-28) CLOSED http://python.org/sf/1306777 opened by Mattias Engdeg?rd =, not = (2005-09-28) CLOSED http://python.org/sf/1307188 opened by Davide Andrea Datagram Socket Timeouts (2005-09-28) http://python.org/sf/1307357 opened by Tom Vrankar subprocess.Popen locks on Cygwin (2005-09-29) http://python.org/sf/1307798 opened by Jacek Poplawski PCbuild vcproj project files need a cleanup (2005-09-29) http://python.org/sf/1307806 opened by Adal Chiriliuc Unsatisfied symbols: _PyGILState_NoteThreadState (code) (2005-09-29) http://python.org/sf/1307978 opened by Goetz Isenmann Datagram Socket Timeouts (2005-09-29) http://python.org/sf/1308042 opened by Tom Vrankar Py_BuildValue (C/API): "K" format (2005-09-29) CLOSED http://python.org/sf/1308740 opened by Yair Chuchem Bugs Closed ___________ Incorrect return type for search() method (2005-09-20) http://python.org/sf/1297059 closed by nnorwitz build fails on Solaris 10 for Objects/complexobject.c (2005-09-22) http://python.org/sf/1299187 closed by nnorwitz test_ioctl fails (2003-07-26) http://python.org/sf/777867 closed by birkenfeld test_ioctl fails (2003-07-31) http://python.org/sf/780576 closed by birkenfeld subprocesss module retains older license header (2005-02-17) http://python.org/sf/1138653 closed by astrand inspect.getsource() misses single line blocks. (2005-09-20) http://python.org/sf/1295909 closed by arigo Extended slice bug (or feature?) (2005-09-25) http://python.org/sf/1303928 closed by birkenfeld Extended slice bug (or feature?) (2005-09-25) http://python.org/sf/1303928 closed by birkenfeld windows unicode write is messing up the EOL. (2005-09-27) http://python.org/sf/1305706 closed by visike Syntax error in Python Library Reference, 6.1.4. Files ... (2005-09-27) http://python.org/sf/1306043 closed by birkenfeld Python 2.4.2c1 fails to build for 64-bit Solaris 9/10 (2005-09-27) http://python.org/sf/1306211 deleted by jestone PyString_AsStringAndSize() return value documented wrong (2005-09-28) http://python.org/sf/1306449 closed by birkenfeld Augmented assigment to mutable objects in tuples fail (2005-09-28) http://python.org/sf/1306777 closed by pje =, not = (2005-09-28) http://python.org/sf/1307188 closed by birkenfeld Py_BuildValue (C/API): "K" format (2005-09-29) http://python.org/sf/1308740 closed by birkenfeld "Howto RE" link is dead (2005-09-22) http://python.org/sf/1299051 closed by birkenfeld MemoryError in httplib (2005-09-20) http://python.org/sf/1296004 closed by birkenfeld shlex does not support unicode (2005-09-21) http://python.org/sf/1298120 closed by birkenfeld BaseHTTPServer to take host from cmdline? (2004-07-06) http://python.org/sf/985651 closed by birkenfeld New / Reopened RFE __________________ Please include pdb with windows distribution (2005-09-24) http://python.org/sf/1303434 opened by Lyle Thompson From nnorwitz at gmail.com Fri Sep 30 07:02:14 2005 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 29 Sep 2005 22:02:14 -0700 Subject: [Python-Dev] [PATCH][BUG] Segmentation Fault in xml.dom.minidom.parse In-Reply-To: <542c7cf582799e73bfc0a189918125b8@uwaterloo.ca> References: <542c7cf582799e73bfc0a189918125b8@uwaterloo.ca> Message-ID: On 9/29/05, Evan Jones wrote: > > Any chance that someone wants to commit this tiny two line fix? That's reallly not the question you *want* to be asking. :-) > This might be the kind of fix that might be elegible to be backported to > Python 2.4 as well. It passes "make test" on both my Linux system and > my Mac. I've also attached a patch that adds this test case to > test_minidom.py. There was a problem your patch exposed. I fixed that too. I even backported it. Thanks, n From steve at holdenweb.com Fri Sep 30 08:54:46 2005 From: steve at holdenweb.com (Steve Holden) Date: Fri, 30 Sep 2005 07:54:46 +0100 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <20050929114321.GA11086@panix.com> <000d01c5c4f1$085e7e20$d63dc797@oemcomputer> Message-ID: <433CE136.9030609@holdenweb.com> Guido van Rossum wrote: > [Guido] > >>>Let me give you what you expect. If all the "X if C else Y" syntax >>>does is prevent that atrocity from ever being introduced, it would be >>>worth it. :) > > > [Steve] > >>Well, fine. However, it does allow atrocities like >> >>func(f for f in lst if f > -1 if f < 0 else +1) > > > No it doesn't! Inside an 'if' (of any flavor), further ifs have to be > nested. So you'd have to write > > func(f for f in lst if f > (-1 if f < 0 else +1)) > > or perhaps > > func(f for f in lst if (f > -1 if f < 0 else +1)) > > But I doubt you meant to write +1 where True could have sufficed. :) > :) All that said, inside an "if" is hardly the best place for a conditional of any kind. Clearly such usage can go down as abuse. > An if-else expression has lower priority than anything else except > lambda; the expression > > lambda x: x if x >= 0 else -x > > is equivalent to > > lambda x: (x if x >= 0 else -x) > That's about the only way it would make sense, I suppose. > >>I realise that any chosen syntax is subject to abuse, but a conditional >>expression in a (currently allowed) conditional context will be >>guaranteed obscure. Your original instinct to omit conditional >>expressions was right! > > > Now you've pushed me over the edge. I've made up my mind now, "X if C > else Y" it will be. I hope to find time to implement it in Python 2.5. > Let it be as controversial as bool or @decorator, I don't care. > Not before time, either. If this ends the discussion then I'll consider I've done everyone a favour. Sometimes no decision is worse than the wrong decision ;-). regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ From steve at holdenweb.com Fri Sep 30 08:54:46 2005 From: steve at holdenweb.com (Steve Holden) Date: Fri, 30 Sep 2005 07:54:46 +0100 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <20050929114321.GA11086@panix.com> <000d01c5c4f1$085e7e20$d63dc797@oemcomputer> Message-ID: <433CE136.9030609@holdenweb.com> Guido van Rossum wrote: > [Guido] > >>>Let me give you what you expect. If all the "X if C else Y" syntax >>>does is prevent that atrocity from ever being introduced, it would be >>>worth it. :) > > > [Steve] > >>Well, fine. However, it does allow atrocities like >> >>func(f for f in lst if f > -1 if f < 0 else +1) > > > No it doesn't! Inside an 'if' (of any flavor), further ifs have to be > nested. So you'd have to write > > func(f for f in lst if f > (-1 if f < 0 else +1)) > > or perhaps > > func(f for f in lst if (f > -1 if f < 0 else +1)) > > But I doubt you meant to write +1 where True could have sufficed. :) > :) All that said, inside an "if" is hardly the best place for a conditional of any kind. Clearly such usage can go down as abuse. > An if-else expression has lower priority than anything else except > lambda; the expression > > lambda x: x if x >= 0 else -x > > is equivalent to > > lambda x: (x if x >= 0 else -x) > That's about the only way it would make sense, I suppose. > >>I realise that any chosen syntax is subject to abuse, but a conditional >>expression in a (currently allowed) conditional context will be >>guaranteed obscure. Your original instinct to omit conditional >>expressions was right! > > > Now you've pushed me over the edge. I've made up my mind now, "X if C > else Y" it will be. I hope to find time to implement it in Python 2.5. > Let it be as controversial as bool or @decorator, I don't care. > Not before time, either. If this ends the discussion then I'll consider I've done everyone a favour. Sometimes no decision is worse than the wrong decision ;-). regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ From theller at python.net Fri Sep 30 09:17:39 2005 From: theller at python.net (Thomas Heller) Date: Fri, 30 Sep 2005 09:17:39 +0200 Subject: [Python-Dev] linecache problem Message-ID: On several occasions I have seen tracebacks in my code pointing to PIL's __init__.py file. That is strange, because I have installed PIL but it is used nowhere. Finally I traced it down to a problem in the linecache code, which tries to be smart in up updatecache function. When os.stat() on the filename fails with os.error, it walks along sys.path and returns the first file with a matching basename. This *may* make sense for toplevel modules, but never for modules in packages. So, if the traceback's stack contains an entry for a non-existing file (for example because the .py file for a .pyc file is no longer present), linecache returns absolute garbage. Example, on my system (the \a\b\c\__init__.py file doesn't exist): C:\>python -c "import linecache; print linecache.getlines(r'\a\b\c\__init__.py')" ['#\n', '# The Python Imaging Library.\n', '# $Id: //modules/pil/PIL/__init__.py#2 $\n', '#\n', '# package placeholder\n', '#\n', '# Copyright (c) 1999 by Secret Labs AB.\n', '#\n', '# See the README file for information on usage and redistribution.\n', '#\n', '\n', '# ;-)\n'] C:\> The bug is present in 2.3, 2.4, and current CVS. Thomas From ncoghlan at gmail.com Fri Sep 30 10:41:03 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 30 Sep 2005 18:41:03 +1000 Subject: [Python-Dev] PEP 350: Codetags In-Reply-To: <20050929153237.97E1.JCARLSON@uci.edu> References: <20050926223521.GE10940@kitchen.client.attbi.com> <20050928161039.GF10940@kitchen.client.attbi.com> <20050929153237.97E1.JCARLSON@uci.edu> Message-ID: <433CFA1F.4010804@gmail.com> Josiah Carlson wrote: > Further, even if it becomes a spec, it doesn't guarantee implementation > in Python editors (for which you are shooting for). Take a wander > through current implementations of code tags in various editors to get a > feel for what they support. I've read various posts about what code > tags could support, but not what editors which implement code tags > and/or its variants actually currently support; which is a better > indication of what people want than an informal survey via email of > python-dev, python-list, and/or the PEP submission process. An approach to this area that would make sense to me is: 1. Defer PEP 350 2. Publish a simple Python module for finding and processing code tags in a configurable fashion 3. Include a default configuration in the module that provides the behaviour described in PEP 350 4. After this hypothetical code tag processing module has been out in the wild for a while, re-open PEP 350 with an eye to including the module in the standard library The idea is that it should be possible to tailor the processing module in order to textually scan a codebase (possibly C or C++ rather than Python) in accordance with a project-specific system of code tagging, rather than requiring that the project necessarily use the default style included in the processing module (Although using a system other than the default one may result in reduced functionality, naturally). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From ncoghlan at gmail.com Fri Sep 30 10:48:13 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 30 Sep 2005 18:48:13 +1000 Subject: [Python-Dev] Conditional Expression Resolution In-Reply-To: References: Message-ID: <433CFBCD.4000105@gmail.com> Guido van Rossum wrote: > Congratulations gracefully accepted. Hurrah! Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From phd at mail2.phd.pp.ru Fri Sep 30 11:24:06 2005 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Fri, 30 Sep 2005 13:24:06 +0400 Subject: [Python-Dev] linecache problem In-Reply-To: References: Message-ID: <20050930092406.GD3411@phd.pp.ru> On Fri, Sep 30, 2005 at 09:17:39AM +0200, Thomas Heller wrote: > On several occasions I have seen tracebacks in my code pointing to PIL's > __init__.py file. That is strange, because I have installed PIL but it > is used nowhere. [skip] > The bug is present in 2.3, 2.4, and current CVS. I have seen such tracebacks in all versions of Python AFAIR. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From solipsis at pitrou.net Fri Sep 30 11:45:17 2005 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 30 Sep 2005 11:45:17 +0200 Subject: [Python-Dev] Pythonic concurrency - cooperative MT In-Reply-To: References: Message-ID: <1128073517.25688.10.camel@p-dvsi-418-1.rd.francetelecom.fr> > (C) That scheduler is non-preemptive. A single > greedy generator can starve all the others. Instead of looking at this as a problem, you could look at it as a feature. Since generators can't be switched at arbitrary places, the programmer has to define his/her synchronization points explicitly. Let me contrast this: - in preemptive MT, threads can switch at every moment by default, and you have to explicitly wrap some pieces of code in critical sections (or other primitives) to protect the semantics of your program; every time you forget a critical section, you introduce a bug - in cooperative MT, threads can only switch where the programmer explicitly allows it; every time you forget a synchronization point, you give your program worse performance (latency), but you *don't* introduce a bug So you have a scheme where you seek optimal performance (latency) but you take the risk of a huge number of difficult bugs, and you have a scheme where good performance needs more careful coding *but* the paradigm avoids difficult bugs /by construction/. By the way, the cooperative MT is exactly the Twisted approach, except implemented in a different manner (event loop instead of explicit cooperative tasks). Regards Antoine. From theller at python.net Fri Sep 30 11:49:42 2005 From: theller at python.net (Thomas Heller) Date: Fri, 30 Sep 2005 11:49:42 +0200 Subject: [Python-Dev] linecache problem References: <20050930092406.GD3411@phd.pp.ru> Message-ID: Oleg Broytmann writes: > On Fri, Sep 30, 2005 at 09:17:39AM +0200, Thomas Heller wrote: >> On several occasions I have seen tracebacks in my code pointing to PIL's >> __init__.py file. That is strange, because I have installed PIL but it >> is used nowhere. > [skip] >> The bug is present in 2.3, 2.4, and current CVS. > > I have seen such tracebacks in all versions of Python AFAIR. I think it's a severe bug that needs to be fixed. Tracebacks showing the wrong sourcelines leave the impression 'the Python installation is totally broken'. Thomas From p.f.moore at gmail.com Fri Sep 30 12:02:15 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 30 Sep 2005 11:02:15 +0100 Subject: [Python-Dev] Pythonic concurrency In-Reply-To: References: Message-ID: <79990c6b0509300302x2ebc704nc06a73458c604e78@mail.gmail.com> On 9/30/05, Jim Jewett wrote: > Bruce Eckel wrote: > > > 3) Tasks are cheap enough that I can make > > thousands of them, ... > > > 4) Tasks are "self-guarding," so they prevent > > other tasks from interfering with them. The > > only way tasks can communicate with each > > other is through some kind of formal > > mechanism (something queue-ish, > > I'd imagine). > > I think these two are the hardest to reconcile. Agreed. I think the biggest problem is that (4) is too strong. At the OS level, certain issues (such as the current directory, or the stdio streams) are per-process, so that taking (4) at face value requires multiprocessing, which violates (3)... If you constrain (4), you can get something much more effective. For example, if you accept that certain things are volatile (start with OS-specified per-process stuff, plus the Python builtins, maybe) then it's much easier to produce solutions. I don't think that shared state is that big an issue per se - after all, we're all used to the idea that the contents of a file might change "behind our backs". The real issue is not having a *clearly defined* shared state. The big problem with threads is that the shared state is *everything*. There's no partitioning at all. Every thread-based abstraction is based around threads voluntarily restricting themselves to a limited set of "safe" communication operations, and otherwise scrupulously avoiding each other's space ("Don't sit there, that's auntie Mary's chair"...) If we had an abstraction (multiple interpreters, anyone?) which still had a shared state, but a much smaller one, which was defined clearly, such that "unsafe" operations were easy to identify, then day-to-day concurrent programming would be a lot easier. Yes, it's still possible to break things, but people can do that already by hacking about with builtins, or abusing the introspection API, and it's not made Python unusable, because they generally don't... This seems to me to be a perfect case for a "Practicality beats Purity" approach. Paul. From p.f.moore at gmail.com Fri Sep 30 12:25:49 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 30 Sep 2005 11:25:49 +0100 Subject: [Python-Dev] Conditional Expression Resolution In-Reply-To: References: Message-ID: <79990c6b0509300325p6d2f5f88t5c285c2996afc2f5@mail.gmail.com> On 9/30/05, Guido van Rossum wrote: > Flames, pleas to reconsider, etc., to /dev/null. No flames from here. > Congratulations gracefully accepted. Consider them supplied. For both your patience, and for supplying the decision we all desperately needed. > It's still my language! :-) It always was :-) And just to prove it, we ended up with your original instinct after all... Paul. From barry at python.org Fri Sep 30 14:42:07 2005 From: barry at python.org (Barry Warsaw) Date: Fri, 30 Sep 2005 08:42:07 -0400 Subject: [Python-Dev] Conditional Expression Resolution In-Reply-To: References: Message-ID: <1128084127.9281.119.camel@presto.wooz.org> On Thu, 2005-09-29 at 21:21, Guido van Rossum wrote: > Flames, pleas to reconsider, etc., to /dev/null. > > Congratulations gracefully accepted. > > It's still my language! :-) Congratulations! May this be as successful a syntax addition as decorators and print>> (of which I'm a fan too! :). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050930/570948e4/attachment.pgp From skip at pobox.com Fri Sep 30 15:32:57 2005 From: skip at pobox.com (skip@pobox.com) Date: Fri, 30 Sep 2005 08:32:57 -0500 Subject: [Python-Dev] Pythonic concurrency - cooperative MT In-Reply-To: <1128073517.25688.10.camel@p-dvsi-418-1.rd.francetelecom.fr> References: <1128073517.25688.10.camel@p-dvsi-418-1.rd.francetelecom.fr> Message-ID: <17213.16009.926895.414492@montanaro.dyndns.org> >> (C) That scheduler is non-preemptive. A single greedy generator can >> starve all the others. Antoine> Instead of looking at this as a problem, you could look at it Antoine> as a feature. Apple looked at it as a feature for years. Not anymore. Skip From anothermax at gmail.com Fri Sep 30 15:35:44 2005 From: anothermax at gmail.com (Jeremy Maxfield) Date: Fri, 30 Sep 2005 15:35:44 +0200 Subject: [Python-Dev] Pythonic concurrency In-Reply-To: <79990c6b0509300302x2ebc704nc06a73458c604e78@mail.gmail.com> References: <79990c6b0509300302x2ebc704nc06a73458c604e78@mail.gmail.com> Message-ID: <93dc9c320509300635q70d532e4q11426137ae6ed023@mail.gmail.com> Support for multiple interpreters already exists from the C API (mod_python, Java Embedded Python a few other add-ons use them) But: - it's not possible to create new interpreter instances from within Python. - there's no mechanism for passing information between interpreters. - interaction with extension modules instances may be a problem. Apart from these points they actually seem to work pretty well and it might be, as you suggest, a "Practical" approach. Implementing a 'subinterp' module could be interesting... Max On 9/30/05, Paul Moore wrote: > > On 9/30/05, Jim Jewett wrote: > > Bruce Eckel wrote: > > > > > 3) Tasks are cheap enough that I can make > > > thousands of them, ... > > > > > 4) Tasks are "self-guarding," so they prevent > > > other tasks from interfering with them. The > > > only way tasks can communicate with each > > > other is through some kind of formal > > > mechanism (something queue-ish, > > > I'd imagine). > > > > I think these two are the hardest to reconcile. > > Agreed. I think the biggest problem is that (4) is too strong. At the > OS level, certain issues (such as the current directory, or the stdio > streams) are per-process, so that taking (4) at face value requires > multiprocessing, which violates (3)... > > If you constrain (4), you can get something much more effective. For > example, if you accept that certain things are volatile (start with > OS-specified per-process stuff, plus the Python builtins, maybe) then > it's much easier to produce solutions. > > I don't think that shared state is that big an issue per se - after > all, we're all used to the idea that the contents of a file might > change "behind our backs". The real issue is not having a *clearly > defined* shared state. > > The big problem with threads is that the shared state is *everything*. > There's no partitioning at all. Every thread-based abstraction is > based around threads voluntarily restricting themselves to a limited > set of "safe" communication operations, and otherwise scrupulously > avoiding each other's space ("Don't sit there, that's auntie Mary's > chair"...) > > If we had an abstraction (multiple interpreters, anyone?) which still > had a shared state, but a much smaller one, which was defined clearly, > such that "unsafe" operations were easy to identify, then day-to-day > concurrent programming would be a lot easier. Yes, it's still possible > to break things, but people can do that already by hacking about with > builtins, or abusing the introspection API, and it's not made Python > unusable, because they generally don't... > > This seems to me to be a perfect case for a "Practicality beats > Purity" approach. > > Paul. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/anothermax%40gmail.com > -- flickr: http://www.flickr.com/photos/anothermax/sets -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20050930/4627acfb/attachment.html From solipsis at pitrou.net Fri Sep 30 15:57:59 2005 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 30 Sep 2005 15:57:59 +0200 Subject: [Python-Dev] Pythonic concurrency - cooperative MT In-Reply-To: <17213.16009.926895.414492@montanaro.dyndns.org> References: <1128073517.25688.10.camel@p-dvsi-418-1.rd.francetelecom.fr> <17213.16009.926895.414492@montanaro.dyndns.org> Message-ID: <1128088679.26560.7.camel@p-dvsi-418-1.rd.francetelecom.fr> Le vendredi 30 septembre 2005 ? 08:32 -0500, skip at pobox.com a ?crit : > >> (C) That scheduler is non-preemptive. A single greedy generator can > >> starve all the others. > > Antoine> Instead of looking at this as a problem, you could look at it > Antoine> as a feature. > > Apple looked at it as a feature for years. Not anymore. You are missing the context: - at the OS level, it is not good to have cooperative MT because any badly-written app (there are lots of ;-)) can cause lack of responsiveness for the whole machine - at the application level, cooperative MT can be good to enforce robust semantics without disrupting other apps The latter is what we are discussing AFAIK: a different concurrency scheme inside a single application - not accross the whole OS or desktop. Actually, I think different concurrency schemes must be chosen and mixed depending on the semantics. For example, when you have a networked GUI app, it can be good to have the network in one preemptive thread (using e.g. Twisted) and the GUI in another preemptive thread (using GTK, wx...). This is because GUI and network have different latency characteristics and requirements. (of course, you can replace "preemptive thread" with "process" in the above description) So whatever innovatice concurrency scheme Python may come out, it should still be mixable with more traditional concurrency schemes, because required properties vary wildly even inside a single app. Regards Antoine. From guido at python.org Fri Sep 30 16:23:26 2005 From: guido at python.org (Guido van Rossum) Date: Fri, 30 Sep 2005 07:23:26 -0700 Subject: [Python-Dev] PEP 350: Codetags In-Reply-To: <433CFA1F.4010804@gmail.com> References: <20050926223521.GE10940@kitchen.client.attbi.com> <20050928161039.GF10940@kitchen.client.attbi.com> <20050929153237.97E1.JCARLSON@uci.edu> <433CFA1F.4010804@gmail.com> Message-ID: On 9/30/05, Nick Coghlan wrote: > An approach to this area that would make sense to me is: > > 1. Defer PEP 350 > 2. Publish a simple Python module for finding and processing code tags in a > configurable fashion > 3. Include a default configuration in the module that provides the behaviour > described in PEP 350 > 4. After this hypothetical code tag processing module has been out in the wild > for a while, re-open PEP 350 with an eye to including the module in the > standard library > > The idea is that it should be possible to tailor the processing module in > order to textually scan a codebase (possibly C or C++ rather than Python) in > accordance with a project-specific system of code tagging, rather than > requiring that the project necessarily use the default style included in the > processing module (Although using a system other than the default one may > result in reduced functionality, naturally). Maybe I'm just an old fart, but this all seems way over-engineered. Even for projects the size of Python, a simple grep+find is sufficient. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steve at holdenweb.com Fri Sep 30 16:28:18 2005 From: steve at holdenweb.com (Steve Holden) Date: Fri, 30 Sep 2005 15:28:18 +0100 Subject: [Python-Dev] Conditional Expression Resolution In-Reply-To: <1128084127.9281.119.camel@presto.wooz.org> References: <1128084127.9281.119.camel@presto.wooz.org> Message-ID: <433D4B82.8040406@holdenweb.com> Barry Warsaw wrote: > On Thu, 2005-09-29 at 21:21, Guido van Rossum wrote: > > >>Flames, pleas to reconsider, etc., to /dev/null. >> >>Congratulations gracefully accepted. >> >>It's still my language! :-) > > > Congratulations! May this be as successful a syntax addition as > decorators and print>> (of which I'm a fan too! :). > I have to say, now Guido is getting rid of print>> along with the rest of the print statement I have become nostalgically fond of it. Perhaps it's just that I can't help siding with the underdog, doomed for the knacker's yard in Python 3.0. Sniff. The decision on X if C else Y is a cause for much jubilation. Congratulations, Guido! nothing-to-see-here-folks-let's-move-along-ly y'rs - steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ From guido at python.org Fri Sep 30 16:30:06 2005 From: guido at python.org (Guido van Rossum) Date: Fri, 30 Sep 2005 07:30:06 -0700 Subject: [Python-Dev] linecache problem In-Reply-To: References: Message-ID: On 9/30/05, Thomas Heller wrote: > On several occasions I have seen tracebacks in my code pointing to PIL's > __init__.py file. That is strange, because I have installed PIL but it > is used nowhere. > > Finally I traced it down to a problem in the linecache code, which tries > to be smart in up updatecache function. When os.stat() on the filename > fails with os.error, it walks along sys.path and returns the first file > with a matching basename. This *may* make sense for toplevel modules, > but never for modules in packages. > > So, if the traceback's stack contains an entry for a non-existing file > (for example because the .py file for a .pyc file is no longer present), > linecache returns absolute garbage. > > Example, on my system (the \a\b\c\__init__.py file doesn't exist): > > C:\>python -c "import linecache; print linecache.getlines(r'\a\b\c\__init__.py')" > ['#\n', '# The Python Imaging Library.\n', > '# $Id: //modules/pil/PIL/__init__.py#2 $\n', '#\n', > '# package placeholder\n', '#\n', '# Copyright (c) 1999 by Secret Labs AB.\n', > '#\n', '# See the README file for information on usage and redistribution.\n', > '#\n', '\n', '# ;-)\n'] > C:\> > > The bug is present in 2.3, 2.4, and current CVS. Probably my fault (I wrote linecache, 13 years ago (before Python had packages!). But looking at the code I don't understand why this is; there's no code to descend into subdirectories of directories found on sys.path. I wonder if the problem isn't on PIL's end, which puts the PIL directory on sys.path? Anyway, don't hesitate to suggest a patch on sourceforge -- python-dev really isn't the forum for further discussion of this issue. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From simonwittber at gmail.com Fri Sep 30 16:32:55 2005 From: simonwittber at gmail.com (Simon Wittber) Date: Fri, 30 Sep 2005 07:32:55 -0700 Subject: [Python-Dev] Pythonic concurrency - cooperative MT In-Reply-To: <1128073517.25688.10.camel@p-dvsi-418-1.rd.francetelecom.fr> References: <1128073517.25688.10.camel@p-dvsi-418-1.rd.francetelecom.fr> Message-ID: <4e4a11f80509300732q35acbd13t4f9f55a6d942fa86@mail.gmail.com> On 9/30/05, Antoine Pitrou wrote: > > > (C) That scheduler is non-preemptive. A single > > greedy generator can starve all the others. > > Instead of looking at this as a problem, you could look at it as a > feature. Since generators can't be switched at arbitrary places, the > programmer has to define his/her synchronization points explicitly. I use this approach extensively, using tasks which are defined using generators. The scheduler I developed for this can be viewed here: http://metaplay.dyndns.org:82/svn/nanothreads/nanothreads.py Synchronization using yield statements is an important feature, as I use these cooperative threads for game development. These threads are _very_ useful for games, as they have very low overhead, and allow the programmer to exercise more control over their execution. Sw. From steve at holdenweb.com Fri Sep 30 16:28:18 2005 From: steve at holdenweb.com (Steve Holden) Date: Fri, 30 Sep 2005 15:28:18 +0100 Subject: [Python-Dev] Conditional Expression Resolution In-Reply-To: <1128084127.9281.119.camel@presto.wooz.org> References: <1128084127.9281.119.camel@presto.wooz.org> Message-ID: <433D4B82.8040406@holdenweb.com> Barry Warsaw wrote: > On Thu, 2005-09-29 at 21:21, Guido van Rossum wrote: > > >>Flames, pleas to reconsider, etc., to /dev/null. >> >>Congratulations gracefully accepted. >> >>It's still my language! :-) > > > Congratulations! May this be as successful a syntax addition as > decorators and print>> (of which I'm a fan too! :). > I have to say, now Guido is getting rid of print>> along with the rest of the print statement I have become nostalgically fond of it. Perhaps it's just that I can't help siding with the underdog, doomed for the knacker's yard in Python 3.0. Sniff. The decision on X if C else Y is a cause for much jubilation. Congratulations, Guido! nothing-to-see-here-folks-let's-move-along-ly y'rs - steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ From guido at python.org Fri Sep 30 16:38:27 2005 From: guido at python.org (Guido van Rossum) Date: Fri, 30 Sep 2005 07:38:27 -0700 Subject: [Python-Dev] Pythonic concurrency - cooperative MT In-Reply-To: <1128088679.26560.7.camel@p-dvsi-418-1.rd.francetelecom.fr> References: <1128073517.25688.10.camel@p-dvsi-418-1.rd.francetelecom.fr> <17213.16009.926895.414492@montanaro.dyndns.org> <1128088679.26560.7.camel@p-dvsi-418-1.rd.francetelecom.fr> Message-ID: On 9/30/05, Antoine Pitrou wrote: > Le vendredi 30 septembre 2005 ? 08:32 -0500, skip at pobox.com a ?crit : > > >> (C) That scheduler is non-preemptive. A single greedy generator can > > >> starve all the others. > > > > Antoine> Instead of looking at this as a problem, you could look at it > > Antoine> as a feature. > > > > Apple looked at it as a feature for years. Not anymore. > > You are missing the context: > - at the OS level, it is not good to have cooperative MT because any > badly-written app (there are lots of ;-)) can cause lack of > responsiveness for the whole machine > - at the application level, cooperative MT can be good to enforce robust > semantics without disrupting other apps > > The latter is what we are discussing AFAIK: a different concurrency > scheme inside a single application - not accross the whole OS or > desktop. I like this analysis. > Actually, I think different concurrency schemes must be chosen and mixed > depending on the semantics. For example, when you have a networked GUI > app, it can be good to have the network in one preemptive thread (using > e.g. Twisted) and the GUI in another preemptive thread (using GTK, > wx...). This is because GUI and network have different latency > characteristics and requirements. > (of course, you can replace "preemptive thread" with "process" in the > above description) I'm skeptical of this. I think a traditional GUI+network app can combine the network and UI tasks in one process -- usually they interact somewhat and that's where the bugs show up. Also note that this is not an application area where MP is very interesting -- most of the time you're blocked waiting for the user or for a socket (or likely both :) so a single CPU is all you need. I've never heard someone complain that the GIL is in the way for these types of apps. > So whatever innovatice concurrency scheme Python may come out, it should > still be mixable with more traditional concurrency schemes, because > required properties vary wildly even inside a single app. I don't think you've proved that yet. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From solipsis at pitrou.net Fri Sep 30 16:55:25 2005 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 30 Sep 2005 16:55:25 +0200 Subject: [Python-Dev] Pythonic concurrency - cooperative MT In-Reply-To: <4e4a11f80509300732q35acbd13t4f9f55a6d942fa86@mail.gmail.com> References: <1128073517.25688.10.camel@p-dvsi-418-1.rd.francetelecom.fr> <4e4a11f80509300732q35acbd13t4f9f55a6d942fa86@mail.gmail.com> Message-ID: <1128092125.26560.18.camel@p-dvsi-418-1.rd.francetelecom.fr> Le vendredi 30 septembre 2005 ? 07:32 -0700, Simon Wittber a ?crit : > I use this approach extensively, using tasks which are defined using > generators. The scheduler I developed for this can be viewed here: > > http://metaplay.dyndns.org:82/svn/nanothreads/nanothreads.py FWIW, I've coded my own little cooperative scheduler here: https://developer.berlios.de/projects/tasklets/ In contrast to your approach, I don't explicitly link threads. The "yield" keyword is used to wait on a resource (for example a synchronised queue or a timer) and the scheduling loop manages the switching accordingly. This approach is roughly the same as in "gtasklets" (http://www.gnome.org/~gjc/gtasklet/gtasklets.html) except that I don't rely on the GTK event loop. The event loop is generic and supports asynchronous resources (resources which become ready in another system thread: for example timers are managed in a separate helper system thread). It's quite alpha, but there are a few examples in the examples directory. Regards Antoine. From tim.peters at gmail.com Fri Sep 30 16:59:21 2005 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 30 Sep 2005 10:59:21 -0400 Subject: [Python-Dev] Conditional Expression Resolution In-Reply-To: References: Message-ID: <1f7befae0509300759v56c4afdqc9e45dcdab9ebf4c@mail.gmail.com> [Guido] > After a long discussion I've decided to add a shortcut conditional > expression to Python 2.5. > > The syntax will be > > A if C else B ... > The priorities will be such that you can write > > ... > x = A if C else B if D else E I assume this groups as A if C else (B if D else E) rather than as (A if C else B) if D else E ? So that C is evaluated first, and if C is true D isn't evaluated at all. ... > Flames, pleas to reconsider, etc., to /dev/null. Compared to the postfix A then C if B it's a positive delight. > Congratulations gracefully accepted. Congratulations gracefully tendered. > It's still my language! :-) If that was in doubt, you just proved it . From theller at python.net Fri Sep 30 16:59:11 2005 From: theller at python.net (Thomas Heller) Date: Fri, 30 Sep 2005 16:59:11 +0200 Subject: [Python-Dev] linecache problem References: Message-ID: Guido van Rossum writes: > On 9/30/05, Thomas Heller wrote: >> On several occasions I have seen tracebacks in my code pointing to PIL's >> __init__.py file. That is strange, because I have installed PIL but it >> is used nowhere. >> >> Finally I traced it down to a problem in the linecache code, which tries >> to be smart in up updatecache function. When os.stat() on the filename >> fails with os.error, it walks along sys.path and returns the first file >> with a matching basename. This *may* make sense for toplevel modules, >> but never for modules in packages. >> The bug is present in 2.3, 2.4, and current CVS. > > Probably my fault (I wrote linecache, 13 years ago (before Python had > packages!). > > But looking at the code I don't understand why this is; there's no > code to descend into subdirectories of directories found on sys.path. > > I wonder if the problem isn't on PIL's end, which puts the PIL > directory on sys.path? It seems PIL cannot decide if it wants to be a package or not, but better would be to ask the author. /F, ? It installs a PIL.pth file in lib/site-packages, which contains 'PIL' only - that's where the sys.path entry comes from. OTOH, the lib/site-packages/PIL directory contains an __init__.py file. PIL.pth is the only .pth file that I have where the directory contains an __init__.py file. > Anyway, don't hesitate to suggest a patch on sourceforge -- python-dev > really isn't the forum for further discussion of this issue. https://sourceforge.net/tracker/index.php?func=detail&aid=1309567&group_id=5470&atid=105470 Thomas From bjourne at gmail.com Fri Sep 30 17:25:55 2005 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Fri, 30 Sep 2005 17:25:55 +0200 Subject: [Python-Dev] Adding a conditional expression in Py3.0 In-Reply-To: References: <20050929114321.GA11086@panix.com> <000d01c5c4f1$085e7e20$d63dc797@oemcomputer> Message-ID: <740c3aec0509300825v6a50daafme1d3c45864b9a75b@mail.gmail.com> > > I did this for my favorite proposal, and ended up with the list shown > > further down below. > > > > I think they all looks great! > > > The fact that so few were found in whole of the standard library does > put the use case into question, though, no? Though I am sure more could > be found with a more thorough scan. There's lots of code in it that looks like this: def __init__(self, foo = None): if foo is not None: self.foo = foo else: self.foo = self.getFooDefault() With if/else it can be written: def __init__(self, foo = None): self.foo = foo if foo is not None else self.getFooDefault() However, I'm convinced that the latter version is less readable than the former. Often the line becomes more than 80 characters so there's no gain in translating it to an if-else expression because you would still have to split the line, other times the condition is so complex that it deserves its own line. Many opportunities for uses, but atleast as many for abuses. -- mvh Bj?rn From solipsis at pitrou.net Fri Sep 30 17:26:27 2005 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 30 Sep 2005 17:26:27 +0200 Subject: [Python-Dev] Pythonic concurrency - cooperative MT In-Reply-To: References: <1128073517.25688.10.camel@p-dvsi-418-1.rd.francetelecom.fr> <17213.16009.926895.414492@montanaro.dyndns.org> <1128088679.26560.7.camel@p-dvsi-418-1.rd.francetelecom.fr> Message-ID: <1128093987.26560.29.camel@p-dvsi-418-1.rd.francetelecom.fr> Hi, > I've never heard > someone complain that the GIL is in the way for these types of apps. I've never said so either. I was just saying that it can be useful to mix cooperative threading and preemptive threading in the same app, i.e. have different domains of cooperative threading which are preemptively scheduled by the OS. That has nothing to do with the GIL, I think (but I don't know much in Python internals). For instance the mix between Twisted and GUI event loops is a very common topic on the twisted mailing-list, because people have trouble getting it right inside a single system thread. The Twisted people have started advocating something called the ThreadedSelectReactor, which I haven't looked at but whose name suggests that some operations are performed in a helper system thread. Regards Antoine. From jimjjewett at gmail.com Fri Sep 30 17:42:15 2005 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 30 Sep 2005 11:42:15 -0400 Subject: [Python-Dev] bool(container) [was bool(iter([])) changed between 2.3 and 2.4] Message-ID: Python doesn't worry about a precise boolean object, it distinguishes between something and nothing. Is there anything left?" is a pretty good analogy for iterators. It isn't always cheaply available, and having might encourage poor style -- so iterators are going back to the default for non-containers of always True. How general strong is this default-to-true rule? If I submit a documentation patch, should I say that numbers, lists, strings, dictionaries, and tuples are a special case, or should I just warn that some container-like objects (including iterators) are always True? More specific question: A Queue.Queue is always true. Should I submit a bug patch or add it as a beware of custom classes example? def __nonzero__(self): return not self.empty() From p.f.moore at gmail.com Fri Sep 30 17:53:32 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 30 Sep 2005 16:53:32 +0100 Subject: [Python-Dev] Pythonic concurrency In-Reply-To: <93dc9c320509300635q70d532e4q11426137ae6ed023@mail.gmail.com> References: <79990c6b0509300302x2ebc704nc06a73458c604e78@mail.gmail.com> <93dc9c320509300635q70d532e4q11426137ae6ed023@mail.gmail.com> Message-ID: <79990c6b0509300853w610b853fm62cad05724360331@mail.gmail.com> On 9/30/05, Jeremy Maxfield wrote: > Support for multiple interpreters already exists from the C API > (mod_python, Java Embedded Python a few other add-ons use them) I'm aware of that (didn't I mention it in my message - sorry). > But: > - it's not possible to create new interpreter instances from within Python. Yet... > - there's no mechanism for passing information between interpreters. But one could possibly be added when exposing the functionality to Python. > - interaction with extension modules instances may be a problem. This is the issue I have heard mentioned before as the problematic one. Unfortunately, I don't know enough about the problems to be able to comment. If it's as simple as saying that any global state maintained by an extension module is part of the "shared state", then I'm OK with that (although it would be nice to encourage extension authors to document their global state a bit more clearly :-)) Actually, it might be necessary to use global extension state (and the fact that it's shared) to implement inter-interpreter communication... > Apart from these points they actually seem to work pretty well and it might > be, as you suggest, a "Practical" approach. > > Implementing a 'subinterp' module could be interesting... Indeed. Maybe I'll have a go (in my copious free time :-)) Paul. From jimjjewett at gmail.com Fri Sep 30 18:00:58 2005 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 30 Sep 2005 12:00:58 -0400 Subject: [Python-Dev] C API doc fix Message-ID: On 9/29/05, Robey Pointer wrote: > Yesterday I ran into a bug in the C API docs. The top of this page: > > http://docs.python.org/api/unicodeObjects.html > > says: > > Py_UNICODE > This type represents a 16-bit unsigned storage type which is > used by Python internally as basis for holding Unicode ordinals. On > platforms where wchar_t is available and also has 16-bits, Py_UNICODE > is a typedef alias for wchar_t to enhance native platform > compatibility. On all other platforms, Py_UNICODE is a typedef alias > for unsigned short. Steven Bethard wrote: > I believe this is the same issue that was brought up in May[1]. My > impression was that people could not agree on a documentation patch. > [1] http://www.python.org/dev/summary/2005-05-01_2005-05-15.html I thought the problem was disagreement over how the system *should* pick an underlying type to alias. Given the current policy, are there objections to a patch that at least steers people away from assuming they can use the underlying type directly? ======== Py_UNICODE Python uses this type to store Unicode ordinals. It is typically a typedef alias, but the underlying type -- and the size of that type -- varies across different systems. ======== -jJ From fumanchu at amor.org Fri Sep 30 18:16:00 2005 From: fumanchu at amor.org (Robert Brewer) Date: Fri, 30 Sep 2005 09:16:00 -0700 Subject: [Python-Dev] Adding a conditional expression in Py3.0 Message-ID: Guido: > Now you've pushed me over the edge. I've made up my mind > now, "X if C else Y" it will be. I hope to find time to > implement it in Python 2.5. Steve Holden: > Not before time, either. If this ends the discussion then > I'll consider I've done everyone a favour. Sometimes no > decision is worse than the wrong decision ;-). Exactly how I felt about bringing the decorator decision to a close. ;) Congratulations to you both! Robert Brewer System Architect Amor Ministries fumanchu at amor.org From guido at python.org Fri Sep 30 18:16:39 2005 From: guido at python.org (Guido van Rossum) Date: Fri, 30 Sep 2005 09:16:39 -0700 Subject: [Python-Dev] bool(container) [was bool(iter([])) changed between 2.3 and 2.4] In-Reply-To: References: Message-ID: On 9/30/05, Jim Jewett wrote: > If I submit a documentation patch, should I say that > numbers, lists, strings, dictionaries, and tuples are > a special case, or should I just warn that some > container-like objects (including iterators) are always > True? You seem to be going at this from the wrong direction. Boolean value are defined by calling __nonzero__ or __len__, whichever exists; if neither exists the answer is true (except for None which is special-cased only for historical reasons -- there's no reason why it couldn't have a __nonzero__ method. __nonzero__ is intended for object types that either want to have number-like behavior or have a special reason for wanting to act like a Boolean. __len__ is for sequences and mappings specifically -- everything that supports __getitem__ should have __len__ and everything that has __len__ should have __getitem__. (This is what broke for iterators in 2.4.) So if anything's an "exception", it's numbers -- strings, lists, tuples are sequences and dicts are mappings, and that's where they get their definition of Booleanness from. Always remember, user-defined classes can define __nonzero__ any way they wish, and they get what they deserve. Library designers however should try to follow established patterns. Testing for Queue emptiness via __nonzero__ seems unwarranted since a Queue doesn't have any other sequence behavior. "Containerish" behavior isn't enough to warrant empty <--> false; in some sense every object is a container (at least every object with a __dict__ attribute) and you sure don't want to map __len__ to self.__dict__.__len__... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From exarkun at divmod.com Fri Sep 30 18:20:07 2005 From: exarkun at divmod.com (Jp Calderone) Date: Fri, 30 Sep 2005 12:20:07 -0400 Subject: [Python-Dev] Pythonic concurrency - cooperative MT In-Reply-To: <1128093987.26560.29.camel@p-dvsi-418-1.rd.francetelecom.fr> Message-ID: <20050930162007.3914.1732779229.divmod.quotient.22926@ohm> On Fri, 30 Sep 2005 17:26:27 +0200, Antoine Pitrou wrote: > >Hi, > >> I've never heard >> someone complain that the GIL is in the way for these types of apps. > >I've never said so either. >I was just saying that it can be useful to mix cooperative threading and >preemptive threading in the same app, i.e. have different domains of >cooperative threading which are preemptively scheduled by the OS. That >has nothing to do with the GIL, I think (but I don't know much in Python >internals). > >For instance the mix between Twisted and GUI event loops is a very >common topic on the twisted mailing-list, because people have trouble >getting it right inside a single system thread. The Twisted people have >started advocating something called the ThreadedSelectReactor, which I >haven't looked at but whose name suggests that some operations are >performed in a helper system thread. > "Advocating" might be putting it too strongly :) "Experimenting with" describes the current state of things most accurately. The problem it aims to solve is integration with cooperative threading systems which don't work very well. An example of such a loop is the wx event loop. Whenever a modal dialog is displayed or a menu is activated, wx's loop stops cooerating. The only way we've found thus far to avoid this is to run wx in a separate thread, so even when it stops cooperating, network code can still get scheduled. Jp From mcherm at mcherm.com Fri Sep 30 18:23:10 2005 From: mcherm at mcherm.com (Michael Chermside) Date: Fri, 30 Sep 2005 09:23:10 -0700 Subject: [Python-Dev] bool(container) [was bool(iter([])) changedbetween 2.3 and 2.4] Message-ID: <20050930092310.n2vxfbfl1bocosg4@login.werra.lunarpages.com> Jim Jewett writes: > Python doesn't worry about a precise boolean object, it > distinguishes between something and nothing. > > Is there anything left?" is a pretty good analogy for iterators. [...] > A Queue.Queue is always true. Should I submit a bug patch I would have phrased this very differently. I would have said it as follows: All Python objects can be converted to a boolean value. Most objects are True, except for False (of course), None, any number equal to 0, an empty string (""), and the empty containers (), [], and {}. User defined objects can override __len__ or __nonzero__ to return False; everything else is True. I don't believe that there is an overarching rule that all "containers" must be false when empty. I'm not sure there is even a clear definition of "container" that would satisfy everyone. It makes complete sense to me that empty iterators and Queue.Queues are True... it follows the general rule that everything is true except the above (short) list of objects and user-defined (or library) classes that want to mimic the behavior of one of these or have some obvious meaning for non-true. -- Michael Chermside From raymond.hettinger at verizon.net Fri Sep 30 18:25:01 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Fri, 30 Sep 2005 12:25:01 -0400 Subject: [Python-Dev] bool(container) [was bool(iter([])) changedbetween 2.3 and 2.4] In-Reply-To: Message-ID: <000201c5c5db$82407820$91f9a244@oemcomputer> [Guido van Rossum] > __len__ is for sequences and mappings specifically -- everything that > supports __getitem__ should have __len__ and everything that has > __len__ should have __getitem__. That's going a bit far. Unordered collections (like sets and bags) are a good counter-example. Raymond From fredrik at pythonware.com Fri Sep 30 18:29:58 2005 From: fredrik at pythonware.com (Fredrik Lundh) Date: Fri, 30 Sep 2005 18:29:58 +0200 Subject: [Python-Dev] bool(container) [was bool(iter([])) changedbetween 2.3 and 2.4] References: Message-ID: Guido van Rossum wrote: > "Containerish" behavior isn't enough to warrant empty <--> false; in > some sense every object is a container (at least every object with a > __dict__ attribute) and you sure don't want to map __len__ to > self.__dict__.__len__... the ElementTree experience shows that doing def __getitem__(self, index): return self.items[index] def __len__(self): return len(self.items) def __nonzero__(self): return True is sometimes a good idea, even for objects that are mostly sequences. (ET 1.2.X lacks the __nonzero__ method, and accidentally treating elements with no subelements as if the element itself doesn't exist is by far the most common gotcha in ET code). From solipsis at pitrou.net Fri Sep 30 18:33:29 2005 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 30 Sep 2005 18:33:29 +0200 Subject: [Python-Dev] Pythonic concurrency - cooperative MT In-Reply-To: <20050930162007.3914.1732779229.divmod.quotient.22926@ohm> References: <20050930162007.3914.1732779229.divmod.quotient.22926@ohm> Message-ID: <1128098009.26560.56.camel@p-dvsi-418-1.rd.francetelecom.fr> Hi Jp, Le vendredi 30 septembre 2005 ? 12:20 -0400, Jp Calderone a ?crit : > "Advocating" might be putting it too strongly :) "Experimenting with" > describes the current state of things most accurately. Ok :) > The problem it aims to solve is integration with cooperative threading > systems which don't work very well. An example of such a loop is the > wx event loop. > > Whenever a modal dialog is displayed or a menu is activated, wx's loop > stops cooerating. This specific problem hides the more general problem, which is that GUI and network activities have different typical latencies. As I explained on the twisted ML, a GUI needs very good response times to feel friendly (typically below 20 or even 10 ms.), whereas some network protocols have non-trivial calculations which can go above 100 ms. Moreover, you don't want a sudden flood of network events to block events in the GUI. This is why even without considering wx's specificities, it is still useful to keep GUI and network activities in separate threads. Regards Antoine. From BruceEckel-Python3234 at mailblocks.com Fri Sep 30 19:47:52 2005 From: BruceEckel-Python3234 at mailblocks.com (Bruce Eckel) Date: Fri, 30 Sep 2005 11:47:52 -0600 Subject: [Python-Dev] Pythonic concurrency - cooperative MT In-Reply-To: <1128093987.26560.29.camel@p-dvsi-418-1.rd.francetelecom.fr> References: <1128073517.25688.10.camel@p-dvsi-418-1.rd.francetelecom.fr> <17213.16009.926895.414492@montanaro.dyndns.org> <1128088679.26560.7.camel@p-dvsi-418-1.rd.francetelecom.fr> <1128093987.26560.29.camel@p-dvsi-418-1.rd.francetelecom.fr> Message-ID: <1165119966.20050930114752@MailBlocks.com> > I was just saying that it can be useful to mix cooperative threading and > preemptive threading in the same app, i.e. have different domains of > cooperative threading which are preemptively scheduled by the OS. That > has nothing to do with the GIL, I think (but I don't know much in Python > internals). This is one of the interesting questions I'd like to answer (but which probably requires some kind of mathematician to prove one way or another): Does a "complete" concurrency solution require both a preemptive task management system and a cooperative one? OS Processes are generally quite limited in number (although I understand that BeOS was designed to create tons of lightweight processes, which changed that particular picture). In Java, you can usually create several hundred threads before it craps out. Simulations and games could easily have thousands or even hundreds of thousands of independent simulation units. If there are limits to the number of concurrency drivers (threads, etc.), then that would suggest that at some point you have to move to a cooperative system. Which would suggest that a "complete" concurrency solution might require both. That wouldn't be my ideal. My ideal would be a single solution that would scale up to large numbers of concurrency units, and that wouldn't require the programmer to remember to explicitly yield control. Whether my ideal is possible is a question I'd like to answer. Bruce Eckel http://www.BruceEckel.com mailto:BruceEckel-Python3234 at mailblocks.com Contains electronic books: "Thinking in Java 3e" & "Thinking in C++ 2e" Web log: http://www.artima.com/weblogs/index.jsp?blogger=beckel Subscribe to my newsletter: http://www.mindview.net/Newsletter My schedule can be found at: http://www.mindview.net/Calendar From dalcinl at gmail.com Fri Sep 30 22:21:47 2005 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Fri, 30 Sep 2005 17:21:47 -0300 Subject: [Python-Dev] bool(container) [was bool(iter([])) changedbetween 2.3 and 2.4] In-Reply-To: <000201c5c5db$82407820$91f9a244@oemcomputer> References: <000201c5c5db$82407820$91f9a244@oemcomputer> Message-ID: On 9/30/05, Raymond Hettinger wrote: > [Guido van Rossum] > > __len__ is for sequences and mappings specifically -- everything that > > supports __getitem__ should have __len__ and everything that has > > __len__ should have __getitem__. > > That's going a bit far. Unordered collections (like sets and bags) are > a good counter-example. > I've implemented many extension types in C, always to interface external libraries (the main examples, MPI and PETSc). In those libraries, some object are invalid (MPI_COMM_NULL) or can get deallocated in the C side. I always implement a __nonzero__ method for testing those situations. It sounds a bit bizarre to me to have to implement a number method for a object that is far from being a number, and of course __len__ do not make any sense. Perhaps Py3k could have a more explicit mechanism for definig the truth value of an object? Something like __bool__ method to be filled in type object? Any opinions? Lisandro Dalc?n From bcannon at gmail.com Fri Sep 30 23:38:41 2005 From: bcannon at gmail.com (Brett Cannon) Date: Fri, 30 Sep 2005 14:38:41 -0700 Subject: [Python-Dev] Conditional Expression Resolution In-Reply-To: References: Message-ID: On 9/29/05, Guido van Rossum wrote: [SNIP] > Flames, pleas to reconsider, etc., to /dev/null. > > Congratulations gracefully accepted. > And gladly given! All proposals should be resolved this cleanly in the end. > It's still my language! :-) > Yes it is, thank goodness! -Brett From gjc at inescporto.pt Fri Sep 30 23:55:44 2005 From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro) Date: Fri, 30 Sep 2005 22:55:44 +0100 Subject: [Python-Dev] Pythonic concurrency - cooperative MT In-Reply-To: <1128098009.26560.56.camel@p-dvsi-418-1.rd.francetelecom.fr> References: <20050930162007.3914.1732779229.divmod.quotient.22926@ohm> <1128098009.26560.56.camel@p-dvsi-418-1.rd.francetelecom.fr> Message-ID: <1128117344.11707.7.camel@localhost.localdomain> On Fri, 2005-09-30 at 18:33 +0200, Antoine Pitrou wrote: > Hi Jp, > > Le vendredi 30 septembre 2005 ? 12:20 -0400, Jp Calderone a ?crit : > > "Advocating" might be putting it too strongly :) "Experimenting with" > > describes the current state of things most accurately. > > Ok :) > > > The problem it aims to solve is integration with cooperative threading > > systems which don't work very well. An example of such a loop is the > > wx event loop. > > > > Whenever a modal dialog is displayed or a menu is activated, wx's loop > > stops cooerating. That is wx's problem. Try PyGTK some day; I hear it's really good! ;-) > > This specific problem hides the more general problem, which is that GUI > and network activities have different typical latencies. As I explained > on the twisted ML, a GUI needs very good response times to feel friendly > (typically below 20 or even 10 ms.), whereas some network protocols have > non-trivial calculations which can go above 100 ms. With PyGTK a typical solution for this is to use a generator function executing an "idle function", which would make the non-trivial calculations, but yield control back to the main loop periodically, in order to process GUI events. For example, see the last code block in http://www.async.com.br/faq/pygtk/index.py?req=show&file=faq23.020.htp > Moreover, you don't > want a sudden flood of network events to block events in the GUI. Process one event at a time, after each event give back control to the main loop, and give low priority to socket IO events. Problem solved. > > This is why even without considering wx's specificities, it is still > useful to keep GUI and network activities in separate threads. You are considering a scenario that seldom happens to design a solution that is far too complicated for most cases. Regards. -- Gustavo J. A. M. Carneiro The universe is always one step beyond logic From ms at cerenity.org Fri Sep 30 23:13:55 2005 From: ms at cerenity.org (Michael Sparks (home address)) Date: Fri, 30 Sep 2005 22:13:55 +0100 Subject: [Python-Dev] Active Objects in Python References: <397621172.20050927111836@MailBlocks.com> Message-ID: [ I don't post often, but hopefully the following is of interest in this discussion ] Bruce Eckel wrote: > Yes, defining an class as "active" would: > 1) Install a worker thread and concurrent queue in each object of that > class. > 2) Automatically turn method calls into tasks and enqueue them > 3) Prevent any other interaction other than enqueued messages Here I think the point is an object that has some form of thread with enforced communications preventing (or limiting) shared data. This essentially describes the project we've been working on with Kamaelia. Kamaelia is divided into two halves: * A simple framework (Axon) for building components with the following characteristics: * Have a main() method which is a generator. * Have inbound and outbound named queues implemented using lists. (akin to stdin/out) * Access to an environmental system which is key/value lookup (similar purpose to the unix environment). (This could easily grow into a much more linda like system) * A collection of components which form a library. We also have a threaded component which uses a thread and queues rather than a generator & lists. With regard to the criteria you listed in a later post: 1) Has been tested against (so far) 2 novice programmers, who just picked up the framework and ran with in. (learnt python one week, axon the next, then implemented interesting things rapidly) 2) We don't deal with distribution (yet), but this is on the cards. Probably before EuroOSCON. (I want to have a program with two top level pygame windows) 3) We use python generators and have experimented initially with several tens of thousands of them, which works suprisingly well, even on an old 500Mhz machine. 4) Self guarding is difficult. However generators are pretty opaque, and getting at values inside their stack frame is beyond my python skills. 5) Deadlock can be forced in all scenarios, but you would actually have to work at it. In practice if we yield at all points or use the threaded components for everything then we'd simply livelock. 6) Regarding an actor. Our pygame sprite component actually is very, very actor like. This is to the extent we've considered implementing a script reading and director component for orchestrating them. (Time has prevented that though) 7) Since our target has been novices, this has come by default. 8) We don't do mobility of components/tasks as yet, however a colleague at work interested in mobility & agent software has proposed looking at that using Kamaelia for the coming year. We also ran the framework by people at the WoTUG (*) conference recently, and the general viewpoint came back that it's tractable from a formal analysis and mobility perspective(if we change x,y,z). (*) http://www.wotug.org/ I wrote a white paper based on my Python UK talk, which is here: * http://www.bbc.co.uk/rd/pubs/whp/whp11.shtml (BBC R&D white papers are aimed at imparting info in a such a way that it's useful and readable, unlike some white papers :) So far we have components for building things from network servers through to audio/video decoders and players, and presentation tools. (Using pygame and tkinter) Also we have a graphical editting tool for putting together systems and introspecting systems. An aim in terms of API was to enable novices to implement a microcosm version of the system. Specifically we tested this idea on a pre-university trainee who built a fairly highly parallel system which took video from an mpg file (simulating a PVR), and sent periodic snapshots to a mobile phone based client. He had minimal programming experience at the start, and implemented this over a 3 months period. We've since repeated that experience with a student who has had 2 years at university, but no prior concurrency, python or networks experience. In his 6 weeks with us he was able to do some interesting things. (Such as implement a system for joining multicast islands and sending content over a simple reliable multicast protocol). The tutorial we use to get people up to speed rapidly is here: * http://kamaelia.sourceforge.net/MiniAxon/ However that doesn't really cover the ideas of how to write components, use pipelines, graphlines and services. At the moment we're adding in the concept of a chassis into which you plug in existing components. The latest concept we're bouncing round is this: # (This was fleshed out in respect to a question by a novice on c.l.p - # periodically send the the same message to all connected clients) # start from Kamaelia.Chassis.ConnectedServer import SimpleServer from Kamaelia.Chassis.Splitter import splitter, publish, Subscriber class message_source(component): [ ... snippety ... ] def main(self): last = self.scheduler.time while 1: yield 1 if self.scheduler.time - last > 1: self.send(self.generateMessage, "outbox") last = self.scheduler.time splitter("client_fanout").activate() source = message_source().activate() publish("client_fanout", source) # perhaps something else? def client_protocol(): return Subscriber("client_fanout") SimpleServer(protocol=client_protocol, port=1400).run() # end >From the perspective of using multiple CPUs, currently we're thinking along the lines of this. I'lll use existing examples to make things more concrete. Currently to build a player to decode and playback dirac [1] encoded video in python, we're doing this: [1] http://dirac.sf.net/ pipeline( ReadFileAdaptor(file, readmode="bitrate", bitrate = 300000*8/5), DiracDecoder(), RateLimit(framerate), VideoOverlay(), ).run() Sometimes, however, it would be nice to take a YUV file, encode, decode and view (since it simplifies the process of seeing changes): pipeline( ReadFileAdaptor(FILENAME, readmode="bitrate", bitrate= 1000000), RawYUVFramer( size=SIZE ), DiracEncoder(preset=DIRACPRESET, encParams=ENCPARAMS ), DiracDecoder(), VideoOverlay() ).run() Clearly on a multiple CPU machine, it would be useful to have the processing cross processes, so we're intending on using a process chassis (a chassis is a thing with holes or can have things attached) to run the components in a different python process. The likely syntax we'll be playing with would be: pipeline( ReadFileAdaptor(FILENAME, readmode="bitrate", bitrate= 1000000), RawYUVFramer( size=SIZE ), SubProcess( DiracEncoder(preset=DIRACPRESET, encParams=ENCPARAMS ), ), SubProcess( DiracDecoder()), VideoOverlay() ).run() This is intended to run over 3 processes. (One parent, two children) For the degenerate case: pipeline( SubProcess( ReadFileAdaptor(FILENAME, readmode="bitrate", bitrate= 1000000)), SubProcess( RawYUVFramer( size=SIZE ),) SubProcess( DiracEncoder(preset=DIRACPRESET, encParams=ENCPARAMS ), ), SubProcess( DiracDecoder()), SubProcess( VideoOverlay()) ).run() We're actively considering the possible sugar: DistributedProcessPipeline( ... original list of components ... ) But we're looking to walk before running there. Much like we implemented the underlying functionality before pipelines and that before implementing graphines. One issue you potentially raise in this scenario is how you find existing components that may exist without having tight coupling. The way we're working on this is to have named services which you look up in a linda-type way. One nasty thing we found there is that the code can start looking snarled up. That was until we realised because we have a scheduler that we can implementing almost-but-not-quite-greenlet style switching using generators. Specifically a pygame based component can do a lookup for an actve display thus: def main(self): yield WaitComplete( self.requestDisplay(DISPLAYREQUEST=True, callback = (self,"control"), size = (self.render_area.width, self.render_area.height), position = self.position ) ) display = self.display Where requestDisplay does this: def requestDisplay(self, **argd): displayservice = PygameDisplay.getDisplayService() self.link((self,"signal"), displayservice) self.send(argd, "signal") for _ in self.waitBox("control"): yield 1 display = self.recv("control") self.display = display That said, having seen your post further down the thread about a set of requirements you post, I think on many fronts we hit much of the aims you list. This is largely for one simple reason - our primary aim is that of making concurrency __easy__ to deal with, whilst not sacrificing the portable scalability. This is also why we've worked on graphical tools for putting together these pipelines, as well as looking inside. We have a deliberately naive C++ implementation based on the mini-axon tutorial which uses Duffs device to implement generators for example. I'd love to see that implementation expand since I think it'd generate a whole set of questions that need answering and would probably benefit the python implementation. Mind you hopefully pypy and shedskin will mitigate /our/ need to do that translation manually :-) If anyone's interested further, I'm happy to talk more (I'm going to Euro OSCON is anyone wants to discuss this there), but this feels like spamming the list so I'll leave things at that. However, so far we're finding the approach is a good one for our needs. The reason I'm posting about it here is because I think Kamaelia directly hits every point you made, but mainly in the hope it's of use to others (either directly or for cherry picking ideas). Best Regards, Michael. -- Michael Sparks, Senior R&D Engineer, Digital Media Group Michael.Sparks at rd.bbc.co.uk, http://kamaelia.sourceforge.net/ British Broadcasting Corporation, Research and Development Kingswood Warren, Surrey KT20 6NP This e-mail may contain personal views which are not the views of the BBC. From vshah at virtc.com Thu Sep 29 18:14:03 2005 From: vshah at virtc.com (Viren Shah) Date: Thu, 29 Sep 2005 12:14:03 -0400 Subject: [Python-Dev] 64-bit bytecode compatibility (was Re: [PEAK] ez_setup on 64-bit linux problem) In-Reply-To: <5.1.1.6.0.20050929114339.01f7ea28@mail.telecommunity.com> References: <5.1.1.6.0.20050929114339.01f7ea28@mail.telecommunity.com> Message-ID: <433C12CB.9070600@virtc.com> Phillip J. Eby wrote: > At 09:49 AM 9/29/2005 -0400, Viren Shah wrote: > >> [I sent this earlier without being a subscriber and it was sent to the >> moderation queue so I'm resending it after subscribing] >> >> Hi, >> I'm running a 64-bit Fedora Core 3 with python 2.3.4. I'm trying to >> install setuptools to use with Trac, and get the following error: >> >> [root at Mrdumpling ~]$ python ez_setup.py >> Downloading >> http://cheeseshop.python.org/packages/2.3/s/setuptools/setuptools-0.6a4-py2.3.egg >> >> Traceback (most recent call last): >> File "ez_setup.py", line 206, in ? >> main(sys.argv[1:]) >> File "ez_setup.py", line 141, in main >> from setuptools.command.easy_install import main >> OverflowError: signed integer is greater than maximum >> >> >> I get the same type of error if I try installing setuptools manually. >> I figure this has to do with the 64-bit nature of the OS and python, >> but not being a python person, don't know what a workaround would be. >> >> Any ideas? > > > Hm. It sounds like perhaps the 64-bit Python in question isn't able to > read bytecode for Python from a 32-bit Python version. You'll need to > download the setuptools source archive from PyPI and install it using > "python setup.py install" instead. > [Thanks for the quick response] I tried downloading and installing setuptools-0.6a4.zip with the same type of result: [root at Mrdumpling setuptools-0.6a4]# python setup.py install running install running bdist_egg running egg_info writing ./setuptools.egg-info/PKG-INFO writing top-level names to ./setuptools.egg-info/top_level.txt writing entry points to ./setuptools.egg-info/entry_points.txt installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build creating build/lib copying pkg_resources.py -> build/lib copying easy_install.py -> build/lib creating build/lib/setuptools copying setuptools/depends.py -> build/lib/setuptools copying setuptools/archive_util.py -> build/lib/setuptools copying setuptools/dist.py -> build/lib/setuptools copying setuptools/__init__.py -> build/lib/setuptools copying setuptools/extension.py -> build/lib/setuptools copying setuptools/sandbox.py -> build/lib/setuptools copying setuptools/package_index.py -> build/lib/setuptools creating build/lib/setuptools/tests copying setuptools/tests/doctest.py -> build/lib/setuptools/tests copying setuptools/tests/__init__.py -> build/lib/setuptools/tests copying setuptools/tests/test_resources.py -> build/lib/setuptools/tests creating build/lib/setuptools/command copying setuptools/command/test.py -> build/lib/setuptools/command copying setuptools/command/saveopts.py -> build/lib/setuptools/command copying setuptools/command/easy_install.py -> build/lib/setuptools/command copying setuptools/command/build_ext.py -> build/lib/setuptools/command copying setuptools/command/egg_info.py -> build/lib/setuptools/command copying setuptools/command/install_lib.py -> build/lib/setuptools/command copying setuptools/command/develop.py -> build/lib/setuptools/command copying setuptools/command/alias.py -> build/lib/setuptools/command copying setuptools/command/sdist.py -> build/lib/setuptools/command copying setuptools/command/bdist_egg.py -> build/lib/setuptools/command copying setuptools/command/bdist_rpm.py -> build/lib/setuptools/command copying setuptools/command/rotate.py -> build/lib/setuptools/command copying setuptools/command/build_py.py -> build/lib/setuptools/command copying setuptools/command/upload.py -> build/lib/setuptools/command copying setuptools/command/setopt.py -> build/lib/setuptools/command copying setuptools/command/__init__.py -> build/lib/setuptools/command copying setuptools/command/install.py -> build/lib/setuptools/command creating build/bdist.linux-x86_64 creating build/bdist.linux-x86_64/egg copying build/lib/pkg_resources.py -> build/bdist.linux-x86_64/egg copying build/lib/easy_install.py -> build/bdist.linux-x86_64/egg creating build/bdist.linux-x86_64/egg/setuptools copying build/lib/setuptools/depends.py -> build/bdist.linux-x86_64/egg/setuptools creating build/bdist.linux-x86_64/egg/setuptools/tests copying build/lib/setuptools/tests/doctest.py -> build/bdist.linux-x86_64/egg/setuptools/tests copying build/lib/setuptools/tests/__init__.py -> build/bdist.linux-x86_64/egg/setuptools/tests copying build/lib/setuptools/tests/test_resources.py -> build/bdist.linux-x86_64/egg/setuptools/tests copying build/lib/setuptools/archive_util.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib/setuptools/dist.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib/setuptools/__init__.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib/setuptools/extension.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib/setuptools/sandbox.py -> build/bdist.linux-x86_64/egg/setuptools creating build/bdist.linux-x86_64/egg/setuptools/command copying build/lib/setuptools/command/test.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib/setuptools/command/saveopts.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib/setuptools/command/easy_install.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib/setuptools/command/build_ext.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib/setuptools/command/egg_info.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib/setuptools/command/install_lib.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib/setuptools/command/develop.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib/setuptools/command/alias.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib/setuptools/command/sdist.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib/setuptools/command/bdist_egg.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib/setuptools/command/bdist_rpm.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib/setuptools/command/rotate.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib/setuptools/command/build_py.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib/setuptools/command/upload.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib/setuptools/command/setopt.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib/setuptools/command/__init__.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib/setuptools/command/install.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib/setuptools/package_index.py -> build/bdist.linux-x86_64/egg/setuptools byte-compiling build/bdist.linux-x86_64/egg/pkg_resources.py to pkg_resources.pyc byte-compiling build/bdist.linux-x86_64/egg/easy_install.py to easy_install.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/depends.py to depends.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/tests/doctest.py to doctest.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/tests/__init__.py to __init__.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/tests/test_resources.py to test_resources.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/archive_util.py to archive_util.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/dist.py to dist.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/__init__.py to __init__.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/extension.py to extension.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/sandbox.py to sandbox.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/test.py to test.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/saveopts.py to saveopts.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py to easy_install.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/build_ext.py to build_ext.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py to egg_info.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/install_lib.py to install_lib.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/develop.py to develop.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/alias.py to alias.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/sdist.py to sdist.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/bdist_egg.py to bdist_egg.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/bdist_rpm.py to bdist_rpm.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/rotate.py to rotate.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/build_py.py to build_py.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/upload.py to upload.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/setopt.py to setopt.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/__init__.py to __init__.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/install.py to install.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/package_index.py to package_index.pyc creating build/bdist.linux-x86_64/egg/EGG-INFO installing scripts to build/bdist.linux-x86_64/egg/EGG-INFO/scripts running install_scripts running build_scripts creating build/scripts-2.3 copying and adjusting easy_install.py -> build/scripts-2.3 changing mode of build/scripts-2.3/easy_install.py from 644 to 755 creating build/bdist.linux-x86_64/egg/EGG-INFO/scripts copying build/scripts-2.3/easy_install.py -> build/bdist.linux-x86_64/egg/EGG-INFO/scripts changing mode of build/bdist.linux-x86_64/egg/EGG-INFO/scripts/easy_install.py to 755 copying ./setuptools.egg-info/entry_points.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying ./setuptools.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying ./setuptools.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO creating dist creating 'dist/setuptools-0.6a4-py2.3.egg' and adding 'build/bdist.linux-x86_64/egg' to it removing 'build/bdist.linux-x86_64/egg' (and everything under it) Processing setuptools-0.6a4-py2.3.egg Traceback (most recent call last): File "setup.py", line 81, in ? scripts = scripts, File "/usr/lib64/python2.3/distutils/core.py", line 149, in setup dist.run_commands() File "/usr/lib64/python2.3/distutils/dist.py", line 907, in run_commands self.run_command(cmd) File "/usr/lib64/python2.3/distutils/dist.py", line 927, in run_command cmd_obj.run() File "/root/svn-install-apps/setuptools-0.6a4/setuptools/command/install.py", line 50, in run cmd.run() File "/root/svn-install-apps/setuptools-0.6a4/setuptools/command/easy_install.py", line 225, in run self.easy_install(spec, True) File "/root/svn-install-apps/setuptools-0.6a4/setuptools/command/easy_install.py", line 303, in easy_install return self.install_item(None, spec, tmpdir, deps, True) File "/root/svn-install-apps/setuptools-0.6a4/setuptools/command/easy_install.py", line 338, in install_item dists = self.install_eggs(spec, download, tmpdir) File "/root/svn-install-apps/setuptools-0.6a4/setuptools/command/easy_install.py", line 537, in install_eggs return [self.install_egg(dist_filename, tmpdir)] File "/root/svn-install-apps/setuptools-0.6a4/setuptools/command/easy_install.py", line 589, in install_egg self.check_conflicts(dist) File "/root/svn-install-apps/setuptools-0.6a4/setuptools/command/easy_install.py", line 711, in check_conflicts names = dict.fromkeys(dist._get_metadata('top_level.txt')) # XXX private attr File "/root/svn-install-apps/setuptools-0.6a4/pkg_resources.py", line 1834, in _get_metadata for line in self.get_metadata_lines(name): File "/root/svn-install-apps/setuptools-0.6a4/pkg_resources.py", line 894, in get_metadata_lines return yield_lines(self.get_metadata(name)) File "/root/svn-install-apps/setuptools-0.6a4/pkg_resources.py", line 891, in get_metadata return self._get(self._fn(self.egg_info,name)) File "/root/svn-install-apps/setuptools-0.6a4/pkg_resources.py", line 949, in _get return self.loader.get_data(path) OverflowError: signed integer is greater than maximum Thanks Viren -- Viren R Shah Sr. Technical Advisor Virtual Technology Corporation vshah at virtc.com P: 703-333-6246 From vshah at virtc.com Thu Sep 29 18:57:31 2005 From: vshah at virtc.com (Viren Shah) Date: Thu, 29 Sep 2005 12:57:31 -0400 Subject: [Python-Dev] 64-bit bytecode compatibility (was Re: [PEAK] ez_setup on 64-bit linux problem) In-Reply-To: <5.1.1.6.0.20050929122931.02f3cbe0@mail.telecommunity.com> References: <5.1.1.6.0.20050929114339.01f7ea28@mail.telecommunity.com> <5.1.1.6.0.20050929114339.01f7ea28@mail.telecommunity.com> <5.1.1.6.0.20050929122931.02f3cbe0@mail.telecommunity.com> Message-ID: <433C1CFB.6060207@virtc.com> Phillip J. Eby wrote: > At 12:14 PM 9/29/2005 -0400, Viren Shah wrote: > >> File "/root/svn-install-apps/setuptools-0.6a4/pkg_resources.py", >> line 949, in _get >> return self.loader.get_data(path) >> OverflowError: signed integer is greater than maximum > > > Interesting. That looks like it might be a bug in the Python zipimport > module, which is what implements get_data(). Apparently it happens upon > importing as well; I assumed that it was a bytecode incompatibility. > > Checking the revision log, I find that there's a 64-bit fix for > zipimport.c in Python 2.4 that looks like it would fix this issue, but > it has not been backported to any revision of Python 2.3. You're going > to either have to backport the fix yourself and rebuild Python 2.3, or > upgrade to Python 2.4. Sorry. :( Cool! Thanks for the solution. I'll upgrade to python 2.4 and hope it works :-) Thanks for all your help Viren -- Viren R Shah Sr. Technical Advisor Virtual Technology Corporation vshah at virtc.com P: 703-333-6246
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4