I noticed your email address on a list serve related to technology and web development. Our company has developed a simple, risk-free and cost effective method of generating leads and creating awareness for your Company through targeted email marketing. Please read on to find out more about this awesome service.
The process:
Results:
We typically provide our clients with anywhere from 30 – 200 leads per week with our standard package depending on:
We will help develop a customized system for your company that will ensure maximum return.
Pricing:
Our standard package includes the following:
Cost: $750 per Month
The above price is all-inclusive, and no other charges will be incurred. We can also provide higher quantities of distribution if required. Please contact us for details.
If you would like more information on our services or would like to get started please click here
Cordially,
Gary Michaels
From paul@svensson.org Wed Jan 16 00:37:56 2002 From: paul@svensson.org (Paul Svensson) Date: Tue, 15 Jan 2002 19:37:56 -0500 (EST) Subject: [Python-Dev] Utopian String Interpolation In-Reply-To: <3C44B755.6F2251E5@prescod.net> Message-ID: On Tue, 15 Jan 2002, Paul Prescod wrote: >Paul Svensson wrote: >> >>.... >> >> +1 on no prefix, -0 on \$. >> To my eyes, \(whatever) looks much cleaner, tho I'm not sure how >> that would work with the evaluate_expressions flag in (5). > >An offline correspond suggested that and also suggested perhaps \`. \` >is nicely reminicent of `abc` and it does basically the same thing, only >in strings, so I kind of like it. > >>>> `5+3` >'8' >>>> "\`5 + 3` is enough" >8 is enough > >The downside is that larger characters like $ and % are much more clear >to my eyes. Plus there is the whole apos-backtick confusion. I thought of \` as well, but didn't suggest it, mainly for those reasons. >The problem with \( is that that is likely to already be a popular >string in regular expressions. In which case it should either be a raw string, or spelled \\(. (We _really_ need to issue syntax errors on undefined \-sequences) >>... >> > 4. Between the $-sign and the opening paren, it should be possible to >> >put a C-style formatting specification. >> > >> >"pi = \$5.3f(math.pi)". >> > >> >There is no reason to force people to switch to a totally different >> >language feature to get that functionality. I never use it myself but >> >presume that scientists do! >> >> Eek -- feeping creaturism. -2. > >The feature is already there and sometimes used. We either keep two >different ways to spell interpolation or we incorporate it. I don't think interpolation and variable formatting are similar enough to conflate in a single notation -- wasn't it the ungainliness of using the existing variable formatting to interpolate that started this thread ? >> The only reason to add this here is to be able to remove the % operator >> on strings, and I'm not convinced that is the right way to go. >> Anyways, this just begs to be spelled something like \%5.3f(math.pi). >> Printf-like format specifications without a %-character seems just weird. > >The offline correspondant also had this idea and I'm coming around to >it. I'm not particularly happy with that idea; simply mimicking the syntax it was supposed to replace, for little gain. I also think there could be some cause for confusion between \%(foo)s looking in vars() and %(foo)s using the other side of the % operator. >>... >> -0. Here I think is a good place to draw the line before the returns >> diminish too far. I see the major part of the usefulness of string >> interpolation coming from compile time usage, and that also nicely matches >> how all other \-sequences are handled. > >And do what to do templating at runtime? Modulo? string.replace? Or just >don't provide that feature? Also, how to handle interpolation in raw >strings? Since the whole point of raw strings is to _not_ touch what's inside the quotes, I don't see how string interpolation makes much sense there. As for runtime templating, a string method to replace \-sequences seems like a very straightforward idea, that shouldn't need much discussion. Call it "".eval([globals, [locals]]), to get some educational synergy from teaching all the newbies not to give unchecked user input to eval(). I still think compile-time templating would be the more common use, and thus should be the driving issue behind the design. /Paul From nhodgson@bigpond.net.au Wed Jan 16 01:08:34 2002 From: nhodgson@bigpond.net.au (Neil Hodgson) Date: Wed, 16 Jan 2002 12:08:34 +1100 Subject: [Python-Dev] Re: PEP 277: Unicode file name support for Windows NT, was PEP-time ? ... References: <15412.29829.302192.219332@12-248-41-177.client.attbi.com> <006e01c1949c$7631d1b0$0acc8490@neil> <200201032209.g03M9vn01498@mira.informatik.hu-berlin.de> <01ab01c194a9$237b6dc0$0acc8490@neil> <200201032316.g03NGTB02137@mira.informatik.hu-berlin.de> <020601c194b3$c85a4320$0acc8490@neil> <3C357405.CC439CBE@lemburg.com> <200201041046.g04AkKR05898@mira.informatik.hu-berlin.de> <016e01c19639$94c909b0$0acc8490@neil> <200201060033.g060X8c14491@mira.informatik.hu-berlin.de> <021901c19654$21f2e3f0$0acc8490@neil> <200201061214.g06CEtc01656@mira.informatik.hu-berlin.de> <036e01c1972d$dbc88a80$0acc8490@neil> <200201070728.g077SmZ01967@mira.informatik.hu-berlin.de> <003b01c1975d$e7dd3070$0acc8490@neil> <200201072317.g07NHEh01830@mira.informatik.hu-berlin.de> <3C3AC26A.D40842FB@lemburg.com> <02ff01c19cc3$92514540$0acc8490@neil> <200201140711.g0E7BsV01370@mira.informatik.hu-berlin.de> <3C44059C.CFC09899@lemburg.com> <200201152124.g0FLOV702247@mira.informatik.hu-berlin.de> Message-ID: <057101c19e2a$5217efc0$0acc8490@neil> Martin v. Loewis: > I'm still not certain what the meaning of this function is, if it > means "Unicode file names are only restricted by the file system > conventions", then on Unix, it may change at run-time, if a user or > the application sets an UTF-8 locale, switching from the original "C" > locale. The underlying motivation of the function is for code to be able to ask "Is it better to pass Unicode strings to file operations"? For me the main criterion for "better" is whether all files are accessible. It is best to determine this through a test that does not require writing or that is dependent on the user's setup, such as having a "C:" drive. Switching to a UTF-8 locale on Unix will make files inaccessible where their names contain illegal UTF-8 sequences. Neil From jason@jorendorff.com Wed Jan 16 02:53:08 2002 From: jason@jorendorff.com (Jason Orendorff) Date: Tue, 15 Jan 2002 20:53:08 -0600 Subject: [Python-Dev] PEP_215_ (string interpolation) alternative EvalDict In-Reply-To: Message-ID: > But your example suggests to me: > > >>> input('?: ') > ?: $'$os.system("rm -rm *" )' > > I guess you need to special case that out of the compiler also. > ( Are there any others lurking about ? ) The user could just as well type ?: os.system("rm -rf *") and save some keystrokes. input() is totally insecure. Always has been. Nothing new here. ## Jason Orendorff http://www.jorendorff.com/ From guido@python.org Wed Jan 16 03:05:49 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 15 Jan 2002 22:05:49 -0500 Subject: [Python-Dev] RELEASED - Python 2.1.2 (final) Message-ID: <200201160305.WAA21292@cj20424-a.reston1.va.home.com> I've released the final version of Python 2.1.2 - a bugfix release for Python 2.1. I recommend everyone who is using Python 2.1 or 2.1.1 to upgrade to 2.1.2 -- this release fixes a few crashes. Read about it and download it here: http://www.python.org/2.1.2/ My special thanks go out to Anthony Baxter, the relentless 2.1.2 releasemeister (and for the use of his timezone so I can call this a January 16 release without having to stay up until after midnight :-). --Guido van Rossum (home page: http://www.python.org/~guido/) From rsc@plan9.bell-labs.com Wed Jan 16 03:26:51 2002 From: rsc@plan9.bell-labs.com (Russ Cox) Date: Tue, 15 Jan 2002 22:26:51 -0500 Subject: [Python-Dev] thread_foobar.h routines Message-ID: I'm writing thread routines for the Plan 9 port of Python. Is it correct that: PyThread_acquire_lock returns 1 on success, 0 on failure. PyThread_down_sema returns 0 on success, -1 on failure. It appears that way, but the inconsistency bothers me. Thanks. Russ From guido@python.org Wed Jan 16 03:33:08 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 15 Jan 2002 22:33:08 -0500 Subject: [Python-Dev] thread_foobar.h routines In-Reply-To: Your message of "Tue, 15 Jan 2002 22:26:51 EST." References: Message-ID: <200201160333.WAA22447@cj20424-a.reston1.va.home.com> > I'm writing thread routines for the Plan 9 port of Python. > > Is it correct that: > > PyThread_acquire_lock returns 1 on success, 0 on failure. > PyThread_down_sema returns 0 on success, -1 on failure. > > It appears that way, but the inconsistency bothers me. Me too. The PyThread_*_sema routines are not used, and I would recommend that you not bother implementing them at all. (If anyone used them, we would have heard a complaint -- in some thread implementations these return -1 for failure, in others 0. :-) We should cut these out of the sources. --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Wed Jan 16 04:00:38 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 15 Jan 2002 23:00:38 -0500 (EST) Subject: [Python-Dev] thread_foobar.h routines In-Reply-To: <200201160333.WAA22447@cj20424-a.reston1.va.home.com> References: <200201160333.WAA22447@cj20424-a.reston1.va.home.com> Message-ID: <15428.64230.707183.14133@cj42289-a.reston1.va.home.com> Guido van Rossum writes: > Me too. The PyThread_*_sema routines are not used, and I would > recommend that you not bother implementing them at all. (If anyone > used them, we would have heard a complaint -- in some thread > implementations these return -1 for failure, in others 0. :-) > > We should cut these out of the sources. I'll be glad to do this. A quick grep seems to show that this really does apply to *all* PyThread_*_sema() routines. If there are no objections, I'll have this done quickly. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From jepler@inetnebr.com Wed Jan 16 04:21:16 2002 From: jepler@inetnebr.com (jepler@inetnebr.com) Date: Tue, 15 Jan 2002 22:21:16 -0600 Subject: PEP 215 (was Re: [Python-Dev] PEP 216 (string interpolation) alternative EvalDict) In-Reply-To: <15427.54378.963060.448829@anthem.wooz.org> References: <3C439693.7A2A3724@prescod.net> <15427.54378.963060.448829@anthem.wooz.org> Message-ID: <20020115222113.A987@unpythonic.dhs.org> On Tue, Jan 15, 2002 at 02:04:10AM -0500, Barry A. Warsaw wrote: > [1] I use _() ostensibly to mark translatable strings, but it has a > side benefit in that it interpolates into the string named variables > from the locals and globals of the calling context. It does this by > using sys._getframe(1) in Python 2.1 and try/except hackery in older > versions of Python. I find it quite handy, and admittedly magical, > but then I'm not suggesting it become a standard Python feature. :) This caught my eye. How will programs that use PEP215 for string interpolation be translatable? All translation systems use some method of identifying the strings in source code, then permitting mapping from the string identifiers to the real strings at runtime. With "gettext", the "string identifier" is typically the original-language string, and the marker/mapper is spelled _("string literal"). Given that short introduction, it's obvious how _("hi there, %s") % yourname works, and why _("hi there, %s" % yourname) doesn't work, but how will I use a similar scheme to translate $"hi there, $yourname" ? Obviously, _($"hi there, $yourname") won't work, because it's equivalent to the second, non-working translation example. Well, I guess we could add _ and $_ strings to Python, right? grumble-grumble'ly yours, Jeff From tim.one@home.com Wed Jan 16 05:00:44 2002 From: tim.one@home.com (Tim Peters) Date: Wed, 16 Jan 2002 00:00:44 -0500 Subject: [Python-Dev] thread_foobar.h routines In-Reply-To: <15428.64230.707183.14133@cj42289-a.reston1.va.home.com> Message-ID: [Fred L. Drake, Jr., on removing the unused PyThread_*_sema routines] > If there are no objections, I'll have this done quickly. +1, and you patch looks fine from a skim (and I'd rather fix it retroactively if necessary than bother to apply it first -- live a little, check it in, we're still pre-alpha-1 for 2.3 ). From fdrake@acm.org Wed Jan 16 05:17:36 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 16 Jan 2002 00:17:36 -0500 (EST) Subject: [Python-Dev] thread_foobar.h routines In-Reply-To: References: <15428.64230.707183.14133@cj42289-a.reston1.va.home.com> Message-ID: <15429.3312.503420.50275@cj42289-a.reston1.va.home.com> Tim Peters writes: > +1, and you patch looks fine from a skim (and I'd rather fix it > retroactively if necessary than bother to apply it first -- live a little, > check it in, we're still pre-alpha-1 for 2.3 ). Guido asked me to wait a day in case any legitimate reasons to keep those routines popped up from python-dev, otherwise it would be in already! -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From fdrake@acm.org Wed Jan 16 05:22:18 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 16 Jan 2002 00:22:18 -0500 (EST) Subject: [Python-Dev] Intel C/C++ compiler evaluation version Message-ID: <15429.3594.897580.664110@cj42289-a.reston1.va.home.com> Has anyone tried the evaluation version of the Intel C/C++ compiler for Linux 32-bit platforms? They distributed a CD in the most recent version of Linux Magazine, and it appears to be available for download as well. I had trouble getting it going; the evaluation license file they sent me didn't work out of the box with the license manager that got installed. If anyone has gotten it to work, please send instructions around! -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From mhammond@skippinet.com.au Wed Jan 16 05:40:03 2002 From: mhammond@skippinet.com.au (Mark Hammond) Date: Wed, 16 Jan 2002 16:40:03 +1100 Subject: [Python-Dev] guidance sought: merging port related changes to Library modules In-Reply-To: <15426.60010.318856.560347@cj42289-a.reston1.va.home.com> Message-ID: Fred writes: > Guido van Rossum writes: > > The various modules ntpath, posixpath, macpath etc. are not just their > > to support their own platform on itself. They are also there to > > Note that ntpath.abspath() relies on nt._getfullpathname(). It is not > unreasonable for this particular function to require that it actually > be running on NT, so I'm not going to suggest changing this. On the > other hand, it means the portable portions of the module are (mostly) > not tested when the regression test is run on a platform other than > Windows; the ntpath.abspath() test raises an ImportError since > ntpath.abspath() imports the "nt" module within the function, and the > resulting ImportError causes the rest of the unit test to be skipped > and regrtest.py reports that the test is skipped. > > I'd like to change the test so that the abspath() test is only run > if the "nt" module is available: Sigh - this too would be my fault :( Before _getfullpathname() was added to the 'nt' module, there was an attempt to import 'win32api', and if OK, use the equivilent function from that. When I added the new function to 'nt', I removed that import check, in the belief it would now always succeed. This was obviously a bad call ;) (FYI, that was rev 1.35 of ntpath.py) A patch that reinstates the code would be: Index: ntpath.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/ntpath.py,v retrieving revision 1.44 diff -u -r1.44 ntpath.py --- ntpath.py 2001/11/05 21:25:02 1.44 +++ ntpath.py 2002/01/16 05:35:19 @@ -457,8 +457,18 @@ # Return an absolute path. def abspath(path): """Return the absolute version of a path""" - if path: # Empty path must return current working directory. + try: from nt import _getfullpathname + except ImportError: # Not running on Windows - mock up something sensible. + global abspath + def _abspath(path): + if not isabs(path): + path = join(os.getcwd(), path) + return normpath(path) + abspath = _abspath + return _abspath(path) + + if path: # Empty path must return current working directory. try: path = _getfullpathname(path) except WindowsError: This should also solve the test case problem. Thoughts? Mark. From fdrake@acm.org Wed Jan 16 05:53:46 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 16 Jan 2002 00:53:46 -0500 (EST) Subject: [Python-Dev] guidance sought: merging port related changes to Library modules In-Reply-To: References: <15426.60010.318856.560347@cj42289-a.reston1.va.home.com> Message-ID: <15429.5482.746441.920073@cj42289-a.reston1.va.home.com> Mark Hammond writes: > Before _getfullpathname() was added to the 'nt' module, there was an attempt > to import 'win32api', and if OK, use the equivilent function from that. > When I added the new function to 'nt', I removed that import check, in the > belief it would now always succeed. This was obviously a bad call ;) (FYI, > that was rev 1.35 of ntpath.py) > > A patch that reinstates the code would be: ... > This should also solve the test case problem. I haven't tested this, but it looks OK to me. Feel free to check it in. Thanks! -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From martin@v.loewis.de Wed Jan 16 07:08:33 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: Wed, 16 Jan 2002 08:08:33 +0100 Subject: [Python-Dev] Re: PEP 277: Unicode file name support for Windows NT, was PEP-time ? ... In-Reply-To: <01e701c19e11$567f71f0$0acc8490@neil> (nhodgson@bigpond.net.au) References: <15412.29829.302192.219332@12-248-41-177.client.attbi.com> <006e01c1949c$7631d1b0$0acc8490@neil> <200201032209.g03M9vn01498@mira.informatik.hu-berlin.de> <01ab01c194a9$237b6dc0$0acc8490@neil> <200201032316.g03NGTB02137@mira.informatik.hu-berlin.de> <020601c194b3$c85a4320$0acc8490@neil> <3C357405.CC439CBE@lemburg.com> <200201041046.g04AkKR05898@mira.informatik.hu-berlin.de> <016e01c19639$94c909b0$0acc8490@neil> <200201060033.g060X8c14491@mira.informatik.hu-berlin.de> <021901c19654$21f2e3f0$0acc8490@neil> <200201061214.g06CEtc01656@mira.informatik.hu-berlin.de> <036e01c1972d$dbc88a80$0acc8490@neil> <200201070728.g077SmZ01967@mira.informatik.hu-berlin.de> <003b01c1975d$e7dd3070$0acc8490@neil> <200201072317.g07NHEh01830@mira.informatik.hu-berlin.de> <3C3AC26A.D40842FB@lemburg.com> <02ff01c19cc3$92514540$0acc8490@neil> <200201140711.g0E7BsV01370@mira.informatik.hu-berlin.de> <01e701c19e11$567f71f0$0acc8490@neil> Message-ID: <200201160708.g0G78Xr01736@mira.informatik.hu-berlin.de> > Won't this lead to a less useful result as Py_FileSystemDefaultEncoding > will be NULL on, for example, Linux, so if there are names containing > non-ASCII characters then it will either raise an exception or stick '?'s in > the names. So it would be better to use narrow strings there as that will > pass through all file names. On Linux, if the user has set LANG to a reasonable value, and the Python application has invoked setlocale(), Py_FileSystemDefaultEncoding will not be NULL. It still might happen that an individual file name cannot be decoded from the file system encoding, e.g. if the locale is set to UTF-8, but you have a Latin-1 file name (created by a different user). In that exceptional case, I would neither expect an exception, nor expect replacement characters in the Unicode string, but instead use a byte string *for this specific file name*. Just because there is there is the rare chance that you cannot meaningfully interpret a certain file name does not mean that all other installation have to suffer. > You have probably already realised, but Windows 9x will also need a > Unicode preserving listdir but it will have to encode using mbcs. Exactly. Unfortunately, we cannot do anything to avoid replacement characters here, since it is already Windows who will introduce them. In turn, we know that decoding from "mbcs" will always succeed. Regards, Martin From martin@v.loewis.de Wed Jan 16 07:34:12 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: Wed, 16 Jan 2002 08:34:12 +0100 Subject: [Python-Dev] Intel C/C++ compiler evaluation version In-Reply-To: <15429.3594.897580.664110@cj42289-a.reston1.va.home.com> (fdrake@acm.org) References: <15429.3594.897580.664110@cj42289-a.reston1.va.home.com> Message-ID: <200201160734.g0G7YCD01815@mira.informatik.hu-berlin.de> > Has anyone tried the evaluation version of the Intel C/C++ compiler > for Linux 32-bit platforms? They distributed a CD in the most recent > version of Linux Magazine, and it appears to be available for download > as well. > > I had trouble getting it going; the evaluation license file they sent > me didn't work out of the box with the license manager that got > installed. If anyone has gotten it to work, please send instructions > around! We had no problems installing it. The compiler goes into /opt/intel/compiler50/ia32/*, the license into /opt/intel/license/l_cpp.lic. On the Debian system with a alien RPM installation, the RPM postinstall scripts did not execute properly, so we adjusted the configuration files ourselves (in particular, the postinstall script would have created a broken .csh file, anyway). Looking at the iccvars.csh script, make sure the following settings are correct: setenv IA32ROOT /opt/intel/compiler50/ia32 setenv INTEL_FLEXLM_LICENSE /opt/intel/licenses (iccvars.sh accordingly). I don't think we run flexlm; sourcing the appropriate settings is enough. HTH, Martin From nhodgson@bigpond.net.au Wed Jan 16 11:38:54 2002 From: nhodgson@bigpond.net.au (Neil Hodgson) Date: Wed, 16 Jan 2002 22:38:54 +1100 Subject: [Python-Dev] Re: PEP 277: Unicode file name support for Windows NT, was PEP-time ? ... References: <15412.29829.302192.219332@12-248-41-177.client.attbi.com> <006e01c1949c$7631d1b0$0acc8490@neil> <200201032209.g03M9vn01498@mira.informatik.hu-berlin.de> <01ab01c194a9$237b6dc0$0acc8490@neil> <200201032316.g03NGTB02137@mira.informatik.hu-berlin.de> <020601c194b3$c85a4320$0acc8490@neil> <3C357405.CC439CBE@lemburg.com> <200201041046.g04AkKR05898@mira.informatik.hu-berlin.de> <016e01c19639$94c909b0$0acc8490@neil> <200201060033.g060X8c14491@mira.informatik.hu-berlin.de> <021901c19654$21f2e3f0$0acc8490@neil> <200201061214.g06CEtc01656@mira.informatik.hu-berlin.de> <036e01c1972d$dbc88a80$0acc8490@neil> <200201070728.g077SmZ01967@mira.informatik.hu-berlin.de> <003b01c1975d$e7dd3070$0acc8490@neil> <200201072317.g07NHEh01830@mira.informatik.hu-berlin.de> <3C3AC26A.D40842FB@lemburg.com> <02ff01c19cc3$92514540$0acc8490@neil> <200201140711.g0E7BsV01370@mira.informatik.hu-berlin.de> <3C44059C.CFC09899@lemburg.com> Message-ID: <018101c19e82$60c42950$0acc8490@neil> M.-A. Lemburg: > The restriction when compiling Python in wide mode on Windows > should be lifted: The PyUnicode_AsWideChar() API should be used > to convert 4-byte Unicode to wchar_t (which is 2-byte on Windows). I'd prefer not to include this as it adds complexity for little benefit but am prepared to do the implementation if it is required. Neil From barry@zope.com Wed Jan 16 13:17:54 2002 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 16 Jan 2002 08:17:54 -0500 Subject: PEP 215 (was Re: [Python-Dev] PEP 216 (string interpolation) alternative EvalDict) References: <3C439693.7A2A3724@prescod.net> <15427.54378.963060.448829@anthem.wooz.org> <20020115222113.A987@unpythonic.dhs.org> Message-ID: <15429.32130.692161.521257@anthem.wooz.org> >>>>> "jepler" == writes: jepler> Well, I guess we could add _ and $_ strings to Python, jepler> right? Ug. t'' strings have been discussed before w.r.t. i18n markup, but I don't like it. I think it's a mistake to proliferate string prefixes. But search the i18n-sig for more discussion on the topic. -Barry From skip@pobox.com Wed Jan 16 14:22:41 2002 From: skip@pobox.com (Skip Montanaro) Date: Wed, 16 Jan 2002 08:22:41 -0600 Subject: [Python-Dev] deprecate input()? Message-ID: <15429.36017.387707.78193@12-248-41-177.client.attbi.com> I just responded to a question on c.l.py a user had about feeding empty strings to input(). While he didn't say why he called input(), I suspect he thought the semantics were more like raw_input(). In these days of widespread Internet nastiness, shouldn't input() be deprecated? -- Skip Montanaro (skip@pobox.com - http://www.mojam.com/) From mal@lemburg.com Wed Jan 16 18:22:37 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 16 Jan 2002 19:22:37 +0100 Subject: [Python-Dev] Utopian String Interpolation References: <3C446A5B.2E7A22CD@prescod.net> Message-ID: <3C45C4ED.806E64F3@lemburg.com> Paul Prescod wrote: > > I think that if we're going to do string interpolation we might as go > all of the way and have one unified string interpolation model. > > 1. There should be no string-prefix. Instead the string \$ should be > magical in all non-raw literal strings as \x, \n etc. are. (if you want > to do string interpolation on a raw string, you could do it using the > method version below) > > >>> from __future__ import string_interp > > >>> a = "acos(.5) = \$(acos(.5))" > > Embrace the __future__! -1. Too dangerous. If string interpolation makes it into the core, then please use a *new* construct. '\$' is currently interpreted as '\$' and this should not be changed (heck, just think what would happen to all the shell script snippets encoded in Python strings). BTW, why don't you wrap all this interpolation stuff into a module and then call a function to have it apply all the magic you want. If I remember correctly, someone else has already written such a module for Python. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mal@lemburg.com Wed Jan 16 18:48:49 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 16 Jan 2002 19:48:49 +0100 Subject: [Python-Dev] Re: PEP 277: Unicode file name support for Windows NT, was PEP-time ? ... References: <15412.29829.302192.219332@12-248-41-177.client.attbi.com> <006e01c1949c$7631d1b0$0acc8490@neil> <200201032209.g03M9vn01498@mira.informatik.hu-berlin.de> <01ab01c194a9$237b6dc0$0acc8490@neil> <200201032316.g03NGTB02137@mira.informatik.hu-berlin.de> <020601c194b3$c85a4320$0acc8490@neil> <3C357405.CC439CBE@lemburg.com> <200201041046.g04AkKR05898@mira.informatik.hu-berlin.de> <016e01c19639$94c909b0$0acc8490@neil> <200201060033.g060X8c14491@mira.informatik.hu-berlin.de> <021901c19654$21f2e3f0$0acc8490@neil> <200201061214.g06CEtc01656@mira.informatik.hu-berlin.de> <036e01c1972d$dbc88a80$0acc8490@neil> <200201070728.g077SmZ01967@mira.informatik.hu-berlin.de> <003b01c1975d$e7dd3070$0acc8490@neil> <200201072317.g07NHEh01830@mira.informatik.hu-berlin.de> <3C3AC26A.D40842FB@lemburg.com> <02ff01c19cc3$92514540$0acc8490@neil> <200201140711.g0E7BsV01370@mira.informatik.hu-berlin.de> <3C44059C.CFC09899@lemburg.com> <200201152124.g0FLOV702247@mira.informatik.hu-berlin.de> Message-ID: <3C45CB11.ACB2CEE6@lemburg.com> "Martin v. Loewis" wrote: > > > The restriction when compiling Python in wide mode on Windows > > should be lifted: The PyUnicode_AsWideChar() API should be used > > to convert 4-byte Unicode to wchar_t (which is 2-byte on Windows). > > While I agree that this restriction ought to be removed eventually, I > doubt that Python will be usable on Windows with a four-byte Unicode > type in any foreseeable future. Perhaps Neil ought to copy your notes to the PEP, so that we don't forget about this issue. > Just have a look at unicodeobject.c:PyUnicode_DecodeMBCS; it makes the > assumption that a Py_UNICODE* is the same thing as a WCHAR*. That > means that the "mbcs" encoding goes away on Windows if > HAVE_USABLE_WCHAR_T does not hold anymore. > > Also, I believe most of PythonWin also assumes HAVE_USABLE_WCHAR_T > (didn't check, though). > > > Why is "unicodefilenames" a function and not a constant ? > > In the Windows binary, you need a run-time check to see whether this > is DOS/W9x, or NT/W2k/XP; on DOS, the Unicode API is not available > (you still can pass Unicode file names to open and listdir, but they > will get converted through the MBCS encoding). So it clearly is not a > compile time constant. I see. > I'm still not certain what the meaning of this function is, if it > means "Unicode file names are only restricted by the file system > conventions", then on Unix, it may change at run-time, if a user or > the application sets an UTF-8 locale, switching from the original "C" > locale. Doesn't it mean: "posix functions and file() can accept Unicode file names" ? That's what I thought, at least; whether they succeed or not is another question and could well be handled by run-time errors (e.g. on Unix it is not at all clear whether NFS, Samba or some other more exotic file system can handle the encoding chosen by Python or the program). Perhaps we ought to drop that function altogether and let the various file IO functions raise run-time errors instead ?! -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mal@lemburg.com Wed Jan 16 18:54:00 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 16 Jan 2002 19:54:00 +0100 Subject: [Python-Dev] Re: PEP 277: Unicode file name support for Windows NT, was PEP-time ? ... References: <15412.29829.302192.219332@12-248-41-177.client.attbi.com> <006e01c1949c$7631d1b0$0acc8490@neil> <200201032209.g03M9vn01498@mira.informatik.hu-berlin.de> <01ab01c194a9$237b6dc0$0acc8490@neil> <200201032316.g03NGTB02137@mira.informatik.hu-berlin.de> <020601c194b3$c85a4320$0acc8490@neil> <3C357405.CC439CBE@lemburg.com> <200201041046.g04AkKR05898@mira.informatik.hu-berlin.de> <016e01c19639$94c909b0$0acc8490@neil> <200201060033.g060X8c14491@mira.informatik.hu-berlin.de> <021901c19654$21f2e3f0$0acc8490@neil> <200201061214.g06CEtc01656@mira.informatik.hu-berlin.de> <036e01c1972d$dbc88a80$0acc8490@neil> <200201070728.g077SmZ01967@mira.informatik.hu-berlin.de> <003b01c1975d$e7dd3070$0acc8490@neil> <200201072317.g07NHEh01830@mira.informatik.hu-berlin.de> <3C3AC26A.D40842FB@lemburg.com> <02ff01c19cc3$92514540$0acc8490@neil> <200201140711.g0E7BsV01370@mira.informatik.hu-berlin.de> <3C44059C.CFC09899@lemburg.com> <018101c19e82$60c42950$0acc8490@neil> Message-ID: <3C45CC48.A71EEA8A@lemburg.com> Neil Hodgson wrote: > > M.-A. Lemburg: > > > The restriction when compiling Python in wide mode on Windows > > should be lifted: The PyUnicode_AsWideChar() API should be used > > to convert 4-byte Unicode to wchar_t (which is 2-byte on Windows). > > I'd prefer not to include this as it adds complexity for little benefit > but am prepared to do the implementation if it is required. Point taken, but please mention this in the PEP. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From martin@v.loewis.de Wed Jan 16 19:09:24 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: Wed, 16 Jan 2002 20:09:24 +0100 Subject: [Python-Dev] Re: PEP 277: Unicode file name support for Windows NT, was PEP-time ? ... In-Reply-To: <3C45CB11.ACB2CEE6@lemburg.com> (mal@lemburg.com) References: <15412.29829.302192.219332@12-248-41-177.client.attbi.com> <006e01c1949c$7631d1b0$0acc8490@neil> <200201032209.g03M9vn01498@mira.informatik.hu-berlin.de> <01ab01c194a9$237b6dc0$0acc8490@neil> <200201032316.g03NGTB02137@mira.informatik.hu-berlin.de> <020601c194b3$c85a4320$0acc8490@neil> <3C357405.CC439CBE@lemburg.com> <200201041046.g04AkKR05898@mira.informatik.hu-berlin.de> <016e01c19639$94c909b0$0acc8490@neil> <200201060033.g060X8c14491@mira.informatik.hu-berlin.de> <021901c19654$21f2e3f0$0acc8490@neil> <200201061214.g06CEtc01656@mira.informatik.hu-berlin.de> <036e01c1972d$dbc88a80$0acc8490@neil> <200201070728.g077SmZ01967@mira.informatik.hu-berlin.de> <003b01c1975d$e7dd3070$0acc8490@neil> <200201072317.g07NHEh01830@mira.informatik.hu-berlin.de> <3C3AC26A.D40842FB@lemburg.com> <02ff01c19cc3$92514540$0acc8490@neil> <200201140711.g0E7BsV01370@mira.informatik.hu-berlin.de> <3C44059C.CFC09899@lemburg.com> <200201152124.g0FLOV702247@mira.informatik.hu-berlin.de> <3C45CB11.ACB2CEE6@lemburg.com> Message-ID: <200201161909.g0GJ9OK01822@mira.informatik.hu-berlin.de> > > I'm still not certain what the meaning of this function is, if it > > means "Unicode file names are only restricted by the file system > > conventions", then on Unix, it may change at run-time, if a user or > > the application sets an UTF-8 locale, switching from the original "C" > > locale. > > Doesn't it mean: "posix functions and file() can accept Unicode file > names" ? Neil has given his own interpretation (return true if it is *better* to pass Unicode strings than to pass byte strings). You property (accepts Unicode) is true on all Python installations since 2.2: if you pass a Unicode string, it will try the file system encoding; if that is NULL, it will try the system encoding. So on all Python systems, open(u"foo.txt","w") currently succeeds everywhere (unless Unicode was completely disabled in the port). > That's what I thought, at least; whether they succeed or not > is another question and could well be handled by run-time errors > (e.g. on Unix it is not at all clear whether NFS, Samba or some > other more exotic file system can handle the encoding chosen by > Python or the program). For NFS, it is clear - file names are null-terminated byte strings (AFAIK). For Samba, I believe it depends on the installation, specifically whether the encoding of Samba matches the one of the user. For more exotic file systems, it is not all that clear. > Perhaps we ought to drop that function altogether and let the > various file IO functions raise run-time errors instead ?! That was my suggestion as well. However, Neil points out that, on Windows, passing Unicode is sometimes better: For some files, there is no byte string file name to identify the file (if the file name is not representable in MBCS). OTOH, on Unix, some files cannot be accessed with a Unicode string, if the file name is invalid in the user's encoding. It turns out that only OS X really got it right: For each file, there is both a byte string name, and a Unicode name. Regards, Martin From paul@prescod.net Wed Jan 16 19:43:49 2002 From: paul@prescod.net (Paul Prescod) Date: Wed, 16 Jan 2002 11:43:49 -0800 Subject: [Python-Dev] Utopian String Interpolation References: <3C446A5B.2E7A22CD@prescod.net> <3C45C4ED.806E64F3@lemburg.com> Message-ID: <3C45D7F5.BF260898@prescod.net> "M.-A. Lemburg" wrote: > >... > > Embrace the __future__! > > -1. > > Too dangerous. It isn't dangerous. That's precisely what __future__ is for! It is no more dangerous than any other feature that uses __future__. > ... If string interpolation makes it into the core, > then please use a *new* construct. '\$' is currently interpreted > as '\$' and this should not be changed (heck, just think what would > happen to all the shell script snippets encoded in Python strings). No, this should be changed. Completely ignoring string interpolation, I am strongly in favour of changing the behaviour of the literal string parser so that unknown \-combinations raise a SyntaxError. If you don't want a backslash to be interpreted as an escape sequence start, you should use a raw string. The Python documentation and grammar already says: escapeseq ::= "\" The documentation says: "Unlike Standard , all unrecognized escape sequences are left in the string unchanged, i.e., the backslash is left in the string. (This behavior is useful when debugging: if an escape sequence is mistyped, the resulting output is more easily recognized as broken.)" That's a weird thing to say. What could be more helpful for debugging than a good old SyntaxError??? > BTW, why don't you wrap all this interpolation stuff into > a module and then call a function to have it apply all the > magic you want. We've been through that in this discussion already. In fact, that's how the discussion started. Paul Prescod From paul@svensson.org Wed Jan 16 20:26:23 2002 From: paul@svensson.org (Paul Svensson) Date: Wed, 16 Jan 2002 15:26:23 -0500 (EST) Subject: [Python-Dev] Utopian String Interpolation In-Reply-To: <3C45D7F5.BF260898@prescod.net> Message-ID: On Wed, 16 Jan 2002, Paul Prescod wrote: >The documentation says: > >"Unlike Standard , all unrecognized escape sequences are left in the >string unchanged, i.e., the backslash is left in the string. (This >behavior is useful when debugging: if an escape sequence is mistyped, >the resulting output is more easily recognized as broken.)" > >That's a weird thing to say. What could be more helpful for debugging >than a good old SyntaxError??? The usefulness is relative; it's arguably easier to find the problem and fix it if the \ remains in the string than if it's simply removed (as C does, tho most compilers issue a warning). It could also be argued that you get more nutritinal value by eating only the black raisins from the cake then by eating just the golden raisins... /Paul From paul@prescod.net Wed Jan 16 20:40:39 2002 From: paul@prescod.net (Paul Prescod) Date: Wed, 16 Jan 2002 12:40:39 -0800 Subject: [Python-Dev] Utopian String Interpolation References: Message-ID: <3C45E547.8B8FDEFF@prescod.net> Paul Svensson wrote: > >... > > The usefulness is relative; it's arguably easier to find the > problem and fix it if the \ remains in the string than if it's > simply removed (as C does, tho most compilers issue a warning). Yeah, I understood that. I just don't understand why it isn't like most other things in Python. Python tends to be strict about things that are likely mistakes, rather than helping you "debug them" after passing them through silently. Paul Prescod From guido@python.org Wed Jan 16 21:31:02 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 16 Jan 2002 16:31:02 -0500 Subject: [Python-Dev] Utopian String Interpolation In-Reply-To: Your message of "Wed, 16 Jan 2002 12:40:39 PST." <3C45E547.8B8FDEFF@prescod.net> References: <3C45E547.8B8FDEFF@prescod.net> Message-ID: <200201162131.QAA26108@cj20424-a.reston1.va.home.com> > Yeah, I understood that. I just don't understand why it isn't like most > other things in Python. Python tends to be strict about things that are > likely mistakes, rather than helping you "debug them" after passing them > through silently. > > Paul Prescod The "why" is that long ago Python didn't have raw strings but it did have regular expressions. I thought it would be painful to have to double all backslashes used for the regex syntax. It would be hard to change this policy now. --Guido van Rossum (home page: http://www.python.org/~guido/) From paul@prescod.net Wed Jan 16 21:59:17 2002 From: paul@prescod.net (Paul Prescod) Date: Wed, 16 Jan 2002 13:59:17 -0800 Subject: [Python-Dev] Utopian String Interpolation References: <3C45E547.8B8FDEFF@prescod.net> <200201162131.QAA26108@cj20424-a.reston1.va.home.com> Message-ID: <3C45F7B5.7603F4D1@prescod.net> Guido van Rossum wrote: > >... > > The "why" is that long ago Python didn't have raw strings but it did > have regular expressions. I thought it would be painful to have to > double all backslashes used for the regex syntax. Aha. > It would be hard to change this policy now. How about an optional warning which, after a year or so, would be turned on by default, and then a year or so after that would be an error? This same issue may effect some eventual merging of literal strings and Unicode literals because \N, \u etc. are treated differently in strings than in Unicode literals. And even if literal strings and Unicode strings are never merged, \N could be useful in ordinary strings. Paul Prescod From jepler@inetnebr.com Wed Jan 16 22:09:29 2002 From: jepler@inetnebr.com (Jeff Epler) Date: Wed, 16 Jan 2002 16:09:29 -0600 Subject: PEP 215 (was Re: [Python-Dev] PEP 216 (string interpolation) alternative EvalDict) In-Reply-To: <15429.32130.692161.521257@anthem.wooz.org> References: <3C439693.7A2A3724@prescod.net> <15427.54378.963060.448829@anthem.wooz.org> <20020115222113.A987@unpythonic.dhs.org> <15429.32130.692161.521257@anthem.wooz.org> Message-ID: <20020116160928.A473@unpythonic.dhs.org> On Wed, Jan 16, 2002 at 08:17:54AM -0500, Barry A. Warsaw wrote: > > >>>>> "jepler" == writes: > > jepler> Well, I guess we could add _ and $_ strings to Python, > jepler> right? > > Ug. t'' strings have been discussed before w.r.t. i18n markup, but I > don't like it. ... and you like $'' strings? That suggestion was intended to bring a bad taste to *everybody*'s mouth, as much as t'' alone does to yours. (Hmm, and then I might need a raw unicode interpolated translated string ... is that spelled $_ur'' or r_$u'' ?) Jeff From guido@python.org Wed Jan 16 22:19:53 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 16 Jan 2002 17:19:53 -0500 Subject: [Python-Dev] Utopian String Interpolation In-Reply-To: Your message of "Wed, 16 Jan 2002 13:59:17 PST." <3C45F7B5.7603F4D1@prescod.net> References: <3C45E547.8B8FDEFF@prescod.net> <200201162131.QAA26108@cj20424-a.reston1.va.home.com> <3C45F7B5.7603F4D1@prescod.net> Message-ID: <200201162219.RAA26286@cj20424-a.reston1.va.home.com> > How about an optional warning which, after a year or so, would be turned > on by default, and then a year or so after that would be an error? > > This same issue may effect some eventual merging of literal strings and > Unicode literals because \N, \u etc. are treated differently in strings > than in Unicode literals. And even if literal strings and Unicode > strings are never merged, \N could be useful in ordinary strings. -1 I don't find this enough of a problem to invoke the heavy gun of a language change. --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Wed Jan 16 22:28:18 2002 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 16 Jan 2002 17:28:18 -0500 Subject: PEP 215 (was Re: [Python-Dev] PEP 216 (string interpolation) alternative EvalDict) References: <3C439693.7A2A3724@prescod.net> <15427.54378.963060.448829@anthem.wooz.org> <20020115222113.A987@unpythonic.dhs.org> <15429.32130.692161.521257@anthem.wooz.org> <20020116160928.A473@unpythonic.dhs.org> Message-ID: <15429.65154.514355.230711@anthem.wooz.org> On Wed, Jan 16, 2002 at 08:17:54AM -0500, Barry A. Warsaw wrote: > Ug. t'' strings have been discussed before w.r.t. i18n markup, but I > don't like it. >>>>> "JE" == Jeff Epler writes: JE> ... and you like $'' strings? No! :) JE> That suggestion was intended to bring a bad taste to JE> *everybody*'s mouth, as much as t'' alone does to yours. Ah, no wonder I've had to drink 3 sodas today. I wondered what that foul flavor was, especially since I made sure to brush my teeth this morning! JE> (Hmm, and then I might need a raw unicode interpolated JE> translated string ... is that spelled $_ur'' or r_$u'' ?) Exactly why I'm against adding more string prefixes. Remember that the _ thingie we currently recommend for gettext /isn't/ prefix proliferation. E.g.: _(u'translate this') _(ru'and this') It's just a function call with a convenient name (and even that's just a convention, of course). -Barry From paul@svensson.org Wed Jan 16 22:29:13 2002 From: paul@svensson.org (Paul Svensson) Date: Wed, 16 Jan 2002 17:29:13 -0500 (EST) Subject: [Python-Dev] Utopian String Interpolation In-Reply-To: <200201162131.QAA26108@cj20424-a.reston1.va.home.com> Message-ID: On Wed, 16 Jan 2002, Guido van Rossum wrote: >> Yeah, I understood that. I just don't understand why it isn't like most >> other things in Python. Python tends to be strict about things that are >> likely mistakes, rather than helping you "debug them" after passing them >> through silently. >> >> Paul Prescod > >The "why" is that long ago Python didn't have raw strings but it did >have regular expressions. I thought it would be painful to have to >double all backslashes used for the regex syntax. > >It would be hard to change this policy now. Yeah, it would be like, say, changing the semantics of integer division. Sometimes it's better to do what's right than what's easy. /Paul From paul@svensson.org Wed Jan 16 22:43:48 2002 From: paul@svensson.org (Paul Svensson) Date: Wed, 16 Jan 2002 17:43:48 -0500 (EST) Subject: [Python-Dev] Utopian String Interpolation In-Reply-To: <3C45F7B5.7603F4D1@prescod.net> Message-ID: On Wed, 16 Jan 2002, Paul Prescod wrote: >Guido van Rossum wrote: >> >>... >> >> The "why" is that long ago Python didn't have raw strings but it did >> have regular expressions. I thought it would be painful to have to >> double all backslashes used for the regex syntax. > >Aha. > >> It would be hard to change this policy now. > >How about an optional warning which, after a year or so, would be turned >on by default, and then a year or so after that would be an error? Such a warning might prove to be a useful debugging tool, even if the language never changed. Maybe it would be a useful addition to PyChecker or some similar tool ? foot-in-the-door-ly, /Paul From Jack.Jansen@oratrix.nl Wed Jan 16 22:56:23 2002 From: Jack.Jansen@oratrix.nl (Jack Jansen) Date: Wed, 16 Jan 2002 23:56:23 +0100 Subject: [Python-Dev] Extending types in C - help needed Message-ID: <43ABBA6E-0AD4-11D6-A4BB-003065517236@oratrix.nl> In the discussion on my request for an ("O@", typeobject, void **) format for PyArg_Parse and Py_BuildValue MAL suggested that I could get the same functionality by creating a type WrapperTypeObject, which would be a subtype of TypeObject with extra fields pointing to the _New() and _Convert() routines to convert Python objects from/to C pointers. This would be good enough for me, because then types wanting to participate in the wrapper protocol would subtype WrapperTypeObject in stead of TypeObject, and two global routines could return the _New and _Convert routines given the type object, and we wouldn't need yet another PyArg_Parse format specifier. However, after digging high and low I haven't been able to deduce how I would then use this WrapperType in C as the type for my extension module objects. Are there any examples? If not, could someone who understands the new inheritance scheme give me some clues as to how to do this? -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From mhammond@skippinet.com.au Thu Jan 17 00:53:37 2002 From: mhammond@skippinet.com.au (Mark Hammond) Date: Thu, 17 Jan 2002 11:53:37 +1100 Subject: [Python-Dev] Re: PEP 277: Unicode file name support for Windows NT, was PEP-time ? ... In-Reply-To: <200201152124.g0FLOV702247@mira.informatik.hu-berlin.de> Message-ID: > Also, I believe most of PythonWin also assumes HAVE_USABLE_WCHAR_T > (didn't check, though). FYI, all the win32 extensions use their own Unicode API. These extensions had Unicode before Python did! These wrapper functions are abstract enough that they should be able to withstand any changes to Python's Unicode implementation quite simply - probably at the cost of extra copies and transformations in those wrappers. Mark. From guido@python.org Thu Jan 17 06:28:03 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 17 Jan 2002 01:28:03 -0500 Subject: [Python-Dev] deprecate input()? In-Reply-To: Your message of "Wed, 16 Jan 2002 08:22:41 CST." <15429.36017.387707.78193@12-248-41-177.client.attbi.com> References: <15429.36017.387707.78193@12-248-41-177.client.attbi.com> Message-ID: <200201170628.BAA28567@cj20424-a.reston1.va.home.com> > I just responded to a question on c.l.py a user had about feeding empty > strings to input(). While he didn't say why he called input(), I suspect he > thought the semantics were more like raw_input(). > > In these days of widespread Internet nastiness, shouldn't input() be > deprecated? Why? I imagine this is only used for interactive input, and then it's the computer's owner who is typing. --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Thu Jan 17 10:11:08 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 17 Jan 2002 11:11:08 +0100 Subject: [Python-Dev] Utopian String Interpolation References: <3C446A5B.2E7A22CD@prescod.net> <3C45C4ED.806E64F3@lemburg.com> <3C45D7F5.BF260898@prescod.net> Message-ID: <3C46A33C.D97C831C@lemburg.com> Paul Prescod wrote: > > "M.-A. Lemburg" wrote: > > > >... > > > Embrace the __future__! > > > > -1. > > > > Too dangerous. > > It isn't dangerous. That's precisely what __future__ is for! It is no > more dangerous than any other feature that uses __future__. It is. Currently Python strings are just that: immutable strings. Now, you suddenly add dynamics to then. This will cause nightmares in terms of security. Note that Python hasn't really had a need for Perl's "taint" because of this. I wouldn't want to see that change in any way. If you really need this, either use a string prefix or call a specific function which implements string interpolation. At least then things are obvious and explicit. > > ... If string interpolation makes it into the core, > > then please use a *new* construct. '\$' is currently interpreted > > as '\$' and this should not be changed (heck, just think what would > > happen to all the shell script snippets encoded in Python strings). > > No, this should be changed. Huh ? I bet RedHat and thousands of sysadmins who have switched from shell or Perl to Python would have strong objections. > Completely ignoring string interpolation, I > am strongly in favour of changing the behaviour of the literal string > parser so that unknown \-combinations raise a SyntaxError. If you don't > want a backslash to be interpreted as an escape sequence start, you > should use a raw string. > > The Python documentation and grammar already says: > > escapeseq ::= "\" > > The documentation says: > > "Unlike Standard , all unrecognized escape sequences are left in the > string unchanged, i.e., the backslash is left in the string. (This > behavior is useful when debugging: if an escape sequence is mistyped, > the resulting output is more easily recognized as broken.)" > > That's a weird thing to say. What could be more helpful for debugging > than a good old SyntaxError??? If there's nothing wrong with the escape why raise a SyntaxError ? > > BTW, why don't you wrap all this interpolation stuff into > > a module and then call a function to have it apply all the > > magic you want. > > We've been through that in this discussion already. In fact, that's how > the discussion started. I've jumped in at a rather late point. Perhaps you ought to rewind the discussion then and start discussing in a different direction :-) E.g. about the syntax to be used in the interpolation and where, when and in which context to evaluate the strings. There are so many options that I can't really see any benefit from chosing only one and hard-coding it into the language. Other users will have other requirement which are likely not to combine well with the one implementation you have in mind. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mal@lemburg.com Thu Jan 17 10:19:33 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 17 Jan 2002 11:19:33 +0100 Subject: [Python-Dev] Re: PEP 277: Unicode file name support for Windows NT, was PEP-time ? ... References: <15412.29829.302192.219332@12-248-41-177.client.attbi.com> <01ab01c194a9$237b6dc0$0acc8490@neil> <200201032316.g03NGTB02137@mira.informatik.hu-berlin.de> <020601c194b3$c85a4320$0acc8490@neil> <3C357405.CC439CBE@lemburg.com> <200201041046.g04AkKR05898@mira.informatik.hu-berlin.de> <016e01c19639$94c909b0$0acc8490@neil> <200201060033.g060X8c14491@mira.informatik.hu-berlin.de> <021901c19654$21f2e3f0$0acc8490@neil> <200201061214.g06CEtc01656@mira.informatik.hu-berlin.de> <036e01c1972d$dbc88a80$0acc8490@neil> <200201070728.g077SmZ01967@mira.informatik.hu-berlin.de> <003b01c1975d$e7dd3070$0acc8490@neil> <200201072317.g07NHEh01830@mira.informatik.hu-berlin.de> <3C3AC26A.D40842FB@lemburg.com> <02ff01c19cc3$92514540$0acc8490@neil> <200201140711.g0E7BsV01370@mira.informatik.hu-berlin.de> <3C44059C.CFC09899@lemburg.com> <200201152124.g0FLOV702247@mira.informatik.hu-berlin.de> <3C45CB11.ACB2CEE6@lemburg.com> <200201161909.g0GJ9OK01822@mira.informatik.hu-berlin.de> Message-ID: <3C46A535.3C579501@lemburg.com> "Martin v. Loewis" wrote: > > [unicodefilenames()] > > Perhaps we ought to drop that function altogether and let the > > various file IO functions raise run-time errors instead ?! > > That was my suggestion as well. However, Neil points out that, on > Windows, passing Unicode is sometimes better: For some files, there is > no byte string file name to identify the file (if the file name is not > representable in MBCS). OTOH, on Unix, some files cannot be accessed > with a Unicode string, if the file name is invalid in the user's > encoding. Sounds like the run-time error solution would at least "solve" the issue in terms of making it depend on the used file name and underlying OS or file system. I'd say: let the different file name based APIs try hard enough and then have them bail out if they can't handle the particular case. > It turns out that only OS X really got it right: For each file, there > is both a byte string name, and a Unicode name. I suppose this is due to the fact that Mac file systems store extended attributes (much like what OS/2 does too) along with the file -- that's a really nice way of being able to extend file system semantics on a per-file basis; much better than the Windows Registry or the MIME guess-by-extension mechanisms. Oh well. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mal@lemburg.com Thu Jan 17 10:29:45 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 17 Jan 2002 11:29:45 +0100 Subject: [Python-Dev] Extending types in C - help needed References: <43ABBA6E-0AD4-11D6-A4BB-003065517236@oratrix.nl> Message-ID: <3C46A799.2E7AD7DB@lemburg.com> Jack Jansen wrote: > > In the discussion on my request for an ("O@", typeobject, > void **) format for PyArg_Parse and Py_BuildValue MAL suggested Thomas Heller suggested this. I am more in favour of exposing the pickle reduce API through "O@", that is have PyArgTuple_Parse() call the .__reduce__() method of the object. This will then return (factory, state_tuple) and these could then be exposed to the C function via two PyObject*. Note that there's no need for any type object magic. If this becomes a common case, it may be worthwhile to add a tp_reduce slot to type objects though. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From nhodgson@bigpond.net.au Thu Jan 17 11:04:53 2002 From: nhodgson@bigpond.net.au (Neil Hodgson) Date: Thu, 17 Jan 2002 22:04:53 +1100 Subject: [Python-Dev] Re: PEP 277: Unicode file name support for Windows NT, was PEP-time ? ... References: <15412.29829.302192.219332@12-248-41-177.client.attbi.com> <01ab01c194a9$237b6dc0$0acc8490@neil> <200201032316.g03NGTB02137@mira.informatik.hu-berlin.de> <020601c194b3$c85a4320$0acc8490@neil> <3C357405.CC439CBE@lemburg.com> <200201041046.g04AkKR05898@mira.informatik.hu-berlin.de> <016e01c19639$94c909b0$0acc8490@neil> <200201060033.g060X8c14491@mira.informatik.hu-berlin.de> <021901c19654$21f2e3f0$0acc8490@neil> <200201061214.g06CEtc01656@mira.informatik.hu-berlin.de> <036e01c1972d$dbc88a80$0acc8490@neil> <200201070728.g077SmZ01967@mira.informatik.hu-berlin.de> <003b01c1975d$e7dd3070$0acc8490@neil> <200201072317.g07NHEh01830@mira.informatik.hu-berlin.de> <3C3AC26A.D40842FB@lemburg.com> <02ff01c19cc3$92514540$0acc8490@neil> <200201140711.g0E7BsV01370@mira.informatik.hu-berlin.de> <3C44059C.CFC09899@lemburg.com> <200201152124.g0FLOV702247@mira.informatik.hu-berlin.de> <3C45CB11.ACB2CEE6@lemburg.com> <200201161909.g0GJ9OK01822@mira.informatik.hu-berlin.de> <3C46A535.3C579501@lemburg.com> Message-ID: <08e201c19f46$cad5f070$0acc8490@neil> M.-A. Lemburg, regarding unicodefilenames(): > Sounds like the run-time error solution would at least "solve" > the issue in terms of making it depend on the used file name > and underlying OS or file system. It is much better to choose a technique that will always work rather than try to recover from a technique that may fail. unicodefilenames() can be dropped in favour of explicit OS and version checks but this is replacing a simple robust check with a more fragile one. unicodefilenames() will allow other environments to declare that client code will be more robust by choosing to use Unicode strings as file name arguments. This could include UTF-8 based systems such as OS X and BeOS, as well as Windows variants like CE. Neil From mal@lemburg.com Thu Jan 17 11:36:21 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 17 Jan 2002 12:36:21 +0100 Subject: [Python-Dev] Re: PEP 277: Unicode file name support for Windows NT, was PEP-time ? ... References: <15412.29829.302192.219332@12-248-41-177.client.attbi.com> <020601c194b3$c85a4320$0acc8490@neil> <3C357405.CC439CBE@lemburg.com> <200201041046.g04AkKR05898@mira.informatik.hu-berlin.de> <016e01c19639$94c909b0$0acc8490@neil> <200201060033.g060X8c14491@mira.informatik.hu-berlin.de> <021901c19654$21f2e3f0$0acc8490@neil> <200201061214.g06CEtc01656@mira.informatik.hu-berlin.de> <036e01c1972d$dbc88a80$0acc8490@neil> <200201070728.g077SmZ01967@mira.informatik.hu-berlin.de> <003b01c1975d$e7dd3070$0acc8490@neil> <200201072317.g07NHEh01830@mira.informatik.hu-berlin.de> <3C3AC26A.D40842FB@lemburg.com> <02ff01c19cc3$92514540$0acc8490@neil> <200201140711.g0E7BsV01370@mira.informatik.hu-berlin.de> <3C44059C.CFC09899@lemburg.com> <200201152124.g0FLOV702247@mira.informatik.hu-berlin.de> <3C45CB11.ACB2CEE6@lemburg.com> <200201161909.g0GJ9OK01822@mira.informatik.hu-berlin.de> <3C46A535.3C579501@lemburg.com> <08e201c19f46$cad5f070$0acc8490@neil> Message-ID: <3C46B735.9C433F60@lemburg.com> Neil Hodgson wrote: > > M.-A. Lemburg, regarding unicodefilenames(): > > > Sounds like the run-time error solution would at least "solve" > > the issue in terms of making it depend on the used file name > > and underlying OS or file system. > > It is much better to choose a technique that will always work rather than > try to recover from a technique that may fail. Is it really ? The problem is that under some OSes it is possible to work with multiple very different file system from within a single Python program. In those cases, the unicodefilename() API wouldn't really help all that much. > unicodefilenames() can be dropped in favour of explicit OS and version > checks but this is replacing a simple robust check with a more fragile one. What kind of checks do you have in mind then ? If possible, it should be possible to pass unicodefilenames() a path to check for Unicode- capability, since on Unix (and probably Mac OS X as well), the path decides which file system get's the ioctrl calls. > unicodefilenames() will allow other environments to declare that client code > will be more robust by choosing to use Unicode strings as file name > arguments. This could include UTF-8 based systems such as OS X and BeOS, as > well as Windows variants like CE. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From martin@v.loewis.de Thu Jan 17 11:42:21 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: Thu, 17 Jan 2002 12:42:21 +0100 Subject: [Python-Dev] Re: PEP 277: Unicode file name support for Windows NT, was PEP-time ? ... In-Reply-To: <3C46A535.3C579501@lemburg.com> (mal@lemburg.com) References: <15412.29829.302192.219332@12-248-41-177.client.attbi.com> <01ab01c194a9$237b6dc0$0acc8490@neil> <200201032316.g03NGTB02137@mira.informatik.hu-berlin.de> <020601c194b3$c85a4320$0acc8490@neil> <3C357405.CC439CBE@lemburg.com> <200201041046.g04AkKR05898@mira.informatik.hu-berlin.de> <016e01c19639$94c909b0$0acc8490@neil> <200201060033.g060X8c14491@mira.informatik.hu-berlin.de> <021901c19654$21f2e3f0$0acc8490@neil> <200201061214.g06CEtc01656@mira.informatik.hu-berlin.de> <036e01c1972d$dbc88a80$0acc8490@neil> <200201070728.g077SmZ01967@mira.informatik.hu-berlin.de> <003b01c1975d$e7dd3070$0acc8490@neil> <200201072317.g07NHEh01830@mira.informatik.hu-berlin.de> <3C3AC26A.D40842FB@lemburg.com> <02ff01c19cc3$92514540$0acc8490@neil> <200201140711.g0E7BsV01370@mira.informatik.hu-berlin.de> <3C44059C.CFC09899@lemburg.com> <200201152124.g0FLOV702247@mira.informatik.hu-berlin.de> <3C45CB11.ACB2CEE6@lemburg.com> <200201161909.g0GJ9OK01822@mira.informatik.hu-berlin.de> <3C46A535.3C579501@lemburg.com> Message-ID: <200201171142.g0HBgLk01405@mira.informatik.hu-berlin.de> > Sounds like the run-time error solution would at least "solve" > the issue in terms of making it depend on the used file name > and underlying OS or file system. Such a solution is impossible to implement in some case. E.g. on Windows, if you use the ANSI (*A) APIs to list the directory contents, Windows will *silently* (AFAIK) give you incorrect file names, i.e. it will replace unrepresentable characters with the replacement char (QUESTION MARK). OTOH, on Unix, there is a better approach for listdir and unconvertable names: just return the byte strings to the user. > I'd say: let the different file name based APIs try hard enough > and then have them bail out if they can't handle the particular > case. That is a good idea. However, in case of the WinNT replacement strategy, the application may still want to know. Passing *in* Unicode objects is no issue at all: If they cannot be converted to a reasonable file name, you clearly get an exception. > > It turns out that only OS X really got it right: For each file, there > > is both a byte string name, and a Unicode name. > > I suppose this is due to the fact that Mac file systems store > extended attributes (much like what OS/2 does too) along with the > file -- that's a really nice way of being able to extend file > system semantics on a per-file basis; much better than the Windows > Registry or the MIME guess-by-extension mechanisms. I'd assume it is different: They just *define* that all local file systems they have control over use UTF-8 on disk, atleast for BSD ufs; for HFS, it might be that they 'just know' what encoding is used on an HFS partition. I doubt they use extended attributes for this, as they reportedly return UTF-8 even for file systems they've never seen before; this may be either due to static knowledge (e.g. that VFAT is UCS-2LE), or through guessing. It may be that there are also limitations and restrictions, but atleast they remove the burden from the application. Regards, Martin From martin@v.loewis.de Thu Jan 17 12:06:54 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: Thu, 17 Jan 2002 13:06:54 +0100 Subject: [Python-Dev] Re: PEP 277: Unicode file name support for Windows NT, was PEP-time ? ... In-Reply-To: <3C46B735.9C433F60@lemburg.com> (mal@lemburg.com) References: <15412.29829.302192.219332@12-248-41-177.client.attbi.com> <020601c194b3$c85a4320$0acc8490@neil> <3C357405.CC439CBE@lemburg.com> <200201041046.g04AkKR05898@mira.informatik.hu-berlin.de> <016e01c19639$94c909b0$0acc8490@neil> <200201060033.g060X8c14491@mira.informatik.hu-berlin.de> <021901c19654$21f2e3f0$0acc8490@neil> <200201061214.g06CEtc01656@mira.informatik.hu-berlin.de> <036e01c1972d$dbc88a80$0acc8490@neil> <200201070728.g077SmZ01967@mira.informatik.hu-berlin.de> <003b01c1975d$e7dd3070$0acc8490@neil> <200201072317.g07NHEh01830@mira.informatik.hu-berlin.de> <3C3AC26A.D40842FB@lemburg.com> <02ff01c19cc3$92514540$0acc8490@neil> <200201140711.g0E7BsV01370@mira.informatik.hu-berlin.de> <3C44059C.CFC09899@lemburg.com> <200201152124.g0FLOV702247@mira.informatik.hu-berlin.de> <3C45CB11.ACB2CEE6@lemburg.com> <200201161909.g0GJ9OK01822@mira.informatik.hu-berlin.de> <3C46A535.3C579501@lemburg.com> <08e201c19f46$cad5f070$0acc8490@neil> <3C46B735.9C433F60@lemburg.com> Message-ID: <200201171206.g0HC6sa01572@mira.informatik.hu-berlin.de> > Is it really ? The problem is that under some OSes it is possible > to work with multiple very different file system from within a > single Python program. In those cases, the unicodefilename() > API wouldn't really help all that much. If you are thinking of Unix: It seems unicodefilename has to return 0 on Unix, meaning that you need to use byte-oriented file names if you want to access all files (not that you will be able to display all file names to the user, though ... there is nothing we can do to achieve *that*). > > unicodefilenames() can be dropped in favour of explicit OS and version > > checks but this is replacing a simple robust check with a more fragile one. > > What kind of checks do you have in mind then ? If possible, it should > be possible to pass unicodefilenames() a path to check for Unicode- > capability, since on Unix (and probably Mac OS X as well), the path > decides which file system get's the ioctrl calls. I think you are missing the point that unicodefilenames, as defined, does not take any parameters. It says either yay or nay. So it could be replaced in application code with if sys.platform == "win32": use_unicode_for_filenames = windowsversion in ['nt','w2k','xp'] elif sys.platform.startswith("darwin"): use_unicode_for_filenames = 1 else: use_unicode_for_filenames = 0 I would not use such code in my applications, nor would I ever use unicodefilenames. Instead, I would just use Unicode file names all the time, and risk that some users have problems with some files. Those users I would tell to fix their systems (i.e. use NT instead of Windows, or use a UTF-8 locale on Unix). Most users will never notice any problem (except for Neil, who likes to put funny file names on his disk :-), so this is a typical 80-20 problem here (or maybe rather 99-1). Regards, Martin From mal@lemburg.com Thu Jan 17 12:29:54 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 17 Jan 2002 13:29:54 +0100 Subject: [Python-Dev] Re: PEP 277: Unicode file name support for Windows NT, was PEP-time ? ... References: <15412.29829.302192.219332@12-248-41-177.client.attbi.com> <200201041046.g04AkKR05898@mira.informatik.hu-berlin.de> <016e01c19639$94c909b0$0acc8490@neil> <200201060033.g060X8c14491@mira.informatik.hu-berlin.de> <021901c19654$21f2e3f0$0acc8490@neil> <200201061214.g06CEtc01656@mira.informatik.hu-berlin.de> <036e01c1972d$dbc88a80$0acc8490@neil> <200201070728.g077SmZ01967@mira.informatik.hu-berlin.de> <003b01c1975d$e7dd3070$0acc8490@neil> <200201072317.g07NHEh01830@mira.informatik.hu-berlin.de> <3C3AC26A.D40842FB@lemburg.com> <02ff01c19cc3$92514540$0acc8490@neil> <200201140711.g0E7BsV01370@mira.informatik.hu-berlin.de> <3C44059C.CFC09899@lemburg.com> <200201152124.g0FLOV702247@mira.informatik.hu-berlin.de> <3C45CB11.ACB2CEE6@lemburg.com> <200201161909.g0GJ9OK01822@mira.informatik.hu-berlin.de> <3C46A535.3C579501@lemburg.com> <08e201c19f46$cad5f070$0acc8490@neil> <3C46B735.9C433F60@lemburg.com> <200201171206.g0HC6sa01572@mira.informatik.hu-berlin.de> Message-ID: <3C46C3C2.984F6227@lemburg.com> "Martin v. Loewis" wrote: > > > Is it really ? The problem is that under some OSes it is possible > > to work with multiple very different file system from within a > > single Python program. In those cases, the unicodefilename() > > API wouldn't really help all that much. > > If you are thinking of Unix: It seems unicodefilename has to return 0 > on Unix, meaning that you need to use byte-oriented file names if you > want to access all files (not that you will be able to display all > file names to the user, though ... there is nothing we can do to > achieve *that*). Right. I am starting to believe that unicodefilenames() doesn't really provide enough information to make it useful for cross-platform programming. > > > unicodefilenames() can be dropped in favour of explicit OS and version > > > checks but this is replacing a simple robust check with a more fragile one. > > > > What kind of checks do you have in mind then ? If possible, it should > > be possible to pass unicodefilenames() a path to check for Unicode- > > capability, since on Unix (and probably Mac OS X as well), the path > > decides which file system get's the ioctrl calls. > > I think you are missing the point that unicodefilenames, as defined, > does not take any parameters. It says either yay or nay. So it could > be replaced in application code with > > if sys.platform == "win32": > use_unicode_for_filenames = windowsversion in ['nt','w2k','xp'] > elif sys.platform.startswith("darwin"): > use_unicode_for_filenames = 1 > else: > use_unicode_for_filenames = 0 Sounds like this would be a good candidate for platform.py which I'll check into CVS soon. With its many platform querying APIs it should easily be possible to add a function which returns the above information based on the platform Python is running on. > I would not use such code in my applications, nor would I ever use > unicodefilenames. Instead, I would just use Unicode file names all the > time, and risk that some users have problems with some files. Those > users I would tell to fix their systems (i.e. use NT instead of > Windows, or use a UTF-8 locale on Unix). Most users will never notice > any problem (except for Neil, who likes to put funny file names on his > disk :-), so this is a typical 80-20 problem here (or maybe rather > 99-1). I doubt that you'll have any luck in trying to convince a user to switch OSes just because Python applications don't cope with native file names. The UTF-8 locale on Unix is also hard to push: e.g. existing latin-1 file names will probably stop working the minute you switch to that locale. (I always leave the setting to "C" and simply don't use locale based file names -- that way I don't run into problems; non-[a-zA-Z0-9\-\._]+ file names are a no-go for cross-platform-code if you ask me...) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mal@lemburg.com Thu Jan 17 12:36:27 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 17 Jan 2002 13:36:27 +0100 Subject: [Python-Dev] Re: PEP 277: Unicode file name support for Windows NT, was PEP-time ? ... References: <15412.29829.302192.219332@12-248-41-177.client.attbi.com> <020601c194b3$c85a4320$0acc8490@neil> <3C357405.CC439CBE@lemburg.com> <200201041046.g04AkKR05898@mira.informatik.hu-berlin.de> <016e01c19639$94c909b0$0acc8490@neil> <200201060033.g060X8c14491@mira.informatik.hu-berlin.de> <021901c19654$21f2e3f0$0acc8490@neil> <200201061214.g06CEtc01656@mira.informatik.hu-berlin.de> <036e01c1972d$dbc88a80$0acc8490@neil> <200201070728.g077SmZ01967@mira.informatik.hu-berlin.de> <003b01c1975d$e7dd3070$0acc8490@neil> <200201072317.g07NHEh01830@mira.informatik.hu-berlin.de> <3C3AC26A.D40842FB@lemburg.com> <02ff01c19cc3$92514540$0acc8490@neil> <200201140711.g0E7BsV01370@mira.informatik.hu-berlin.de> <3C44059C.CFC09899@lemburg.com> <200201152124.g0FLOV702247@mira.informatik.hu-berlin.de> <3C45CB11.ACB2CEE6@lemburg.com> <200201161909.g0GJ9OK01822@mira.informatik.hu-berlin.de> <3C46A535.3C579501@lemburg.com> <200201171142.g0HBgLk01405@mira.informatik.hu-berlin.de> Message-ID: <3C46C54B.72D0746A@lemburg.com> "Martin v. Loewis" wrote: > > > Sounds like the run-time error solution would at least "solve" > > the issue in terms of making it depend on the used file name > > and underlying OS or file system. > > Such a solution is impossible to implement in some case. E.g. on > Windows, if you use the ANSI (*A) APIs to list the directory contents, > Windows will *silently* (AFAIK) give you incorrect file names, i.e. it > will replace unrepresentable characters with the replacement char > (QUESTION MARK). Samba does the same for mounted Windows shares, BTW. > OTOH, on Unix, there is a better approach for listdir and > unconvertable names: just return the byte strings to the user. Sigh. > > I'd say: let the different file name based APIs try hard enough > > and then have them bail out if they can't handle the particular > > case. > > That is a good idea. However, in case of the WinNT replacement > strategy, the application may still want to know. > > Passing *in* Unicode objects is no issue at all: If they cannot be > converted to a reasonable file name, you clearly get an exception. True and that's good :-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From paul@svensson.org Thu Jan 17 13:43:02 2002 From: paul@svensson.org (Paul Svensson) Date: Thu, 17 Jan 2002 08:43:02 -0500 (EST) Subject: [Python-Dev] Utopian String Interpolation In-Reply-To: <3C46A33C.D97C831C@lemburg.com> Message-ID: On Thu, 17 Jan 2002, M.-A. Lemburg wrote: >Paul Prescod wrote: >> >> The documentation says: >> >> "Unlike Standard , all unrecognized escape sequences are left in the >> string unchanged, i.e., the backslash is left in the string. (This >> behavior is useful when debugging: if an escape sequence is mistyped, >> the resulting output is more easily recognized as broken.)" >> >> That's a weird thing to say. What could be more helpful for debugging >> than a good old SyntaxError??? > >If there's nothing wrong with the escape why raise a >SyntaxError ? I would certainly claim that an unrecognized escape sequence _is_ wrong. /Paul From mal@lemburg.com Thu Jan 17 14:02:11 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 17 Jan 2002 15:02:11 +0100 Subject: [Python-Dev] Utopian String Interpolation References: Message-ID: <3C46D963.2BC4F2A1@lemburg.com> Paul Svensson wrote: > > On Thu, 17 Jan 2002, M.-A. Lemburg wrote: > > >Paul Prescod wrote: > >> > >> The documentation says: > >> > >> "Unlike Standard , all unrecognized escape sequences are left in the > >> string unchanged, i.e., the backslash is left in the string. (This > >> behavior is useful when debugging: if an escape sequence is mistyped, > >> the resulting output is more easily recognized as broken.)" > >> > >> That's a weird thing to say. What could be more helpful for debugging > >> than a good old SyntaxError??? > > > >If there's nothing wrong with the escape why raise a > >SyntaxError ? > > I would certainly claim that an unrecognized escape sequence _is_ wrong. Depending on how you see it, an "unrecognized escape sequence" is not an escape sequence to begin with :-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From guido@python.org Thu Jan 17 14:15:00 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 17 Jan 2002 09:15:00 -0500 Subject: [Python-Dev] Utopian String Interpolation In-Reply-To: Your message of "Thu, 17 Jan 2002 08:43:02 EST." References: Message-ID: <200201171415.JAA30650@cj20424-a.reston1.va.home.com> > I would certainly claim that an unrecognized escape sequence _is_ wrong. Then you are wrong. Go away and design your own language. --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Thu Jan 17 15:39:30 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 17 Jan 2002 10:39:30 -0500 Subject: [Python-Dev] Utopian String Interpolation References: <3C446A5B.2E7A22CD@prescod.net> <3C45C4ED.806E64F3@lemburg.com> <3C45D7F5.BF260898@prescod.net> <3C46A33C.D97C831C@lemburg.com> Message-ID: <15430.61490.271542.258065@anthem.wooz.org> >>>>> "MAL" == M writes: MAL> It is. Currently Python strings are just that: immutable MAL> strings. Now, you suddenly add dynamics to then. This will MAL> cause nightmares in terms of security. Note that Python MAL> hasn't really had a need for Perl's "taint" because of MAL> this. I wouldn't want to see that change in any way. Bingo! MAL> I've jumped in at a rather late point. Perhaps you ought to MAL> rewind the discussion then and start discussing in a MAL> different direction :-) E.g. about the syntax to be used in MAL> the interpolation and where, when and in which context to MAL> evaluate the strings. Proponants of this feature can start by updating the PEP. -Barry From paul@svensson.org Thu Jan 17 16:32:11 2002 From: paul@svensson.org (Paul Svensson) Date: Thu, 17 Jan 2002 11:32:11 -0500 (EST) Subject: [Python-Dev] Utopian String Interpolation In-Reply-To: <200201171415.JAA30650@cj20424-a.reston1.va.home.com> Message-ID: On Thu, 17 Jan 2002, Guido van Rossum wrote: >> I would certainly claim that an unrecognized escape sequence _is_ wrong. > >Then you are wrong. (---) Then maybe the Python Referece Manual (2.4.1) needs to be updated, since the paragraph concerning unrecognized escape sequences doesn't mention them other than being "mistyped" or "broken". (Does "mistyped" and "broken" qualify as "wrong" ?) /Paul From skip@pobox.com Thu Jan 17 17:40:19 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 17 Jan 2002 11:40:19 -0600 Subject: [Python-Dev] deprecate input()? In-Reply-To: <200201170628.BAA28567@cj20424-a.reston1.va.home.com> References: <15429.36017.387707.78193@12-248-41-177.client.attbi.com> <200201170628.BAA28567@cj20424-a.reston1.va.home.com> Message-ID: <15431.3203.415990.602525@beluga.mojam.com> >> I just responded to a question on c.l.py a user had about feeding >> empty strings to input(). While he didn't say why he called input(), >> I suspect he thought the semantics were more like raw_input(). >> >> In these days of widespread Internet nastiness, shouldn't input() be >> deprecated? Guido> Why? I imagine this is only used for interactive input, and then Guido> it's the computer's owner who is typing. Yes, but what if the program containing calls to input() get shipped to someone else's computer? It just seems to me that a) input is almost never what you want to call and that b) it would seem to a naive programmer to be the correct way to ask the user for a line of input. Skip From guido@python.org Thu Jan 17 17:49:26 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 17 Jan 2002 12:49:26 -0500 Subject: [Python-Dev] deprecate input()? In-Reply-To: Your message of "Thu, 17 Jan 2002 11:40:19 CST." <15431.3203.415990.602525@beluga.mojam.com> References: <15429.36017.387707.78193@12-248-41-177.client.attbi.com> <200201170628.BAA28567@cj20424-a.reston1.va.home.com> <15431.3203.415990.602525@beluga.mojam.com> Message-ID: <200201171749.MAA00493@cj20424-a.reston1.va.home.com> > Guido> Why? I imagine this is only used for interactive input, > Guido> and then it's the computer's owner who is typing. > > Yes, but what if the program containing calls to input() get shipped > to someone else's computer? It just seems to me that a) input is > almost never what you want to call and that b) it would seem to a > naive programmer to be the correct way to ask the user for a line of > input. I don't see the security problem. Can you explain a scenario where this causes a security risk? If the user of the program types something evil in the input box they screw themselves! --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@rahul.net Thu Jan 17 17:56:46 2002 From: aahz@rahul.net (Aahz Maruch) Date: Thu, 17 Jan 2002 09:56:46 -0800 (PST) Subject: [Python-Dev] Utopian String Interpolation In-Reply-To: <200201171415.JAA30650@cj20424-a.reston1.va.home.com> from "Guido van Rossum" at Jan 17, 2002 09:15:00 AM Message-ID: <20020117175646.034B5E8C8@waltz.rahul.net> Guido van Rossum wrote: >Paul Svensson: >> >> I would certainly claim that an unrecognized escape sequence _is_ wrong. > > Then you are wrong. Go away and design your own language. Hey! That's a bit harsh. I'm not going to campaign to make unrecognized escape sequences a syntax error, but not raising a syntax error does seem to be against Python's principles. -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista We must not let the evil of a few trample the freedoms of the many. From rsc@plan9.bell-labs.com Thu Jan 17 18:01:21 2002 From: rsc@plan9.bell-labs.com (Russ Cox) Date: Thu, 17 Jan 2002 13:01:21 -0500 Subject: [Python-Dev] deprecate input()? Message-ID: > Yes, but what if the program containing calls to input() get shipped to > someone else's computer? It just seems to me that a) input is almost never > what you want to call and that b) it would seem to a naive programmer to be > the correct way to ask the user for a line of input. Since most arbitrary lines of input generate syntax errors, wouldn't the naive programmer quickly figure out that input isn't the "read a line" function? (Unless you're trying to input numbers, I suppose.) Russ From entropiamax@jazzfree.com Thu Jan 17 18:25:46 2002 From: entropiamax@jazzfree.com (Andres Tuells) Date: Thu, 17 Jan 2002 19:25:46 +0100 Subject: [Python-Dev] Re: Stackless Python is DEAD! Long live Stackless Python References: <3C470552.3040802@tismer.com> Message-ID: <006f01c19f84$620dde20$9d76393e@integralabzenon> Thats great !!! ----- Original Message ----- From: "Christian Tismer" To: ; ; Sent: Thursday, January 17, 2002 6:09 PM Subject: Ann: Stackless Python is DEAD! Long live Stackless Python > > ####################################### > > Announcement: > > ####################################### > > > The end of an era has come: > --------------------------- > Stackless Python, in the form provided upto Python 2.0, is DEAD. > > I am abandoning the whole implementation. > > > A new era has begun: > -------------------- > A completely new implementation is in development for > Python 2.2 and up which gives you the following features: > > - There are no restrictions any longer for uthread/coroutine > switching. Switching is possible at *any* time, in *any* > context. > > - There are no significant changes to the Python core any > longer. The new patches are of minimum size, and they > will probably survive unchanged until Python 3.0 . > > - Maintenance work for Stackless Python is reduced to the > bare minimum. There is no longer a need to incorporate > Stackless into the standard, since there is no work to > be shared. > > - Stackless breaks its major axiom now. It is no longer > platform independent, since it *does* modify the C stack. > I will support all Intel platforms by myself. For other > platforms, I'm asking for volunteers. > > * The basic elements of Stackless are now switchable chains > of frames. We have to define an interface that turns these > chains into microthreads and coroutines. > > Everybody is invited to come to the Stackless mailing > list and discuss the layout of this new design. > Especially we need to decide about (*). > > http://starship.python.net/mailman/listinfo/stackless > > see you there - chris > > -- > Christian Tismer :^) > Mission Impossible 5oftware : Have a break! Take a ride on Python's > Kaunstr. 26 : *Starship* http://starship.python.net/ > 14163 Berlin : PGP key -> http://wwwkeys.pgp.net/ > PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF > where do you want to jump today? http://www.stackless.com/ > > > > -- > http://mail.python.org/mailman/listinfo/python-list From guido@python.org Thu Jan 17 18:25:46 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 17 Jan 2002 13:25:46 -0500 Subject: [Python-Dev] Utopian String Interpolation In-Reply-To: Your message of "Thu, 17 Jan 2002 09:56:46 PST." <20020117175646.034B5E8C8@waltz.rahul.net> References: <20020117175646.034B5E8C8@waltz.rahul.net> Message-ID: <200201171825.NAA00602@cj20424-a.reston1.va.home.com> > >Paul Svensson: > >> > >> I would certainly claim that an unrecognized escape sequence _is_ wrong. > > > Guido van Rossum wrote: > > Then you are wrong. Go away and design your own language. > Aahz: > Hey! That's a bit harsh. I'm not going to campaign to make > unrecognized escape sequences a syntax error, but not raising a syntax > error does seem to be against Python's principles. Whatever. Who is Paul Svensson and what is he doing in python-dev? --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Thu Jan 17 18:36:59 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 17 Jan 2002 12:36:59 -0600 Subject: [Python-Dev] deprecate input()? In-Reply-To: <200201171749.MAA00493@cj20424-a.reston1.va.home.com> References: <15429.36017.387707.78193@12-248-41-177.client.attbi.com> <200201170628.BAA28567@cj20424-a.reston1.va.home.com> <15431.3203.415990.602525@beluga.mojam.com> <200201171749.MAA00493@cj20424-a.reston1.va.home.com> Message-ID: <15431.6603.470764.669139@beluga.mojam.com> Guido> Why? I imagine this is only used for interactive input, and then Guido> it's the computer's owner who is typing. >> Yes, but what if the program containing calls to input() get shipped >> to someone else's computer? It just seems to me that a) input is >> almost never what you want to call and that b) it would seem to a >> naive programmer to be the correct way to ask the user for a line of >> input. Guido> I don't see the security problem. Can you explain a scenario Guido> where this causes a security risk? If the user of the program Guido> types something evil in the input box they screw themselves! Fine. Let's drop it. Skip From nhodgson@bigpond.net.au Thu Jan 17 19:31:36 2002 From: nhodgson@bigpond.net.au (Neil Hodgson) Date: Fri, 18 Jan 2002 06:31:36 +1100 Subject: [Python-Dev] Re: PEP 277: Unicode file name support for Windows NT, was PEP-time ? ... References: <15412.29829.302192.219332@12-248-41-177.client.attbi.com> <020601c194b3$c85a4320$0acc8490@neil> <3C357405.CC439CBE@lemburg.com> <200201041046.g04AkKR05898@mira.informatik.hu-berlin.de> <016e01c19639$94c909b0$0acc8490@neil> <200201060033.g060X8c14491@mira.informatik.hu-berlin.de> <021901c19654$21f2e3f0$0acc8490@neil> <200201061214.g06CEtc01656@mira.informatik.hu-berlin.de> <036e01c1972d$dbc88a80$0acc8490@neil> <200201070728.g077SmZ01967@mira.informatik.hu-berlin.de> <003b01c1975d$e7dd3070$0acc8490@neil> <200201072317.g07NHEh01830@mira.informatik.hu-berlin.de> <3C3AC26A.D40842FB@lemburg.com> <02ff01c19cc3$92514540$0acc8490@neil> <200201140711.g0E7BsV01370@mira.informatik.hu-berlin.de> <3C44059C.CFC09899@lemburg.com> <200201152124.g0FLOV702247@mira.informatik.hu-berlin.de> <3C45CB11.ACB2CEE6@lemburg.com> <200201161909.g0GJ9OK01822@mira.informatik.hu-berlin.de> <3C46A535.3C579501@lemburg.com> <08e201c19f46$cad5f070$0acc8490@neil> <3C46B735.9C433F60@lemburg.com> Message-ID: <00c401c19f8d$941e3fa0$0acc8490@neil> M.-A. Lemburg: > Is it really ? The problem is that under some OSes it is possible > to work with multiple very different file system from within a > single Python program. In those cases, the unicodefilename() > API wouldn't really help all that much. On NT the core file system calls are Unicode based with the narrow string calls being shims on top of this. When mounting non-native file systems, NT may perform name mapping, but that name mapping is 'complete and consistent' in that it is not possible to do anything with the narrow APIs that cannot be achieved with the Unicode APIs. > > unicodefilenames() can be dropped in favour of explicit OS and version > > checks but this is replacing a simple robust check with a more fragile one. > > What kind of checks do you have in mind then ? If possible, it should > be possible to pass unicodefilenames() a path to check for Unicode- > capability, since on Unix (and probably Mac OS X as well), the path > decides which file system get's the ioctrl calls. Any platform experts know how this works on MacOS X or BeOS? Do non-native file systems get mapped to Unicode names so that UTF-8 will always work? Neil From thomas.heller@ion-tof.com Thu Jan 17 19:23:00 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 17 Jan 2002 20:23:00 +0100 Subject: [Python-Dev] Extending types in C - help needed References: <43ABBA6E-0AD4-11D6-A4BB-003065517236@oratrix.nl> Message-ID: <08c501c19f8c$72631b20$e000a8c0@thomasnotebook> From: "Jack Jansen" > In the discussion on my request for an ("O@", typeobject, > void **) format for PyArg_Parse and Py_BuildValue MAL suggested (as MAL already explained, that we suggested by me) > that I could get the same functionality by creating a type > WrapperTypeObject, which would be a subtype of TypeObject with > extra fields pointing to the _New() and _Convert() routines to > convert Python objects from/to C pointers. This would be good > enough for me, because then types wanting to participate in the > wrapper protocol would subtype WrapperTypeObject in stead of > TypeObject, and two global routines could return the _New and > _Convert routines given the type object, and we wouldn't need > yet another PyArg_Parse format specifier. > > However, after digging high and low I haven't been able to > deduce how I would then use this WrapperType in C as the type > for my extension module objects. Are there any examples? If not, > could someone who understands the new inheritance scheme give me > some clues as to how to do this? Currently (after quite some time) I have the impression that you cannot create a subtype of PyType_Type in C because PyType_Type ends in a variable sized array, at least not in this way: struct { PyTypeObject type; ...additional fields... } WrapperType_Type; Can someone confirm this? (I have to find out what to do with the tp_members slot, which seems to be correspond to the Python level __slots__ class variable) Thomas From guido@python.org Thu Jan 17 19:51:36 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 17 Jan 2002 14:51:36 -0500 Subject: [Python-Dev] Extending types in C - help needed In-Reply-To: Your message of "Thu, 17 Jan 2002 20:23:00 +0100." <08c501c19f8c$72631b20$e000a8c0@thomasnotebook> References: <43ABBA6E-0AD4-11D6-A4BB-003065517236@oratrix.nl> <08c501c19f8c$72631b20$e000a8c0@thomasnotebook> Message-ID: <200201171951.OAA00909@cj20424-a.reston1.va.home.com> > Currently (after quite some time) I have the impression that you > cannot create a subtype of PyType_Type in C because PyType_Type > ends in a variable sized array, at least not in this way: > > struct { > PyTypeObject type; > ...additional fields... > } WrapperType_Type; > > Can someone confirm this? Yes, alas. The type you would have to declare is 'etype', a private type in typeobject.c. --Guido van Rossum (home page: http://www.python.org/~guido/) From nhodgson@bigpond.net.au Thu Jan 17 20:07:30 2002 From: nhodgson@bigpond.net.au (Neil Hodgson) Date: Fri, 18 Jan 2002 07:07:30 +1100 Subject: [Python-Dev] Re: PEP 277: Unicode file name support for Windows NT, was PEP-time ? ... References: <15412.29829.302192.219332@12-248-41-177.client.attbi.com> <3C357405.CC439CBE@lemburg.com> <200201041046.g04AkKR05898@mira.informatik.hu-berlin.de> <016e01c19639$94c909b0$0acc8490@neil> <200201060033.g060X8c14491@mira.informatik.hu-berlin.de> <021901c19654$21f2e3f0$0acc8490@neil> <200201061214.g06CEtc01656@mira.informatik.hu-berlin.de> <036e01c1972d$dbc88a80$0acc8490@neil> <200201070728.g077SmZ01967@mira.informatik.hu-berlin.de> <003b01c1975d$e7dd3070$0acc8490@neil> <200201072317.g07NHEh01830@mira.informatik.hu-berlin.de> <3C3AC26A.D40842FB@lemburg.com> <02ff01c19cc3$92514540$0acc8490@neil> <200201140711.g0E7BsV01370@mira.informatik.hu-berlin.de> <3C44059C.CFC09899@lemburg.com> <200201152124.g0FLOV702247@mira.informatik.hu-berlin.de> <3C45CB11.ACB2CEE6@lemburg.com> <200201161909.g0GJ9OK01822@mira.informatik.hu-berlin.de> <3C46A535.3C579501@lemburg.com> <08e201c19f46$cad5f070$0acc8490@neil> <3C46B735.9C433F60@lemburg.com> <200201171206.g0HC6sa01572@mira.informatik.hu-berlin.de> Message-ID: <016601c19f92$9a049e00$0acc8490@neil> Martin v. Loewis: > Most users will never notice > any problem (except for Neil, who likes to put funny file names on his > disk :-), so this is a typical 80-20 problem here (or maybe rather > 99-1). While Martin is referring to the rarity of having non-native file names on Windows 9x, the problem adressed by PEP 277 is real. Already this year, there have been two enquiries [from Michael Ebert and Guenter Radestock] to comp.lang.python about Unicode file name use on NT. Neil From paul@prescod.net Thu Jan 17 20:10:27 2002 From: paul@prescod.net (Paul Prescod) Date: Thu, 17 Jan 2002 12:10:27 -0800 Subject: [Python-Dev] Utopian String Interpolation References: <3C446A5B.2E7A22CD@prescod.net> <3C45C4ED.806E64F3@lemburg.com> <3C45D7F5.BF260898@prescod.net> <3C46A33C.D97C831C@lemburg.com> Message-ID: <3C472FB3.81EAA5D5@prescod.net> "M.-A. Lemburg" wrote: > >... > > It is. Currently Python strings are just that: immutable strings. > Now, you suddenly add dynamics to then. I don't want to go through this whole thread from the beginning again. PEP 215 does not add "dynamics" to anything. In fact, PEP 215 is a more static mechanism than the current idiom. Even if we make PEP 215's behaviour the default for strings, it is still NOT DYNAMIC. >... This will cause nightmares > in terms of security. There is a thread called "PEP 215 does not introduce security issues". Please read it. Everyone involved who initially thought that PEP 215 had security issues backed down and agreed that it did not. Once again, whether there is a string prefix or not is irrelevant to this question. PEP 215's semantics are *not dynamic*. > ... Note that Python hasn't really had a need > for Perl's "taint" because of this. I wouldn't want to see that > change in any way. I am certainly not a Perl programmer but Python is also attackable through the sorts of holes that "taint" is intended to avoid. username = raw_input() os.system("cp %s.new %s.old" % (username, username)) Perl considers this "dangerous" and so it has taint. It has *nothing* to do with interpolation syntax. >... > Huh ? I bet RedHat and thousands of sysadmins who have switched > from shell or Perl to Python would have strong objections. Python has a construct called a "raw string" which is perfect for when you don't want backslashes treated specially. Paul Prescod From DavidA@ActiveState.com Thu Jan 17 17:46:23 2002 From: DavidA@ActiveState.com (David Ascher) Date: Thu, 17 Jan 2002 09:46:23 -0800 Subject: [Python-Dev] deprecate input()? References: <15429.36017.387707.78193@12-248-41-177.client.attbi.com> <200201170628.BAA28567@cj20424-a.reston1.va.home.com> Message-ID: <3C470DEF.B5AD08EE@ActiveState.com> Guido van Rossum wrote: > > > I just responded to a question on c.l.py a user had about feeding empty > > strings to input(). While he didn't say why he called input(), I suspect he > > thought the semantics were more like raw_input(). > > > > In these days of widespread Internet nastiness, shouldn't input() be > > deprecated? > > Why? I imagine this is only used for interactive input, and then it's > the computer's owner who is typing. input() can also be used effectively in interactive apps (calculators, scripting engines for GUI apps) in contexts where the users can be trusted. Not _everything_ is on the web, luckily, and not everything needs to be evildoer-proof... That doesn't mean that I think the naming choices for input() and raw_input() have withstood the test of hindsight, but few things do... --david From Jack.Jansen@oratrix.nl Thu Jan 17 20:59:13 2002 From: Jack.Jansen@oratrix.nl (Jack Jansen) Date: Thu, 17 Jan 2002 21:59:13 +0100 Subject: [Python-Dev] Extending types in C - help needed In-Reply-To: <3C46A799.2E7AD7DB@lemburg.com> Message-ID: <0F82FB1B-0B8D-11D6-B884-003065517236@oratrix.nl> On Thursday, January 17, 2002, at 11:29 AM, M.-A. Lemburg wrote: > Jack Jansen wrote: >> >> In the discussion on my request for an ("O@", typeobject, >> void **) format for PyArg_Parse and Py_BuildValue MAL suggested > > Thomas Heller suggested this. Oops, you're right. I should be careful not to mix up my Germans;-) > I am more in favour of > exposing the pickle reduce API through "O@", that is > have PyArgTuple_Parse() call the .__reduce__() method > of the object. This will then return (factory, state_tuple) > and these could then be exposed to the C function via two > PyObject*. You've suggested this before, but at that time I ignored it because it made absolutely no sense to me. "pickle" triggers one set of ideas for me, "reduce" triggers a different set, "factory function" yet another different set. None of these sets of ideas have the least resemblance to what I'm trying to do:-) I gave a fairly complete example (using calldll from Python to wrap a function that returns a Mac WindowObject) last week, could you explain how you would implement this with pickle, reduce and factory functions? -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From Jack.Jansen@oratrix.nl Thu Jan 17 21:03:25 2002 From: Jack.Jansen@oratrix.nl (Jack Jansen) Date: Thu, 17 Jan 2002 22:03:25 +0100 Subject: [Python-Dev] Re: PEP 277: Unicode file name support for Windows NT, was PEP-time ? ... In-Reply-To: <200201171142.g0HBgLk01405@mira.informatik.hu-berlin.de> Message-ID: On Thursday, January 17, 2002, at 12:42 PM, Martin v. Loewis wrote: >> I suppose this is due to the fact that Mac file systems store >> extended attributes (much like what OS/2 does too) along with the >> file -- that's a really nice way of being able to extend file >> system semantics on a per-file basis; much better than the Windows >> Registry or the MIME guess-by-extension mechanisms. > > I'd assume it is different: They just *define* that all local file > systems they have control over use UTF-8 on disk, atleast for BSD ufs; > for HFS, it might be that they 'just know' what encoding is used on an > HFS partition. I doubt they use extended attributes for this, as they > reportedly return UTF-8 even for file systems they've never seen > before; this may be either due to static knowledge (e.g. that VFAT is > UCS-2LE), or through guessing. It's actually a whole lot simpler: for filesystems with an encoding that is open to interpretation the user specifies it during mount:-) -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From Jack.Jansen@oratrix.nl Thu Jan 17 21:09:20 2002 From: Jack.Jansen@oratrix.nl (Jack Jansen) Date: Thu, 17 Jan 2002 22:09:20 +0100 Subject: [Python-Dev] Re: PEP 277: Unicode file name support for Windows NT, was PEP-time ? ... In-Reply-To: <00c401c19f8d$941e3fa0$0acc8490@neil> Message-ID: <79BF50EE-0B8E-11D6-B884-003065517236@oratrix.nl> On Thursday, January 17, 2002, at 08:31 PM, Neil Hodgson wrote: >> What kind of checks do you have in mind then ? If possible, it should >> be possible to pass unicodefilenames() a path to check for Unicode- >> capability, since on Unix (and probably Mac OS X as well), the path >> decides which file system get's the ioctrl calls. > > Any platform experts know how this works on MacOS X or BeOS? Do > non-native file systems get mapped to Unicode names so that UTF-8 will > always work? For Mac OS X: yes, that is how it is supposed to work. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From mal@lemburg.com Fri Jan 18 09:47:03 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 18 Jan 2002 10:47:03 +0100 Subject: [Python-Dev] Extending types in C - help needed References: <0F82FB1B-0B8D-11D6-B884-003065517236@oratrix.nl> Message-ID: <3C47EF17.37CE6126@lemburg.com> Jack Jansen wrote: > > On Thursday, January 17, 2002, at 11:29 AM, M.-A. Lemburg wrote: > > > I am more in favour of > > exposing the pickle reduce API through "O@", that is > > have PyArgTuple_Parse() call the .__reduce__() method > > of the object. This will then return (factory, state_tuple) > > and these could then be exposed to the C function via two > > PyObject*. > > You've suggested this before, but at that time I ignored it > because it made absolutely no sense to me. "pickle" triggers one > set of ideas for me, "reduce" triggers a different set, "factory > function" yet another different set. None of these sets of ideas > have the least resemblance to what I'm trying to do:-) The idea is simple but extends what you are trying to achieve (I gave an example on how to use this somewhere in the "wrapper" thread). Basically, you'll just want to use the state tuple to access the underlying void* C pointer via a PyCObject which does the wrapping of the pointer. The "pickle" mechanism would store the PyCObject in the state tuple which you could then access to get at the C pointer. This may sound complicated at first, but it provides much more flexibility w/r to more complex objects, e.g. the method you have in mind only supports wrapping a single C pointer; the "pickle" mechanism can potentially handle any serializable object. > I gave a fairly complete example (using calldll from Python to > wrap a function that returns a Mac WindowObject) last week, > could you explain how you would implement this with pickle, > reduce and factory functions? Sorry, no time for that ... I've got an important business trip next week which needs to be prepared. Please bring this up again after next week. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From jack@oratrix.com Fri Jan 18 10:24:01 2002 From: jack@oratrix.com (Jack Jansen) Date: Fri, 18 Jan 2002 11:24:01 +0100 Subject: [Python-Dev] Extending types in C - help needed In-Reply-To: <3C47EF17.37CE6126@lemburg.com> Message-ID: <7D823B3B-0BFD-11D6-B669-0030655234CE@oratrix.com> On Friday, January 18, 2002, at 10:47 , M.-A. Lemburg wrote: > Jack Jansen wrote: >> >> On Thursday, January 17, 2002, at 11:29 AM, M.-A. Lemburg wrote: >> >>> I am more in favour of >>> exposing the pickle reduce API through "O@", that is >>> have PyArgTuple_Parse() call the .__reduce__() method >>> of the object. This will then return (factory, state_tuple) >>> and these could then be exposed to the C function via two >>> PyObject*. >> >> You've suggested this before, but at that time I ignored it >> because it made absolutely no sense to me. "pickle" triggers one >> set of ideas for me, "reduce" triggers a different set, "factory >> function" yet another different set. None of these sets of ideas >> have the least resemblance to what I'm trying to do:-) > > The idea is simple but extends what you are trying to > achieve (I gave an example on how to use this somewhere > in the "wrapper" thread). Basically, you'll just want to > use the state tuple to access the underlying void* C pointer > via a PyCObject which does the wrapping of the pointer. > The "pickle" mechanism would store the PyCObject in the > state tuple which you could then access to get at the > C pointer. > I think you're missing a few points here. First of all, my objects aren't PyCObjects but other extension objects. While the main pointer in the object could be wrapped in a PyCObject there may be other information in my objects that is important, such as a pointer to the dispose routine to call on the c-pointer when the Python object reaches refcount zero (and this pointer may change over time as ownership of, say, a button is passed from Python to the system). The _New and _Convert routines will know how to get from the C pointer to the *correct* object, i.e. normally there will be only one Python object for every C object. Also, the method seems rather complicated for doing a simple thing. The only thing I really want is a way to refer to an _New or _Convert method from Python code. The most reasonable way to do that seems to be by creating a way to get from te type object (which is available in Python) to those routines. Thomas' suggestion looked very promising, and simple too, until Guido said that unfortunately it couldn't be done. Your suggestion, as far as I understand it, looks complicated and probably inefficient too (remember the code will have to go through all these hoops every time it needs to convert an object from Python to C or vice versa). Correct me if I'm wrong, From mal@lemburg.com Fri Jan 18 11:27:11 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 18 Jan 2002 12:27:11 +0100 Subject: [Python-Dev] Extending types in C - help needed References: <7D823B3B-0BFD-11D6-B669-0030655234CE@oratrix.com> Message-ID: <3C48068F.5A239D16@lemburg.com> Jack Jansen wrote: > > >> On Thursday, January 17, 2002, at 11:29 AM, M.-A. Lemburg wrote: > >> > >>> I am more in favour of > >>> exposing the pickle reduce API through "O@", that is > >>> have PyArgTuple_Parse() call the .__reduce__() method > >>> of the object. This will then return (factory, state_tuple) > >>> and these could then be exposed to the C function via two > >>> PyObject*. > >> > >> You've suggested this before, but at that time I ignored it > >> because it made absolutely no sense to me. "pickle" triggers one > >> set of ideas for me, "reduce" triggers a different set, "factory > >> function" yet another different set. None of these sets of ideas > >> have the least resemblance to what I'm trying to do:-) > > > > The idea is simple but extends what you are trying to > > achieve (I gave an example on how to use this somewhere > > in the "wrapper" thread). Basically, you'll just want to > > use the state tuple to access the underlying void* C pointer > > via a PyCObject which does the wrapping of the pointer. > > The "pickle" mechanism would store the PyCObject in the > > state tuple which you could then access to get at the > > C pointer. > > > I think you're missing a few points here. First of all, my objects > aren't PyCObjects but other extension objects. I know. The idea is that either you add a .__reduce__ method to the extension objects or register their types with a registry comparable to copyreg. > While the main pointer in > the object could be wrapped in a PyCObject there may be other > information in my objects that is important, such as a pointer to the > dispose routine to call on the c-pointer when the Python object reaches > refcount zero (and this pointer may change over time as ownership of, > say, a button is passed from Python to the system). Note that PyCObjects support all of this. It's not important in this context, though. The PyCObject is only used to wrap the raw pointer; the factory function then takes this pointer and creates one of your extension object out of it. > The _New and > _Convert routines will know how to get from the C pointer to the > *correct* object, i.e. normally there will be only one Python object for > every C object. That's also possible using the "pickle" approach. > Also, the method seems rather complicated for doing a simple thing. The > only thing I really want is a way to refer to an _New or _Convert method > from Python code. The most reasonable way to do that seems to be by > creating a way to get from te type object (which is available in Python) > to those routines. Thomas' suggestion looked very promising, and simple > too, until Guido said that unfortunately it couldn't be done. Your > suggestion, as far as I understand it, looks complicated and probably > inefficient too (remember the code will have to go through all these > hoops every time it needs to convert an object from Python to C or vice > versa). It is more complicated, but also more flexible. Plus it builds on techniques which are already applied in Python's pickle mechanism. Note that by adding a tp_reduce slot, the overhead of calling a Python function could be kept reasonable. Helper functions could aid in accessing the C pointer which is stored in the state tuple. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From guido@python.org Fri Jan 18 15:09:30 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 18 Jan 2002 10:09:30 -0500 Subject: [niemeyer@conectiva.com: Re: [Python-Dev] Python's footprint] In-Reply-To: Your message of "Mon, 14 Jan 2002 15:04:17 CST." <15427.18385.906669.456387@12-248-41-177.client.attbi.com> References: <20020110224908.C884@ibook.distro.conectiva> <200201111334.g0BDYLh01331@mira.informatik.hu-berlin.de> <20020111122105.B1808@ibook.distro.conectiva> <200201112347.g0BNlWk01567@mira.informatik.hu-berlin.de> <3C402BBA.1040806@lemburg.com> <20020114093053.C1325@ibook.distro.conectiva> <3C42C9A5.975FA5B8@lemburg.com> <20020114104146.A2607@ibook.distro.conectiva> <3C4326D2.F2A82030@lemburg.com> <200201141847.NAA10894@cj20424-a.reston1.va.home.com> <3C4337D5.B54330B1@lemburg.com> <15427.15272.171558.1993@12-248-41-177.client.attbi.com> <200201142017.PAA12356@cj20424-a.reston1.va.home.com> <15427.18385.906669.456387@12-248-41-177.client.attbi.com> Message-ID: <200201181509.KAA09399@cj20424-a.reston1.va.home.com> What's the current thinking about making docstrings optional? Does everybody agree on Gustavo's patch? http://sourceforge.net/tracker/?func=detail&atid=305470&aid=505375&group_id=5470 --Guido van Rossum (home page: http://www.python.org/~guido/) From nas@python.ca Fri Jan 18 15:15:54 2002 From: nas@python.ca (Neil Schemenauer) Date: Fri, 18 Jan 2002 07:15:54 -0800 Subject: [niemeyer@conectiva.com: Re: [Python-Dev] Python's footprint] In-Reply-To: <200201181509.KAA09399@cj20424-a.reston1.va.home.com>; from guido@python.org on Fri, Jan 18, 2002 at 10:09:30AM -0500 References: <20020114093053.C1325@ibook.distro.conectiva> <3C42C9A5.975FA5B8@lemburg.com> <20020114104146.A2607@ibook.distro.conectiva> <3C4326D2.F2A82030@lemburg.com> <200201141847.NAA10894@cj20424-a.reston1.va.home.com> <3C4337D5.B54330B1@lemburg.com> <15427.15272.171558.1993@12-248-41-177.client.attbi.com> <200201142017.PAA12356@cj20424-a.reston1.va.home.com> <15427.18385.906669.456387@12-248-41-177.client.attbi.com> <200201181509.KAA09399@cj20424-a.reston1.va.home.com> Message-ID: <20020118071554.A17496@glacier.arctrix.com> Guido van Rossum wrote: > What's the current thinking about making docstrings optional? > > Does everybody agree on Gustavo's patch? 10% space saving? That doesn't seem to be worth the effort. OTOH, I'm not dealing with any platforms that are memory constrained right now. Neil From mal@lemburg.com Fri Jan 18 15:23:23 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 18 Jan 2002 16:23:23 +0100 Subject: [niemeyer@conectiva.com: Re: [Python-Dev] Python's footprint] References: <20020110224908.C884@ibook.distro.conectiva> <200201111334.g0BDYLh01331@mira.informatik.hu-berlin.de> <20020111122105.B1808@ibook.distro.conectiva> <200201112347.g0BNlWk01567@mira.informatik.hu-berlin.de> <3C402BBA.1040806@lemburg.com> <20020114093053.C1325@ibook.distro.conectiva> <3C42C9A5.975FA5B8@lemburg.com> <20020114104146.A2607@ibook.distro.conectiva> <3C4326D2.F2A82030@lemburg.com> <200201141847.NAA10894@cj20424-a.reston1.va.home.com> <3C4337D5.B54330B1@lemburg.com> <15427.15272.171558.1993@12-248-41-177.client.attbi.com> <200201142017.PAA12356@cj20424-a.reston1.va.home.com> <15427.18385.906669.456387@12-248-41-177.client.attbi.com> <200201181509.KAA09399@cj20424-a.reston1.va.home.com> Message-ID: <3C483DEB.5B9D12A6@lemburg.com> Guido van Rossum wrote: > > What's the current thinking about making docstrings optional? > > Does everybody agree on Gustavo's patch? > > http://sourceforge.net/tracker/?func=detail&atid=305470&aid=505375&group_id=5470 +1. This will help Python embedders and porters to embedded systems a lot. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From barry@zope.com Fri Jan 18 15:26:06 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 18 Jan 2002 10:26:06 -0500 Subject: [niemeyer@conectiva.com: Re: [Python-Dev] Python's footprint] References: <20020114093053.C1325@ibook.distro.conectiva> <3C42C9A5.975FA5B8@lemburg.com> <20020114104146.A2607@ibook.distro.conectiva> <3C4326D2.F2A82030@lemburg.com> <200201141847.NAA10894@cj20424-a.reston1.va.home.com> <3C4337D5.B54330B1@lemburg.com> <15427.15272.171558.1993@12-248-41-177.client.attbi.com> <200201142017.PAA12356@cj20424-a.reston1.va.home.com> <15427.18385.906669.456387@12-248-41-177.client.attbi.com> <200201181509.KAA09399@cj20424-a.reston1.va.home.com> <20020118071554.A17496@glacier.arctrix.com> Message-ID: <15432.16014.869439.363615@anthem.wooz.org> >>>>> "NS" == Neil Schemenauer writes: >> What's the current thinking about making docstrings optional? >> Does everybody agree on Gustavo's patch? NS> 10% space saving? That doesn't seem to be worth the effort. NS> OTOH, I'm not dealing with any platforms that are memory NS> constrained right now. Personally I don't care either for the same reasons. I'll just note that what Emacs used to do (maybe it still does, I dunno), is extract all its inlined docstrings into a separate file which could be thrown away if you didn't want to pay for the bloat. All that complexity was built in a time when 300KB or so of docstrings really could make a huge difference for download times or storage resources. -Barry From mal@lemburg.com Fri Jan 18 15:42:09 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 18 Jan 2002 16:42:09 +0100 Subject: [niemeyer@conectiva.com: Re: [Python-Dev] Python's footprint] References: <20020114093053.C1325@ibook.distro.conectiva> <3C42C9A5.975FA5B8@lemburg.com> <20020114104146.A2607@ibook.distro.conectiva> <3C4326D2.F2A82030@lemburg.com> <200201141847.NAA10894@cj20424-a.reston1.va.home.com> <3C4337D5.B54330B1@lemburg.com> <15427.15272.171558.1993@12-248-41-177.client.attbi.com> <200201142017.PAA12356@cj20424-a.reston1.va.home.com> <15427.18385.906669.456387@12-248-41-177.client.attbi.com> <200201181509.KAA09399@cj20424-a.reston1.va.home.com> <20020118071554.A17496@glacier.arctrix.com> <15432.16014.869439.363615@anthem.wooz.org> Message-ID: <3C484251.9579FEDF@lemburg.com> "Barry A. Warsaw" wrote: > > >>>>> "NS" == Neil Schemenauer writes: > > >> What's the current thinking about making docstrings optional? > >> Does everybody agree on Gustavo's patch? > > NS> 10% space saving? That doesn't seem to be worth the effort. > NS> OTOH, I'm not dealing with any platforms that are memory > NS> constrained right now. > > Personally I don't care either for the same reasons. I'll just note > that what Emacs used to do (maybe it still does, I dunno), is extract > all its inlined docstrings into a separate file which could be thrown > away if you didn't want to pay for the bloat. All that complexity was > built in a time when 300KB or so of docstrings really could make a > huge difference for download times or storage resources. You should also consider the possibility of using the macros for translating the docs-strings. They are a form of markup. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From barry@zope.com Fri Jan 18 15:46:48 2002 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 18 Jan 2002 10:46:48 -0500 Subject: [niemeyer@conectiva.com: Re: [Python-Dev] Python's footprint] References: <20020114093053.C1325@ibook.distro.conectiva> <3C42C9A5.975FA5B8@lemburg.com> <20020114104146.A2607@ibook.distro.conectiva> <3C4326D2.F2A82030@lemburg.com> <200201141847.NAA10894@cj20424-a.reston1.va.home.com> <3C4337D5.B54330B1@lemburg.com> <15427.15272.171558.1993@12-248-41-177.client.attbi.com> <200201142017.PAA12356@cj20424-a.reston1.va.home.com> <15427.18385.906669.456387@12-248-41-177.client.attbi.com> <200201181509.KAA09399@cj20424-a.reston1.va.home.com> <20020118071554.A17496@glacier.arctrix.com> <15432.16014.869439.363615@anthem.wooz.org> <3C484251.9579FEDF@lemburg.com> Message-ID: <15432.17256.768942.156692@anthem.wooz.org> >>>>> "MAL" == M writes: MAL> You should also consider the possibility of using the macros MAL> for translating the docs-strings. They are a form of markup. Good point! -Barry From jack@oratrix.com Fri Jan 18 16:23:30 2002 From: jack@oratrix.com (Jack Jansen) Date: Fri, 18 Jan 2002 17:23:30 +0100 Subject: [niemeyer@conectiva.com: Re: [Python-Dev] Python's footprint] In-Reply-To: <3C483DEB.5B9D12A6@lemburg.com> Message-ID: On Friday, January 18, 2002, at 04:23 , M.-A. Lemburg wrote: > Guido van Rossum wrote: >> >> What's the current thinking about making docstrings optional? >> >> Does everybody agree on Gustavo's patch? >> >> http://sourceforge.net/tracker/?func=detail&atid=305470&aid=505375&group_id= >> 5470 > > +1. > > This will help Python embedders and porters to embedded systems > a lot. +1. Same reasoning. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From paul@prescod.net Fri Jan 18 18:38:05 2002 From: paul@prescod.net (Paul Prescod) Date: Fri, 18 Jan 2002 10:38:05 -0800 Subject: [Python-Dev] Utopian String Interpolation References: <20020117175646.034B5E8C8@waltz.rahul.net> <200201171825.NAA00602@cj20424-a.reston1.va.home.com> Message-ID: <3C486B8D.1E30C93D@prescod.net> I think that something in particular that Paul S. said got under your skin (and there was something he said that could certainly get under a person's skin). I'm pretty sure it isn't now a policy to rudely reject suggestions from people you haven't heard of! Until I went back through the thread I felt as Aahz did that your rejection was somewhat severe in tone. I think you (still) agree that people should not be afraid of (politely) stating their opinions in python-dev, even when those opinions disagree with yours. Or if there is an unspoken rule that unproven developers shouldn't be in python-dev then maybe we should just make it a spoken rule. But I'm most confident of the theory that you snapped at one person in particular because of something he said. Paul Prescod Guido van Rossum wrote: > > > >Paul Svensson: > > >> > > >> I would certainly claim that an unrecognized escape sequence _is_ wrong. > > > > > Guido van Rossum wrote: > > > Then you are wrong. Go away and design your own language. > > > Aahz: > > Hey! That's a bit harsh. I'm not going to campaign to make > > unrecognized escape sequences a syntax error, but not raising a syntax > > error does seem to be against Python's principles. > > Whatever. Who is Paul Svensson and what is he doing in python-dev? > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev From martin@v.loewis.de Fri Jan 18 18:42:22 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: Fri, 18 Jan 2002 19:42:22 +0100 Subject: [Python-Dev] Extending types in C - help needed In-Reply-To: <7D823B3B-0BFD-11D6-B669-0030655234CE@oratrix.com> (message from Jack Jansen on Fri, 18 Jan 2002 11:24:01 +0100) References: <7D823B3B-0BFD-11D6-B669-0030655234CE@oratrix.com> Message-ID: <200201181842.g0IIgMc01444@mira.informatik.hu-berlin.de> From thomas.heller@ion-tof.com Fri Jan 18 18:56:45 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri, 18 Jan 2002 19:56:45 +0100 Subject: [Python-Dev] Extending types in C - help needed References: <7D823B3B-0BFD-11D6-B669-0030655234CE@oratrix.com> <200201181842.g0IIgMc01444@mira.informatik.hu-berlin.de> Message-ID: <05fc01c1a051$f1c4db90$e000a8c0@thomasnotebook> From: "Martin v. Loewis" To: Cc: ; ; Sent: Friday, January 18, 2002 7:42 PM Subject: Re: [Python-Dev] Extending types in C - help needed > Hmm, not very much help ;-) Thomas From thomas.heller@ion-tof.com Fri Jan 18 19:01:16 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri, 18 Jan 2002 20:01:16 +0100 Subject: [Python-Dev] Extending types in C - help needed References: <43ABBA6E-0AD4-11D6-A4BB-003065517236@oratrix.nl> <08c501c19f8c$72631b20$e000a8c0@thomasnotebook> <200201171951.OAA00909@cj20424-a.reston1.va.home.com> Message-ID: <060801c1a052$93d5a860$e000a8c0@thomasnotebook> > > Currently (after quite some time) I have the impression that you > > cannot create a subtype of PyType_Type in C because PyType_Type > > ends in a variable sized array, at least not in this way: > > > > struct { > > PyTypeObject type; > > ...additional fields... > > } WrapperType_Type; > > > > Can someone confirm this? > > Yes, alas. The type you would have to declare is 'etype', a private > type in typeobject.c. Does this mean this is the wrong route, or is it absolute impossible to create a subtype of PyType_Type in C with additional slots? Any tips about the route to take? Thanks, Thomas From martin@v.loewis.de Fri Jan 18 19:36:20 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: Fri, 18 Jan 2002 20:36:20 +0100 Subject: [Python-Dev] Extending types in C - help needed In-Reply-To: <7D823B3B-0BFD-11D6-B669-0030655234CE@oratrix.com> (message from Jack Jansen on Fri, 18 Jan 2002 11:24:01 +0100) References: <7D823B3B-0BFD-11D6-B669-0030655234CE@oratrix.com> Message-ID: <200201181936.g0IJaKu01691@mira.informatik.hu-berlin.de> > Also, the method seems rather complicated for doing a simple thing. The > only thing I really want is a way to refer to an _New or _Convert method > from Python code. I believe the attached code implements your requirements. In particular, see PyArg_GenericCopy for an application that extracts a void* from an object through a type-safe protocol, then creates a clone of the original object through the same protocol. Both extractor and creator function are associated with the type object. To see this work in Python, run >>> import handle >>> x=handle.new(10) >>> x >>> y=handle.copy(x) >>> y Regards, Martin #include "Python.h" /************* Generic Converters ***************/ struct converters{ PyObject* (*create)(void*); int (*extract)(PyObject*, void**); }; char descr_string[] = "calldll converter structure"; void PyArg_AddConverters(PyTypeObject *type, struct converters* convs) { PyObject *cobj = PyCObject_FromVoidPtrAndDesc(convs, descr_string, NULL); PyDict_SetItemString(type->tp_dict, "__calldll__", cobj); Py_DECREF(cobj); } struct converters* PyArg_GetConverters(PyTypeObject *type) { PyObject *cobj; void *descr; cobj = PyObject_GetAttrString((PyObject*)type, "__calldll__"); if (!cobj) return NULL; descr = PyCObject_GetDesc(cobj); if (!descr) return NULL; if (descr != descr_string){ PyErr_SetString(PyExc_TypeError, "invalid cobj"); return NULL; } return (struct converters*)PyCObject_AsVoidPtr(cobj); } PyObject *PyArg_Create(PyTypeObject* type, void * value) { struct converters *convs = PyArg_GetConverters(type); if (!convs) return NULL; return convs->create(value); } int PyArg_Extract(PyObject* obj, void** value) { struct converters *convs = PyArg_GetConverters(obj->ob_type); if (!convs) return -1; convs->extract(obj, value); return 0; } PyObject* PyArg_GenericCopy(PyObject* obj) { void *tmp; if (PyArg_Extract(obj, &tmp)) return NULL; return PyArg_Create(obj->ob_type, tmp); } /************* End Generic Converters ***************/ typedef struct { PyObject_HEAD int handle; } HandleObject; staticforward PyTypeObject Handle_Type; #define HandleObject_Check(v) ((v)->ob_type == &Handle_Type) static HandleObject * newHandleObject(int i) { HandleObject *self; self = PyObject_New(HandleObject, &Handle_Type); if (self == NULL) return NULL; self->handle = i; return self; } /* Handle methods */ static void Handle_dealloc(HandleObject *self) { PyObject_Del(self); } /**************** Generic Converters: Handle support ***************/ static PyObject* handle_conv_new(void *s){ return (PyObject*)newHandleObject((int)s); } static int handle_conv_extract(PyObject *o, void **dest){ HandleObject *h = (HandleObject*)o; *dest = (void*)h->handle; return 0; } struct converters HandleConvs = { handle_conv_new, handle_conv_extract }; /**************** Generic Converters: Handle support ***************/ statichere PyTypeObject Handle_Type = { /* The ob_type field must be initialized in the module init function * to be portable to Windows without using C++. */ PyObject_HEAD_INIT(NULL) 0, /*ob_size*/ "handle.Handle", /*tp_name*/ sizeof(HandleObject), /*tp_basicsize*/ 0, /*tp_itemsize*/ /* methods */ (destructor)Handle_dealloc, /*tp_dealloc*/ 0, /*tp_print*/ 0, /*tp_getattr*/ 0, /*tp_setattr*/ 0, /*tp_compare*/ 0, /*tp_repr*/ 0, /*tp_as_number*/ 0, /*tp_as_sequence*/ 0, /*tp_as_mapping*/ 0, /*tp_hash*/ 0, /*tp_call*/ 0, /*tp_str*/ 0, /*tp_getattro*/ 0, /*tp_setattro*/ 0, /*tp_as_buffer*/ Py_TPFLAGS_DEFAULT, /*tp_flags*/ }; /* --------------------------------------------------------------------- */ static PyObject * xx_new(PyObject *self, PyObject *args) { HandleObject *rv; int h; if (!PyArg_ParseTuple(args, "i:new", &h)) return NULL; rv = newHandleObject(h); if ( rv == NULL ) return NULL; return (PyObject *)rv; } static PyObject * xx_copy(PyObject *self, PyObject *args) { PyObject *obj; if (!PyArg_ParseTuple(args, "O:copy", &obj)) return NULL; return PyArg_GenericCopy(obj); } static PyMethodDef xx_methods[] = { {"new", xx_new, METH_VARARGS}, {"copy", xx_copy, METH_VARARGS}, {NULL, NULL} /* sentinel */ }; DL_EXPORT(void) inithandle(void) { PyObject *m; Handle_Type.ob_type = &PyType_Type; PyType_Ready(&Handle_Type); PyArg_AddConverters(&Handle_Type, &HandleConvs); /* Create the module and add the functions */ m = Py_InitModule("handle", xx_methods); } From sdm7g@Virginia.EDU Fri Jan 18 19:52:18 2002 From: sdm7g@Virginia.EDU (Steven Majewski) Date: Fri, 18 Jan 2002 14:52:18 -0500 (EST) Subject: [Python-Dev] (PyMapping|PyDict|PyObject)_DelItemString [was: [Pythonmac-SIG] pyobjc.so ] In-Reply-To: Message-ID: [ Background note for cc: to python-dev: pyobjc.so builds under both python2.1.2 and python2.2. It works under 2.1.2, but under 2.2, it gives a 'Failure linking new module' error. ] Added a call to NSLinkEditError to get back more info from the error (I'll submit this as a patch to SF after I clean it up a bit.): >>> import pyobjc Traceback (most recent call last): File " ", line 1, in ? ImportError: dyld: /usr/local/src/Python-2.2/python.exe Undefined symbols: _PyObject_DelItemString Failure linking new module >>> grepping for this in 2.1.2 finds nothing. In 2.2, there seems to be one occurance: grep PyObject_DelItemString */*.[ch] Include/abstract.h:#define PyMapping_DelItemString(O,K) PyObject_DelItemString((O),(K)) Searching for PyMapping_DelItemString, it looks like this changed from PyDict_DelItemString() in 2.1.2 to PyObject_DelItemString() in 2.2: dm7g% grep PyMapping_DelItemString Python-2.*/*/*.[ch] Python-2.1.2/Include/abstract.h: int PyMapping_DelItemString(PyObject *o, char *key); Python-2.1.2/Include/abstract.h:#define PyMapping_DelItemString(O,K) PyDict_DelItemString((O),(K)) Python-2.2/Include/abstract.h: int PyMapping_DelItemString(PyObject *o, char *key); Python-2.2/Include/abstract.h:#define PyMapping_DelItemString(O,K) PyObject_DelItemString((O),(K)) Is this change of name an inadvertant bug, or is it something that was intentionally changed, but incompletely? -- Steve From thomas.heller@ion-tof.com Fri Jan 18 20:06:23 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri, 18 Jan 2002 21:06:23 +0100 Subject: [Python-Dev] Extending types in C - help needed References: <7D823B3B-0BFD-11D6-B669-0030655234CE@oratrix.com> <200201181936.g0IJaKu01691@mira.informatik.hu-berlin.de> Message-ID: <072001c1a05b$ac8a08c0$e000a8c0@thomasnotebook> > > Also, the method seems rather complicated for doing a simple thing. The > > only thing I really want is a way to refer to an _New or _Convert method > > from Python code. > > I believe the attached code implements your requirements. Yes, this looks very much like what I had in mind, except that you demonstrate how to store and retrieve a C structure in the type's tp_dict. Nice intro into PyCObject! Thanks, Thomas From thomas.heller@ion-tof.com Fri Jan 18 20:23:11 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri, 18 Jan 2002 21:23:11 +0100 Subject: [Python-Dev] Extending types in C - help needed References: <7D823B3B-0BFD-11D6-B669-0030655234CE@oratrix.com> <200201181936.g0IJaKu01691@mira.informatik.hu-berlin.de> Message-ID: <07a001c1a05e$05170810$e000a8c0@thomasnotebook> [sorry if this is duplicated, I'm having mailer problems] > > Also, the method seems rather complicated for doing a simple thing. The > > only thing I really want is a way to refer to an _New or _Convert method > > from Python code. > > I believe the attached code implements your requirements. Yes, this looks very much like what I had in mind, except that you demonstrate how to store and retrieve a C structure in the type's tp_dict. Nice intro into PyCObject! Thanks, Thomas From martin@v.loewis.de Fri Jan 18 20:24:43 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: Fri, 18 Jan 2002 21:24:43 +0100 Subject: [niemeyer@conectiva.com: Re: [Python-Dev] Python's footprint] In-Reply-To: <200201181509.KAA09399@cj20424-a.reston1.va.home.com> (message from Guido van Rossum on Fri, 18 Jan 2002 10:09:30 -0500) References: <20020110224908.C884@ibook.distro.conectiva> <200201111334.g0BDYLh01331@mira.informatik.hu-berlin.de> <20020111122105.B1808@ibook.distro.conectiva> <200201112347.g0BNlWk01567@mira.informatik.hu-berlin.de> <3C402BBA.1040806@lemburg.com> <20020114093053.C1325@ibook.distro.conectiva> <3C42C9A5.975FA5B8@lemburg.com> <20020114104146.A2607@ibook.distro.conectiva> <3C4326D2.F2A82030@lemburg.com> <200201141847.NAA10894@cj20424-a.reston1.va.home.com> <3C4337D5.B54330B1@lemburg.com> <15427.15272.171558.1993@12-248-41-177.client.attbi.com> <200201142017.PAA12356@cj20424-a.reston1.va.home.com> <15427.18385.906669.456387@12-248-41-177.client.attbi.com> <200201181509.KAA09399@cj20424-a.reston1.va.home.com> Message-ID: <200201182024.g0IKOhL01893@mira.informatik.hu-berlin.de> > What's the current thinking about making docstrings optional? > > Does everybody agree on Gustavo's patch? Looks good to me. Martin From martin@v.loewis.de Fri Jan 18 20:27:24 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: Fri, 18 Jan 2002 21:27:24 +0100 Subject: [niemeyer@conectiva.com: Re: [Python-Dev] Python's footprint] In-Reply-To: <3C484251.9579FEDF@lemburg.com> (mal@lemburg.com) References: <20020114093053.C1325@ibook.distro.conectiva> <3C42C9A5.975FA5B8@lemburg.com> <20020114104146.A2607@ibook.distro.conectiva> <3C4326D2.F2A82030@lemburg.com> <200201141847.NAA10894@cj20424-a.reston1.va.home.com> <3C4337D5.B54330B1@lemburg.com> <15427.15272.171558.1993@12-248-41-177.client.attbi.com> <200201142017.PAA12356@cj20424-a.reston1.va.home.com> <15427.18385.906669.456387@12-248-41-177.client.attbi.com> <200201181509.KAA09399@cj20424-a.reston1.va.home.com> <20020118071554.A17496@glacier.arctrix.com> <15432.16014.869439.363615@anthem.wooz.org> <3C484251.9579FEDF@lemburg.com> Message-ID: <200201182027.g0IKROV01896@mira.informatik.hu-berlin.de> > You should also consider the possibility of using the macros > for translating the docs-strings. They are a form of markup. While that is true, most of the current strings are marked-up already, by means of having an __doc__ suffix. I have an extractor that understands this form of markup, and the Python .pot file in CVS has those strings extracted. Regards, Martin From martin@v.loewis.de Fri Jan 18 20:53:30 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: Fri, 18 Jan 2002 21:53:30 +0100 Subject: [Python-Dev] Extending types in C - help needed In-Reply-To: <072001c1a05b$ac8a08c0$e000a8c0@thomasnotebook> (thomas.heller@ion-tof.com) References: <7D823B3B-0BFD-11D6-B669-0030655234CE@oratrix.com> <200201181936.g0IJaKu01691@mira.informatik.hu-berlin.de> <072001c1a05b$ac8a08c0$e000a8c0@thomasnotebook> Message-ID: <200201182053.g0IKrU001987@mira.informatik.hu-berlin.de> > Yes, this looks very much like what I had in mind, except that you > demonstrate how to store and retrieve a C structure in the type's > tp_dict. Indeed. I also think it is more appropriate than either a new metatype or a ParseTuple extension for the problem at hand (supporting arbitrary types in calldll), for the following reasons: - There may be different ways of how an object converts to a "native" type. In particular, in some cases, ParseTuple may need to return (fill out) something more complex than a void*, something that calldll cannot support by nature. - A type may need to provide various independent extensions to the standard protocols, e.g. it may provide "give me a Unicode doc string" in addition to "give me a conversion function to void*". In this case, you'd need multiple inheritance on the metatype level, something that does not reflect well in C. For Python, it is much more common not to care at all about inheritance. Instead, just access the protocol, and expect an exception if it is not supported. Also notice that this *does* make use of new-style classes: In 2.1, types did not have a tp_dict slot. Of course, the PyType_Ready call should go immediately before the place where tp_dict is accessed, and a check should be added whether tp_flags contains Py_TPFLAGS_HAVE_CLASS. Regards, Martin From guido@python.org Fri Jan 18 20:57:21 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 18 Jan 2002 15:57:21 -0500 Subject: [Python-Dev] (PyMapping|PyDict|PyObject)_DelItemString [was: [Pythonmac-SIG] pyobjc.so ] In-Reply-To: Your message of "Fri, 18 Jan 2002 14:52:18 EST." References: Message-ID: <200201182057.PAA21273@cj20424-a.reston1.va.home.com> > >>> import pyobjc > Traceback (most recent call last): > File " ", line 1, in ? > ImportError: dyld: /usr/local/src/Python-2.2/python.exe Undefined symbols: > _PyObject_DelItemString > Failure linking new module > >>> > > grepping for this in 2.1.2 finds nothing. > > In 2.2, there seems to be one occurance: > > grep PyObject_DelItemString */*.[ch] > Include/abstract.h:#define PyMapping_DelItemString(O,K) PyObject_DelItemString((O),(K)) > > Searching for PyMapping_DelItemString, it looks like this changed from > PyDict_DelItemString() in 2.1.2 to PyObject_DelItemString() in 2.2: > > dm7g% grep PyMapping_DelItemString Python-2.*/*/*.[ch] > Python-2.1.2/Include/abstract.h: int PyMapping_DelItemString(PyObject *o, char *key); > Python-2.1.2/Include/abstract.h:#define PyMapping_DelItemString(O,K) PyDict_DelItemString((O),(K)) > Python-2.2/Include/abstract.h: int PyMapping_DelItemString(PyObject *o, char *key); > Python-2.2/Include/abstract.h:#define PyMapping_DelItemString(O,K) PyObject_DelItemString((O),(K)) > > > Is this change of name an inadvertant bug, or is it something that > was intentionally changed, but incompletely? The latter. See: http://sourceforge.net/tracker/?group_id=5470&atid=105470&func=detail&aid=498915 --Guido van Rossum (home page: http://www.python.org/~guido/) From martin@v.loewis.de Fri Jan 18 20:57:48 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: Fri, 18 Jan 2002 21:57:48 +0100 Subject: [Python-Dev] (PyMapping|PyDict|PyObject)_DelItemString [was: [Pythonmac-SIG] pyobjc.so ] In-Reply-To: (message from Steven Majewski on Fri, 18 Jan 2002 14:52:18 -0500 (EST)) References: Message-ID: <200201182057.g0IKvms01998@mira.informatik.hu-berlin.de> > Is this change of name an inadvertant bug, or is it something that > was intentionally changed, but incompletely? This is bug #498915, fixed in abstract.h 2.43 and 2.42.6.1, abstract.c 2.94 and 2.93.6.1 Regards, Martin From thomas.heller@ion-tof.com Fri Jan 18 21:21:59 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri, 18 Jan 2002 22:21:59 +0100 Subject: [Python-Dev] Extending types in C - help needed References: <7D823B3B-0BFD-11D6-B669-0030655234CE@oratrix.com> <200201181936.g0IJaKu01691@mira.informatik.hu-berlin.de> <072001c1a05b$ac8a08c0$e000a8c0@thomasnotebook> <200201182053.g0IKrU001987@mira.informatik.hu-berlin.de> Message-ID: <094901c1a066$3c54e6f0$e000a8c0@thomasnotebook> > Also notice that this *does* make use of new-style classes: In 2.1, > types did not have a tp_dict slot. Of course, the PyType_Ready call > should go immediately before the place where tp_dict is accessed, and > a check should be added whether tp_flags contains > Py_TPFLAGS_HAVE_CLASS. Wouldn't it suffice to check for tp_dict != NULL (after the call to PyType_Ready of course)? Hm. What does Py_TPFLAGS_HAVE_CLASS mean exactly? Or, better, since TPFLAGS_DEFAULT contains TPFLAGS_HAVE_CLASS, what does it mean when Py_TPFLAGS_HAVE_CLASS is NOT in tp_flags? Does it mean that this is a 'new style' type object? Thomas From martin@v.loewis.de Fri Jan 18 21:32:23 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: Fri, 18 Jan 2002 22:32:23 +0100 Subject: [Python-Dev] Extending types in C - help needed In-Reply-To: <094901c1a066$3c54e6f0$e000a8c0@thomasnotebook> (thomas.heller@ion-tof.com) References: <7D823B3B-0BFD-11D6-B669-0030655234CE@oratrix.com> <200201181936.g0IJaKu01691@mira.informatik.hu-berlin.de> <072001c1a05b$ac8a08c0$e000a8c0@thomasnotebook> <200201182053.g0IKrU001987@mira.informatik.hu-berlin.de> <094901c1a066$3c54e6f0$e000a8c0@thomasnotebook> Message-ID: <200201182132.g0ILWNf02114@mira.informatik.hu-berlin.de> > Wouldn't it suffice to check for tp_dict != NULL (after the call > to PyType_Ready of course)? No, see below (although I must admit that I wrote "Right" here first :-) > Hm. What does Py_TPFLAGS_HAVE_CLASS mean exactly? According to the documentation, it means that the underlying TypeObject structure has the necessary fields in its C declaration. > Or, better, since TPFLAGS_DEFAULT contains TPFLAGS_HAVE_CLASS, > what does it mean when Py_TPFLAGS_HAVE_CLASS is NOT in tp_flags? It means you have been loading a module from an earlier Python version, which had a different setting for TPFLAGS_DEFAULTS, and a shorter definition of the TypeObject. If you try to access tp_dict in such an object, you are accessing random memory. This may immediately crash, or only crash when you pass the pointer you got to the dictionary functions. Regards, Martin From guido@python.org Fri Jan 18 22:12:29 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 18 Jan 2002 17:12:29 -0500 Subject: [Python-Dev] Utopian String Interpolation In-Reply-To: Your message of "Fri, 18 Jan 2002 10:38:05 PST." <3C486B8D.1E30C93D@prescod.net> References: <20020117175646.034B5E8C8@waltz.rahul.net> <200201171825.NAA00602@cj20424-a.reston1.va.home.com> <3C486B8D.1E30C93D@prescod.net> Message-ID: <200201182212.RAA26732@cj20424-a.reston1.va.home.com> > I think that something in particular that Paul S. said got under your > skin (and there was something he said that could certainly get under a > person's skin). I'm pretty sure it isn't now a policy to rudely reject > suggestions from people you haven't heard of! Until I went back through > the thread I felt as Aahz did that your rejection was somewhat severe in > tone. I think you (still) agree that people should not be afraid of > (politely) stating their opinions in python-dev, even when those > opinions disagree with yours. Or if there is an unspoken rule that > unproven developers shouldn't be in python-dev then maybe we should just > make it a spoken rule. But I'm most confident of the theory that you > snapped at one person in particular because of something he said. > > Paul Prescod He harped at the same issue in three consecutive message without explaining his position. --Guido van Rossum (home page: http://www.python.org/~guido/) From sdm7g@Virginia.EDU Fri Jan 18 22:49:38 2002 From: sdm7g@Virginia.EDU (Steven Majewski) Date: Fri, 18 Jan 2002 17:49:38 -0500 (EST) Subject: [Python-Dev] Re: several messages In-Reply-To: <200201182057.g0IKvms01998@mira.informatik.hu-berlin.de> Message-ID: On Fri, 18 Jan 2002, Guido van Rossum wrote: > http://sourceforge.net/tracker/?group_id=5470&atid=105470&func=detail&aid=498915 On Fri, 18 Jan 2002, Martin v. Loewis wrote: > This is bug #498915, fixed in abstract.h 2.43 and 2.42.6.1, > abstract.c 2.94 and 2.93.6.1 Thanks. I changed it back to PyDict_... With that patch, pyobjc seems to build and work with Python-2.2 as well as 2.1.2. -- Steve. From jason@jorendorff.com Fri Jan 18 23:18:27 2002 From: jason@jorendorff.com (Jason Orendorff) Date: Fri, 18 Jan 2002 17:18:27 -0600 Subject: [Python-Dev] Utopian String Interpolation In-Reply-To: <200201182212.RAA26732@cj20424-a.reston1.va.home.com> Message-ID: Paul Prescod: > > [...] But I'm most confident of the theory that you > > snapped at one person in particular because of something he said. Guido: > He harped at the same issue in three consecutive message without > explaining his position. Actually I was quite happy with the thread. At runtime, Python tends to complain about iffy situations, even situations that other languages might silently accept. For example: print 50 + " percent" # TypeError x = [1, 2, 3]; x.remove(4) # ValueError x = {}; print x[3] # KeyError a, b = "x,y,z,z,y".split() # ValueError x.append(1, 2) # TypeError, recently print u"\N{EURO SIGN}" # UnicodeError I'm not complaining. I like the pickiness. But the Python compiler (that is, Python's syntax) tends to be more forgiving. Examples: - Inconsistent use of tabs and spaces. (Originally handled by tabnanny.py; now an optional warning in Python itself.) - Useless or probably-useless expressions, like these: def g(f): os.environ['EDITOR'] # does nothing with value f.write(xx), f.write(yy) # should be ; not , f.close # obvious mistake (PyChecker catches the last one.) - Non-escaping backslashes in strings (there is a well-known reason for this one; but the reason no longer exists, in new code anyway, since 1.5.) So we catch things like this with static analysis tools like tabnanny.py, or lately PyChecker. If Guido finds any of these syntax-checks compelling enough, he can always incorporate them into Python whenever (but don't hold your breath). Again, you'll get no complaints from me on this. But I am curious. Is this apparent difference in pickiness a design choice? Or is it just harder to write picky compilers than picky libraries? Or am I seeing something that's not really there? ## Jason Orendorff http://www.jorendorff.com/ From Jack.Jansen@oratrix.nl Sat Jan 19 00:07:56 2002 From: Jack.Jansen@oratrix.nl (Jack Jansen) Date: Sat, 19 Jan 2002 01:07:56 +0100 Subject: [Python-Dev] Extending types in C - help needed In-Reply-To: <200201181936.g0IJaKu01691@mira.informatik.hu-berlin.de> Message-ID: <9709BBF2-0C70-11D6-BED2-003065517236@oratrix.nl> On Friday, January 18, 2002, at 08:36 PM, Martin v. Loewis wrote: >> Also, the method seems rather complicated for doing a simple >> thing. The >> only thing I really want is a way to refer to an _New or >> _Convert method >> from Python code. > > I believe the attached code implements your requirements. > Martin, hats off! This does exactly what I want, and it does so in a pretty generalized way. Actually in _such_ a generalized way that I think this should be documented loud and clear. Looking at it a bit more, how about storing each function pointer in a separate PyCObject, and adding general APIs somewhere in the core void PyType_SetAnnotation(PyTypeObject *tp, char *name, char *descr, void *); void *PyType_GetAnnotation(PyTypeObject *tp, char *name, char *descr); (I've picked the name annotation here, because it sort-of feels like that, another name may bring the idea across better). > -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From tim.one@home.com Sat Jan 19 00:10:26 2002 From: tim.one@home.com (Tim Peters) Date: Fri, 18 Jan 2002 19:10:26 -0500 Subject: [Python-Dev] deprecate input()? In-Reply-To: <15431.3203.415990.602525@beluga.mojam.com> Message-ID: [Skip Montanaro] > Yes, but what if the program containing calls to input() get shipped to > someone else's computer? It just seems to me that a) input is almost > never what you want to call and that b) it would seem to a naive > programmer to be the correct way to ask the user for a line of input. One of my favorite papers for the upcoming Python Conference describes the use of Python in a CAD system for chip design. The authors had indeed used input(), and didn't know that it eval'ed expressions. The program's users discovered it first, succumbing to a natural urge to type expressions in the input fields. One of the things that made this paper a favorite is that the authors didn't whine about this: to the contrary, they were delighted to get the kudos for Guido's good intuition about what a kick-ass input() function should do. guido-never-drives-before-a-few-stiff-drinks-either -ly y'rs - tim From martin@v.loewis.de Sat Jan 19 00:28:23 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: Sat, 19 Jan 2002 01:28:23 +0100 Subject: [Python-Dev] Extending types in C - help needed In-Reply-To: <9709BBF2-0C70-11D6-BED2-003065517236@oratrix.nl> (message from Jack Jansen on Sat, 19 Jan 2002 01:07:56 +0100) References: <9709BBF2-0C70-11D6-BED2-003065517236@oratrix.nl> Message-ID: <200201190028.g0J0SN102701@mira.informatik.hu-berlin.de> > Martin, hats off! > > This does exactly what I want, and it does so in a pretty > generalized way. Actually in _such_ a generalized way that I > think this should be documented loud and clear. Thanks! > Looking at it a bit more, how about storing each function > pointer in a separate PyCObject, and adding general APIs > somewhere in the core > void PyType_SetAnnotation(PyTypeObject *tp, char *name, char > *descr, void *); > void *PyType_GetAnnotation(PyTypeObject *tp, char *name, char *descr); I'll happily add that to some recipe collection. However, before generalizing it, I'd like to see more use cases. There should, atleast, be a *second* application beyond calldll (or, perhaps even beyond MacPython). Generalizing from a single use case is not good. Regards, Martin From guido@python.org Sat Jan 19 03:38:56 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 18 Jan 2002 22:38:56 -0500 Subject: [Python-Dev] When to signal an error In-Reply-To: Your message of "Fri, 18 Jan 2002 17:18:27 CST." References: Message-ID: <200201190338.WAA28421@cj20424-a.reston1.va.home.com> (I'm changing the topic :-) > At runtime, Python tends to complain about iffy situations, > even situations that other languages might silently accept. "Other languages" being Perl or JavaScript? The situations you show here would all be errors in most languages that are compiled to machine code. > For example: > > print 50 + " percent" # TypeError > x = [1, 2, 3]; x.remove(4) # ValueError > x = {}; print x[3] # KeyError > a, b = "x,y,z,z,y".split() # ValueError > x.append(1, 2) # TypeError, recently > print u"\N{EURO SIGN}" # UnicodeError > > I'm not complaining. I like the pickiness. That's why you're using Python. :-) > But the Python compiler (that is, Python's syntax) tends to be > more forgiving. Examples: > > - Inconsistent use of tabs and spaces. (Originally handled > by tabnanny.py; now an optional warning in Python itself.) > - Useless or probably-useless expressions, like these: > def g(f): > os.environ['EDITOR'] # does nothing with value > f.write(xx), f.write(yy) # should be ; not , > f.close # obvious mistake > (PyChecker catches the last one.) > - Non-escaping backslashes in strings (there is a well-known > reason for this one; but the reason no longer exists, in new > code anyway, since 1.5.) > > So we catch things like this with static analysis tools like > tabnanny.py, or lately PyChecker. If Guido finds any of these > syntax-checks compelling enough, he can always incorporate them > into Python whenever (but don't hold your breath). > > Again, you'll get no complaints from me on this. But I am > curious. Is this apparent difference in pickiness a design > choice? Or is it just harder to write picky compilers than > picky libraries? Or am I seeing something that's not really > there? There's no unifying reason why thes examples are not errors. The first and last can be considered historical raisins -- the tabs/spaces mix was considered a good thing in the days when Python only ran on Unixoid systems where nobody would think about changing the display size for tabs; we know the reason for the last. But it's hard to change these without inconveniencing users, and there are other ways to deal with them (like picky tools). The three examples in the second item have in common that they are syntactically expressions but are used in a statement context. The problem here that any language designer is faced with: you would want to allow expressions with an obvious side-effect, but you would want to disallow expressions that obviously have no side-effects. But where to draw the line? Traditional parsing technology such as used in Python makes it hard to be very differentiating here; a good analysis of which expressions "make sense" and which ones don't can only be done during a later pass of the compiler. I believe that evertually some PyChecker-like technology will be incorporated in the Python compiler. The same happened to C compilers: the lint program became useless once GCC incorporated the same technology. But these warnings will always have a different status than purely syntactical error: there are often cases where the user knows better (for example, sometimes an attribute reference can have a desirable side effect). --Guido van Rossum (home page: http://www.python.org/~guido/) From neal@metaslash.com Sat Jan 19 19:25:18 2002 From: neal@metaslash.com (Neal Norwitz) Date: Sat, 19 Jan 2002 14:25:18 -0500 Subject: [Python-Dev] When to signal an error References: <200201190338.WAA28421@cj20424-a.reston1.va.home.com> Message-ID: <3C49C81E.C2D2F1BD@metaslash.com> Guido van Rossum wrote: > I believe that evertually some PyChecker-like technology will be > incorporated in the Python compiler. The same happened to C > compilers: the lint program became useless once GCC incorporated the > same technology. pychecker was (and still is) an experiment to me. But I think it would be great if the lessons from pychecker could be integrated into the compiler. Currently, I think there are 2 or 3 warnings which definitely fit this class: No global found, using ++/--, and expressions with no effect as Jason described. I have posted a patch on SF to demonstrate the feasibility of expressions with no effect: https://sourceforge.net/tracker/index.php?func=detail&aid=505826&group_id=5470&atid=305470 It should be pretty easy to warn about ++ and --. No global found would probably require another pass of the code after compilation. I'd be happy to help the process of integrating warnings into the compiler, however, I'm not sure how to proceed. Should pychecker be put into the standard library (users can now do: import pychecker.checker and all modules imported are checked by installing an __import__)? Should pychecker be added as a tool? Should a PEP be written? etc. > But these warnings will always have a different status than purely > syntactical error: there are often cases where the user knows better > (for example, sometimes an attribute reference can have a desirable > side effect). I agree. Neal From jason@jorendorff.com Sat Jan 19 23:16:42 2002 From: jason@jorendorff.com (Jason Orendorff) Date: Sat, 19 Jan 2002 17:16:42 -0600 Subject: [Python-Dev] When to signal an error In-Reply-To: <3C49C81E.C2D2F1BD@metaslash.com> Message-ID: Neal Norwitz: > Guido van Rossum: > > But these warnings will always have a different status than purely > > syntactical error: there are often cases where the user knows better > > (for example, sometimes an attribute reference can have a desirable > > side effect). > > I agree. Here's what Pychecker finds in the standard library (as of 2.2). In each case, the expression is intended to raise an exception if the named variable or attribute doesn't exist. Each one could be rewritten (I'm curious as to the prevailing stylistic opinions on this): === code.py (lines 217 and 221) try: sys.ps1 except AttributeError: sys.ps1 = ">>> " try: sys.ps2 except AttributeError: sys.ps2 = "... " Could be rewritten: if not hasattr(sys, 'ps1'): sys.ps1 = ">>> " if not hasattr(sys, 'ps2'): sys.ps2 = "... " === locale.py (line 721) try: LC_MESSAGES except: pass else: __all__.append("LC_MESSAGES") Could be rewritten: if globals().has_key("LC_MESSAGES"): __all__.append("LC_MESSAGES") === pickle.py (line 58) try: UnicodeType except NameError: UnicodeType = None Could be rewritten: globals().setdefault('UnicodeType', None) ## Jason Orendorff http://www.jorendorff.com/ From jason@jorendorff.com Sat Jan 19 23:34:12 2002 From: jason@jorendorff.com (Jason Orendorff) Date: Sat, 19 Jan 2002 17:34:12 -0600 Subject: [Python-Dev] When to signal an error In-Reply-To: <200201190338.WAA28421@cj20424-a.reston1.va.home.com> Message-ID: Guido van Rossum wrote: > Jason Orendorff wrote: > > At runtime, Python tends to complain about iffy situations, > > even situations that other languages might silently accept. > > "Other languages" being Perl or JavaScript? The situations you show > here would all be errors in most languages that are compiled to > machine code. > > > For example: > > print 50 + " percent" # TypeError > > x = [1, 2, 3]; x.remove(4) # ValueError > > x = {}; print x[3] # KeyError > > a, b = "x,y,z,z,y".split() # ValueError > > x.append(1, 2) # TypeError, recently > > print u"\N{EURO SIGN}" # UnicodeError Not to bicker, but Java only manages to reject 2 of the 6, both at compile time. The other 4 silently pass through the standard library without complaint. None cause exceptions during execution. ML makes no distinction between append(1, 2) and append((1, 2)), but that's a syntax thing... C++ STL remove() doesn't complain if it doesn't find anything to remove; nor does the C++ map<>::operator[]() complain if no entry exists. > > I'm not complaining. I like the pickiness. > > That's why you're using Python. :-) (laugh) You sell yourself short, Guido. :) I would still use Python even if (50 + " percent") started evaluating to "50 percent" tomorrow. ## Jason Orendorff http://www.jorendorff.com/ From martin@v.loewis.de Sun Jan 20 00:02:10 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: Sun, 20 Jan 2002 01:02:10 +0100 Subject: [Python-Dev] When to signal an error In-Reply-To: References: Message-ID: <200201200002.g0K02Ao03439@mira.informatik.hu-berlin.de> > Each one could be rewritten (I'm curious as to the prevailing > stylistic opinions on this): I think those rewrites do not improve the code, see detailed comments below. > Could be rewritten: > if not hasattr(sys, 'ps1'): > sys.ps1 = ">>> " > if not hasattr(sys, 'ps2'): > sys.ps2 = "... " Using string literals when you mean attribute names is bad style. It just helps to trick the checker. Sometimes, you cannot avoid this style, but if you can, you should. > if globals().has_key("LC_MESSAGES"): > __all__.append("LC_MESSAGES") This combines the previous issue with the usage of globals(). I find it confusing to perform function calls to check for the presence of names. > try: > UnicodeType > except NameError: > UnicodeType = None > > Could be rewritten: > globals().setdefault('UnicodeType', None) Same issue here. If this needs to be rewritten, I'd prefer try: from types import UnicodeType except ImportError: UnicodeType = None Somebody might also change the "from types import *" to explicitly list the set of names that are requested, when changing this fragment. Regards, Martin From guido@python.org Sun Jan 20 00:53:41 2002 From: guido@python.org (Guido van Rossum) Date: Sat, 19 Jan 2002 19:53:41 -0500 Subject: [Python-Dev] Extending types in C - help needed In-Reply-To: Your message of "Fri, 18 Jan 2002 20:01:16 +0100." <060801c1a052$93d5a860$e000a8c0@thomasnotebook> References: <43ABBA6E-0AD4-11D6-A4BB-003065517236@oratrix.nl> <08c501c19f8c$72631b20$e000a8c0@thomasnotebook> <200201171951.OAA00909@cj20424-a.reston1.va.home.com> <060801c1a052$93d5a860$e000a8c0@thomasnotebook> Message-ID: <200201200053.TAA30250@cj20424-a.reston1.va.home.com> > > Yes, alas. The type you would have to declare is 'etype', a private > > type in typeobject.c. > > Does this mean this is the wrong route, or is it absolute impossible > to create a subtype of PyType_Type in C with additional slots? I wish I had time to explain this, but I don't. For now, you'll have to read how types are initialized in typeobject.c -- maybe there's a way, maybe there isn't. > Any tips about the route to take? It can be done easily dynamically. --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Sun Jan 20 12:11:57 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Sun, 20 Jan 2002 13:11:57 +0100 Subject: [Python-Dev] Extending types in C - help needed References: <7D823B3B-0BFD-11D6-B669-0030655234CE@oratrix.com> <200201181936.g0IJaKu01691@mira.informatik.hu-berlin.de> Message-ID: <3C4AB40D.47E3E35@lemburg.com> [Martin's PyCObject based Handle object] This seems to be very close to the __reduce__ idea I posted on this thread a couple of days ago. Why not extend it to fully support this standard Python protocol ? -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From Boris_Lipner Sun Jan 20 12:49:52 2002 From: Boris_Lipner (Boris_Lipner) Date: Sun, 20 Jan 2002 15:49:52 +0300 Subject: [Python-Dev] cooperation Message-ID: <8515280648.20020120154952@nm.ru> Dear Sirs, For some technical reasons we have partially lost our Data Bank of art galleries. Please, write the address of your website, so that we could continue our cooperation. Our site is http://www.gallery-a.ru/ Best regards, Boris_Lipner v2004@nm.ru From martin@v.loewis.de Sun Jan 20 19:13:22 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: Sun, 20 Jan 2002 20:13:22 +0100 Subject: [Python-Dev] Extending types in C - help needed In-Reply-To: <3C4AB40D.47E3E35@lemburg.com> (mal@lemburg.com) References: <7D823B3B-0BFD-11D6-B669-0030655234CE@oratrix.com> <200201181936.g0IJaKu01691@mira.informatik.hu-berlin.de> <3C4AB40D.47E3E35@lemburg.com> Message-ID: <200201201913.g0KJDMO01304@mira.informatik.hu-berlin.de> > This seems to be very close to the __reduce__ idea I posted > on this thread a couple of days ago. Why not extend it to > fully support this standard Python protocol ? Because it is not clear, to me, what specifically the semantics of this protocol is. I wrote it to support MacOS calldll. I cannot see applicability beyond this API. One of the strength of OO and polymorphism is precisely that users can freely extend the protocols that their objects support, without requiring *all* objects to support the protocol. A standard protocol should be clearly useful cross-platform, for many different types, in different applications. Regards, Martin From res0peyy@verizon.net Sun Jan 20 21:48:24 2002 From: res0peyy@verizon.net (Joshua 'The List' S.) Date: Sun, 20 Jan 2002 13:48:24 -0800 Subject: [Python-Dev] (no subject) Message-ID: <20020120214832.LDFQ4908.out020.verizon.net@there> From ping@lfw.org Sun Jan 20 22:23:15 2002 From: ping@lfw.org (Ka-Ping Yee) Date: Sun, 20 Jan 2002 16:23:15 -0600 (CST) Subject: [Python-Dev] Python and Security In-Reply-To: <3C472FB3.81EAA5D5@prescod.net> Message-ID: "M.-A. Lemburg" wrote: > ... Note that Python hasn't really had a need > for Perl's "taint" because of this. I wouldn't want to see that > change in any way. On Thu, 17 Jan 2002, Paul Prescod wrote: > I am certainly not a Perl programmer but Python is also attackable > through the sorts of holes that "taint" is intended to avoid. Paul is right on the money. Tainting is a completely separate issue. That said, however, i wonder why security rarely comes up as an issue for Python. Is it because nobody expects security properties from the language? Does anyone know how much the restricted execution feature gets used? Is there anyone here that would use a tainting feature if it existed? It would be interesting to explore the possibilities for safe distributed programming in Python. Restricted execution mode and the ability to hook __import__ seem like a pretty strong starting point, and given a suitable cryptographic comm library, it might be feasible to get from there to capability-style distributed programming. IMHO, simplicity and readability are extremely important for a secure programming language, so that gives Python a great head start. (By the way, i'm planning to be at Python 10, and hope to see many of you there. As i'm looking for ways to keep costs down, would anyone be interested in splitting the cost of a hotel room in exchange for a roommate with a strange hairstyle? I'll be there Feb 4 to 7, three nights.) -- ?!ng From martin@v.loewis.de Sun Jan 20 22:37:11 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: Sun, 20 Jan 2002 23:37:11 +0100 Subject: [Python-Dev] Python and Security In-Reply-To: (message from Ka-Ping Yee on Sun, 20 Jan 2002 16:23:15 -0600 (CST)) References: Message-ID: <200201202237.g0KMbBY02366@mira.informatik.hu-berlin.de> > That said, however, i wonder why security rarely comes up as an > issue for Python. Is it because nobody expects security properties > from the language? Does anyone know how much the restricted > execution feature gets used? Is there anyone here that would use > a tainting feature if it existed? In my understanding, tainting is needed if you allow data received from remote to invoke arbitrary operations. In Python, there is only a short list where this might cause a problem: - invoking exec or eval on a string of unknown origin - unpickling an arbitrary string - performing getattr with a parameter of unknown origin. Because there are so few places where tainted data may cause problems, it never is an issue: people just intuitively know to avoid them. > It would be interesting to explore the possibilities for safe > distributed programming in Python. Not sure what this has to do with tainting, though: if you want to execute code you receive from untrusted sources, a sandbox is closer to what you need. Regards, Martin From barry@zope.com Sun Jan 20 23:01:44 2002 From: barry@zope.com (Barry A. Warsaw) Date: Sun, 20 Jan 2002 18:01:44 -0500 Subject: [Python-Dev] Python and Security References: <200201202237.g0KMbBY02366@mira.informatik.hu-berlin.de> Message-ID: <15435.19544.899789.631148@anthem.wooz.org> >>>>> "MvL" == Martin v Loewis writes: | - invoking exec or eval on a string of unknown origin | - unpickling an arbitrary string | - performing getattr with a parameter of unknown origin. Don't forget os.system(), popen(), and friends, i.e. passing unsanitized strings to the shell. In my my long rusty Perl experience, this was the most common reason to use taint strings. Python OTOH really has very little need to call out to the shell; almost everything you'd want to do that way can be done in pure Python. There are some opportunties for improving string sanitization for the few instances where os.system() is necessary. Most of the security issues I've had to deal with in Mailman have been in library modules -- or the use thereof, not in the language itself. Things like vulnerabilies in Cookie.py or pickle/marshal, or cross-site scripting exploits, that kind of thing. There are also more subtle issues that would be interesting to explore, like DoS attacks with thru-the-web regular expression searching, deliberate form confuddling, and some of the ttw code execution stuff that e.g. Zope gets into. Rexec is an incomplete solution to the latter. -Barry From paul@prescod.net Sun Jan 20 23:49:58 2002 From: paul@prescod.net (Paul Prescod) Date: Sun, 20 Jan 2002 15:49:58 -0800 Subject: [Python-Dev] Re: Python and Security References: Message-ID: <3C4B57A6.702BFC36@prescod.net> Ka-Ping Yee wrote: > >... > > That said, however, i wonder why security rarely comes up as an > issue for Python. I guess you didn't read comp.lang.python this week. ;) http://www.securityfocus.com/archive/1/250580 > ... Is it because nobody expects security properties > from the language? Remember that people for a long time thought of Perl as a "CGI language". And early uses of CGI would probably have depended heavily on the Perl equivalents of "popen" and "system". Plus, those features are so easy to get at in the language. Compare: print `ls` versus: import os print os.popen("ls").read() If you were a newbie in each of these languages what are the percentage chance of you using either of these features versus the list-dir equivalent. List-dir is available in each language. > ... Does anyone know how much the restricted > execution feature gets used? I personally would not trust it because I don't know if anyone is following its progress from one version of Python to another. I also know that even languages that are designed from scratch to be safe (Java and JavaScript) have had leaky implemetations so I don't really hold out much hope for Python until I hear that someone is actively researching this. > ... Is there anyone here that would use > a tainting feature if it existed? I'd like to think I've internalized taints rules by osmosis... > (By the way, i'm planning to be at Python 10, and hope to see many > of you there. As i'm looking for ways to keep costs down, would > anyone be interested in splitting the cost of a hotel room in > exchange for a roommate with a strange hairstyle? I'll be there > Feb 4 to 7, three nights.) Maybe there should be a bulletin board or something for people to find each other. I think one of the Python conferences had something like that...for hotels and also to share cabs from the airport. Paul Prescod From simon@netthink.co.uk Mon Jan 21 00:11:27 2002 From: simon@netthink.co.uk (Simon Cozens) Date: Mon, 21 Jan 2002 00:11:27 +0000 Subject: [Python-Dev] Python and Security In-Reply-To: <200201202237.g0KMbBY02366@mira.informatik.hu-berlin.de> References: <200201202237.g0KMbBY02366@mira.informatik.hu-berlin.de> Message-ID: <20020121001127.GA5014@netthink.co.uk> On Sun, Jan 20, 2002 at 11:37:11PM +0100, Martin v. Loewis wrote: > In my understanding, tainting is needed if you allow data received > from remote to invoke arbitrary operations. In Python, there is only a > short list where this might cause a problem: > > - invoking exec or eval on a string of unknown origin > - unpickling an arbitrary string > - performing getattr with a parameter of unknown origin. >From a Perl point of view, tainting is there to stop data received from outside to do *anything* related to the system. This includes what you say, but goes further: - open - os.popen (in fact, most of os.*) - socket (no, really) and everything that depends on it (urllib, etc.) Since Python has rexec for this sort of thing, tainting may not be so important, but I think rexec goes too far. The idea of tainting is not to *disallow* using, say, arbitrary user input from CGI scripts as filenames - it's help the programmer segregate which pieces of data need special treatment before being passed to these kinds of functions. -- Rule the Empire through force. -- Shogun Tokugawa From aahz@rahul.net Mon Jan 21 01:38:59 2002 From: aahz@rahul.net (Aahz Maruch) Date: Sun, 20 Jan 2002 17:38:59 -0800 (PST) Subject: [Python-Dev] Python and Security In-Reply-To: <15435.19544.899789.631148@anthem.wooz.org> from "Barry A. Warsaw" at Jan 20, 2002 06:01:44 PM Message-ID: <20020121013900.62CFCE8CD@waltz.rahul.net> Barry A. Warsaw wrote: > >>>>> "MvL" == Martin v Loewis writes: > > | - invoking exec or eval on a string of unknown origin > | - unpickling an arbitrary string > | - performing getattr with a parameter of unknown origin. > > Don't forget os.system(), popen(), and friends, i.e. passing > unsanitized strings to the shell. In my my long rusty Perl > experience, this was the most common reason to use taint strings. More precisely, because Perl culture developed as a superset of shell scripts, it used to be all-too-common for Perl scripts to get their data by parsing the output of a Unix utility (instead of calling a library function directly). This necessarily spawned a subshell where malicious input could be a security problem. (When I was learning Perl, the available books often taught this programming style.) I've heard that Perl culture has changed, but the taint capability is still there because too many Perlers stick to their trusty poor habits. Pythonistas, of course, never learned bad habits. ;-) -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista We must not let the evil of a few trample the freedoms of the many. From simon@netthink.co.uk Mon Jan 21 02:06:53 2002 From: simon@netthink.co.uk (Simon Cozens) Date: Mon, 21 Jan 2002 02:06:53 +0000 Subject: [Python-Dev] Python and Security In-Reply-To: <20020121013900.62CFCE8CD@waltz.rahul.net> References: <15435.19544.899789.631148@anthem.wooz.org> <20020121013900.62CFCE8CD@waltz.rahul.net> Message-ID: <20020121020653.GA5885@netthink.co.uk> On Sun, Jan 20, 2002 at 05:38:59PM -0800, Aahz Maruch wrote: > More precisely, because Perl culture developed as a superset of shell > scripts, it used to be all-too-common for Perl scripts to get their data > by parsing the output of a Unix utility (instead of calling a library > function directly). This necessarily spawned a subshell where malicious > input could be a security problem. Not so. This is what taint is: Taint tells you where there's some shit you want to clean up. If you ask the user for a filename to write to, taint tells you that you'd better check for leading slashes, double dots and the like before writing to it. If you're about to run an external program, taint tells you that you might not want to believe the user's idea of what $PATH ought to be. If you're getting a URL from somewhere, taint tells you that you should probably think twice before happily passing back file:///etc/shadow. And so on and so forth. None of these examples are about input to a subshell. I'm not in a position to say whether or not Python needs taint; if it had it, I probably wouldn't use the feature. But let's not misunderstand what it's for. -- Thermodynamics in a nutshell: 1st Law: You can't win. (Energy is conserved) 2nd Law: You can't break even. (Entropy) 0th Law: You can't even quit the game. (Closed systems) -- Taki Kogoma From paul@prescod.net Mon Jan 21 02:27:59 2002 From: paul@prescod.net (Paul Prescod) Date: Sun, 20 Jan 2002 18:27:59 -0800 Subject: [Python-Dev] When to signal an error References: <200201200002.g0K02Ao03439@mira.informatik.hu-berlin.de> Message-ID: <3C4B7CAF.F6D4909B@prescod.net> "Martin v. Loewis" wrote: > >... > > > Could be rewritten: > > if not hasattr(sys, 'ps1'): > > sys.ps1 = ">>> " > > if not hasattr(sys, 'ps2'): > > sys.ps2 = "... " > > Using string literals when you mean attribute names is bad style. It > just helps to trick the checker. Just for the record, I think that Jason's rewrites were clearer in every case because they said exactly what he was trying to do. "If the sys module has the attribute ps1 then ..." This is much clearer than "Get the ps1 attribute from the sys module and throw it away.". Python has a functions specifically for checking for the existance of attributes and keys. Why not use them? Plus, I think that exceptions should be (as far as possible) reserved for exceptional situations. Using them to as tests is not as compact, not as readable and not as runtime efficient. But more to the point, any of these could have been rewritten as: _junk = sys.ps1 That would shut up compiler messages without forcing you to use the haskey/hasattr style. Paul Prescod From mwh@python.net Mon Jan 21 10:24:54 2002 From: mwh@python.net (Michael Hudson) Date: 21 Jan 2002 10:24:54 +0000 Subject: [Python-Dev] When to signal an error In-Reply-To: Neal Norwitz's message of "Sat, 19 Jan 2002 14:25:18 -0500" References: <200201190338.WAA28421@cj20424-a.reston1.va.home.com> <3C49C81E.C2D2F1BD@metaslash.com> Message-ID: <2mn0z8xaq1.fsf@starship.python.net> Neal Norwitz writes: > Currently, I think there are 2 or 3 warnings which definitely fit this class: > No global found, using ++/--, and expressions with no effect as Jason > described. It would sure be nice if using a variable before assignment produced a warning at compile time. However I think this needs flow analysis and you won't catch me trying to add that to compile.c. Cheers, M. -- MGM will not get your whites whiter or your colors brighter. It will, however, sit there and look spiffy while sucking down a major honking wad of RAM. -- http://www.xiph.org/mgm/ From Samuele Pedroni" Hi. Thanks to http://www.pythonware.com/daily/ I landed in http://norvig.com/python/python.html Peter Norvig is about to supply Python versions of the algorithms with the 2nd edition of his AI: A Modern Approach. So far, so good. In the section about coding convetions he says: =A6In general, follow Guido's style conventions, =A6but I have some quirks that I prefer (although I could be talked out o= f them): ... =A6* _ instead of self as first argument to methods: def f(_, x): ... I'm perfectly aware that the 'self' thing it is just a convetion, OTOH much of the cross-programmer readability of code relies on such convention. It is good, bad or irrelevant to have such an authoritative book (although about AI not Python directly) adopting such a line-noisy convention? Maybe nobody cares, but I preferred not to let this go unnoticed. Someone who cares could try to discuss the issue or make it apparent to Mr. Norvig. Opinions? regards, Samuele Pedroni. From jeremy@alum.mit.edu Sun Jan 20 22:43:59 2002 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Sun, 20 Jan 2002 17:43:59 -0500 Subject: [Python-Dev] When to signal an error In-Reply-To: <3C49C81E.C2D2F1BD@metaslash.com> References: <200201190338.WAA28421@cj20424-a.reston1.va.home.com> <3C49C81E.C2D2F1BD@metaslash.com> Message-ID: <15435.18479.961038.136717@gondolin.digicool.com> >>>>> "NN" == Neal Norwitz writes: NN> Guido van Rossum wrote: >> I believe that evertually some PyChecker-like technology will be >> incorporated in the Python compiler. The same happened to C >> compilers: the lint program became useless once GCC incorporated >> the same technology. NN> pychecker was (and still is) an experiment to me. But I think NN> it would be great if the lessons from pychecker could be NN> integrated into the compiler. Me, too. NN> I'd be happy to help the process of integrating warnings into NN> the compiler, however, I'm not sure how to proceed. Should NN> pychecker be put into the standard library (users can now do: NN> import pychecker.checker and all modules imported are checked by NN> installing an __import__)? Should pychecker be added as a tool? NN> Should a PEP be written? etc. How much of pychecker's work could be done by the compiler itself? I'd like to see more of the warnings generated during compilation, but agree with Michael Hudson that extending it is a lot of work. Perhaps it's time to redesign the compiler. A PEP is probably good for more than one reason. One reason is to document the warnings that are generated and the rationale for them. If you integrate it into the compiler, the PEP is a good place to capture some design info. Jeremy From jeremy@alum.mit.edu Sun Jan 20 22:44:39 2002 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Sun, 20 Jan 2002 17:44:39 -0500 Subject: [Python-Dev] When to signal an error In-Reply-To: <3C49C81E.C2D2F1BD@metaslash.com> References: <200201190338.WAA28421@cj20424-a.reston1.va.home.com> <3C49C81E.C2D2F1BD@metaslash.com> Message-ID: <15435.18519.191874.661917@gondolin.digicool.com> We could talk about this at the conference. Jeremy From guido@python.org Mon Jan 21 16:07:59 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 21 Jan 2002 11:07:59 -0500 Subject: [Python-Dev] OT: style convention: self vs. _ in new Norvig's book In-Reply-To: Your message of "Mon, 21 Jan 2002 14:20:15 +0100." <001201c1a27e$5d8ec060$6d94fea9@newmexico> References: <001201c1a27e$5d8ec060$6d94fea9@newmexico> Message-ID: <200201211607.LAA17031@cj20424-a.reston1.va.home.com> > http://norvig.com/python/python.html > > Peter Norvig is about to supply > Python versions of the algorithms with > the 2nd edition of his AI: A Modern Approach. > > So far, so good. In the section about > coding convetions he says: > > ¦In general, follow Guido's style conventions, > ¦but I have some quirks that I prefer (although I could be talked out of them): > ... > ¦* _ instead of self as first argument to methods: def f(_, x): > ... > > I'm perfectly aware that the 'self' thing it is just a convetion, > OTOH much of the cross-programmer readability > of code relies on such convention. > > It is good, bad or irrelevant to have such > an authoritative book (although about AI not > Python directly) adopting such a line-noisy > convention? > > Maybe nobody cares, but I preferred not to > let this go unnoticed. Someone who cares > could try to discuss the issue or make it > apparent to Mr. Norvig. > > Opinions? > > regards, Samuele Pedroni. Peter: My apologies for butting in here without doing full research. I don't know how you reached this set of conventions, so maybe you've got a very good reason; but I don't see it on your webpage. Two of those coding conventions look really ugly to me: 2-space indents and _ for self. I think the code will look horrible! I think everyone should be able to make their own style choices, but I ask you to reconsider. If you have to reconsider one, I would beg you to use 'self' like everybody else. The _ name is already overloaded with multiple meanings in the Python community: it's a shorthand for the last evaluated expression in interactive mode, and some people use it as a dummy variable to assign uninteresting results to. Almost the entire Python community is happy with 4-space indents; if you're worried about your lines getting too long, that's usually a hint that your code can be restructured in a way that's easier on the reader's eye/mind anyway. --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Mon Jan 21 16:10:04 2002 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 21 Jan 2002 11:10:04 -0500 Subject: [Python-Dev] OT: style convention: self vs. _ in new Norvig's book References: <001201c1a27e$5d8ec060$6d94fea9@newmexico> <200201211607.LAA17031@cj20424-a.reston1.va.home.com> Message-ID: <15436.15708.321312.724003@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> The _ name is already overloaded with multiple meanings in GvR> the Python community: it's a shorthand for the last evaluated GvR> expression in interactive mode, and some people use it as a GvR> dummy variable to assign uninteresting results to. It's also the common name of a function in internationalized Python applications (mostly inherited from established conventions in the C world). -Barry From pnorvig@google.com Mon Jan 21 18:16:51 2002 From: pnorvig@google.com (Peter Norvig) Date: Mon, 21 Jan 2002 10:16:51 -0800 Subject: [Python-Dev] OT: style convention: self vs. _ in new Norvig's book References: <001201c1a27e$5d8ec060$6d94fea9@newmexico> <200201211607.LAA17031@cj20424-a.reston1.va.home.com> Message-ID: <3C4C5B13.FA2909A6@google.com> Wow; I didn't expect this to generate such a response. But I did post the code far before it was ready and put the "I could be talked out of it" there for a reason. So, thank you for your feedback! My reactions: 4 spaces: OK.=20 I have no strong feelings on that, and I think its just an accident of the way my emacs was configured that I started using 2 spaces. I agree that I should make it easier for other people to edit my code, so I'll switch to the default. self: OK, I'll try it.=20 My rationale was: I'm used to Java, where self is usually spelled '', and I figured '_' was the next best thing. I find it much nicer to read because 'self' is too intrusive; I want something that disappears.=20 Compare: _.x, _.y, _.z =3D x, y, z self.x, self.y, self.z =3D x, y, z Besides saving 9 characters, I find that the first line I can read at a glance, ignoring the _, while the second I have to look at more carefully. I also like the symmetry of _._ in _._private_slot. However, I recognize I'm doing this as an outsider to the language without much experience reading/writing it. If it is really true that using '_' would be seen as a change to the language and not a personal quirk, then I agree that I shouldn't do it. The first hint I had of this was when I saw something on comp.lang.python (I forget the details) suggesting that an automated tool look for methods with first argument 'self'. So I'll try 'self' for a while, and hope I learn to like it (and learn to read the second sample line above in one glance). If I don't, I'll write here and give you all another chance to innundate me with reasons why I should. -Peter PS - Getting a personal request from Guido reminds me of the time I was at a conference and John McCarthy walked up to the booth of one of the Lisp vendors and said in his usual direct fashion "I hear you have a new version. You should send me one". The booth bimbo had no idea who McCarthy was and politely suggested he pay for a copy. Then someone in the booth with a little more experience came over and said "That's ok -- it's his language, he can have whatever he wants." Guido van Rossum wrote: >=20 > > http://norvig.com/python/python.html > > > > Peter Norvig is about to supply > > Python versions of the algorithms with > > the 2nd edition of his AI: A Modern Approach. > > > > So far, so good. In the section about > > coding convetions he says: > > > > =A6In general, follow Guido's style conventions, > > =A6but I have some quirks that I prefer (although I could be talked o= ut of them): > > ... > > =A6* _ instead of self as first argument to methods: def f(_, x): > > ... > > > > I'm perfectly aware that the 'self' thing it is just a convetion, > > OTOH much of the cross-programmer readability > > of code relies on such convention. > > > > It is good, bad or irrelevant to have such > > an authoritative book (although about AI not > > Python directly) adopting such a line-noisy > > convention? > > > > Maybe nobody cares, but I preferred not to > > let this go unnoticed. Someone who cares > > could try to discuss the issue or make it > > apparent to Mr. Norvig. > > > > Opinions? > > > > regards, Samuele Pedroni. >=20 > Peter: >=20 > My apologies for butting in here without doing full research. I don't > know how you reached this set of conventions, so maybe you've got a > very good reason; but I don't see it on your webpage. >=20 > Two of those coding conventions look really ugly to me: 2-space > indents and _ for self. I think the code will look horrible! >=20 > I think everyone should be able to make their own style choices, but I > ask you to reconsider. If you have to reconsider one, I would beg you > to use 'self' like everybody else. The _ name is already overloaded > with multiple meanings in the Python community: it's a shorthand for > the last evaluated expression in interactive mode, and some people use > it as a dummy variable to assign uninteresting results to. >=20 > Almost the entire Python community is happy with 4-space indents; if > you're worried about your lines getting too long, that's usually a > hint that your code can be restructured in a way that's easier on the > reader's eye/mind anyway. >=20 > --Guido van Rossum (home page: http://www.python.org/~guido/) --=20 _____________________________________________________________________ Peter Norvig, Director of Machine Learning, Google, http://google.com pnorvig@google.com, Voice:650-330-0100 x1248, Fax:650-618-1499 From guido@python.org Mon Jan 21 19:02:49 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 21 Jan 2002 14:02:49 -0500 Subject: [Python-Dev] OT: style convention: self vs. _ in new Norvig's book In-Reply-To: Your message of "Mon, 21 Jan 2002 10:16:51 PST." <3C4C5B13.FA2909A6@google.com> References: <001201c1a27e$5d8ec060$6d94fea9@newmexico> <200201211607.LAA17031@cj20424-a.reston1.va.home.com> <3C4C5B13.FA2909A6@google.com> Message-ID: <200201211902.OAA22868@cj20424-a.reston1.va.home.com> > Wow; I didn't expect this to generate such a response. But I did post > the code far before it was ready and put the "I could be talked out of > it" there for a reason. So, thank you for your feedback! My reactions: You're welcome. I'm always there to save a straying stranger. :-) [snip] > self: OK, I'll try it. > > My rationale was: I'm used to Java, where self is usually spelled '', > and I figured '_' was the next best thing. I find it much nicer to read > because 'self' is too intrusive; I want something that disappears. I hear that in the Lisp world, when someone complains about the parentheses, the standard response is "once you're used to it, the parentheses disappear". So it is for Python's 'self'. :-) > Compare: > > _.x, _.y, _.z = x, y, z > self.x, self.y, self.z = x, y, z > > Besides saving 9 characters, I find that the first line I can read at a > glance, ignoring the _, while the second I have to look at more > carefully. I also like the symmetry of _._ in _._private_slot. However, > I recognize I'm doing this as an outsider to the language without much > experience reading/writing it. If it is really true that using '_' would > be seen as a change to the language and not a personal quirk, then I > agree that I shouldn't do it. The first hint I had of this was when I > saw something on comp.lang.python (I forget the details) suggesting that > an automated tool look for methods with first argument 'self'. So I'll > try 'self' for a while, and hope I learn to like it (and learn to read > the second sample line above in one glance). If I don't, I'll write > here and give you all another chance to innundate me with reasons why I > should. Thanks! > -Peter > > PS - Getting a personal request from Guido reminds me of the time I was > at a conference and John McCarthy walked up to the booth of one of the > Lisp vendors and said in his usual direct fashion "I hear you have a new > version. You should send me one". The booth bimbo had no idea who > McCarthy was and politely suggested he pay for a copy. Then someone in > the booth with a little more experience came over and said "That's ok -- > it's his language, he can have whatever he wants." What's a booth bimbo? :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From jason@jorendorff.com Mon Jan 21 20:47:59 2002 From: jason@jorendorff.com (Jason Orendorff) Date: Mon, 21 Jan 2002 14:47:59 -0600 Subject: [Python-Dev] OT: style convention: self vs. _ in new Norvig's book In-Reply-To: <001201c1a27e$5d8ec060$6d94fea9@newmexico> Message-ID: > ] In general, follow Guido's style conventions, > ] but I have some quirks that I prefer (although I could be talked > ] out of them): > ... > ] * _ instead of self as first argument to methods: def f(_, x): > ... I dunno; I think sample code should (a) stick rather conservatively to typical usage, apart from the concept being illustrated of course; and (b) strive for maximum readability. For Python, both principles demand that one should write: def foo(bar): if is_list(bar): return sum(map(foo, bar)) else: return [bar] instead of: def foo(bar): if is_list(bar): return sum(map(foo, bar)) else: return [bar] This may be one of those things that only makes sense if you've not a Lisp programmer. (wink) To stray from the topic: I find I only disagree with three points in Peter Norvig's enlightening table of Lisp vs. Python features. 1. That "x.slot = y" is not user-extensible. The __setattr__() method does this. 2. That Python's relative lack of control structures is necessarily worse than Lisp's abundance of them. Especially for students, I think this: if is_list(n): return foo_l(n) elif is_str(n) or is_int(n): return foo_a(n) else: raise TypeError is at least as clear, though not as brief, as this: (etypecase n (list (foo-l n)) ((or string integer) (foo-a n))) with the obligatory note in the text to the effect that "'Etypecase' is a form similar to 'case' which selects a clause based on the type..." and so on. 3. That Python doesn't support generic programming. Generic algorithms are expressed as naturally in Python as in any language I know: from operator import add def sum(items): return reduce(add, items) >>> sum([3, 4, 5]) 12 >>> sum([3, 4j, 4-2j]) (7+2j) >>> sum(["py", "th", "o", "n"]) 'python' Likewise it's natural to write functions that can operate on "any sequence", not just lists or tuples, "any file-like object", not just a real file, "any function-like object", etc. Perhaps something more specific is meant by "generic programming". Cheers, ## Jason Orendorff http://www.jorendorff.com/ From martin@v.loewis.de Mon Jan 21 21:57:40 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: Mon, 21 Jan 2002 22:57:40 +0100 Subject: [Python-Dev] OT: style convention: self vs. _ in new Norvig's book In-Reply-To: <001201c1a27e$5d8ec060$6d94fea9@newmexico> (pedronis@bluewin.ch) References: <001201c1a27e$5d8ec060$6d94fea9@newmexico> Message-ID: <200201212157.g0LLve701480@mira.informatik.hu-berlin.de> > Opinions? I dislike it, because _ is already taken for two things: for the last expression in interactive mode, and as a markup of translatable strings. Regards, Martin From pnorvig@google.com Mon Jan 21 23:22:32 2002 From: pnorvig@google.com (Peter Norvig) Date: Mon, 21 Jan 2002 15:22:32 -0800 Subject: [Python-Dev] OT: style convention: self vs. _ in new Norvig's book References: <001201c1a27e$5d8ec060$6d94fea9@newmexico> <200201211607.LAA17031@cj20424-a.reston1.va.home.com> <3C4C5B13.FA2909A6@google.com> <200201211902.OAA22868@cj20424-a.reston1.va.home.com> Message-ID: <3C4CA2B8.4951AB78@google.com> Guido van Rossum wrote: > I hear that in the Lisp world, when someone complains about the > parentheses, the standard response is "once you're used to it, the > parentheses disappear". So it is for Python's 'self'. :-) That may be a good analogy, and as I said, I'm willing to try. But I still think one character is easier to ignore than four, and that there is no compelling argument for 'self' over '_', while there is a positive reason for parens (ease of automated parsing tools). > What's a booth bimbo? :-) "It's not a sexist phenomenon as such, applying equally to the pretty young men and women who work as scenery at various booths. Universally, these people have no clue about the products they represent; instead they hand out buttons and propaganda, smile nicely, and act as props for the larger show that goes on around them." -- http://www.tidbits.com/tb-issues/TidBITS-159.html#lnk6 > > --Guido van Rossum (home page: http://www.python.org/~guido/) From jason@jorendorff.com Tue Jan 22 00:06:51 2002 From: jason@jorendorff.com (Jason Orendorff) Date: Mon, 21 Jan 2002 18:06:51 -0600 Subject: [Python-Dev] OT: style convention: self vs. _ in new Norvig's book In-Reply-To: <3C4CA2B8.4951AB78@google.com> Message-ID: Peter Norvig wrote: > Guido van Rossum wrote: > > I hear that in the Lisp world, when someone complains about the > > parentheses, the standard response is "once you're used to it, the > > parentheses disappear". So it is for Python's 'self'. :-) > > That may be a good analogy, and as I said, I'm willing to try. It's an excellent analogy: both statements are about 1/3 true in my experience. :-) > But I still think one character is easier to ignore than four, > and that there is no compelling argument for 'self' over '_', > while there is a positive reason for parens (ease of automated > parsing tools). There is no especially compelling reason for Python to have 'self' over '_' or 'me' or '@' or ''. However, there is a compelling reason for you to choose 'self': "Prefer the standard to the offbeat." --Strunk and White ## Jason Orendorff http://www.jorendorff.com/ From Anthony Baxter Tue Jan 22 00:14:37 2002 From: Anthony Baxter (Anthony Baxter) Date: Tue, 22 Jan 2002 11:14:37 +1100 Subject: [Python-Dev] OT: style convention: self vs. _ in new Norvig's book In-Reply-To: Message from Peter Norvig of "Mon, 21 Jan 2002 15:22:32 -0800." <3C4CA2B8.4951AB78@google.com> Message-ID: <200201220014.g0M0EbW28603@mbuna.arbhome.com.au> >>> Peter Norvig wrote > That may be a good analogy, and as I said, I'm willing to try. But I > still think one character is easier to ignore than four, and that there > is no compelling argument for 'self' over '_', while there is a positive > reason for parens (ease of automated parsing tools). The primary arguments against '_' are that it already has meaning. I can think of three, off the top of my head. Interactive mode uses this as "result of last expression". The i18n code uses it as a function _('translate me'). Zope uses it in DTML (python) expressions as the default namespace. I'd also add the subjective argument that it's ugly, and looks far too magical and perl-like. I don't _want_ it to disappear into the background, as it's going to cause me pain if I miss it. Anthony From pnorvig@google.com Tue Jan 22 00:27:04 2002 From: pnorvig@google.com (Peter Norvig) Date: Mon, 21 Jan 2002 16:27:04 -0800 Subject: [Python-Dev] OT: style convention: self vs. _ in new Norvig's book References: Message-ID: <3C4CB1D8.B050B002@google.com> OK, OK; When both Guido and E. B. team up against me, I know I'm licked. -Peter Jason Orendorff wrote: > There is no especially compelling reason for Python to have > 'self' over '_' or 'me' or '@' or ''. > > However, there is a compelling reason for you to choose 'self': > "Prefer the standard to the offbeat." --Strunk and White From montanaro@tttech.com Tue Jan 22 14:35:15 2002 From: montanaro@tttech.com (montanaro@tttech.com) Date: Tue, 22 Jan 2002 08:35:15 -0600 Subject: [Python-Dev] Bug? is Tkinter+no threads+Windows supported? Message-ID: <15437.30883.138962.301012@dynamic2.tttech1.ttt> My client is trying to build a version of Python on Windows with Tkinter and pymalloc enabled, and threads disabled (in part because pymalloc is not thread-safe). There appears to be a bug in _tkinter.c:EventHook. It has this code: #if defined(WITH_THREAD) || defined(MS_WINDOWS) Py_BEGIN_ALLOW_THREADS PyThread_acquire_lock(tcl_lock, 1); tcl_tstate = event_tstate; result = Tcl_DoOneEvent(TCL_DONT_WAIT); tcl_tstate = NULL; PyThread_release_lock(tcl_lock); if (result == 0) Sleep(20); Py_END_ALLOW_THREADS #else result = Tcl_DoOneEvent(0); #endif It seems on the surface that the "|| defined(MS_WINDOWS)" bit should be deleted. This code dates from 1998 and comes with this log text: revision 1.72 date: 1998/06/13 13:56:28; author: guido; state: Exp; lines: +26 -6 Fixed the EventHook() code so that it also works on Windows, sort of. (The "sort of" is because it uses kbhit() to detect that the user starts typing, and then no events are processed until they hit return.) Also fixed a nasty locking bug: EventHook() is called without the Tcl lock set, so it can't use the ENTER_PYTHON and LEAVE_PYTHON macros, which manipulate both the Python and the Tcl lock. I now only acquire and release the Python lock. (Haven't tested this on Unix yet...) This suggests that Guido was (rightly) worried about the case of threading on Windows. What about a non-threaded interpreter on Windows? Skip From nas@python.ca Tue Jan 22 15:02:12 2002 From: nas@python.ca (Neil Schemenauer) Date: Tue, 22 Jan 2002 07:02:12 -0800 Subject: [Python-Dev] Bug? is Tkinter+no threads+Windows supported? In-Reply-To: <15437.30883.138962.301012@dynamic2.tttech1.ttt>; from montanaro@tttech.com on Tue, Jan 22, 2002 at 08:35:15AM -0600 References: <15437.30883.138962.301012@dynamic2.tttech1.ttt> Message-ID: <20020122070212.B10448@glacier.arctrix.com> montanaro@tttech.com wrote: > My client is trying to build a version of Python on Windows with Tkinter and > pymalloc enabled, and threads disabled (in part because pymalloc is not > thread-safe). Using pymalloc with threads should be safe as long as you don't have extensions that call pymalloc without the big lock held. Neil From montanaro@tttech.com Wed Jan 23 08:24:27 2002 From: montanaro@tttech.com (montanaro@tttech.com) Date: Wed, 23 Jan 2002 02:24:27 -0600 Subject: [Python-Dev] "This document is locked" message from Sourceforge? Message-ID: <15438.29499.711766.816065@dynamic2.tttech1.ttt> I just logged into Sourceforge. Now every time I visit a page, although that page displays, I also get username/password popup saying the document is locked and giving a server message of "foo". Any idea where this came from? Perhaps a test on SF they forgot to undo before putting some pages into production? Skip From mwh@python.net Wed Jan 23 10:57:23 2002 From: mwh@python.net (Michael Hudson) Date: 23 Jan 2002 10:57:23 +0000 Subject: [Python-Dev] "This document is locked" message from Sourceforge? In-Reply-To: montanaro@tttech.com's message of "Wed, 23 Jan 2002 02:24:27 -0600" References: <15438.29499.711766.816065@dynamic2.tttech1.ttt> Message-ID: <2mbsfl2v3g.fsf@starship.python.net> montanaro@tttech.com writes: > I just logged into Sourceforge. Now every time I visit a page, although > that page displays, I also get username/password popup saying the document > is locked and giving a server message of "foo". Any idea where this came > from? Perhaps a test on SF they forgot to undo before putting some pages > into production? Haven't noticed that, but sf is being nice and snappy this morning, isn't it? It seems to take five minutes for a bug report to finish displaying. Argh! Cheers, M. -- $ head -n 2 src/bash/bash-2.04/unwind_prot.c /* I can't stand it anymore! Please can't we just write the whole Unix system in lisp or something? */ -- spotted by Rich van der Hoff From sdm7g@Virginia.EDU Wed Jan 23 16:26:09 2002 From: sdm7g@Virginia.EDU (Steven Majewski) Date: Wed, 23 Jan 2002 11:26:09 -0500 (EST) Subject: [Python-Dev] VERBOSE and DEBUG conventions. Message-ID: Py_DebugFlag is used for debugging the Python parser. Py_VerboseFlag is used for debugging and tracing imports. (and in some places it wants Py_VerboseFlag > 1 (more than one "-v") for output) Are there any conventions on which to use for other debugging output? (Or did Guido have any particular conventions in mind when he added them? ) Right now, I'm using Py_VerboseFlag to also trigger logging of message sends in pyobjc. Stealing this flag for another use isn't a problem here because [1] the logging goes to a /tmp file, so I don't have to turn off import tracing -- the two logging streams don't get mixed together, and [2] it only functions when you import pyobjc, so it's not going to get in someone else's use. But I may need to add other debug and log output to my module and I'ld like to do it in the least suprising manner if possible. -- Steve Majewski From tim.one@home.com Wed Jan 23 17:51:58 2002 From: tim.one@home.com (Tim Peters) Date: Wed, 23 Jan 2002 12:51:58 -0500 Subject: [Python-Dev] VERBOSE and DEBUG conventions. In-Reply-To: Message-ID: [Steven Majewski] > Py_DebugFlag is used for debugging the Python parser. If Guido had it to do over again, I suspect he'd put that code in #ifdef Py_DEBUG blocks instead. > Py_VerboseFlag is used for debugging and tracing imports. > (and in some places it wants Py_VerboseFlag > 1 (more than one "-v") > for output) > > Are there any conventions on which to use for other debugging output? Py_VerboseFlag is for output about core activities every user of Python may want to see sometimes, and in release builds. It doesn't cover much beyond tracing imports, printing stats about memory cleanup, and some highly dubious fudging: PyThreadState_Clear(PyThreadState *tstate) { if (Py_VerboseFlag && tstate->frame != NULL) fprintf(stderr, "PyThreadState_Clear: warning: thread still has a frame\n"); (that should probably be an error instead -- or be officially blessed). > (Or did Guido have any particular conventions in mind when he added > them? ) > > Right now, I'm using Py_VerboseFlag to also trigger logging of message > sends in pyobjc. Stealing this flag for another use isn't a problem > here because [1] the logging goes to a /tmp file, so I don't have > to turn off import tracing -- the two logging streams don't get mixed > together, and [2] it only functions when you import pyobjc, so it's > not going to get in someone else's use. > > But I may need to add other debug and log output to my module and > I'ld like to do it in the least suprising manner if possible. Supply a "set debug and log options" interface for your module, and then call it . Good example: the gc module. From guido@python.org Wed Jan 23 18:01:14 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 23 Jan 2002 13:01:14 -0500 Subject: [Python-Dev] VERBOSE and DEBUG conventions. In-Reply-To: Your message of "Wed, 23 Jan 2002 12:51:58 EST." References: Message-ID: <200201231801.NAA17967@pcp742651pcs.reston01.va.comcast.net> > [Steven Majewski] > > Py_DebugFlag is used for debugging the Python parser. > > If Guido had it to do over again, I suspect he'd put that code in #ifdef > Py_DEBUG blocks instead. Yes and no. Some of it *is* already only inside #ifdef Py_DEBUG (see parser.c); but it still requires a command line flag because the output is too much to bear in a regular debugging run... --Guido van Rossum (home page: http://www.python.org/~guido/) From sdm7g@Virginia.EDU Wed Jan 23 18:06:30 2002 From: sdm7g@Virginia.EDU (Steven Majewski) Date: Wed, 23 Jan 2002 13:06:30 -0500 (EST) Subject: [Python-Dev] VERBOSE and DEBUG conventions. In-Reply-To: Message-ID: On Wed, 23 Jan 2002, Tim Peters wrote: > Supply a "set debug and log options" interface for your module, and then > call it . Good example: the gc module. Thanks. That mostly makes sense. Except that I needed it to be in trace/debug mode when the module initialization is being done, so I can't import the module and then set it. I suppose I could just use another environment variable: $PYOBJC_DEBUG -- then I could set debug levels. -- Steve FYI: In case you're wondering why I don't just use gdb: It's seems to be a meta level problem between the python runtime and the objective-c runtime, and I suspect the objc extensions in gdb must make use of the objc-runtime ( for 'po' - print object, for example.) because I seem to be causing another objc runtime exception the act of examining things in the debugger. This is not very documented in the gdb manual, so unless I'm going to wade thru the sources, I though it would be easier just to instrument the module. (and maybe Python.) From gmcm@hypernet.com Wed Jan 23 18:26:02 2002 From: gmcm@hypernet.com (Gordon McMillan) Date: Wed, 23 Jan 2002 13:26:02 -0500 Subject: [Python-Dev] VERBOSE and DEBUG conventions. In-Reply-To: References: Message-ID: <3C4EB9EA.25681.2F95B9E4@localhost> On 23 Jan 2002 at 13:06, Steven Majewski wrote: > FYI: In case you're wondering why I don't just use gdb: > It's seems to be a meta level problem between the python > runtime and the objective-c runtime, and I suspect the > objc extensions in gdb must make use of the objc-runtime ( > for 'po' - print object, for example.) because I seem to be > causing another objc runtime exception the act of examining > things in the debugger. Or perhaps chip geometries are getting small enough that simply the act of observing is enough. running-on-stale-Doritos-ly y'rs -- Gordon http://www.mcmillan-inc.com/ From sdm7g@Virginia.EDU Wed Jan 23 18:57:08 2002 From: sdm7g@Virginia.EDU (Steven Majewski) Date: Wed, 23 Jan 2002 13:57:08 -0500 (EST) Subject: [Python-Dev] VERBOSE and DEBUG conventions. In-Reply-To: <3C4EB9EA.25681.2F95B9E4@localhost> Message-ID: On Wed, 23 Jan 2002, Gordon McMillan wrote: > Or perhaps chip geometries are getting small enough > that simply the act of observing is enough. Well: the effect is being magnified by Class Object self reference, which I could probably avoid if objective-C had actual metaclasses. > running-on-stale-Doritos-ly y'rs Gosh. I'm impressed. I'm still running on coffee and donuts here. I usually don't start on stale Doritos until much later in the day! ( Unless I've wrapped around on an all nighter and I'm still on last night's Doritos. ) From DavidA@ActiveState.com Wed Jan 23 23:02:32 2002 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 23 Jan 2002 15:02:32 -0800 Subject: [Python-Dev] largeint.h and ver.h gone from VS.NET Message-ID: <3C4F4108.48D9AEF3@activestate.com> As mentioned in: http://groups.google.com/groups?q=largeint.h&hl=en&selm=epfeXXOYBHA.2060%40tkmsftngp07&rnum=1 largeint.h is gone from the VisualStudio compiler as of the VisualStudio.NET release. Python's build currently fails without the workaround mentioned in that posting. Furthermore, the file "ver.h" used in python_nt.rc appears to be gone as well. Not sure why we needed it. Gettinr dir fo it seems to have no ill effect =). Anyone remember what it's for? I'm having sre problems in the test suite though, which have pretty wide-ranging effects. Is someone else looking at the patches needed for VS.NET, or should I keep digging? --david From DavidA@ActiveState.com Wed Jan 23 23:06:50 2002 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 23 Jan 2002 15:06:50 -0800 Subject: [Python-Dev] largeint.h and ver.h gone from VS.NET References: <3C4F4108.48D9AEF3@activestate.com> Message-ID: <3C4F420A.7F26C7F5@activestate.com> Whoa. test_longexp seems to be causing the python_d process to bloat to almost 80Megs. This is with the VS.NET build. I guess I really have to get a VC6 build going now =). From Jack.Jansen@oratrix.nl Wed Jan 23 23:07:49 2002 From: Jack.Jansen@oratrix.nl (Jack Jansen) Date: Thu, 24 Jan 2002 00:07:49 +0100 Subject: [Python-Dev] PEP 278 - Universal newline support Message-ID: <0537F905-1056-11D6-B9C4-003065517236@oratrix.nl> Folks, there's a new PEP 278 plus an accompanying patch available on the subject of universal newline support (the ability to read and import files that use a different newline convention than what the current platform uses). Please read, apply, try, provide feedback and put me back to work:-) -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From guido@python.org Wed Jan 23 23:14:09 2002 From: guido@python.org (Guido van Rossum) Date: Wed, 23 Jan 2002 18:14:09 -0500 Subject: [Python-Dev] largeint.h and ver.h gone from VS.NET In-Reply-To: Your message of "Wed, 23 Jan 2002 15:06:50 PST." <3C4F420A.7F26C7F5@activestate.com> References: <3C4F4108.48D9AEF3@activestate.com> <3C4F420A.7F26C7F5@activestate.com> Message-ID: <200201232314.SAA19582@pcp742651pcs.reston01.va.comcast.net> > test_longexp seems to be causing the python_d process to bloat to almost > 80Megs. This is with the VS.NET build. I think that's expected -- test_longexp is very memory intensive, we've seen complaints about this on feeble platforms before. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@home.com Wed Jan 23 23:48:47 2002 From: tim.one@home.com (Tim Peters) Date: Wed, 23 Jan 2002 18:48:47 -0500 Subject: [Python-Dev] largeint.h and ver.h gone from VS.NET In-Reply-To: <3C4F4108.48D9AEF3@activestate.com> Message-ID: [David Ascher] > As mentioned in: > > http://groups.google.com/groups?q=largeint.h&hl=en&selm=epfeXXOYBH > A.2060%40tkmsftngp07&rnum=1 > > largeint.h is gone from the VisualStudio compiler as of the > VisualStudio.NET release. > > Python's build currently fails without the workaround mentioned in that > posting. Did they also, e.g., change the signature of QueryPerformanceCounter(), so that largeint.h isn't needed to get at the MS-specific LARGE_INTEGER typedef? Note that the workaround doesn't work unless these files are on MS's list of redistributable files (which always takes me an hour to find, and no time for that now). > Furthermore, the file "ver.h" used in python_nt.rc appears to be gone as > well. Not sure why we needed it. Gettinr dir fo it seems to have no > ill effect =). Anyone remember what it's for? Mark Hammond created all the code in question (here and above), so ActiveState should know who to hire to maintain it . Here's ver.h in its entirety (as of VC6): #ifndef RC_INVOKED #pragma message ("VER.H obsolete, including WINVER.H instead") #endif #include gettinr-dir-fo-it-indeed-ly y'rs - tim From DavidA@ActiveState.com Thu Jan 24 00:45:03 2002 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 23 Jan 2002 16:45:03 -0800 Subject: [Python-Dev] largeint.h and ver.h gone from VS.NET References: Message-ID: <3C4F590E.B4EE9DC@activestate.com> > Did they also, e.g., change the signature of QueryPerformanceCounter(), so > that largeint.h isn't needed to get at the MS-specific LARGE_INTEGER > typedef? Note that the workaround doesn't work unless these files are on > MS's list of redistributable files (which always takes me an hour to find, > and no time for that now). I did not intend that the workaround would be the right way to do it long term. LARGE_INTEGER is now defined in winnt.h, which is included by windows.h. However, the current code does need more than just the typedef, such as LargeIntegerEqualToZero, LargeIntegerSubtract, etc. > > Furthermore, the file "ver.h" used in python_nt.rc appears to be gone as > > well. Not sure why we needed it. Gettinr dir fo it seems to have no > > ill effect =). Anyone remember what it's for? > > Mark Hammond created all the code in question (here and above), so > ActiveState should know who to hire to maintain it . Sigh. I'm not doing this on behalf of ActiveState -- there's no real need for us to move to VS.NET for most of our builds right now. I'm just playing with my new toy. --david-should-have-posted-from-my-hotmail-account?-ascher From tim.one@home.com Thu Jan 24 01:02:53 2002 From: tim.one@home.com (Tim Peters) Date: Wed, 23 Jan 2002 20:02:53 -0500 Subject: [Python-Dev] largeint.h and ver.h gone from VS.NET In-Reply-To: <3C4F590E.B4EE9DC@activestate.com> Message-ID: [David Ascher] > I did not intend that the workaround would be the right way to do it > long term. > > LARGE_INTEGER is now defined in winnt.h, which is included by > windows.h. However, the current code does need more than just the > typedef, such as LargeIntegerEqualToZero, LargeIntegerSubtract, etc. Those can be replaced with "== 0" and "-" etc -- the obvious things, at least under VC6. Don't know about .NET. >> Mark Hammond created all the code in question (here and above), so >> ActiveState should know who to hire to maintain it . > Sigh. I'm not doing this on behalf of ActiveState Neither am I . > -- there's no real need for us to move to VS.NET for most of our builds > right now. I'm just playing with my new toy. Well, then *you* know who to hire -- same thing. BTW, the #include of ver.h is gone in current CVS now. Mucking with LARGE_INTEGER awaits a volunteer. From fredrik@pythonware.com Thu Jan 24 09:10:05 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 24 Jan 2002 10:10:05 +0100 Subject: [Python-Dev] largeint.h and ver.h gone from VS.NET References: <3C4F4108.48D9AEF3@activestate.com> Message-ID: <00db01c1a4b6$eb4008d0$0900a8c0@spiff> david wrote: > I'm having sre problems in the test suite though, which have pretty > wide-ranging effects. SRE uses agressive inlining under MSVC. maybe their new optimizer is slightly broken? (not the first time, in a X.0 release) as a temporary workaround, try changing #if defined(_MSC_VER) to #if 0 && defined(_MSC_VER) if SRE works after this change, try switching on USE_INLINE. if you find a combination that works, change the MSC_VER clause to: #if defined(_MSC_VER) && _MSC_VER >= SOMETHING ... vs.net configuration #elif defined(_MSC_VER) ... msvc 5/6 configuration #elif defined(USE_INLINE) ... and mail me the patch. cheers /F From guido@python.org Thu Jan 24 15:07:34 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 24 Jan 2002 10:07:34 -0500 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Mac/scripts gensuitemodule.py,1.18,1.19 In-Reply-To: Your message of "Thu, 24 Jan 2002 09:34:40 EST." <15440.7040.836670.604742@grendel.zope.com> References: <15440.7040.836670.604742@grendel.zope.com> Message-ID: <200201241507.KAA11788@pcp742651pcs.reston01.va.comcast.net> > The keyword module has an undocumented data object kwlist which is a > list of keywords. Perhaps this should be documented and made part of > the public API? I'd want to change the list to a tuple, but that > seems harmless since it isn't already part of the API. Why make it a tuple? Out of fear someone changes it? Let them change it, and learn about sharing of object references! Agree it should be documented of course. --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Thu Jan 24 16:07:41 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 24 Jan 2002 11:07:41 -0500 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Mac/scripts gensuitemodule.py,1.18,1.19 In-Reply-To: <200201241507.KAA11788@pcp742651pcs.reston01.va.comcast.net> References: <15440.7040.836670.604742@grendel.zope.com> <200201241507.KAA11788@pcp742651pcs.reston01.va.comcast.net> Message-ID: <15440.12621.645554.228182@grendel.zope.com> Guido van Rossum writes: > Why make it a tuple? Out of fear someone changes it? Let them change > it, and learn about sharing of object references! Partly, and partly because it's something that should be changed anyway. Do you seriously object to changing it to a tuple??? > Agree it should be documented of course. OK. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From guido@python.org Thu Jan 24 16:11:23 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 24 Jan 2002 11:11:23 -0500 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Mac/scripts gensuitemodule.py,1.18,1.19 In-Reply-To: Your message of "Thu, 24 Jan 2002 11:07:41 EST." <15440.12621.645554.228182@grendel.zope.com> References: <15440.7040.836670.604742@grendel.zope.com> <200201241507.KAA11788@pcp742651pcs.reston01.va.comcast.net> <15440.12621.645554.228182@grendel.zope.com> Message-ID: <200201241611.LAA15046@pcp742651pcs.reston01.va.comcast.net> > Do you seriously object to changing it to a tuple??? Yes, I don't want to create any more show code examples that use tuples for (conceptually) arbitrary-length arrays of homogeneous data. The data type to use for those is lists. --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@rahul.net Thu Jan 24 16:56:38 2002 From: aahz@rahul.net (Aahz Maruch) Date: Thu, 24 Jan 2002 08:56:38 -0800 (PST) Subject: [Python-Dev] Tuples vs. lists In-Reply-To: <200201241611.LAA15046@pcp742651pcs.reston01.va.comcast.net> from "Guido van Rossum" at Jan 24, 2002 11:11:23 AM Message-ID: <20020124165638.DE31AE8C4@waltz.rahul.net> Guido van Rossum wrote: >Fred: >> >> Do you seriously object to changing it to a tuple??? > > Yes, I don't want to create any more show code examples that use > tuples for (conceptually) arbitrary-length arrays of homogeneous > data. The data type to use for those is lists. Hrm. Even when it's something that's supposed to be immutable? I'm asking because I'm currently using a tuple for the digit list in my BCD module, and I'd like a clearer explanation of why you think that it should be a list (assuming you do). >From my viewpoint, the BCD digit string should be handled like a string; I'm only using a tuple for efficiency of storing numbers instead of characters. -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista We must not let the evil of a few trample the freedoms of the many. From guido@python.org Thu Jan 24 17:01:51 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 24 Jan 2002 12:01:51 -0500 Subject: [Python-Dev] Tuples vs. lists In-Reply-To: Your message of "Thu, 24 Jan 2002 08:56:38 PST." <20020124165638.DE31AE8C4@waltz.rahul.net> References: <20020124165638.DE31AE8C4@waltz.rahul.net> Message-ID: <200201241701.MAA15403@pcp742651pcs.reston01.va.comcast.net> > Hrm. Even when it's something that's supposed to be immutable? I'm > asking because I'm currently using a tuple for the digit list in my BCD > module, and I'd like a clearer explanation of why you think that it > should be a list (assuming you do). > > From my viewpoint, the BCD digit string should be handled like a string; > I'm only using a tuple for efficiency of storing numbers instead of > characters. Can't you trust your users not to change it? --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@rahul.net Thu Jan 24 18:03:35 2002 From: aahz@rahul.net (Aahz Maruch) Date: Thu, 24 Jan 2002 10:03:35 -0800 (PST) Subject: [Python-Dev] Tuples vs. lists In-Reply-To: <200201241701.MAA15403@pcp742651pcs.reston01.va.comcast.net> from "Guido van Rossum" at Jan 24, 2002 12:01:51 PM Message-ID: <20020124180336.4FBE1E8C1@waltz.rahul.net> Guido van Rossum wrote: > Aahz: >> >> Hrm. Even when it's something that's supposed to be immutable? I'm >> asking because I'm currently using a tuple for the digit list in my BCD >> module, and I'd like a clearer explanation of why you think that it >> should be a list (assuming you do). >> >> From my viewpoint, the BCD digit string should be handled like a string; >> I'm only using a tuple for efficiency of storing numbers instead of >> characters. > > Can't you trust your users not to change it? Sure, but then I can't just copy references to the tuple when creating a copy of an instance, I'd have to copy the entire list. That's what I meant by efficiency. There are important semantic differences coming from the fact that tuples are immutable and lists are mutable, and I think that a strict heterogeneous/homogenous distinction loses that. -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista We must not let the evil of a few trample the freedoms of the many. From guido@python.org Thu Jan 24 18:07:24 2002 From: guido@python.org (Guido van Rossum) Date: Thu, 24 Jan 2002 13:07:24 -0500 Subject: [Python-Dev] Tuples vs. lists In-Reply-To: Your message of "Thu, 24 Jan 2002 10:03:35 PST." <20020124180336.4FBE1E8C1@waltz.rahul.net> References: <20020124180336.4FBE1E8C1@waltz.rahul.net> Message-ID: <200201241807.NAA17320@pcp742651pcs.reston01.va.comcast.net> > Sure, but then I can't just copy references to the tuple when creating a > copy of an instance, I'd have to copy the entire list. That's what I > meant by efficiency. There are important semantic differences coming > from the fact that tuples are immutable and lists are mutable, and I > think that a strict heterogeneous/homogenous distinction loses that. Well, as long as you promise not to change it, you *can* copy a reference, right? I guess I don't understand your application enough -- do you intend this to be a starting point that is modified during the program's execution, or is this a constant array? --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@home.com Fri Jan 25 01:22:47 2002 From: tim.one@home.com (Tim Peters) Date: Thu, 24 Jan 2002 20:22:47 -0500 Subject: [Python-Dev] VERBOSE and DEBUG conventions. In-Reply-To: Message-ID: > Supply a "set debug and log options" interface for your module, and then > call it . Good example: the gc module. [Steven Majewski] > Thanks. That mostly makes sense. > Except that I needed it to be in trace/debug mode when the module > initialization is being done, so I can't import the module and then > set it. I suppose I could just use another environment variable: > $PYOBJC_DEBUG -- then I could set debug levels. Sure. Or split out option/logging knobs into a distinct module. > FYI: In case you're wondering why I don't just use gdb: Nope . > It's seems to be a meta level problem between the python runtime > and the objective-c runtime, and I suspect the objc extensions > in gdb must make use of the objc-runtime ( for 'po' - print object, > for example.) because I seem to be causing another objc runtime > exception the act of examining things in the debugger. > This is not very documented in the gdb manual, so unless I'm > going to wade thru the sources, I though it would be easier just > to instrument the module. (and maybe Python.) Upgrade your OS to Windows and all these time-consuming *choices* go away. Got a bug? Great! There's nowhere to report it that isn't a black hole, and you can't even think about patching the sources, so you just live with it and buy another OS next year. Except for all the bugs you have to learn to endure, it makes life much simpler . From nhodgson@bigpond.net.au Fri Jan 25 05:02:12 2002 From: nhodgson@bigpond.net.au (Neil Hodgson) Date: Fri, 25 Jan 2002 16:02:12 +1100 Subject: [Python-Dev] Re: PEP 277: Unicode file name support for Windows NT, was PEP-time ? ... References: <15412.29829.302192.219332@12-248-41-177.client.attbi.com> <200201041046.g04AkKR05898@mira.informatik.hu-berlin.de> <016e01c19639$94c909b0$0acc8490@neil> <200201060033.g060X8c14491@mira.informatik.hu-berlin.de> <021901c19654$21f2e3f0$0acc8490@neil> <200201061214.g06CEtc01656@mira.informatik.hu-berlin.de> <036e01c1972d$dbc88a80$0acc8490@neil> <200201070728.g077SmZ01967@mira.informatik.hu-berlin.de> <003b01c1975d$e7dd3070$0acc8490@neil> <200201072317.g07NHEh01830@mira.informatik.hu-berlin.de> <3C3AC26A.D40842FB@lemburg.com> <02ff01c19cc3$92514540$0acc8490@neil> <200201140711.g0E7BsV01370@mira.informatik.hu-berlin.de> <3C44059C.CFC09899@lemburg.com> <200201152124.g0FLOV702247@mira.informatik.hu-berlin.de> <3C45CB11.ACB2CEE6@lemburg.com> <200201161909.g0GJ9OK01822@mira.informatik.hu-berlin.de> <3C46A535.3C579501@lemburg.com> <08e201c19f46$cad5f070$0acc8490@neil> <3C46B735.9C433F60@lemburg.com> <200201171206.g0HC6sa01572@mira.informatik.hu-berlin.de> <3C46C3C2.984F6227@lemburg.com> Message-ID: <06c901c1a55d$73987f40$0acc8490@neil> M.-A. Lemburg: > "Martin v. Loewis" wrote: > > ... > > if sys.platform == "win32": > > use_unicode_for_filenames = windowsversion in ['nt','w2k','xp'] > > elif sys.platform.startswith("darwin"): > > use_unicode_for_filenames = 1 > > else: > > use_unicode_for_filenames = 0 > > Sounds like this would be a good candidate for platform.py which I'll > check into CVS soon. With its many platform querying APIs it should > easily be possible to add a function which returns the above > information based on the platform Python is running on. OK. I'll remove unicodefilenames() from the PEP and my patch. Neil From tismer@tismer.com Thu Jan 17 17:09:38 2002 From: tismer@tismer.com (Christian Tismer) Date: Thu, 17 Jan 2002 18:09:38 +0100 Subject: [Python-Dev] Ann: Stackless Python is DEAD! Long live Stackless Python Message-ID: <3C470552.3040802@tismer.com> ####################################### Announcement: ####################################### The end of an era has come: --------------------------- Stackless Python, in the form provided upto Python 2.0, is DEAD. I am abandoning the whole implementation. A new era has begun: -------------------- A completely new implementation is in development for Python 2.2 and up which gives you the following features: - There are no restrictions any longer for uthread/coroutine switching. Switching is possible at *any* time, in *any* context. - There are no significant changes to the Python core any longer. The new patches are of minimum size, and they will probably survive unchanged until Python 3.0 . - Maintenance work for Stackless Python is reduced to the bare minimum. There is no longer a need to incorporate Stackless into the standard, since there is no work to be shared. - Stackless breaks its major axiom now. It is no longer platform independent, since it *does* modify the C stack. I will support all Intel platforms by myself. For other platforms, I'm asking for volunteers. * The basic elements of Stackless are now switchable chains of frames. We have to define an interface that turns these chains into microthreads and coroutines. Everybody is invited to come to the Stackless mailing list and discuss the layout of this new design. Especially we need to decide about (*). http://starship.python.net/mailman/listinfo/stackless see you there - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net/ 14163 Berlin : PGP key -> http://wwwkeys.pgp.net/ PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF where do you want to jump today? http://www.stackless.com/ From mal@lemburg.com Fri Jan 25 20:46:29 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 25 Jan 2002 21:46:29 +0100 Subject: [Python-Dev] Using LXR for Python CVS Source Code ? Message-ID: <3C51C425.358637DB@lemburg.com> Browing the Mozilla web-site I came across I nice utility which enables cross-referenced source code browsing: LXR http://lxr.mozilla.org/mozilla/source/webtools/lxr/ For example, see e.g. http://lxr.mozilla.org/mozilla/source/expat/xmlparse/hashtable.c I suppose setting this up on python.org would ease referencing Python C sources a lot and also provide a nice tool for learning to understand the internal structures of the interpreter. What do you think ? -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From guido@python.org Fri Jan 25 20:49:27 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 25 Jan 2002 15:49:27 -0500 Subject: [Python-Dev] Using LXR for Python CVS Source Code ? In-Reply-To: Your message of "Fri, 25 Jan 2002 21:46:29 +0100." <3C51C425.358637DB@lemburg.com> References: <3C51C425.358637DB@lemburg.com> Message-ID: <200201252049.PAA06163@pcp742651pcs.reston01.va.comcast.net> > Browing the Mozilla web-site I came across I nice utility which > enables cross-referenced source code browsing: LXR > > http://lxr.mozilla.org/mozilla/source/webtools/lxr/ > > For example, see e.g. > > http://lxr.mozilla.org/mozilla/source/expat/xmlparse/hashtable.c > > I suppose setting this up on python.org would ease referencing > Python C sources a lot and also provide a nice tool for learning > to understand the internal structures of the interpreter. > > What do you think ? +1 Do you want access to the python.org website and CVS so you can install this yourself? --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Fri Jan 25 21:16:19 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 25 Jan 2002 22:16:19 +0100 Subject: [Python-Dev] Using LXR for Python CVS Source Code ? References: <3C51C425.358637DB@lemburg.com> <200201252049.PAA06163@pcp742651pcs.reston01.va.comcast.net> Message-ID: <3C51CB23.D7E91DD7@lemburg.com> Guido van Rossum wrote: > > > Browing the Mozilla web-site I came across I nice utility which > > enables cross-referenced source code browsing: LXR > > > > http://lxr.mozilla.org/mozilla/source/webtools/lxr/ > > > > For example, see e.g. > > > > http://lxr.mozilla.org/mozilla/source/expat/xmlparse/hashtable.c > > > > I suppose setting this up on python.org would ease referencing > > Python C sources a lot and also provide a nice tool for learning > > to understand the internal structures of the interpreter. > > > > What do you think ? > > +1 > > Do you want access to the python.org website and CVS so you can > install this yourself? I could do that, but would need some help from the admins since LXR requires Perl 5+ and Glimpse to be installed. I'll also need to modify the Apache config files and will probably have to setup a cron job which updates the indexes once a day. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From DavidA@ActiveState.com Fri Jan 25 21:26:04 2002 From: DavidA@ActiveState.com (David Ascher) Date: Fri, 25 Jan 2002 13:26:04 -0800 Subject: [Python-Dev] Using LXR for Python CVS Source Code ? References: <3C51C425.358637DB@lemburg.com> <200201252049.PAA06163@pcp742651pcs.reston01.va.comcast.net> <3C51CB23.D7E91DD7@lemburg.com> Message-ID: <3C51CD6C.3EA845ED@activestate.com> Not wishing to make a science project out of it, but you might consider the newer lxr, which uses a real database (mysql, IIRC). We've used lxr in-house for a while, it's an absolutely wonderful tool. It is quite hard to setup multiple lxr's on a single machine (at least with the 'old' lxr), be forewarned. Also, lxr doesn't really deal especially well with Python code - but for C/C++ code, it rocks. --david "M.-A. Lemburg" wrote: > > Guido van Rossum wrote: > > > > > Browing the Mozilla web-site I came across I nice utility which > > > enables cross-referenced source code browsing: LXR > > > > > > http://lxr.mozilla.org/mozilla/source/webtools/lxr/ > > > > > > For example, see e.g. > > > > > > http://lxr.mozilla.org/mozilla/source/expat/xmlparse/hashtable.c > > > > > > I suppose setting this up on python.org would ease referencing > > > Python C sources a lot and also provide a nice tool for learning > > > to understand the internal structures of the interpreter. > > > > > > What do you think ? > > > > +1 > > > > Do you want access to the python.org website and CVS so you can > > install this yourself? > > I could do that, but would need some help from the admins > since LXR requires Perl 5+ and Glimpse to be installed. I'll > also need to modify the Apache config files and will probably > have to setup a cron job which updates the indexes once a > day. > > -- > Marc-Andre Lemburg > CEO eGenix.com Software GmbH > ______________________________________________________________________ > Company & Consulting: http://www.egenix.com/ > Python Software: http://www.egenix.com/files/python/ > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev From mal@lemburg.com Fri Jan 25 22:57:31 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 25 Jan 2002 23:57:31 +0100 Subject: [Python-Dev] Using LXR for Python CVS Source Code ? References: <3C51C425.358637DB@lemburg.com> <200201252049.PAA06163@pcp742651pcs.reston01.va.comcast.net> <3C51CB23.D7E91DD7@lemburg.com> <3C51CD6C.3EA845ED@activestate.com> Message-ID: <3C51E2DB.85961923@lemburg.com> David Ascher wrote: > > Not wishing to make a science project out of it, but you might consider > the newer lxr, which uses a real database (mysql, IIRC). > > We've used lxr in-house for a while, it's an absolutely wonderful tool. > It is quite hard to setup multiple lxr's on a single machine (at least > with the 'old' lxr), be forewarned. > > Also, lxr doesn't really deal especially well with Python code - but for > C/C++ code, it rocks. Hmm, I was planning to install the Mozilla version of LXR. I'll also look at the latest LXR version 0.9. If it does indeed use MySQL, I'd rather not go down that road -- setting up and maintaining MySQL is not exactly fun... -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From aazmace@aazbooks.com Sat Jan 26 02:24:31 2002 From: aazmace@aazbooks.com (AaZbooks.com) Date: Sat, 26 Jan 2002 03:24:31 +0100 Subject: [Python-Dev] Nouvelles acquisitions / New online Message-ID: <200201260224.DAA05258@www.olm.fr> Chers bibliophiles, Cette semaine nous vous proposons nos nouvelles acquisitions (plus de 500). Vous y trouverez par exemple: SAINTE BEUVE C.A. MADAME DESBORDES VALMORE SA VIE ET SA CORRESPONDANCE Edition originale (rare). Ouvrage contenant un catalogue de Michl Lévy, libraire éditeur, de 36 p.Broché.IN8. LEVY Paris 1870 Réf.: 18337 (107,00 ) ou bien encore: FALLET RENE BANLIEUE SUD EST Roman. Edition originale (rare). DOMAT Paris 1947 Réf.: 18313 (45,00 ) En vous souhaitant bonne lecture Toute l'équipe de AaZbooks.com ------------------------------------------------------- Dear bibliophiles, This week new online 500 books, for example: SAINTE BEUVE C.A. MADAME DESBORDES VALMORE SA VIE ET SA CORRESPONDANCE Edition originale (rare). Ouvrage contenant un catalogue de Michl Lévy, libraire éditeur, de 36 p.Broché.IN8. LEVY Paris 1870 Réf.: 18337 (107,00 ) FALLET RENE BANLIEUE SUD EST Roman. Edition originale (rare). DOMAT Paris 1947 Réf.: 18313 (45,00 ) Regards AaZbooks' Team ---------------------------------------------------------- AaZbooks.com - BP N°1 - La grande Bruyère - F72320 St-Maixent Tel.: +33 (0)2 43 71 00 70 - Fax: +33 (0)2 43 71 29 16 http://www.aazbooks.com ---------------------------------------------------------- Pour vous désinscrire cliquez ci-dessous http://www.aazbooks.com\lnews\desinscription.php From skip@pobox.com Sat Jan 26 22:17:26 2002 From: skip@pobox.com (Skip Montanaro) Date: Sat, 26 Jan 2002 16:17:26 -0600 Subject: [Python-Dev] Using LXR for Python CVS Source Code ? In-Reply-To: <3C51E2DB.85961923@lemburg.com> References: <3C51C425.358637DB@lemburg.com> <200201252049.PAA06163@pcp742651pcs.reston01.va.comcast.net> <3C51CB23.D7E91DD7@lemburg.com> <3C51CD6C.3EA845ED@activestate.com> <3C51E2DB.85961923@lemburg.com> Message-ID: <15443.10998.673581.778224@localhost.localdomain> mal> Hmm, I was planning to install the Mozilla version of LXR. I'll mal> also look at the latest LXR version 0.9. If it does indeed use mal> MySQL, I'd rather not go down that road -- setting up and mal> maintaining MySQL is not exactly fun... I find MySQL fairly straightforward to work with. (I use it on the Mojam & Musi-Cal sites.) If there's a functional difference between the new version and the old, I'd be willing to help out administering the database. Skip From andymac@bullseye.apana.org.au Sun Jan 27 10:48:59 2002 From: andymac@bullseye.apana.org.au (Andrew MacIntyre) Date: Sun, 27 Jan 2002 21:48:59 +1100 (EDT) Subject: [Python-Dev] updated patches for OS/2 EMX port Message-ID: Its taken longer than I'd hoped, however they're finally up for review. The updated bits have been attached to the previous patch entries in the patch manager: 435381: distutils changes http://sf.net/tracker/?func=detail&atid=305470&aid=435381&group_id=5470 450265: build files - self contained subdirectory in PC/ http://sf.net/tracker/?func=detail&atid=305470&aid=450265&group_id=5470 450266: library changes - 3 patch files covering:- - Lib/ (included os2emxpath.py as previously discussed here) - Lib/plat-os2emx/ (new subdirectory) - Lib/test/ (cope with 2 EMX limitations) http://sf.net/tracker/?func=detail&atid=305470&aid=450266&group_id=5470 450267: core changes - 4 patch files covering:- - Include/ - Modules/ (lots of changes; see below for more info) - Objects/ (see below for more info) - Python/ http://sf.net/tracker/?func=detail&atid=305470&aid=450267&group_id=5470 I hope that I got the patch links right... Particular notes wrt #450267: - the patch to Modules/import.c supports VACPP in addition to EMX. Michael Muller has trialled this patch with a VACPP build successfully. It is messy, but OS/2 isn't going to lose the 8.3 naming limit on DLLs anytime soon :-( Although truncating the DLL (PYD) name to 8 characters increases the chances of a name clash, the case-sensitive import support in the same patch alleviates it somewhat, and the fact that the "init " entrypoint is maintained will result in an import failure when there is an actual name clash. - Modules/unicodedata.c is affected by a name clash between the internally defined _getname() and an EMX routine of the same name defined in . The patch renames the internal routine to _getucname() to avoid this, but this change may not be acceptable - advice please. - Objects/stringobject.c and Objects/unicodeobject.c contain changes to handle the EMX runtime library returning "0x" as the prefix for output formatted with a "%X" format. I have tried to minimise the changes in these patches to the minimum needed for the port to function, ie I've tried to eradicate the cosmetic changes in the earlier patches, and avoid picking up unwanted files (such as Modules/Setup). Please let me know if you find any such changes I missed. The patches uploaded apply cleanly to a copy of an anonoymously checked out CVS tree as of 0527 AEST this morning (Jan 27), and have been built and regression tested on both OS/2 EMX and FreeBSD 4.4R with no unexpected test failures. If there are no unresolvable objections, and approval to apply these patches is granted, I propose that the patches be applied as follows:- Stage 1: the build patch (creates+populates PC/os2emx/) Stage 2: the Lib/plat-os2emx/ patch Stage 3: the Lib/ and Lib/test/ patches Stage 4: the distutils patch Stage 5: the Include/, Objects/ and Python/ patches Stage 6: the Modules/ patch I would expect to allow at least 48 hours between stages. Comments/advice on this proposal also appreciated. -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au | Snail: PO Box 370 andymac@pcug.org.au | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From martin@v.loewis.de Sun Jan 27 20:32:44 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 27 Jan 2002 21:32:44 +0100 Subject: [Python-Dev] updated patches for OS/2 EMX port In-Reply-To: References: Message-ID: Andrew MacIntyre writes: > - Modules/unicodedata.c is affected by a name clash between the internally > defined _getname() and an EMX routine of the same name defined in > . The patch renames the internal routine to _getucname() to > avoid this, but this change may not be acceptable - advice please. My advice for renaming things because of name clashes: Always rename in a way that solves this particular problem for good, by using the Py prefix (or _Py to further indicate that this is not public API; it's a static function, anyway). Somebody may have a function _getucname somewhere, whereas it is really unlikely that people add a Py prefix to their functions (if they have been following the last 30 years of C programming). > - Objects/stringobject.c and Objects/unicodeobject.c contain changes to > handle the EMX runtime library returning "0x" as the prefix for output > formatted with a "%X" format. I'd suggest a different approach here, which does not use #ifdefs: Instead of testing for the system, test for the bug. Then, if the bug goes away, or appears on other systems as well, the code will be good. Once formatting is complete, see whether it put in the right letter, and fix that in the result buffer if the native sprintf got it wrong. If you follow this strategy, you should still add a comment indicating that this was added for OS/2, to give people an idea where that came from. Another approach would be to autoconfiscate this particular issue. I'm in general in favour of autoconf'ed bug tests instead of runtime bug tests, but people on systems without /bin/sh might feel differently. > If there are no unresolvable objections, and approval to apply these > patches is granted, I propose that the patches be applied as follows:- > > Stage 1: the build patch (creates+populates PC/os2emx/) > Stage 2: the Lib/plat-os2emx/ patch > Stage 3: the Lib/ and Lib/test/ patches > Stage 4: the distutils patch > Stage 5: the Include/, Objects/ and Python/ patches > Stage 6: the Modules/ patch > > I would expect to allow at least 48 hours between stages. > > Comments/advice on this proposal also appreciated. Sounds good to me (although I'd probably process the "uncritical", i.e. truly platform-specific parts much more quickly). Who's going to work with Andrew to integrate this stuff? Regards, Martin From aahz@rahul.net Mon Jan 28 21:21:58 2002 From: aahz@rahul.net (Aahz Maruch) Date: Mon, 28 Jan 2002 13:21:58 -0800 (PST) Subject: [Python-Dev] Tuples vs. lists In-Reply-To: <200201241807.NAA17320@pcp742651pcs.reston01.va.comcast.net> from "Guido van Rossum" at Jan 24, 2002 01:07:24 PM Message-ID: <20020128212158.D7315E8C3@waltz.rahul.net> Guido van Rossum wrote: > Aahz: >> >> Sure, but then I can't just copy references to the tuple when creating a >> copy of an instance, I'd have to copy the entire list. That's what I >> meant by efficiency. There are important semantic differences coming >> from the fact that tuples are immutable and lists are mutable, and I >> think that a strict heterogeneous/homogenous distinction loses that. > > Well, as long as you promise not to change it, you *can* copy a > reference, right? I guess I don't understand your application > enough -- do you intend this to be a starting point that is modified > during the program's execution, or is this a constant array? It's a constant. The BCD module is Binary Coded Decimal; instances are intended to be as immutable as strings and numbers (well, it *is* a number type). Modifying an instance is guaranteed to produce a new instance. To a large extent, I guess I feel that if a class is intended to be immutable, each of its underlying data attributes should also be immutable. -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista We must not let the evil of a few trample the freedoms of the many. From guido@python.org Mon Jan 28 21:26:23 2002 From: guido@python.org (Guido van Rossum) Date: Mon, 28 Jan 2002 16:26:23 -0500 Subject: [Python-Dev] Tuples vs. lists In-Reply-To: Your message of "Mon, 28 Jan 2002 13:21:58 PST." <20020128212158.D7315E8C3@waltz.rahul.net> References: <20020128212158.D7315E8C3@waltz.rahul.net> Message-ID: <200201282126.QAA30702@pcp742651pcs.reston01.va.comcast.net> > It's a constant. The BCD module is Binary Coded Decimal; instances are > intended to be as immutable as strings and numbers (well, it *is* a > number type). Modifying an instance is guaranteed to produce a new > instance. To a large extent, I guess I feel that if a class is intended > to be immutable, each of its underlying data attributes should also be > immutable. Or you could assign it to a private variable. --Guido van Rossum (home page: http://www.python.org/~guido/) From tismer@tismer.com Tue Jan 29 00:58:09 2002 From: tismer@tismer.com (Christian Tismer) Date: Tue, 29 Jan 2002 01:58:09 +0100 Subject: [Python-Dev] Ann: Stackless 2.2 pre-alpha is ready! Message-ID: <3C55F3A1.607@tismer.com> Dear Python community, Stackless Python 2.2 is alive! This is the first alpha version. It does not have any relevant changes to the interpreter. It does not have any limitation on switching. Support code for uthreads and coroutines is already implemented. And as announced, it is completely platform dependant. This version works on MS Win32 only. I'm going to support other platforms if I can find some sponsors. Let me say, it works great! There is no single problem. This technique can be applied to any software, any interpreter, provided I can support the platform. *** This is a critical phase for Stackless! *** *** I Am Asking For Corporate Sponsorships. *** I don't know how things should go on. I could turn it into a commercial product, Stackless is enabled enough for this. Or I could continue to keep it open-sourced, provided there is enough sponsorship. This decision has to be discussed in the next two weeks, after that I will decide. Anyway: Please check it out of CVS and have a look, it is sooo small code now. cvs -d :pserver:anonymous@tismer.com:/home/cvs co stackless/src You might want to add -z9 since this is a full Python 2.2 checkout. In this state, I don't prepare a distribution. You can build Stackless from CVS. I also put a copy of my python22.dll here for testing: http://www.stackless.com/slpython22.zip It is just almost 2 percent slower on my W2k machine. The trick is to avoid stack switching as much as possible. I do it only on every 8th recursion level, which is more than what's usual. >>> def f(n): ... if n:f(n-1) ... >>> import sys >>> sys.setrecursionlimit(100000+10) >>> f(100000) >>> ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net/ 14163 Berlin : PGP key -> http://wwwkeys.pgp.net/ PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF where do you want to jump today? http://www.stackless.com/ _______________________________________________ Stackless mailing list Stackless@www.tismer.com http://www.tismer.com/mailman/listinfo/stackless From greg@cosc.canterbury.ac.nz Tue Jan 29 05:09:10 2002 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 29 Jan 2002 18:09:10 +1300 (NZDT) Subject: [Python-Dev] Ann: Stackless 2.2 pre-alpha is ready! In-Reply-To: <3C55F3A1.607@tismer.com> Message-ID: <200201290509.SAA18841@s454.cosc.canterbury.ac.nz> Christian Tismer : > I could turn it into a commercial product, Stackless is > enabled enough for this. Or I could continue to keep it > open-sourced, provided there is enough sponsorship. It would be disappointing if it ceased being open-source! I hope enough volunteers can be found to work on ports to other platforms. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tismer@tismer.com Tue Jan 29 12:12:16 2002 From: tismer@tismer.com (Christian Tismer) Date: Tue, 29 Jan 2002 13:12:16 +0100 Subject: [Python-Dev] Ann: Stackless 2.2 pre-alpha is ready! References: <200201290509.SAA18841@s454.cosc.canterbury.ac.nz> Message-ID: <3C5691A0.5000101@tismer.com> Greg Ewing wrote: > Christian Tismer : > > >>I could turn it into a commercial product, Stackless is >>enabled enough for this. Or I could continue to keep it >>open-sourced, provided there is enough sponsorship. >> > > It would be disappointing if it ceased being open-source! > I hope enough volunteers can be found to work on ports to > other platforms. No problem, I was just trying to get more sponsors, which in fact already exist (but not enough for a living). Stackless will stay open source, especially after it has become so few source :-) -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net/ 14163 Berlin : PGP key -> http://wwwkeys.pgp.net/ PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF where do you want to jump today? http://www.stackless.com/ From fdrake@acm.org Tue Jan 29 16:36:16 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 29 Jan 2002 11:36:16 -0500 Subject: [Python-Dev] release22-maint branch strangeness Message-ID: <15446.53120.164519.118410@grendel.zope.com> I've noticed some strangeness with the release22-maint branch. I made a documentation change there this morning, and CVS gave the change a really weird version number when I checked it in. Looking further, it looks like the previous checkin for that file (Doc/tut/tut.tex) has some strangeness as well. The branching tags are also pretty whacked. This is an excerpt of the "cvs log" for the file: ------------------------------------------------------------------------ RCS file: /cvsroot/python/python/dist/src/Doc/tut/tut.tex,v Working file: tut.tex head: 1.158 branch: locks: strict access list: symbolic names: r212: 1.133.2.5 r212c1: 1.133.2.5 release22-mac: 1.156.4.1 release22-maint: 1.156.4.1.0.2 release22: 1.156.4.1 release22-branch: 1.156.0.4 release22-fork: 1.156 ------------------------------------------------------------------------ Note the revision number for release22-maint; it looks like it's a branch created from a branch created from a tag on a branch(!). All the while, I've been thinking that branches, once created, are independent (identified by the third component of the revision number for any given file). I still think they're supposed to be. Using a checkout created with the "-r release22-maint" options, I made two checkins, and the revision numbers & other metadata seem seriously strange: ------------------------------------------------------------------------ revision 1.156.4.1 date: 2001/12/21 03:48:33; author: fdrake; state: Exp; lines: +2 -2 branches: 1.156.4.1.2; Fix up some examples in the tutorial so we don't contradict our own advice on docstrings. This fixes SF bug #495601. ---------------------------- revision 1.156.4.1.2.1 date: 2002/01/29 14:54:18; author: fdrake; state: Exp; lines: +8 -1 Revise cheeseshop example so that the order of the keyword output is completely determined by the example; dict insertion order and the string hash algorithm no longer affect the output. This fixes SF bug #509281. ------------------------------------------------------------------------ For revision 1.156.4.1, note the strange branch number (1.156.4.1.2 -- too many components), and for revision 1.156.4.1.2.1 (too many components again!). The strange branch number on the first indicates that a branch was created from that revision (itself part of the release22-branch branch). Does anyone remember who created these branches? Or what commands were used to create them (using which branch/tag as the source of the working copy being used?)? This pretty much has Barry & I stumped at the moment, and we'd like to get this straightened out. The suspect branches are release22-maint, release22-mac. Thanks! -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From mwh@python.net Tue Jan 29 17:25:05 2002 From: mwh@python.net (Michael Hudson) Date: 29 Jan 2002 17:25:05 +0000 Subject: [Python-Dev] release22-maint branch strangeness In-Reply-To: "Fred L. Drake, Jr."'s message of "Tue, 29 Jan 2002 11:36:16 -0500" References: <15446.53120.164519.118410@grendel.zope.com> Message-ID: <2melk9nk7i.fsf@starship.python.net> "Fred L. Drake, Jr." writes: > I've noticed some strangeness with the release22-maint branch. I made > a documentation change there this morning, and CVS gave the change a > really weird version number when I checked it in. Looking further, it > looks like the previous checkin for that file (Doc/tut/tut.tex) has > some strangeness as well. The branching tags are also pretty > whacked. This is an excerpt of the "cvs log" for the file: Looks to me like the release22 tag for Doc/tut/tut.tex was set on the release22-branch, not the trunk. This is not what happened for, e.g. configure.in. Quite how this happened, or what (if anything) we should do about it, is another question entirely. cvs status -v is quite handy here. $ cvs status -v Doc/tut/tut.tex | head -n 20 =================================================================== File: tut.tex Status: Needs Patch Working revision: 1.157 Repository revision: 1.158 /cvsroot/python/python/dist/src/Doc/tut/tut.tex,v Sticky Tag: (none) Sticky Date: (none) Sticky Options: (none) Existing Tags: r212 (revision: 1.133.2.5) r212c1 (revision: 1.133.2.5) release22-mac (revision: 1.156.4.1) release22-maint (branch: 1.156.4.1.2) release22 (revision: 1.156.4.1) release22-branch (branch: 1.156.4) release22-fork (revision: 1.156) r22c1-mac (revision: 1.156) r22c1 (revision: 1.156) r22rc1-branch (branch: 1.156.2) $ cvs status -v configure.in | head -n 20 =================================================================== File: configure.in Status: Up-to-date Working revision: 1.289 Repository revision: 1.289 /cvsroot/python/python/dist/src/configure.in,v Sticky Tag: (none) Sticky Date: (none) Sticky Options: (none) Existing Tags: r212 (revision: 1.215.2.7) r212c1 (revision: 1.215.2.7) release22-mac (revision: 1.288) release22-maint (branch: 1.288.6) release22 (revision: 1.288) release22-branch (branch: 1.288.4) release22-fork (revision: 1.288) r22c1-mac (revision: 1.288) r22c1 (revision: 1.288) r22rc1-branch (branch: 1.288.2) Did different people create the release22 tags in different bits of the tree? Cheers, M. -- The "of course, while I have no problem with this at all, it's surely too much for a lesser being" flavor of argument always rings hollow to me. -- Tim Peters, 29 Apr 1998 From fdrake@acm.org Tue Jan 29 19:15:48 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 29 Jan 2002 14:15:48 -0500 Subject: [Python-Dev] release22-maint branch strangeness In-Reply-To: <2melk9nk7i.fsf@starship.python.net> References: <15446.53120.164519.118410@grendel.zope.com> <2melk9nk7i.fsf@starship.python.net> Message-ID: <15446.62692.281105.738748@grendel.zope.com> Michael Hudson writes: > Looks to me like the release22 tag for Doc/tut/tut.tex was set on the > release22-branch, not the trunk. Should not the branches be independent, once created? > This is not what happened for, e.g. configure.in. configure.in was not changed on the release22-branch. Take a look at Include/patchlevel.h. It doesn't look as messed up as Doc/tut/tut.tex, but something is definately wrong here as well. > Did different people create the release22 tags in different bits of > the tree? I'm not quite sure how things were handled with the -maint and -mac branches; I wonder if a branch tag was used somewhere a normal tag could have been used. I don't see it, though. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From tismer@tismer.com Tue Jan 29 19:41:45 2002 From: tismer@tismer.com (Christian Tismer) Date: Tue, 29 Jan 2002 20:41:45 +0100 Subject: [Python-Dev] Thread questionlet Message-ID: <3C56FAF9.20109@tismer.com> Dear developers, I'm still a little ignorant to real threads. In order to do the implementation of hard-wired microthreads right, I tried to understand how real threads work. My question, which I could not easily answer by reading the source is: What happens when the main thread ends? Do all threads run until they are eady too, or are they just killed away? And if they are killed, are they just removed, or do they all get an exception for cleanup? I would guess the latter, but I'm not sure. When a thread ends, it may contain several levels of other C calls which might need to finalize, so I thought of a special exception for this, but didn't find such. Many thanks and sorry about my ignorance - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net/ 14163 Berlin : PGP key -> http://wwwkeys.pgp.net/ PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF where do you want to jump today? http://www.stackless.com/ From tim.one@home.com Tue Jan 29 20:29:41 2002 From: tim.one@home.com (Tim Peters) Date: Tue, 29 Jan 2002 15:29:41 -0500 Subject: [Python-Dev] Thread questionlet In-Reply-To: <3C56FAF9.20109@tismer.com> Message-ID: [Christian Tismer] > ... > My question, which I could not easily answer by reading > the source is: > What happens when the main thread ends? Do all threads run > until they are ready too, or are they just killed away? You're walking near the edge of a very steep cliff. There are jagged rocks a kilometer below, so don't slip . It varies by OS, and even by exactly how the main thread exits. Reading OS docs doesn't really help either, because the version of threads exposed by the C libraries may differ from native OS facilities in subtle but crucial ways. > And if they are killed, are they just removed, or do > they all get an exception for cleanup? Can only be answered one platform at a time. They're not going to get a *Python*-level exception, no. Here's a simple test program: import thread import time def f(i): while 1: print "thread %d about to sleep" % i time.sleep(0.5) for i in range(3): thread.start_new_thread(f, (i,)) time.sleep(3) print "main is done" and a typical run on Windows: C:\Code\python\PCbuild>\python22\python.exe tdie.py thread 0 about to sleep thread 1 about to sleep thread 2 about to sleep thread 0 about to sleep thread 1 about to sleep thread 2 about to sleep thread 0 about to sleep thread 1 about to sleep thread 2 about to sleep thread 0 about to sleep thread 1 about to sleep thread 2 about to sleep thread 1 about to sleep thread 0 about to sleep thread 2 about to sleep thread 1 about to sleep thread 0 about to sleep thread 2 about to sleep thread 1 about to sleep main is done C:\Code\python\PCbuild> I expect much the same on Linux (all threads die, no exceptions raised). But, IIRC, the threads would keep going on SGI despite that the main thread is history. > ... > When a thread ends, it may contain several levels of other > C calls which might need to finalize, so I thought of > a special exception for this, but didn't find such. Closing threads cleanly is the programmer's responsiblity across all OSes. It can be very difficult. Python doesn't really help (or hinder). Microsoft helps in that DLLs can define a "call on thread detach" function that's automatically called when a thread detaches from the DLL, but Python doesn't exploit that. The DLL hook may not get called even if it did, depending on exactly how a thread detaches (the Big Hammer last-chance Win32 TerminateProcess/TerminateThread functions generally leave things a mess -- "TerminateThread is a dangerous function that should only be used in the most extreme cases", etc). From guido@python.org Tue Jan 29 21:46:41 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 29 Jan 2002 16:46:41 -0500 Subject: [Python-Dev] Thread questionlet In-Reply-To: Your message of "Tue, 29 Jan 2002 20:41:45 +0100." <3C56FAF9.20109@tismer.com> References: <3C56FAF9.20109@tismer.com> Message-ID: <200201292146.QAA24412@pcp742651pcs.reston01.va.comcast.net> > My question, which I could not easily answer by reading > the source is: > What happens when the main thread ends? Do all threads run > until they are eady too, or are they just killed away? > And if they are killed, are they just removed, or do > they all get an exception for cleanup? If you're talking about the thread module, they are killed without being given notice. The threading module however waits for all non-daemon threads, using the atexit mechanism build on top of sys.exit. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Tue Jan 29 21:51:35 2002 From: guido@python.org (Guido van Rossum) Date: Tue, 29 Jan 2002 16:51:35 -0500 Subject: [Python-Dev] release22-maint branch strangeness In-Reply-To: Your message of "Tue, 29 Jan 2002 11:36:16 EST." <15446.53120.164519.118410@grendel.zope.com> References: <15446.53120.164519.118410@grendel.zope.com> Message-ID: <200201292151.QAA24484@pcp742651pcs.reston01.va.comcast.net> > I've noticed some strangeness with the release22-maint branch. I made > a documentation change there this morning, and CVS gave the change a > really weird version number when I checked it in. Looking further, it > looks like the previous checkin for that file (Doc/tut/tut.tex) has > some strangeness as well. The branching tags are also pretty > whacked. This is an excerpt of the "cvs log" for the file: > > ------------------------------------------------------------------------ > RCS file: /cvsroot/python/python/dist/src/Doc/tut/tut.tex,v > Working file: tut.tex > head: 1.158 > branch: > locks: strict > access list: > symbolic names: > r212: 1.133.2.5 > r212c1: 1.133.2.5 > release22-mac: 1.156.4.1 > release22-maint: 1.156.4.1.0.2 > release22: 1.156.4.1 > release22-branch: 1.156.0.4 > release22-fork: 1.156 > ------------------------------------------------------------------------ > > Note the revision number for release22-maint; it looks like it's a > branch created from a branch created from a tag on a branch(!). All > the while, I've been thinking that branches, once created, are > independent (identified by the third component of the revision number > for any given file). I still think they're supposed to be. I think you must've used a tag to bvase your branch, and that tag was already on a branch. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@pobox.com Tue Jan 29 21:51:29 2002 From: skip@pobox.com (Skip Montanaro) Date: Tue, 29 Jan 2002 15:51:29 -0600 Subject: [Python-Dev] Stevens - still best for Unix system call programming? Message-ID: <15447.6497.563586.973235@beluga.mojam.com> Sorry for the off-topic post. I'm starting in on a little project to create an analog to fopen(3) and friends that provides the illusion of large file support even on systems that don't support large files, so I'm doing more fiddling with Unix system calls than I've done in awhile, and am looking for a little hardcover help. Is Richard Stevens' "Advanced Programming in the UNIX Environment" still the _sine qua non_ in this area? Thx, Skip P.S. OPN: it will have a Python binding... From barry@zope.com Tue Jan 29 22:08:00 2002 From: barry@zope.com (Barry A. Warsaw) Date: Tue, 29 Jan 2002 17:08:00 -0500 Subject: [Python-Dev] Stevens - still best for Unix system call programming? References: <15447.6497.563586.973235@beluga.mojam.com> Message-ID: <15447.7488.861384.586661@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: SM> Is Richard Stevens' "Advanced Programming in the UNIX SM> Environment" still the _sine qua non_ in this area? Indeed! It always seems to answer my questions accurately and in depth. -Barry From aahz@rahul.net Tue Jan 29 23:47:37 2002 From: aahz@rahul.net (Aahz Maruch) Date: Tue, 29 Jan 2002 15:47:37 -0800 (PST) Subject: [Python-Dev] Thread questionlet In-Reply-To: <3C56FAF9.20109@tismer.com> from "Christian Tismer" at Jan 29, 2002 08:41:45 PM Message-ID: <20020129234738.66F9AE8C4@waltz.rahul.net> Christian Tismer wrote: > > I'm still a little ignorant to real threads. > In order to do the implementation of hard-wired microthreads > right, I tried to understand how real threads work. Can't answer your specific question, but you might want to look at my Starship pages if you want to increase your general understanding of Python threads (there probably won't be much new to you; OTOH, it shouldn't take you long to read). http://starship.python.net/crew/aahz/ -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista We must not let the evil of a few trample the freedoms of the many. From tismer@tismer.com Wed Jan 30 00:21:40 2002 From: tismer@tismer.com (Christian Tismer) Date: Wed, 30 Jan 2002 01:21:40 +0100 Subject: [Python-Dev] Thread questionlet References: <20020129234738.66F9AE8C4@waltz.rahul.net> Message-ID: <3C573C94.3090305@tismer.com> Aahz Maruch wrote: > Christian Tismer wrote: > >>I'm still a little ignorant to real threads. >>In order to do the implementation of hard-wired microthreads >>right, I tried to understand how real threads work. >> > > Can't answer your specific question, but you might want to look at my > Starship pages if you want to increase your general understanding of > Python threads (there probably won't be much new to you; OTOH, it > shouldn't take you long to read). > > http://starship.python.net/crew/aahz/ Oh well, 1024 thanks, very helpful. I'm again the clueless implementor. It still feels warm and fuzzy here, although I think there is no rule that I missed to break since -dev knows about me, and now my final sacrileg... ...after all, this is kinda piecemaker, since Stackless is now orthogonal, in a way. I gave up some academic POV, in favor of something pragmatic, and finally we all get rid of a problem. Hey, I want to become a productive contributor (again?) :) thanks - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net/ 14163 Berlin : PGP key -> http://wwwkeys.pgp.net/ PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF where do you want to jump today? http://www.stackless.com/ From tim.one@home.com Wed Jan 30 06:29:39 2002 From: tim.one@home.com (Tim Peters) Date: Wed, 30 Jan 2002 01:29:39 -0500 Subject: [Python-Dev] Stale CVS lock Message-ID: There appears to be a stale anoncvs lock in the Misc directory preventing checkins there (like NEWS); have opened an SF support request: http://sf.net/tracker/?func=detail&aid=510555&group_id=1&atid=200001 From tismer@tismer.com Wed Jan 30 10:54:14 2002 From: tismer@tismer.com (Christian Tismer) Date: Wed, 30 Jan 2002 11:54:14 +0100 Subject: [Python-Dev] Thread questionlet References: Message-ID: <3C57D0D6.8010602@tismer.com> Tim Peters wrote: > [Christian Tismer] > >>... >>My question, which I could not easily answer by reading >>the source is: >>What happens when the main thread ends? Do all threads run >>until they are ready too, or are they just killed away? >> > > You're walking near the edge of a very steep cliff. There are jagged rocks > a kilometer below, so don't slip . Uhmm -- I really didn't want to poke into something problematic, but obviously I have no more simple questions left. ;-) > It varies by OS, and even by exactly how the main thread exits. Reading OS > docs doesn't really help either, because the version of threads exposed by > the C libraries may differ from native OS facilities in subtle but crucial > ways. It does not sound like being designed so, more like just some way through these subtleties, without trying to solve every platform's problems. I don't try to solve this, either. But since I'm writing some kind of platform independant threads (isn't it funny? by using non-portable tricks, I get some portable threads), I'd like to think about how this world *could* look like. Maybe I have a chance to provide an (u)thread implementation which is really what people would want for real threads? >>And if they are killed, are they just removed, or do >>they all get an exception for cleanup? >> > > Can only be answered one platform at a time. They're not going to get a > *Python*-level exception, no. Here's a simple test program: [thanks for the test code] > I expect much the same on Linux (all threads die, no exceptions raised). > But, IIRC, the threads would keep going on SGI despite that the main thread > is history. So threads do force the programmer to write platform-dependant Python code. For sure nothing that Python wants, it just happens. >>... >>When a thread ends, it may contain several levels of other >>C calls which might need to finalize, so I thought of >>a special exception for this, but didn't find such. >> > > Closing threads cleanly is the programmer's responsiblity across all OSes. > It can be very difficult. Python doesn't really help (or hinder). Ok with me, this is really not trivial. (I guessed that from reading the source, but it really was not obvious. So I asked a naive question, but you know me better...) Maybe Python could try to help though an API? > Microsoft helps in that DLLs can define a "call on thread detach" function > that's automatically called when a thread detaches from the DLL, but Python > doesn't exploit that. The DLL hook may not get called even if it did, > depending on exactly how a thread detaches (the Big Hammer last-chance Win32 > TerminateProcess/TerminateThread functions generally leave things a mess -- > "TerminateThread is a dangerous function that should only be used in the > most extreme cases", etc). Now the real question: If you have the oportunity which I have: Define some threads which (mis)behave equally (un)well on every supported platform, once and forever. Would you try to mimick the median real threads behavior as they work today? Or would you try to build something consistent, cross-platform, that makes sense, that would even make sense for new revisions of the real thread modules? I think here is a chance to do a reference implementation of (u)threads since there are absolutely no OS dictated restrictions or MS added doubtful features, we can just do it right. Given that there is a suitable definition of "right", of course. The problem is that I'm not a specialist on threading, therefore I'm asking for suggestions. Please, what do you all think would be "right", given that you have full control of ver your "virtual OS"? contructively-but-trying-not-to-overdo - ly y'rs - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net/ 14163 Berlin : PGP key -> http://wwwkeys.pgp.net/ PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF where do you want to jump today? http://www.stackless.com/ From mwh@python.net Wed Jan 30 14:54:30 2002 From: mwh@python.net (Michael Hudson) Date: 30 Jan 2002 14:54:30 +0000 Subject: [Python-Dev] can someone with purify run test_curses Message-ID: <2m7kpzq47t.fsf@starship.python.net> Please? It segfaults for me, in a confusing way. It only segfaults if I run it under regrtest, not directly. Cheers, M. -- CLiki pages can be edited by anybody at any time. Imagine the most fearsomely comprehensive legal disclaimer you have ever seen, and double it -- http://ww.telent.net/cliki/index From aahz@rahul.net Wed Jan 30 14:56:04 2002 From: aahz@rahul.net (Aahz Maruch) Date: Wed, 30 Jan 2002 06:56:04 -0800 (PST) Subject: [Python-Dev] Thread questionlet In-Reply-To: <3C57D0D6.8010602@tismer.com> from "Christian Tismer" at Jan 30, 2002 11:54:14 AM Message-ID: <20020130145604.AF2F0E8C4@waltz.rahul.net> Christian Tismer wrote: > > I don't try to solve this, either. But since I'm writing some kind of > platform independant threads (isn't it funny? by using non-portable > tricks, I get some portable threads), I'd like to think about how > this world *could* look like. Maybe I have a chance to provide an > (u)thread implementation which is really what people would want for > real threads? No, you don't. Real threads have one killer advantage you just can't emulate: they can parallelize I/O operations (and theoretically parallelize computations on multiple CPUs). The advantage of microthreads has been that they're lightweight, so they're good for applications that require *lots* of threads, such as simulations. I think keeping this advantage would be a Good Idea. You might want to look at Ruby, though, because it does what you're wanting to do. (I think -- I haven't touched Ruby myself.) -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista We must not let the evil of a few trample the freedoms of the many. From neal@metaslash.com Wed Jan 30 15:36:23 2002 From: neal@metaslash.com (Neal Norwitz) Date: Wed, 30 Jan 2002 10:36:23 -0500 Subject: [Python-Dev] can someone with purify run test_curses References: <2m7kpzq47t.fsf@starship.python.net> Message-ID: <3C5812F7.86B78FBC@metaslash.com> Michael Hudson wrote: > > Please? It segfaults for me, in a confusing way. > > It only segfaults if I run it under regrtest, not directly. What version? Current CVS? Any other special instructions? When do you need it done? Neal From mwh@python.net Wed Jan 30 15:43:19 2002 From: mwh@python.net (Michael Hudson) Date: 30 Jan 2002 15:43:19 +0000 Subject: [Python-Dev] can someone with purify run test_curses In-Reply-To: Neal Norwitz's message of "Wed, 30 Jan 2002 10:36:23 -0500" References: <2m7kpzq47t.fsf@starship.python.net> <3C5812F7.86B78FBC@metaslash.com> Message-ID: <2mhep37sko.fsf@starship.python.net> Neal Norwitz writes: > Michael Hudson wrote: > > > > Please? It segfaults for me, in a confusing way. > > > > It only segfaults if I run it under regrtest, not directly. > > What version? Current CVS? Any other special instructions? > When do you need it done? I don't think I do, now (see checkins). But it might be worth running test_curses --with-pymalloc, if it's not too much hassle. I'll have a look to see if there are any other Object/Mem mismatches. Cheers, M. -- Our lecture theatre has just crashed. It will currently only silently display an unexplained line-drawing of a large dog accompanied by spookily flickering lights. -- Dan Sheppard, ucam.chat (from Owen Dunn's summary of the year) From nas@python.ca Wed Jan 30 16:28:07 2002 From: nas@python.ca (Neil Schemenauer) Date: Wed, 30 Jan 2002 08:28:07 -0800 Subject: [Python-Dev] Mixing memory management APIs In-Reply-To: ; from mwh@users.sourceforge.net on Wed, Jan 30, 2002 at 07:47:36AM -0800 References: Message-ID: <20020130082807.B9393@glacier.arctrix.com> Michael Hudson wrote: > Modified Files: > _curses_panel.c > Log Message: > Oh look, another one. > > 2.2.1 candiate (he says, largely talking to himself :) > *** 192,196 **** > Py_DECREF(po->wo); > remove_lop(po); > ! PyMem_DEL(po); > } > > --- 192,196 ---- > Py_DECREF(po->wo); > remove_lop(po); > ! PyObject_DEL(po); > } I think we have to break down and do what Tim suggests. Ie make: free == PyMem_DEL == PyObject_DEL == PyObject_FREE == ... pymalloc needs to use a completely new set of APIs. The only problem I see is coming up with names. NEW, MALLOC, REALLOC, RESIZE, and DEL are all taken. Any suggestions? Neil From mwh@python.net Wed Jan 30 17:01:17 2002 From: mwh@python.net (Michael Hudson) Date: 30 Jan 2002 17:01:17 +0000 Subject: [Python-Dev] Mixing memory management APIs In-Reply-To: Neil Schemenauer's message of "Wed, 30 Jan 2002 08:28:07 -0800" References: <20020130082807.B9393@glacier.arctrix.com> Message-ID: <2mvgdj6aea.fsf@starship.python.net> Neil Schemenauer writes: > Michael Hudson wrote: > > Modified Files: > > _curses_panel.c > > Log Message: > > Oh look, another one. > > > > 2.2.1 candiate (he says, largely talking to himself :) > > > *** 192,196 **** > > Py_DECREF(po->wo); > > remove_lop(po); > > ! PyMem_DEL(po); > > } > > > > --- 192,196 ---- > > Py_DECREF(po->wo); > > remove_lop(po); > > ! PyObject_DEL(po); > > } > > I think we have to break down and do what Tim suggests. Ie make: > > free == PyMem_DEL == PyObject_DEL == PyObject_FREE == ... > > pymalloc needs to use a completely new set of APIs. The only problem I > see is coming up with names. NEW, MALLOC, REALLOC, RESIZE, and DEL are > all taken. Any suggestions? And then change all the current uses of PyObject_Del to the new API? What would that buy us? Unless I misunderstand we *have* to do something different to remove an object as opposed to freeing raw storage (GC, for example). I agree we have too many preprocessor macros, but I don't think we can have free == PyObject_DEL. I don't what we have is so bad; a helpful tip is that if you're using the _Free/_FREE/_Malloc/_REALLOC/etc interfaces, stop. That gets rid of half the problem. Cheers, M. -- We've had a lot of problems going from glibc 2.0 to glibc 2.1. People claim binary compatibility. Except for functions they don't like. -- Peter Van Eynde, comp.lang.lisp From mwh@python.net Wed Jan 30 17:03:56 2002 From: mwh@python.net (Michael Hudson) Date: 30 Jan 2002 17:03:56 +0000 Subject: [Python-Dev] Mixing memory management APIs In-Reply-To: Michael Hudson's message of "30 Jan 2002 17:01:17 +0000" References: <20020130082807.B9393@glacier.arctrix.com> <2mvgdj6aea.fsf@starship.python.net> Message-ID: <2m8zafkbyb.fsf@starship.python.net> Michael Hudson writes: > Neil Schemenauer writes: > > > I think we have to break down and do what Tim suggests. Ie make: > > > > free == PyMem_DEL == PyObject_DEL == PyObject_FREE == ... > > > > pymalloc needs to use a completely new set of APIs. The only problem I > > see is coming up with names. NEW, MALLOC, REALLOC, RESIZE, and DEL are > > all taken. Any suggestions? > > And then change all the current uses of PyObject_Del to the new API? > What would that buy us? Unless I misunderstand we *have* to do > something different to remove an object as opposed to freeing raw > storage (GC, for example). > > I agree we have too many preprocessor macros, but I don't think we can > have free == PyObject_DEL. No, I take that back... -- Just point your web browser at http://www.python.org/search/ and look for "program", "doesn't", "work", or "my". Whenever you find someone else whose program didn't work, don't do what they did. Repeat as needed. -- Tim Peters, on python-help, 16 Jun 1998 From aahz@rahul.net Wed Jan 30 17:13:48 2002 From: aahz@rahul.net (Aahz Maruch) Date: Wed, 30 Jan 2002 09:13:48 -0800 (PST) Subject: [Python-Dev] Mixing memory management APIs In-Reply-To: <20020130082807.B9393@glacier.arctrix.com> from "Neil Schemenauer" at Jan 30, 2002 08:28:07 AM Message-ID: <20020130171348.E8D78E8C4@waltz.rahul.net> Neil Schemenauer wrote: > > pymalloc needs to use a completely new set of APIs. The only problem I > see is coming up with names. NEW, MALLOC, REALLOC, RESIZE, and DEL are > all taken. Any suggestions? >From the Department of Redundancy Department, how about: PyMalloc_New, PyMalloc_Malloc, .... -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista We must not let the evil of a few trample the freedoms of the many. From nas@python.ca Wed Jan 30 17:40:50 2002 From: nas@python.ca (Neil Schemenauer) Date: Wed, 30 Jan 2002 09:40:50 -0800 Subject: [Python-Dev] Mixing memory management APIs In-Reply-To: <2mvgdj6aea.fsf@starship.python.net>; from mwh@python.net on Wed, Jan 30, 2002 at 05:01:17PM +0000 References: <20020130082807.B9393@glacier.arctrix.com> <2mvgdj6aea.fsf@starship.python.net> Message-ID: <20020130094050.B9709@glacier.arctrix.com> Michael Hudson wrote: > I agree we have too many preprocessor macros, but I don't think we can > have free == PyObject_DEL. If we don't then many extension modules will break. The example module Modules/xxmodule.c used to allocate using PyObject_New and deallocate using free(). I believe there are many modules out there that do the same (or use PyMem_Del, etc). Neil From paul@pfdubois.com Wed Jan 30 17:39:45 2002 From: paul@pfdubois.com (Paul Dubois) Date: Wed, 30 Jan 2002 09:39:45 -0800 Subject: [Python-Dev] Odd errors when catching ImportError Message-ID: <000701c1a9b5$1bccb5e0$09860cc0@CLENHAM> Please excuse me if this is in the bug list; I looked through it but the list is too long for old people. I have been running into a number of odd errors caused by code like the following. The behavior seems to be machine dependent. fooflag = 0 try: import foo except ImportError: fooflag = 1 I have had this result in a seg fault upon exit, and also when something like this was in file xxx.py inside a package, and the __init__.py did from xxx import fooflag I've had it tell me xxx had no attribute fooflag. I added "print fooflag" at the bottom of the file and it fixed it. That was on a DEC. On Linux it worked. I suppose I should be testing for the ability to import foo some other way but I don't know what it is. From sdm7g@Virginia.EDU Wed Jan 30 18:00:48 2002 From: sdm7g@Virginia.EDU (Steven Majewski) Date: Wed, 30 Jan 2002 13:00:48 -0500 (EST) Subject: [Python-Dev] next vs darwin Message-ID: I recall having the discussion but I don't quite recall the resolution: Is Next support now officially dropped from the distribution ? I have a revised dynamic loading module that strips out all of the dead branches ( as well as better error reporting ): I was going to call it dynload_darwin.c and add support to configure, but grepping thru configure I only saw darwin as triggering dynload_next.c -- it *looks* like the Next has bee dropped. Should we rename the file anyway ? ( to make it easier for folks to know where to look. ) There has also been some discussion on the pythonmac-sig list about dynamic loading. There are some other problems that this module doesn't fix yet. If someone wants to subit a better one, that's fine by me, but we REALLY need to get the better error reporting in there so we can at least find the problem. The other thing that's been discussed is adding configure support to build with the dlopen compatability libs if that is available. ( doing config with --without-dyld doesn't seem to change anything. ) -- Steve From martin@v.loewis.de Wed Jan 30 19:04:59 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 30 Jan 2002 20:04:59 +0100 Subject: [Python-Dev] next vs darwin In-Reply-To: References: Message-ID: Steven Majewski writes: > I recall having the discussion but I don't quite recall the > resolution: Is Next support now officially dropped from the > distribution ? AFAIR, yes. > Should we rename the file anyway ? ( to make it easier for > folks to know where to look. ) Yes. > The other thing that's been discussed is adding configure > support to build with the dlopen compatability libs if > that is available. Can you please explain what that would provide to module users or end users? Would there be additional modules available that otherwise wouldn't be available? If not, I don't think that this option should be provided. Regards, Martin From mal@lemburg.com Wed Jan 30 19:15:33 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 30 Jan 2002 20:15:33 +0100 Subject: [Python-Dev] Mixing memory management APIs References: <20020130082807.B9393@glacier.arctrix.com> <2mvgdj6aea.fsf@starship.python.net> <20020130094050.B9709@glacier.arctrix.com> Message-ID: <3C584655.33BE8C9D@lemburg.com> Neil Schemenauer wrote: > > Michael Hudson wrote: > > I agree we have too many preprocessor macros, but I don't think we can > > have free == PyObject_DEL. > > If we don't then many extension modules will break. The example module > Modules/xxmodule.c used to allocate using PyObject_New and deallocate > using free(). I believe there are many modules out there that do the > same (or use PyMem_Del, etc). Breaking extensions is not a good idea. After all, these make Python so much fun to work with (since most of the work is usually already done ;-). I do think that we should keep the differentiation between allocating raw memory buffers and space for Python objects. Even though this is not currently used, it clarifies the code somewhat, e.g. to free memory allocated for a Python object you write PyObject_FREE(), for an raw buffer you write PyMem_FREE(). Who knows... perhaps we might want to handle Python object memory blocks differently in the future (e.g. build pymalloc support right into PyObject_NEW() and PyObject_DEL()) while leaving user space memory in the realm of malloc() et al. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From rschaefe@teksystems.com Wed Jan 30 20:41:25 2002 From: rschaefe@teksystems.com (Rebecca Schaefer) Date: Wed, 30 Jan 2002 15:41:25 -0500 Subject: [Python-Dev] Python/Web developer Message-ID: <3C585A75.5831727A@teksystems.com> This is a multi-part message in MIME format. --------------5BF9DEF828C31C37EAF3FA97 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit TEKsystems in Appleton, WI has an opening for a web developer with Python, Zope, html, SQL, UNIX, and Perl experience. It is a long term contract opportunity. Any interested candidates should email Rebecca Schaefer at rschaefe@teksystems.com Thank you, Rebecca --------------5BF9DEF828C31C37EAF3FA97 Content-Type: text/x-vcard; charset=us-ascii; name="rschaefe.vcf" Content-Transfer-Encoding: 7bit Content-Description: Card for Rebecca Schaefer Content-Disposition: attachment; filename="rschaefe.vcf" begin:vcard n:Schaefer;Rebecca tel;fax:920-225-7692 tel;work:888-598-5874 x-mozilla-html:FALSE org: adr:;;;;;; version:2.1 email;internet:rschaefe@teksystems.com title:Senior Recruiter fn:Rebecca Schaefer end:vcard --------------5BF9DEF828C31C37EAF3FA97-- From sdm7g@Virginia.EDU Wed Jan 30 20:42:53 2002 From: sdm7g@Virginia.EDU (Steven Majewski) Date: Wed, 30 Jan 2002 15:42:53 -0500 (EST) Subject: [Python-Dev] next vs darwin In-Reply-To: Message-ID: On 30 Jan 2002, Martin v. Loewis wrote: > > The other thing that's been discussed is adding configure > > support to build with the dlopen compatability libs if > > that is available. > > Can you please explain what that would provide to module users or end > users? Would there be additional modules available that otherwise > wouldn't be available? If not, I don't think that this option should > be provided. dlcompat libs are used by Apple to build Apache and some other programs. The libs are not included in Mac OSX, although the sources are available in the Darwin CVS, and an improved version is distributed on Fink and maybe other places. Since additional libs are required, I would not make that the default. ( unless, since there's already a check for libdl in config, we make it dependent on that. ) The problem is that the current dynload_next is broken, and we've had some problems replicating tests and solutions because, among other problems, of the very poor error reporting in dynload_next, everyone is starting from a differently hacked version of the 2.2 distribution. (The other variable is which modules and packages people are loading.) Reportedly, using the dlcompat libs fixes some problems for some people. Obviously, the best solution would be an even better dynload_darwin that fixes all of the problems. But it the interim, I'ld like to at least get everone debugging from the same baseline. If there's a string objection to adding optional libdl support, I can live without that. Adding it would just make it easier for folks to test that configuration and build. Getting a less broken dynload module is probably more important. -- Steve. From sdm7g@Virginia.EDU Wed Jan 30 23:23:57 2002 From: sdm7g@Virginia.EDU (Steven Majewski) Date: Wed, 30 Jan 2002 18:23:57 -0500 (EST) Subject: [Python-Dev] Apple PublicSource license [was: next vs darwin] In-Reply-To: Message-ID: I did another version of dynload_darwin that took a >10 line function from the dlcompat/dlopen.c code which uses an undocumented (at least in the man pages -- there's probably comments in the Darwin source code) non-public way around the public/private namespace problem we were having with the previous version. I'm waiting for some folks on pythonmac-sig to test it and report back. I'm guessing that this solves the problem without requiring libdl. However it gets into the possible problem of including another license. Could someone who undestands these issues a bit more than I, look at this: Apple Public Source License: http://www.publicsource.apple.com/apsl/ -- Steve BTW: Here's the magic code I added from dlcompat/dlopen.c: ( On the one hand, it's fairly short and trivial. On the other, I wouldn't have had a clue about this without reading the code! ) /* * NSMakePrivateModulePublic() is not part of the public dyld API so we define * it here. The internal dyld function pointer for * __dyld_NSMakePrivateModulePublic is returned so thats all that maters to get * the functionality need to implement the dlopen() interfaces. */ static enum bool NSMakePrivateModulePublic( NSModule module) { static enum bool (*p)(NSModule module) = NULL; if(p == NULL) _dyld_func_lookup("__dyld_NSMakePrivateModulePublic", (unsigned long *)&p); if(p == NULL){ #ifdef DEBUG printf("_dyld_func_lookup of __dyld_NSMakePrivateModulePublic " "failed\n"); #endif return(FALSE); } return(p(module)); } From tim.one@home.com Wed Jan 30 23:49:19 2002 From: tim.one@home.com (Tim Peters) Date: Wed, 30 Jan 2002 18:49:19 -0500 Subject: [Python-Dev] Mixing memory management APIs In-Reply-To: <2mvgdj6aea.fsf@starship.python.net> Message-ID: [NeilS, growing older but wiser, embraces the wisdom of giving up ] [Michael Hudson] > And then change all the current uses of PyObject_Del to the new API? It would mean changing all uses of all memory macros in the core to use new macros. > What would that buy us? The possibility to move to Vladimir's malloc implementation without breaking any extension modules (none: no breakage at all). I want the Python core to use Vladimir's malloc/free, and until the fabled free-threading gets implemented, to use a version that *exploits* the GIL to eliminate malloc/free lock overhead too. We know for a fact that some major extension modules misused the existing memory API (via mismatching "get memory"->"free memory" pairs), and it's so Byzantine and ill-documented that this shouldn't be a surprise. Beyond that, I don't believe we've ever said anything about thread safety wrt the existing memory API, simply because we relied on the platform malloc/free to provide thread safety even in the worst of cases. But if the Python core switches to a gimmick that relies on the GIL, then even extensions that use the current API properly (wrt correct matching pairs) may get into huge trouble if the underlying allocator stops doing its own layer of locking. The intent with new macros would be to spell out the rules. Extensions that wanted to play along could switch, while extensions that ignored the issues would continue to work with the existing seven ways to spell "malloc" (and seven to spell "free"). > ... > I don't what we have is so bad; a helpful tip is that if you're using > the _Free/_FREE/_Malloc/_REALLOC/etc interfaces, stop. That gets rid > of half the problem. But only for extensions that are actively maintained by people who are keen to dig into how they've abused the current API. It's likely easier for the core to move to new macros than to fully debug even one large extension module that's been working so far by luck. a-big-hammer-wouldn't-be-called-that-if-it-weren't-big -ly y'rs - tim From tim.one@home.com Wed Jan 30 23:51:56 2002 From: tim.one@home.com (Tim Peters) Date: Wed, 30 Jan 2002 18:51:56 -0500 Subject: [Python-Dev] Mixing memory management APIs In-Reply-To: <20020130082807.B9393@glacier.arctrix.com> Message-ID: [NeilS] > pymalloc needs to use a completely new set of APIs. The only problem I > see is coming up with names. NEW, MALLOC, REALLOC, RESIZE, and DEL are > all taken. Any suggestions? I liked Aahz's suggestion to start them with "PyMalloc_" well enough. Most of us use editors with word-completion anyway . From neal@metaslash.com Thu Jan 31 00:13:58 2002 From: neal@metaslash.com (Neal Norwitz) Date: Wed, 30 Jan 2002 19:13:58 -0500 Subject: [Python-Dev] Mixing memory management APIs References: Message-ID: <3C588C46.2BF27BBE@metaslash.com> Because of Michael Hudson's request, I tried running Purify --with-pymalloc enabled. The results were a bit surprising: 13664 errors! All the errors were in unicodeobject.c. There were 3 types of errors: Free Memory Reads, Array Bounds Reads, and Unitialized Memory Reads. The line #s were in strange places (e.g., in a function declaration and accessing self->length in an if clause, after it was accessed w/o error). The line #s are primarily: unicodeobject.c:2875, and unicodeobject.c:2214. Has anyone run else used Purify and/or Insure --with-pymalloc? BTW, I test_curses fails: test test_curses crashed -- _curses.error: curs_set() returned ERR Solaris 2.8, Purify 2002. Neal -- Problems (error lines begin with =>) PyUnicode_TranslateCharmap [unicodeobject.c:2214] PyObject *PyUnicode_EncodeASCII(const Py_UNICODE *p, int size, => const char *errors) { PyObject *repr; char *s, *start; split_char [unicodeobject.c:2875] if (end > self->length) end = self->length; if (end < 0) => end += self->length; if (end < 0) end = 0; From tim.one@home.com Thu Jan 31 00:33:48 2002 From: tim.one@home.com (Tim Peters) Date: Wed, 30 Jan 2002 19:33:48 -0500 Subject: [Python-Dev] Odd errors when catching ImportError In-Reply-To: <000701c1a9b5$1bccb5e0$09860cc0@CLENHAM> Message-ID: [Paul Dubois] > ... > I have been running into a number of odd errors caused by code like the > following. The behavior seems to be machine dependent. Which version(s) of Python? (Released, current CVS, all, ...?) > fooflag = 0 > try: > import foo > except ImportError: > fooflag = 1 > > I have had this result in a seg fault upon exit, Does or does not "foo" exist? Or does it segfault both ways? Either way, run Python -vv to get a trace of what it's trying during the import attempt. The last line displayed before the segfault may be a useful clue. You may even discover you're really importing a compiled foo extension module with a hardcoded segfault in module init . > and also when something like this was in file xxx.py inside a package, > and the __init__.py did > > from xxx import fooflag > > I've had it tell me xxx had no attribute fooflag. I added "print > fooflag" at the bottom of the file and it fixed it. That was on a DEC. > On Linux it worked. > > I suppose I should be testing for the ability to import foo some other > way but I don't know what it is. That's "the usual" way to check imports; if it were a widespread problem under any version of Python, I expect we would have heard about it before. If you have useful followups, you should record them in a bug report on SourceForge (Python-Dev is a black hole for bug reports). From andymac@bullseye.apana.org.au Wed Jan 30 21:15:11 2002 From: andymac@bullseye.apana.org.au (Andrew MacIntyre) Date: Thu, 31 Jan 2002 08:15:11 +1100 (EDT) Subject: [Python-Dev] updated patches for OS/2 EMX port In-Reply-To: Message-ID: I've let this lie for a few days to see whether any other comments were forthcoming, but nothing's turned up... On 27 Jan 2002, Martin v. Loewis wrote: > Andrew MacIntyre writes: > > > - Modules/unicodedata.c is affected by a name clash between the internally > > defined _getname() and an EMX routine of the same name defined in > > . The patch renames the internal routine to _getucname() to > > avoid this, but this change may not be acceptable - advice please. > > My advice for renaming things because of name clashes: Always rename > in a way that solves this particular problem for good, by using the Py > prefix (or _Py to further indicate that this is not public API; it's a > static function, anyway). Somebody may have a function _getucname > somewhere, whereas it is really unlikely that people add a Py prefix > to their functions (if they have been following the last 30 years of C > programming). Fair enough. I was trying to minimise stylistic differences in the fix, but if using _Py_getname is the canonical solution, that's easy fixed. > > - Objects/stringobject.c and Objects/unicodeobject.c contain changes to > > handle the EMX runtime library returning "0x" as the prefix for output > > formatted with a "%X" format. > > I'd suggest a different approach here, which does not use #ifdefs: > Instead of testing for the system, test for the bug. Then, if the bug > goes away, or appears on other systems as well, the code will be good. I did it the way I did because there's already code dealing with other brokeness in this area which doesn't solve the EMX issue, and the #ifdef solution minimises the risk of EMX fixes breaking something else which I can't test. At this stage I can't see this bug being fixed in EMX :-( > Once formatting is complete, see whether it put in the right letter, > and fix that in the result buffer if the native sprintf got it wrong. > > If you follow this strategy, you should still add a comment indicating > that this was added for OS/2, to give people an idea where that came > from. Definitely a more general approach, which I'll look at in detail. > Another approach would be to autoconfiscate this particular issue. I'm > in general in favour of autoconf'ed bug tests instead of runtime bug > tests, but people on systems without /bin/sh might feel differently. While there are sh/bash shells for OS/2, they're not standard equipment. Autoconf also has a very spotty record on OS/2, although there are people trying to improve that. > > If there are no unresolvable objections, and approval to apply these > > patches is granted, I propose that the patches be applied as follows:- > > > > Stage 1: the build patch (creates+populates PC/os2emx/) > > Stage 2: the Lib/plat-os2emx/ patch > > Stage 3: the Lib/ and Lib/test/ patches > > Stage 4: the distutils patch > > Stage 5: the Include/, Objects/ and Python/ patches > > Stage 6: the Modules/ patch > > > > I would expect to allow at least 48 hours between stages. > > > > Comments/advice on this proposal also appreciated. > > Sounds good to me (although I'd probably process the "uncritical", > i.e. truly platform-specific parts much more quickly). Who's going to > work with Andrew to integrate this stuff? The last I heard, Guido expected I was going to commit my own patches (after review), so I was allowing time for my initial attempts to commit to be checked by regular builders/testers of the tree before getting to changes that affect non-EMX specific parts of Python. -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au | Snail: PO Box 370 andymac@pcug.org.au | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From tim.one@home.com Thu Jan 31 04:41:13 2002 From: tim.one@home.com (Tim Peters) Date: Wed, 30 Jan 2002 23:41:13 -0500 Subject: [Python-Dev] Thread questionlet In-Reply-To: <3C57D0D6.8010602@tismer.com> Message-ID: If you ask Guido, the only reason to use threads is to do overlapped I/O. And if you come up with a good counter-example, he'll find a way to *call* it overlapped I/O, so there's no opposing him on this . That's clearly a huge reason in practice to use threads, and that reason requires using platform threads (to get true overlap). Another huge reason is to play nice with threaded libraries written in other languages, and again that requires playing along with platform threads. So what most thread users want is what Python gives them: a thin wrapper around native threads, complete with platform quirks. The threading.py module adds *some* sanity to that, providing portable APIs for some important synch primitives, and uniform thread shutdown semantics (as Guido pointed out, when you use threading.py's thread wrappers, when the main thread exits it waits for all (non-daemon) threads to quit). What people seem to ask for most often now is a way for one thread to tell another thread to stop. In practice I've always done this by having each thread poll a "time to stop?" variable from time to time. That's not what people want, though. They want a way to *force* another thread to stop, even if (e.g.) the target thread is stuck in a blocking read, or in the middle of doing an extraordinarily expensive regexp search. There simply isn't a portable way to do that. Java initially spec'ed a way to do it in its thread model, but declared that deprecated after obtaining experience with it: not only did it prove impossible to implement in all cases, but even when it worked, the thread that got killed had no way to leave global invariants in a sane state (e.g., the thread may have had any number of synch gimmicks-- like locks --in various states, and global invariants for synch gimmicks can't tolerate a participant vanishing without both extreme care and a way for a thread to declare itself unstoppable at times). So that's a mess, but that's still what people want. OTOH, they won't want it for long if they get it (just as Java ran screaming from it). I'm not sure the audience for cororoutine-style threads even overlaps. You could try to marry both models, by running coroutine-style threads in a pool of OS threads. Then, e.g., provided you knew a potentially blocking I/O call when you saw one, you could farm it out to one of the real threads. If you can't do that, then I doubt the "real thread" crowd will have any interest in uthreads (or will, but treat them as an entirely distinct facility -- which, for their purposes, they would be). For purposes of computational parallelism (more my background than Guido's -- the idea that you might want to use a thread to avoid blocking on I/O was novel to me ), the global interpreter lock renders Python useless except for prototyping, so there's not much point digging into the hundreds of higher-level parallelism models that have been developed. IOW, uthreads are their own universe. I haven't used them, so don't know what would be useful. What do the current uthread users ask for? That's where I'd start. From mal@lemburg.com Thu Jan 31 09:45:24 2002 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 31 Jan 2002 10:45:24 +0100 Subject: [Python-Dev] updated patches for OS/2 EMX port References: Message-ID: <3C591234.4F3B2FCE@lemburg.com> Andrew MacIntyre wrote: > > On 27 Jan 2002, Martin v. Loewis wrote: > > Andrew MacIntyre writes: > > > > > - Modules/unicodedata.c is affected by a name clash between the internally > > > defined _getname() and an EMX routine of the same name defined in > > > . The patch renames the internal routine to _getucname() to > > > avoid this, but this change may not be acceptable - advice please. > > > > My advice for renaming things because of name clashes: Always rename > > in a way that solves this particular problem for good, by using the Py > > prefix (or _Py to further indicate that this is not public API; it's a > > static function, anyway). Somebody may have a function _getucname > > somewhere, whereas it is really unlikely that people add a Py prefix > > to their functions (if they have been following the last 30 years of C > > programming). > > Fair enough. I was trying to minimise stylistic differences in the fix, > but if using _Py_getname is the canonical solution, that's easy fixed. +1 > > > - Objects/stringobject.c and Objects/unicodeobject.c contain changes to > > > handle the EMX runtime library returning "0x" as the prefix for output > > > formatted with a "%X" format. > > > > I'd suggest a different approach here, which does not use #ifdefs: > > Instead of testing for the system, test for the bug. Then, if the bug > > goes away, or appears on other systems as well, the code will be good. > > I did it the way I did because there's already code dealing with other > brokeness in this area which doesn't solve the EMX issue, and the #ifdef > solution minimises the risk of EMX fixes breaking something else which I > can't test. At this stage I can't see this bug being fixed in EMX :-( I'd go with Martin's suggestion here: there already is code in formatint() which tests for '%#X' adding '0x' or not. This code should be made to handle the special case by testing for it -- who knows: there may be other platforms where this doesn't work as expected either. BTW, could you point me to your patch for this ? Thanks, -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From mwh@python.net Thu Jan 31 10:32:29 2002 From: mwh@python.net (Michael Hudson) Date: 31 Jan 2002 10:32:29 +0000 Subject: [Python-Dev] test_curses In-Reply-To: Neal Norwitz's message of "Wed, 30 Jan 2002 19:13:58 -0500" References: <3C588C46.2BF27BBE@metaslash.com> Message-ID: <2mlmeex136.fsf_-_@starship.python.net> Neal Norwitz writes: > BTW, I test_curses fails: > test test_curses crashed -- _curses.error: curs_set() returned ERR Hmm, yes I get that too (I'd commented it out, because I thought it had something to do with the crashes, but it didn't AFAICT). It's very strange. I tried wrestling with gdb to find out how it was failing, but didn't get very far. It's hard to see how curs_set can fail. Maybe I need to build ncurses from source and link to that. Cheers, M. -- ARTHUR: Yes. It was on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying "Beware of the Leopard". -- The Hitch-Hikers Guide to the Galaxy, Episode 1 From aahz@rahul.net Thu Jan 31 10:44:39 2002 From: aahz@rahul.net (Aahz Maruch) Date: Thu, 31 Jan 2002 02:44:39 -0800 (PST) Subject: [Python-Dev] Thread questionlet In-Reply-To: from "Tim Peters" at Jan 30, 2002 11:41:13 PM Message-ID: <20020131104439.13FA4E8D1@waltz.rahul.net> Tim Peters wrote: > > For purposes of computational parallelism (more my background than > Guido's -- the idea that you might want to use a thread to avoid > blocking on I/O was novel to me ), the global interpreter lock > renders Python useless except for prototyping, so there's not much > point digging into the hundreds of higher-level parallelism models > that have been developed. Well, maybe. I'm still hoping to prove you at least partly wrong one of these years. ;-) (The long-term plan for my BCD module is to turn it into a C extension that releases the GIL. If that's successful, I'll start working on ways to have Numeric release the GIL.) -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista We must not let the evil of a few trample the freedoms of the many. From mwh@python.net Thu Jan 31 11:18:30 2002 From: mwh@python.net (Michael Hudson) Date: 31 Jan 2002 11:18:30 +0000 Subject: [Python-Dev] test_curses In-Reply-To: Michael Hudson's message of "31 Jan 2002 10:32:29 +0000" References: <3C588C46.2BF27BBE@metaslash.com> <2mlmeex136.fsf_-_@starship.python.net> Message-ID: <2mpu3qlqex.fsf@starship.python.net> Michael Hudson writes: > Neal Norwitz writes: > > > BTW, I test_curses fails: > > test test_curses crashed -- _curses.error: curs_set() returned ERR > > Hmm, yes I get that too (I'd commented it out, because I thought it > had something to do with the crashes, but it didn't AFAICT). > > It's very strange. I tried wrestling with gdb to find out how it was > failing, but didn't get very far. > > It's hard to see how curs_set can fail. Maybe I need to build ncurses > from source and link to that. Heh, well I worked that one out. The terminfo database didn't contain entries for cursor visibility for the $TERM I had (xterm-color, for some forgotten reason). Setting it to xterm made test_curses.py pass for me. much-python-dev-noise-not-much-content-from-me-ly y'rs M. -- That's why the smartest companies use Common Lisp, but lie about it so all their competitors think Lisp is slow and C++ is fast. (This rumor has, however, gotten a little out of hand. :) -- Erik Naggum, comp.lang.lisp From jeremy@alum.mit.edu Thu Jan 31 08:25:54 2002 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Thu, 31 Jan 2002 03:25:54 -0500 Subject: [Python-Dev] opcode performance measurements Message-ID: <15448.65426.470757.274910@gondolin.digicool.com> I've made some simple measurements of how long opcodes take to execute and how long it takes to go around the mainloop, using the Pentim timestamp counter, which measures processor cycles. The results aren't particularly surprising, but they provide some empirical validation of what we've believed all along. I don't have time to go into all the gory details here, though I plan to at Spam 10 developers day next week. I put together a few Web pages that summarize the data I've collected on some simple benchmarks: http://www.zope.org/Members/jeremy/CurrentAndFutureProjects/PerformanceMeasurements Comments and questions are welcome. I've got a little time to do more measurement and analysis before devday. Jeremy From skip@pobox.com Thu Jan 31 18:37:16 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 31 Jan 2002 12:37:16 -0600 Subject: [Python-Dev] Re: opcode performance measurements In-Reply-To: <15448.65426.470757.274910@gondolin.digicool.com> References: <15448.65426.470757.274910@gondolin.digicool.com> Message-ID: <15449.36572.854311.657565@beluga.mojam.com> Jeremy> I've made some simple measurements of how long opcodes take to Jeremy> execute and how long it takes to go around the mainloop ... Jeremy> Comments and questions are welcome. I've got a little time to Jeremy> do more measurement and analysis before devday. Interesting results. I've been working on my {TRACK,UNTRACK}_GLOBAL opcode implementations. I have an optimizer filter that sets up tracking for all LOAD_GLOBAL,{LOAD_ATTR}* combinations. It's still not quite working and will only be a proof of concept by devday if I do get it working, but I expect most of these expensive opcode combinations to collapse into a LOAD_FAST, with the addition of a TRACK_GLOBAL/UNTRACK_GLOBAL pair executed at function start and end, respectively. Skip From jepler@unpythonic.dhs.org Thu Jan 31 19:48:03 2002 From: jepler@unpythonic.dhs.org (Jeff Epler) Date: Thu, 31 Jan 2002 13:48:03 -0600 Subject: [Python-Dev] Re: opcode performance measurements In-Reply-To: <15449.36572.854311.657565@beluga.mojam.com> References: <15448.65426.470757.274910@gondolin.digicool.com> <15449.36572.854311.657565@beluga.mojam.com> Message-ID: <20020131134802.A5269@unpythonic.dhs.org> On Thu, Jan 31, 2002 at 12:37:16PM -0600, Skip Montanaro wrote: > Interesting results. I've been working on my {TRACK,UNTRACK}_GLOBAL opcode > implementations. I have an optimizer filter that sets up tracking for all > LOAD_GLOBAL,{LOAD_ATTR}* combinations. It's still not quite working and > will only be a proof of concept by devday if I do get it working, but I > expect most of these expensive opcode combinations to collapse into a > LOAD_FAST, with the addition of a TRACK_GLOBAL/UNTRACK_GLOBAL pair executed > at function start and end, respectively. Won't there be code that this slows down? For instance, the code generated by print "f = lambda: 0" print "def g():" print "\tif f():" # prevent optimization of 'if 0:' print "\t\tx = []" for i in range(10000): print "\t\tx.append(global_%d)" % i print "\t\treturn x" print "\treturn []" (10001 TRACK_GLOBALs, one LOAD_GLOBAL) not to mention, will it even work? TRACK_GLOBAL will have to make special note of globals that didn't exist yet when the function prologue is executed, and either not subsequently execute the load as a LOAD_FAST or else have a special value that causes the same NameError "global name 'global_666' is not defined" message, not an UnboundLocalError... The latter sounds easy enough to solve, but how do you make sure that this optimization is never a pessimization (aside from sending programmers such as myself to the retraining camps of the PSU)? Jeff PS Hey, that's remarkable .. usually people get unexpectedly cut off when they try to mentio From jeremy@alum.mit.edu Thu Jan 31 10:14:28 2002 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Thu, 31 Jan 2002 05:14:28 -0500 Subject: [Python-Dev] Re: opcode performance measurements In-Reply-To: <15449.36572.854311.657565@beluga.mojam.com> References: <15448.65426.470757.274910@gondolin.digicool.com> <15449.36572.854311.657565@beluga.mojam.com> Message-ID: <15449.6404.210447.679063@gondolin.digicool.com> >>>>> "SM" == Skip Montanaro writes: SM> Interesting results. I've been working on my SM> {TRACK,UNTRACK}_GLOBAL opcode implementations. I have an SM> optimizer filter that sets up tracking for all SM> LOAD_GLOBAL,{LOAD_ATTR}* combinations. It's still not quite SM> working and will only be a proof of concept by devday if I do SM> get it working, but I expect most of these expensive opcode SM> combinations to collapse into a LOAD_FAST, with the addition of SM> a TRACK_GLOBAL/UNTRACK_GLOBAL pair executed at function start SM> and end, respectively. I won't have any implementation done at all, but should have finished the design for LOAD_FAST-style access to globals and module attributes. I also have some ideas about Python bytecode specializer that would work essentially like a JIT but generated specialized bytecode instead of machine code. Jeremy PS Skip-- Sorry the PEP isn't clear, but the only dictionary lookups that need to occur are at function creation time. MAKE_FUNCTION would need to lookup the offsets of the globals used by the functions, so that a LOAD_FAST_GLOBAL opcode would take an int argument. From jepler@unpythonic.dhs.org Thu Jan 31 20:17:25 2002 From: jepler@unpythonic.dhs.org (Jeff Epler) Date: Thu, 31 Jan 2002 14:17:25 -0600 Subject: [Python-Dev] Re: opcode performance measurements In-Reply-To: <15449.6404.210447.679063@gondolin.digicool.com> References: <15448.65426.470757.274910@gondolin.digicool.com> <15449.36572.854311.657565@beluga.mojam.com> <15449.6404.210447.679063@gondolin.digicool.com> Message-ID: <20020131141725.B5269@unpythonic.dhs.org> On Thu, Jan 31, 2002 at 05:14:28AM -0500, Jeremy Hylton wrote: > PS Skip-- Sorry the PEP isn't clear, but the only dictionary lookups > that need to occur are at function creation time. MAKE_FUNCTION would > need to lookup the offsets of the globals used by the functions, so > that a LOAD_FAST_GLOBAL opcode would take an int argument. So how does this work for mutually-recursive functions? def f(x): if x==1: return 1 return g(x) def g(x): return x * f(x-1) can f not optimize the load of the global g into a LOAD_FAST_GLOBAL? Jeff PS Which PEP? I only see 266 From skip@pobox.com Thu Jan 31 20:27:59 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 31 Jan 2002 14:27:59 -0600 Subject: [Python-Dev] Re: opcode performance measurements In-Reply-To: <20020131134802.A5269@unpythonic.dhs.org> References: <15448.65426.470757.274910@gondolin.digicool.com> <15449.36572.854311.657565@beluga.mojam.com> <20020131134802.A5269@unpythonic.dhs.org> Message-ID: <15449.43215.961559.895079@beluga.mojam.com> Jeff> Won't there be code that this slows down? For instance, the code Jeff> generated by ... Sure, there will be code that slows down. That's why I said what I am working on is a proof of concept. Right now, each function the optimizer operates on converts it to the equivalent of TRACK_GLOBAL x.foo TRACK_GLOBAL y.bar TRACK_GLOBAL z try: original function body using x.foo, y.bar and z finally: UNTRACK_GLOBAL z UNTRACK_GLOBAL y.bar UNTRACK_GLOBAL x.foo There are no checks for obvious potential problems at the moment. Such problems include (but are not limited to): * Only track globals that are accessed in loops. This would eliminate your corner case and should be easily handled (only work between SETUP_LOOP and its jump target). * Only track globals when there are <= 256 globals (half an oparg - the other half being an index into the fastlocals array). This would also cure your problem. * Only track globals that are valid at the start of function execution, or defer tracking setup until they are. This can generally be avoided by not tracking globals that are written during the function's execution, but other safeguards will probably be necessary to insure that it works properly. Jeff> ... how do you make sure that this optimization is never a Jeff> pessimization ... I expect in the majority of cases either my idea or Jeremy's will be a net win, especially after seeing his timing data. I'm willing to accept that in some situations the code will run slower. I'm confident they will be a small minority. Tim Peters can construct cases where dicts perform badly. Does that mean Python shouldn't have dicts? ;-) Skip From skip@pobox.com Thu Jan 31 20:29:47 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 31 Jan 2002 14:29:47 -0600 Subject: [Python-Dev] Re: opcode performance measurements In-Reply-To: <20020131141725.B5269@unpythonic.dhs.org> References: <15448.65426.470757.274910@gondolin.digicool.com> <15449.36572.854311.657565@beluga.mojam.com> <15449.6404.210447.679063@gondolin.digicool.com> <20020131141725.B5269@unpythonic.dhs.org> Message-ID: <15449.43323.97437.978245@beluga.mojam.com> Jeff> PS Which PEP? I only see 266 Mine is 266. Jeremy's is 267. Skip From skip@pobox.com Thu Jan 31 20:39:10 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 31 Jan 2002 14:39:10 -0600 Subject: [Python-Dev] What's up w/ CVS? Message-ID: <15449.43886.970376.594507@beluga.mojam.com> Anyone have any idea what's going on w/ SF CVS? I tried to "cvs up" earlier and it just hung. I just tried again now and got the dreaded "WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!" from ssh. Skip From Samuele Pedroni" <15449.36572.854311.657565@beluga.mojam.com> <15449.6404.210447.679063@gondolin.digicool.com> Message-ID: <00c801c1aa96$2b521320$6d94fea9@newmexico> Hi. Q about PEP 267 Does the PEP mechanims adress only import a use a.x cases. How does it handle things like import a.b use a.b.x Thanks, Samuele Pedroni. From jeremy@alum.mit.edu Thu Jan 31 11:02:17 2002 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Thu, 31 Jan 2002 06:02:17 -0500 Subject: [Python-Dev] Re: opcode performance measurements In-Reply-To: <20020131141725.B5269@unpythonic.dhs.org> References: <15448.65426.470757.274910@gondolin.digicool.com> <15449.36572.854311.657565@beluga.mojam.com> <15449.6404.210447.679063@gondolin.digicool.com> <20020131141725.B5269@unpythonic.dhs.org> Message-ID: <15449.9273.809378.72361@gondolin.digicool.com> >>>>> "JE" == Jeff Epler writes: JE> On Thu, Jan 31, 2002 at 05:14:28AM -0500, Jeremy Hylton wrote: >> PS Skip-- Sorry the PEP isn't clear, but the only dictionary >> lookups that need to occur are at function creation time. >> MAKE_FUNCTION would need to lookup the offsets of the globals >> used by the functions, so that a LOAD_FAST_GLOBAL opcode would >> take an int argument. JE> So how does this work for mutually-recursive functions? JE> def f(x): JE> if x==1: return 1 return g(x) JE> def g(x): return x * f(x-1) JE> can f not optimize the load of the global g into a JE> LOAD_FAST_GLOBAL? JE> PS Which PEP? I only see 266 PEP 267. (They gesture at each other.) So you've got a module with two globals f() and g(). They're stored in slots 0 and 1 of the module globals array. When f() and g() are compiled, the symbol table for the module can note the location of f() and g() and that f() and g() contain references to globals. Instead of emitting LOAD_GLOBAL "f" in g(), you can emit LOAD_GLOBAL 0 ("f"). The complication here is that a code object isn't tied to a single module. It would be possible to to exec f.func_code in some other environment where "g" was not stored in the module global array. The dictionary lookups may occur in MAKE_FUNCTION in order to verify that the code object and the module object agree on the layout of the globals array. Jeremy From tim.one@home.com Thu Jan 31 20:52:47 2002 From: tim.one@home.com (Tim Peters) Date: Thu, 31 Jan 2002 15:52:47 -0500 Subject: [Python-Dev] What's up w/ CVS? In-Reply-To: <15449.43886.970376.594507@beluga.mojam.com> Message-ID: [Skip] > Anyone have any idea what's going on w/ SF CVS? I tried to "cvs > up" earlier and it just hung. I just tried again now and got the > dreaded "WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! IT IS > POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!" from ssh. I get this too. SF is also showing other signs of flakiness today. Take a nap . From barry@zope.com Thu Jan 31 20:59:02 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 31 Jan 2002 15:59:02 -0500 Subject: [Python-Dev] What's up w/ CVS? References: <15449.43886.970376.594507@beluga.mojam.com> Message-ID: <15449.45078.573517.218139@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: SM> Anyone have any idea what's going on w/ SF CVS? I tried to SM> "cvs up" earlier and it just hung. I just tried again now and SM> got the dreaded "WARNING: REMOTE HOST IDENTIFICATION HAS SM> CHANGED! IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING SM> NASTY!" from ssh. Same here. No word from the SF web pages. napping-ly y'rs, -Barry From jeremy@alum.mit.edu Thu Jan 31 11:12:26 2002 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Thu, 31 Jan 2002 06:12:26 -0500 Subject: [Python-Dev] Re: opcode performance measurements In-Reply-To: <00c801c1aa96$2b521320$6d94fea9@newmexico> References: <15448.65426.470757.274910@gondolin.digicool.com> <15449.36572.854311.657565@beluga.mojam.com> <15449.6404.210447.679063@gondolin.digicool.com> <00c801c1aa96$2b521320$6d94fea9@newmexico> Message-ID: <15449.9882.393698.701265@gondolin.digicool.com> >>>>> "SP" == Samuele Pedroni writes: SP> Hi. Q about PEP 267 Does the PEP mechanims adress only SP> import a SP> use a.x SP> cases. How does it handle things like SP> import a.b SP> use a.b.x You're a smart guy, can you tell me? :-). Seriously, I haven't gotten that far. import mod.sub creates a binding for "mod" in the global namespace The compiler can detect that the import statement is a package import -- and mark "mod.sub" as a candidate for optimization. A use of "mod.sub.attr" in function should be treated just as "mod.attr". The globals array (dict-list hybrid, technically) has the publicly visible binding for "mod" but also has an internal binding for "mod.sub" and "mod.sub.attr". Every module or submodule attribute in a function gets an internal slot in the globals. The internal slot gets initialized the first time it is used and then shared by all the functions in the module. So I think this case isn't special enough to need a special case. Jeremy From skip@pobox.com Thu Jan 31 21:03:51 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 31 Jan 2002 15:03:51 -0600 Subject: [Python-Dev] distutils & stderr Message-ID: <15449.45367.46625.175691@beluga.mojam.com> If I could "cvs up" I would submit a patch, but in the meantime, is there any good reason that distutils shouldn't write its output to stderr? I'm using PyInline to execute a little bit of C code that returns some information about the system to the calling Python code. This code then sends some output to stdout. I've patched my local directory tree so that distutils sends its output to sys.stderr. Is there some overriding reason distutils messages should go to sys.stdout? BTW, Python + PyInline makes a hell of a lot easier to understand configure script... ;-) Skip From barry@zope.com Thu Jan 31 21:07:48 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 31 Jan 2002 16:07:48 -0500 Subject: [Python-Dev] distutils & stderr References: <15449.45367.46625.175691@beluga.mojam.com> Message-ID: <15449.45604.994144.291947@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: SM> If I could "cvs up" I would submit a patch SF seems happy again. From tim.one@home.com Thu Jan 31 21:17:25 2002 From: tim.one@home.com (Tim Peters) Date: Thu, 31 Jan 2002 16:17:25 -0500 Subject: [Python-Dev] distutils & stderr In-Reply-To: <15449.45367.46625.175691@beluga.mojam.com> Message-ID: [Skip Montanaro] > If I could "cvs up" I would submit a patch, but in the meantime, is there > any good reason that distutils shouldn't write its output to stderr? Win9X (command.com) users can't redirect stderr, and the DOS box there has a 50-line maximum output history. So stuff going to stderr is often lost forever. stdout can be redirected. I don't know whether disutils had that in mind, but it is "a reason" to leave it alone. > I'm using PyInline to execute a little bit of C code that returns some > information about the system to the calling Python code. This code then > sends some output to stdout. If there's a connection between this and disutils, it's not apparent from what you wrote. From skip@pobox.com Thu Jan 31 21:16:27 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 31 Jan 2002 15:16:27 -0600 Subject: [Python-Dev] Re: opcode performance measurements In-Reply-To: <15449.9882.393698.701265@gondolin.digicool.com> References: <15448.65426.470757.274910@gondolin.digicool.com> <15449.36572.854311.657565@beluga.mojam.com> <15449.6404.210447.679063@gondolin.digicool.com> <00c801c1aa96$2b521320$6d94fea9@newmexico> <15449.9882.393698.701265@gondolin.digicool.com> Message-ID: <15449.46123.751909.428436@beluga.mojam.com> SP> cases. How does it handle things like SP> import a.b SP> use a.b.x Jeremy> You're a smart guy, can you tell me? :-). Seriously, I haven't Jeremy> gotten that far. My stuff does handle this, as long as the first name is global. It just gobbles up all LOAD_GLOBALS and any immediately following LOAD_ATTRs. For instance, this trivial function: def f(): return distutils.core.setup compiles to: 0 LOAD_GLOBAL 0 (distutils) 3 LOAD_ATTR 1 (core) 6 LOAD_ATTR 2 (setup) 9 RETURN_VALUE 10 LOAD_CONST 0 (None) 13 RETURN_VALUE My TrackGlobalOptimizer class currently transforms this to 0 SETUP_FINALLY 11 (to 14) 3 TRACK_GLOBAL 3 (distutils.core.setup, distutils.core.setup) 6 POP_BLOCK 7 LOAD_CONST 0 (None) 10 LOAD_FAST 0 (distutils.core.setup) 13 RETURN_VALUE >> 14 UNTRACK_GLOBAL 3 (distutils.core.setup, distutils.core.setup) 17 END_FINALLY 18 LOAD_CONST 0 (None) 21 RETURN_VALUE which is obviously not an improvement because distutils.core.setup is only accessed once. As people make more use of packages, such multiple attribute loads might become more common. Skip From skip@pobox.com Thu Jan 31 21:33:37 2002 From: skip@pobox.com (Skip Montanaro) Date: Thu, 31 Jan 2002 15:33:37 -0600 Subject: [Python-Dev] distutils & stderr In-Reply-To: References: <15449.45367.46625.175691@beluga.mojam.com> Message-ID: <15449.47153.267248.439654@beluga.mojam.com> Tim> Win9X (command.com) users can't redirect stderr, and the DOS box Tim> there has a 50-line maximum output history. So stuff going to Tim> stderr is often lost forever. stdout can be redirected. I don't Tim> know whether disutils had that in mind, but it is "a reason" to Tim> leave it alone. Perhaps it would be friendlier if all distutils messages were hidden in "if verbose:" statements (many already are). PyInline could then dial down the verbosity before calling distutils. >> I'm using PyInline to execute a little bit of C code that returns >> some information about the system to the calling Python code. This >> code then sends some output to stdout. Tim> If there's a connection between this and disutils, it's not Tim> apparent from what you wrote. Sorry about the missing link. PyInline uses distutils to compile the C code. How PyInline does its think doesn't really matter to me, so I'm not going to be interested in distutils' messages. Skip From tim.one@home.com Thu Jan 31 21:31:37 2002 From: tim.one@home.com (Tim Peters) Date: Thu, 31 Jan 2002 16:31:37 -0500 Subject: [Python-Dev] Re: opcode performance measurements In-Reply-To: <15449.43215.961559.895079@beluga.mojam.com> Message-ID: [Skip Montanaro] > ... > Tim Peters can construct cases where dicts perform badly. Does that mean > Python shouldn't have dicts? ;-) I've thought about that hard over the years. The answer is no . From tim.one@home.com Thu Jan 31 21:54:00 2002 From: tim.one@home.com (Tim Peters) Date: Thu, 31 Jan 2002 16:54:00 -0500 Subject: [Python-Dev] Thread questionlet In-Reply-To: <20020131104439.13FA4E8D1@waltz.rahul.net> Message-ID: [Tim] > For purposes of computational parallelism ... the global interpreter > lock> renders Python useless except for prototyping, so there's not much > point digging into the hundreds of higher-level parallelism models > that have been developed. [Aahz] > Well, maybe. I'm still hoping to prove you at least partly wrong one of > these years. ;-) WRT higher-level parallelism models, you already have in a small way, by your good championing of the Queue module. Queue-based approaches are a step above the morass of low-level home-grown locking protocols people routinely screw up; it's almost *hard* to screw up a Queue-based approach. The GIL issue is distinct, and it plainly stops computational parallelism from doing any good so long as we're talking about Python code. > (The long-term plan for my BCD module is to turn it into a C extension > that releases the GIL. Well, that's not Python code. It's unclear whether it will actually help: Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS aren't free, and a typical BCD calculation may be so cheap that it's a net loss to release and reacquire the GIL across one. Effective use of fine-grained parallelism usually requires something cheaper to build on, like very lightweight critical sections mediating otherwise free-running threads. > If that's successful, I'll start working on ways to have Numeric release > the GIL.) I expect that's more promising because matrix ops are much coarser-grained, but also much harder to do safely: BCD objects are immutable (IIRC), so a routine crunching one doesn't have to worry about another thread mutating it midstream if the GIL is released. A Numeric array probably does have to worry about that. From Jack.Jansen@oratrix.nl Thu Jan 31 21:55:20 2002 From: Jack.Jansen@oratrix.nl (Jack Jansen) Date: Thu, 31 Jan 2002 22:55:20 +0100 Subject: [Python-Dev] Fwd: [Pythonmac-SIG] sys.exit() functionality Message-ID: <38A3EBA8-1695-11D6-9DB9-003065517236@oratrix.nl> This discussion started on pythonmac-SIG, but someone suggested that it isn't really a MacPython-specific issue (even though the implementation will be different for MacPython from unix-Python). Any opinions? Begin forwarded message: > From: Martin Miller > Date: Wed Jan 30, 2002 08:14:13 PM Europe/Amsterdam > To: pythonmac-sig@python.org > Subject: Re: [Pythonmac-SIG] sys.exit() functionality > > On Wed, 30 Jan 2002 15:29:21 +0100, Jack Jansen wrote: >> >> On Tuesday, January 29, 2002, at 08:54 , Jon Bradley wrote: >> >>> hey all, >>> >>> In embedded Python - why does sys.exit() quit out of the application >>> that's >>> embedding the interpreter? Is there any way to trap or >>> disregard this? >>> >>> If a user creates an application with Python and runs it through the >>> embedded interpreter, calling quit or exit on the Python application >>> itself >>> is more than ok, but allowing it to force out of the parent >>> application >>> isn't. >> >> Sounds reasonable. How about a routine PyMac_SetExitFunc() that you >> could call to set your own exit function, (similar to >> PyMac_SetConsoleHandler())? MacPython would then do all it's normal >> cleanup, but at the very end call your routine in stead of exit(). > > With an approach like the above, wouldn't it be better to have a > platform-independent way of defining a custom exit function, > rather than > calling a Mac-only system function -- or is this whole thing only an > issue with MacPython embedding? > > Martin > > _______________________________________________ > Pythonmac-SIG maillist - Pythonmac-SIG@python.org > http://mail.python.org/mailman/listinfo/pythonmac-sig > -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From aahz@rahul.net Thu Jan 31 22:07:30 2002 From: aahz@rahul.net (Aahz Maruch) Date: Thu, 31 Jan 2002 14:07:30 -0800 (PST) Subject: [Python-Dev] opcode performance measurements In-Reply-To: <15448.65426.470757.274910@gondolin.digicool.com> from "Jeremy Hylton" at Jan 31, 2002 03:25:54 AM Message-ID: <20020131220731.5EFD1E8C6@waltz.rahul.net> Jeremy Hylton wrote: > > I've made some simple measurements of how long opcodes take to execute > and how long it takes to go around the mainloop, using the Pentim > timestamp counter, which measures processor cycles. > > The results aren't particularly surprising, but they provide some > empirical validation of what we've believed all along. I don't have > time to go into all the gory details here, though I plan to at > Spam 10 developers day next week. My suggestion WRT SET_LINENO is to encourage the use of python -O and PYTHONOPTIMIZE. -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista We must not let the evil of a few trample the freedoms of the many. From jeremy@alum.mit.edu Thu Jan 31 12:34:28 2002 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Thu, 31 Jan 2002 07:34:28 -0500 Subject: [Python-Dev] opcode performance measurements In-Reply-To: <20020131220731.5EFD1E8C6@waltz.rahul.net> References: <15448.65426.470757.274910@gondolin.digicool.com> <20020131220731.5EFD1E8C6@waltz.rahul.net> Message-ID: <15449.14804.850939.223697@gondolin.digicool.com> >>>>> "AM" == Aahz Maruch writes: AM> My suggestion WRT SET_LINENO is to encourage the use of python AM> -O and PYTHONOPTIMIZE. Vladimir submitted a patch long ago to dynamically recompile bytecode to add/remove SET_LINENO as needed. I find that approach much more appealing, because you don't have to pay the SET_LINENO penalty just because there's some chance you'd want to connect with a debugger. A long running server process is the prime use case; it benefits from -O but may need to be debugged. Jeremy From tim.one@home.com Thu Jan 31 22:30:22 2002 From: tim.one@home.com (Tim Peters) Date: Thu, 31 Jan 2002 17:30:22 -0500 Subject: [Python-Dev] opcode performance measurements In-Reply-To: <20020131220731.5EFD1E8C6@waltz.rahul.net> Message-ID: [Aahz] > My suggestion WRT SET_LINENO is to encourage the use of python -O and > PYTHONOPTIMIZE. What SET_LINENO does isn't even used in normal Python operation anymore (line numbers in tracebacks are obtained via a different means, the co_lnotab member of PyCodeObjects). They're needed now only to call back to user-supplied tracing routines, and that's rarely needed. The Python debugger is the most visible example of a tool that uses the line tracing hook. There are others way to get that to work, but they require real thought and effort to implement. There's a patch on SourceForge (IIRC, from Vladimir) that may have worked at one time, but nobody has picked it up (I tried to for 2.2, but couldn't make time for it then; I don't expect to have time for it for 2.3 either, alas). From Jack.Jansen@oratrix.nl Thu Jan 31 22:35:54 2002 From: Jack.Jansen@oratrix.nl (Jack Jansen) Date: Thu, 31 Jan 2002 23:35:54 +0100 Subject: [Python-Dev] next vs darwin In-Reply-To: Message-ID: On Wednesday, January 30, 2002, at 09:42 PM, Steven Majewski wrote: > dlcompat libs are used by Apple to build Apache and some other > programs. > The libs are not included in Mac OSX, although the sources are > available > in the Darwin CVS, and an improved version is distributed on Fink and > maybe other places. Since additional libs are required, I would not > make that the default. ( unless, since there's already a check for > libdl in config, we make it dependent on that. ) > > The problem is that the current dynload_next is broken, and we've > had some problems replicating tests and solutions because, among other > problems, of the very poor error reporting in dynload_next, everyone > is starting from a differently hacked version of the 2.2 distribution. > (The other variable is which modules and packages people are loading.) > > Reportedly, using the dlcompat libs fixes some problems for > some people. I'm not too thrilled with dlcompat. First and foremost, it fixes some problems for some people but may introduce problems for others (if I understand correctly). And then there's the issue of it not being part of the base MacOSX distribution. I now have a dynload_next.c (that I'll check in tomorrow) that can behave in two ways based on a #define. With the define off it loads every extension module in a separate namespace, i.e. two independent modules can never break each other by supplying external symbols the other module expected to load from a completely different place. With the define on it loads all extension modules into the application namespace. Some people want this (despite the problems sketched above) because they have modules that refer to external symbols defined in modules that have been loaded earlier (and I assume there's magic that ensures their modules are loaded in the right order). While I think this is an accident waiting to happen [*] the latter behaviour is more-or-less the standard unix behaviour, so it should probably be supportable in some way. I prefer the new (OSX 10.1) preferred Apple way of linking plugins (which is also the common way to do so on all other non-unix platforms) where the plugin has to be linked against the application and dynamic libraries it is going to be plugged into, so none of this dynamic behaviour goes on. [*] I know of two cases where this already happened: both the curses library and the SGI gl library defined a function clear(), so you were hosed when you used both in the same Python script. And the SGI compression library contains a private version of libjpeg with no symbol renaming, so if you used the cl module and a module which linked against the normal libjpeg you were also hosed. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From jepler@unpythonic.dhs.org Thu Jan 31 22:54:52 2002 From: jepler@unpythonic.dhs.org (jepler@unpythonic.dhs.org) Date: Thu, 31 Jan 2002 16:54:52 -0600 Subject: [Python-Dev] Re: opcode performance measurements In-Reply-To: <15449.9273.809378.72361@gondolin.digicool.com> References: <15448.65426.470757.274910@gondolin.digicool.com> <15449.36572.854311.657565@beluga.mojam.com> <15449.6404.210447.679063@gondolin.digicool.com> <20020131141725.B5269@unpythonic.dhs.org> <15449.9273.809378.72361@gondolin.digicool.com> Message-ID: <20020131165451.A987@unpythonic.dhs.org> On Thu, Jan 31, 2002 at 06:02:17AM -0500, Jeremy Hylton wrote: > JE> can f not optimize the load of the global g into a > JE> LOAD_FAST_GLOBAL? > > So you've got a module with two globals f() and g(). They're stored > in slots 0 and 1 of the module globals array. When f() and g() are > compiled, the symbol table for the module can note the location of f() > and g() and that f() and g() contain references to globals. Instead > of emitting LOAD_GLOBAL "f" in g(), you can emit LOAD_GLOBAL 0 ("f"). But isn't what happens in this module something like LOAD_CONST MAKE_FUNCTION STORE_GLOBAL 0 (f) LOAD_CONST MAKE_FUNCTION STORE_GLOBAL 1 (g) so if you convert LOAD_GLOBAL into LOAD_FAST_GLOBAL when you MAKE_FUNCTION on code1, there is not yet a "g" in the dlict. Are you populating the "names" part of the dlict as an earlier "pass" of module compilation, then? So the optimization doesn't apply if I create the globals from within a function? (Of course, in that case it would work if I set the attributes to 'None' in the module scope, right?): def make_fg(): global f, g def f(x): pass def g(x): pass Jeff From barry@zope.com Thu Jan 31 22:52:43 2002 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 31 Jan 2002 17:52:43 -0500 Subject: [Python-Dev] Attention Mailman list administrators Message-ID: <15449.51899.451313.87015@anthem.wooz.org> You will soon notice (if you haven't already) that your list admin passwords on mail.python.org are broken. This happened due to an upgrade of the version of Python running on that system. The old list passwords can't be recovered, so they have to be reset. List administrators can contact me to get this done. If you know the old password, send it to me and I'll reset the list to it. Otherwise, let me know and I'll generate a new password for you. Sorry for the inconvenience, -Barry From sdm7g@Virginia.EDU Thu Jan 31 22:55:22 2002 From: sdm7g@Virginia.EDU (Steven Majewski) Date: Thu, 31 Jan 2002 17:55:22 -0500 (EST) Subject: [Python-Dev] next vs darwin In-Reply-To: Message-ID: On Thu, 31 Jan 2002, Jack Jansen wrote: > I'm not too thrilled with dlcompat. First and foremost, it fixes > some problems for some people but may introduce problems for > others (if I understand correctly). And then there's the issue > of it not being part of the base MacOSX distribution. > > I now have a dynload_next.c (that I'll check in tomorrow) that > can behave in two ways based on a #define. > > With the define off it loads every extension module in a > separate namespace, i.e. two independent modules can never break > each other by supplying external symbols the other module > expected to load from a completely different place. > > With the define on it loads all extension modules into the > application namespace. [...] Did you see the version I posted a day or two ago: If I fixed up the #ifdef macros, you could compile that three ways (at least): Global public symbols, Private Symbols, or the dlcompat trick. ( But it uses that magic hook into the non-public API from dlcompat. ) My main requirement is the better error reporting. -- Steve From jeremy@alum.mit.edu Thu Jan 31 13:09:59 2002 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Thu, 31 Jan 2002 08:09:59 -0500 Subject: [Python-Dev] Re: opcode performance measurements In-Reply-To: <20020131165451.A987@unpythonic.dhs.org> References: <15448.65426.470757.274910@gondolin.digicool.com> <15449.36572.854311.657565@beluga.mojam.com> <15449.6404.210447.679063@gondolin.digicool.com> <20020131141725.B5269@unpythonic.dhs.org> <15449.9273.809378.72361@gondolin.digicool.com> <20020131165451.A987@unpythonic.dhs.org> Message-ID: <15449.16935.134346.826927@gondolin.digicool.com> >>>>> "JE" == jepler writes: JE> But isn't what happens in this module something like JE> LOAD_CONST MAKE_FUNCTION STORE_GLOBAL 0 (f) JE> LOAD_CONST MAKE_FUNCTION STORE_GLOBAL 1 (g) JE> so if you convert LOAD_GLOBAL into LOAD_FAST_GLOBAL when you JE> MAKE_FUNCTION on code1, there is not yet a "g" in the dlict. JE> Are you populating the "names" part of the dlict as an earlier JE> "pass" of module compilation, then? Yes. The compiler can do a pretty good job of establishing all the globals in a module at compile time. When a module is loaded, the interpreter would allocate space for all the expected globals. JE> "pass" of module compilation, then? So the optimization doesn't JE> apply if I create the globals from within a function? It still applies. The function is compiled at the same time as the module, so the module symbol table can account for globals assigned to only in functions. The harder cases are much more dynamic -- exec using a module's globals, assignment to create new attributes on an imported module, etc. Example: import foo assert not hasattr(foo, 'bar') # just to illustrate the example foo.bar = 12 There's no way for the compiler to know that foo will have a bar attribute when it compiles foo. Jeremy From martin@v.loewis.de Thu Jan 31 23:00:34 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 01 Feb 2002 00:00:34 +0100 Subject: [Python-Dev] Fwd: [Pythonmac-SIG] sys.exit() functionality In-Reply-To: <38A3EBA8-1695-11D6-9DB9-003065517236@oratrix.nl> References: <38A3EBA8-1695-11D6-9DB9-003065517236@oratrix.nl> Message-ID: Jack Jansen writes: > This discussion started on pythonmac-SIG, but someone suggested that > it isn't really a MacPython-specific issue (even though the > implementation will be different for MacPython from unix-Python). > > Any opinions? I think allowing to replace Py_Exit is the right way to go. Make it a function pointer, initialized to _Py_Exit, and let the embedding context change its value (through a setter, or through direct assignment). Double-check that all callers of Py_Exit behave well when it actually does return (which currently is not the case), and don't forget to bump the API version. Regards, Martin From martin@v.loewis.de Thu Jan 31 23:09:55 2002 From: martin@v.loewis.de (Martin v. Loewis) Date: 01 Feb 2002 00:09:55 +0100 Subject: [Python-Dev] next vs darwin In-Reply-To: References: Message-ID: Jack Jansen writes: > With the define on it loads all extension modules into the application > namespace. Some people want this (despite the problems sketched above) > because they have modules that refer to external symbols defined in > modules that have been loaded earlier (and I assume there's magic that > ensures their modules are loaded in the right order). On Unix, this is a runtime option via sys.setdlopenflags (RTLD_GLOBAL turns on import into application namespace). Do you think you could emulate this API? > While I think this is an accident waiting to happen [*] the latter > behaviour is more-or-less the standard unix behaviour, so it should > probably be supportable in some way. It is not at all standard unix behaviour. Since Python 1.5.2, Python loads extensions with RTLD_LOCAL on systems, so that each module has its own namespace. People often requested that this is changed, but we successfully managed to turn down all these requests. Eventually, somebody came up with sys.setdlopenflags; this was good enough for me. > I prefer the new (OSX 10.1) preferred Apple way of linking plugins > (which is also the common way to do so on all other non-unix > platforms) where the plugin has to be linked against the application > and dynamic libraries it is going to be plugged into, so none of > this dynamic behaviour goes on. I'm not sure linking with a libpython.so is desirable, I'm quite fond of the approach to let the executable export symbols to the extensions. If that is possible on OS X, I'd encourage you to follow such a strategy (in unix gcc/ld, this is enabled through -Wl,--export-dynamic). > [*] I know of two cases where this already happened: both the curses > library and the SGI gl library defined a function clear(), so you > were hosed when you used both in the same Python script. On Unix, the originally trigger might have been the problem with initsocket, which was also exported in an Oracle library, thus breaking Oracle (the Python symbol is now init_socket, but that does not change the principle). Regards, Martin From jeremy@alum.mit.edu Thu Jan 31 14:43:22 2002 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Thu, 31 Jan 2002 09:43:22 -0500 Subject: [Python-Dev] distutils & stderr In-Reply-To: References: <15449.47153.267248.439654@beluga.mojam.com> Message-ID: <15449.22538.192214.765110@gondolin.digicool.com> >>>>> "TP" == Tim Peters writes: TP> [Skip] >> Sorry about the missing link. PyInline uses distutils to compile >> the C code. How PyInline does its think doesn't really matter to >> me, so I'm not going to be interested in distutils' messages. TP> If distutils output isn't interesting to PyInline users, TP> shouldn't PyInline be changed to run setup.py with its TP> -q/--quiet option? I started a thread on similar issues on the distutils-sig mailing list a week or two ago. There's agreement that output is a problem. The code has no consistent way of generating messages or of interpreting the notion of verbose or quiet. I think the right solution is to have several levels of verbosity and have a single function or method to use for output. (Perhaps a print statement with appropriate >>.) This makes it easier to control the amount of information you get and where it gets printed to. Michael Hudson has signed up to implement it and whatever else we can pile on when he's not looking. Further discussion should probably go to the sig. Jeremy From andymac@bullseye.apana.org.au Thu Jan 31 21:13:52 2002 From: andymac@bullseye.apana.org.au (Andrew MacIntyre) Date: Fri, 1 Feb 2002 08:13:52 +1100 (EDT) Subject: [Python-Dev] updated patches for OS/2 EMX port In-Reply-To: <3C591234.4F3B2FCE@lemburg.com> Message-ID: On Thu, 31 Jan 2002, M.-A. Lemburg wrote: > > > > - Objects/stringobject.c and Objects/unicodeobject.c contain changes to > > > > handle the EMX runtime library returning "0x" as the prefix for output > > > > formatted with a "%X" format. > > > > > > I'd suggest a different approach here, which does not use #ifdefs: > > > Instead of testing for the system, test for the bug. Then, if the bug > > > goes away, or appears on other systems as well, the code will be good. > > > > I did it the way I did because there's already code dealing with other > > brokeness in this area which doesn't solve the EMX issue, and the #ifdef > > solution minimises the risk of EMX fixes breaking something else which I > > can't test. At this stage I can't see this bug being fixed in EMX :-( > > I'd go with Martin's suggestion here: there already is code in > formatint() which tests for '%#X' adding '0x' or not. This code > should be made to handle the special case by testing for it -- > who knows: there may be other platforms where this doesn't work > as expected either. There are sure to be other platforms that have this bogosity. I'll look into this some more. > BTW, could you point me to your patch for this ? The Objects patch in patch #450267, at http://sf.net/tracker/?func=detail&atid=305470&aid=450267&group_id=5470 -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au | Snail: PO Box 370 andymac@pcug.org.au | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From jeremy@alum.mit.edu Thu Jan 31 22:36:37 2002 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Thu, 31 Jan 2002 17:36:37 -0500 Subject: [Python-Dev] distutils & stderr In-Reply-To: <20020201031143.GC9864@gerg.ca> References: <15449.47153.267248.439654@beluga.mojam.com> <15449.22538.192214.765110@gondolin.digicool.com> <20020201031143.GC9864@gerg.ca> Message-ID: <15449.50933.89877.252954@gondolin.digicool.com> >>>>> "GW" == Greg Ward writes: GW> Oh wait: most of the low-level worker code in the Distutils GW> falls outside the main class hierarchy, so the verbose flag GW> isn't *quite* so readily available; it gets passed in to a heck GW> of a lot of functions. Crap. I wish it were so clean and simple, Greg . In a lot of places, the binary verbose flag that is stored in the main class hierarchy is compared to some a var named "level". The result of that comparison is passed to functions, which ignore it and just use print. At least sometimes. Jeremy From aahz@rahul.net Wed Jan 30 23:51:30 2002 From: aahz@rahul.net (Aahz Maruch) Date: Wed, 30 Jan 2002 15:51:30 -0800 (PST) Subject: [Python-Dev] Python/Web developer In-Reply-To: <3C585A75.5831727A@teksystems.com> from "Rebecca Schaefer" at Jan 30, 2002 03:41:25 PM Message-ID: <20020130235130.765BCE8C4@waltz.rahul.net> Rebecca Schaefer wrote: > > TEKsystems in Appleton, WI has an opening for a web developer with > Python, Zope, html, SQL, UNIX, and Perl experience. It is a long term > contract opportunity. Any interested candidates should email Rebecca > Schaefer at rschaefe@teksystems.com python-dev is the wrong place for job ads. Please use either python-list or send a message to jobs@python.org. -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista We must not let the evil of a few trample the freedoms of the many. From Samuele Pedroni" <15449.36572.854311.657565@beluga.mojam.com><15449.6404.210447.679063@gondolin.digicool.com><00c801c1aa96$2b521320$6d94fea9@newmexico> <15449.9882.393698.701265@gondolin.digicool.com> Message-ID: <014001c1aa9a$94b8a960$6d94fea9@newmexico> From: Jeremy Hylton > >>>>> "SP" == Samuele Pedroni writes: > > SP> Hi. Q about PEP 267 Does the PEP mechanims adress only > SP> import a > SP> use a.x > > SP> cases. How does it handle things like > SP> import a.b > SP> use a.b.x > > You're a smart guy, can you tell me? :-). Seriously, I haven't > gotten that far. > > import mod.sub > creates a binding for "mod" in the global namespace > > The compiler can detect that the import statement is a package import > -- and mark "mod.sub" as a candidate for optimization. A use of > "mod.sub.attr" in function should be treated just as "mod.attr". > > The globals array (dict-list hybrid, technically) has the publicly > visible binding for "mod" but also has an internal binding for > "mod.sub" and "mod.sub.attr". Every module or submodule attribute in > a function gets an internal slot in the globals. The internal slot > gets initialized the first time it is used and then shared by all the > functions in the module. > > So I think this case isn't special enough to need a special case. > OK, I stated the wrong question. What happens if I do the following: import a.b def f(): print a.b.x a.g() print a.b.x f() Now a.g() change a.b from a submodule to an object with a x attribute. Maybe this case does not make sense, but the point is that the PEP is quite vague about imported stuff. Samuele (more puzzled than smart).RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4