Apologize if this is not the right = group for this=20 question. I am trying to build PyXPCOM from Mozilla 0.9.6 source tree on = Win=20 2000. I have managed to do make successfully and I can import xpcom. = When I=20 import xpcom._xpcom, I get "ImportError: cannot import name _xpcom" and = the path=20 includes Mozilla bin directory. The configure.html file hints at = something being=20 wrong with _xpcom(_d).pyd.
How do I fix this problem?
TIA, Praki
------=_NextPart_000_0005_01C17D10.60B38110-- From MarkH@ActiveState.com Wed Dec 5 06:38:38 2001 From: MarkH@ActiveState.com (Mark Hammond) Date: Wed, 5 Dec 2001 17:38:38 +1100 Subject: [Python-Dev] Building PyXPCOM from Mozilla source/ImportError: cannot import name _xpcom In-Reply-To: <000801c17d53$6fd608a0$3953afd8@popcorn> Message-ID: The correct forum for this is the Python XPCOM mailing list - details at http://listserv.ActiveState.com/mailman/listinfo/pyxpcom Please direct all followups there. Your PYTHONPATH needs to be set so that the main "xpcom" Python package is set on it. This directory should have an "_xpcom.pyd" (or xpcom_d.pyd for debug builds). I am guessing that this .pyd simply does not exist in that directory. If you receive another message that you omitted mentioning, regarding a required .dll file not being found, then the problem is likely that the PATH is not set correctly - but it appears from your message that it is OK. Hope this helps, Mark. -----Original Message----- From: python-dev-admin@python.org [mailto:python-dev-admin@python.org]On Behalf Of Nalini Prakash Sent: Wednesday, 5 December 2001 5:10 PM To: python-dev@python.org Subject: [Python-Dev] Building PyXPCOM from Mozilla source/ImportError: cannot import name _xpcom Apologize if this is not the right group for this question. I am trying to build PyXPCOM from Mozilla 0.9.6 source tree on Win 2000. I have managed to do make successfully and I can import xpcom. When I import xpcom._xpcom, I get "ImportError: cannot import name _xpcom" and the path includes Mozilla bin directory. The configure.html file hints at something being wrong with _xpcom(_d).pyd. How do I fix this problem? TIA, Praki From anthony@interlink.com.au Wed Dec 5 07:05:29 2001 From: anthony@interlink.com.au (Anthony Baxter) Date: Wed, 05 Dec 2001 18:05:29 +1100 Subject: [Python-Dev] python2.1.2 plans, plus the curse of PEP-006. Message-ID: <200112050705.fB575UN09520@mbuna.arbhome.com.au> Ok, after a bit of an unfortunate break I'm back into the backporting. My current plan is to have a release candidate for 2.1.2 due a couple of days before Christmas, and then try to get it tested as much as possible before a release either in the week between christmas/new year (not optimal) or shortly after. I'd hoped to get this stuff done much sooner, but unfortunately a combination of various personal matters has knocked me out for a couple of weeks. (I'm not sure if it's just me, but it's possible the reason no-one's done two bugfix releases in a row is that PEP-006 is accursed, and brings down the wrath of gods on anyone foolish enough to get involved. I'm hoping that the last few weeks horror is over and done now...) Anthony -- Anthony Baxter It's never too late to have a happy childhood. From mal@lemburg.com Wed Dec 5 08:28:18 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 05 Dec 2001 09:28:18 +0100 Subject: [Python-Dev] nested scopes confusion References: <083a01c17cfa$e13f6120$e000a8c0@thomasnotebook> <08dc01c17cfc$a5ef22c0$e000a8c0@thomasnotebook> Message-ID: <3C0DDAA2.F12F365C@lemburg.com> Thomas Heller wrote: > > From: "Thomas Heller" > > How do I achieve the desired effect? Note that the default argument > > trick (def mth(i=i): ...) does not work because *args is present. > Answering my own question (sorry), this works: > > def functions(): > result = [] > for i in range(10): > def make_function(index): > def mth(): > return index > return mth > mth = make_function(i) > result.append(mth) > i = 25 > return result > > for mth in functions(): > print mth() > > But how will I understand this 3 months from now? Why do you put the function definitions *inside* the functions() constructor ? AFAICTL the confusion is caused by this overly complicated nesting which doesn't seem necessary. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From guido@python.org Wed Dec 5 13:43:02 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 05 Dec 2001 08:43:02 -0500 Subject: [Python-Dev] python2.1.2 plans, plus the curse of PEP-006. In-Reply-To: Your message of "Wed, 05 Dec 2001 18:05:29 +1100." <200112050705.fB575UN09520@mbuna.arbhome.com.au> References: <200112050705.fB575UN09520@mbuna.arbhome.com.au> Message-ID: <200112051343.IAA13534@cj20424-a.reston1.va.home.com> Welcome back, Anthony! Glad to hear you haven't given up. If there's anything we can do to alleviate the curse, let us know... --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Wed Dec 5 14:49:04 2001 From: barry@zope.com (Barry A. Warsaw) Date: Wed, 5 Dec 2001 09:49:04 -0500 Subject: [Python-Dev] python2.1.2 plans, plus the curse of PEP-006. References: <200112050705.fB575UN09520@mbuna.arbhome.com.au> Message-ID: <15374.13280.512488.799375@anthem.wooz.org> >>>>> "AB" == Anthony Baxter writes: AB> (I'm not sure if it's just me, but it's possible the reason AB> no-one's done two bugfix releases in a row is that PEP-006 is AB> accursed, and brings down the wrath of gods on anyone foolish AB> enough to get involved. I'm hoping that the last few weeks AB> horror is over and done now...) Anthony, First, thanks for taking on the responsibility, and glad you're back! Second, remember that PEP 6 is a work in progress, especially since you're breaking new ground. If PEP 6 actually hinders patch releases rather than helps them, then IMHO the PEP is probably broken. Is your problem with it the prohibitions? Or is it the procedure? Is the PEP too incomplete to be helpful? Please do suggest changes! If you want to co-own it so you can update it yourself, we can do that too. Patch releases should be easy(er). -Barry From fdrake@acm.org Wed Dec 5 16:36:19 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 5 Dec 2001 11:36:19 -0500 Subject: [Python-Dev] pychecker on 2.2 current In-Reply-To: <3C0C3806.B7325138@metaslash.com> References: <3C0C3806.B7325138@metaslash.com> Message-ID: <15374.19715.930571.37939@grendel.zope.com> Neal Norwitz writes: > mimetypes.py:405: No global (sys) found > > This is inside usage() which presumably should only be used internally. > sys is imported if __name__ == '__main__'. > So the problem can only be triggered if usage() is called by external source. > > Possible fixes are: > sys could be locally imported in usage() > sys could be imported at the module scope > usage could be made _usage() to denote it's private Neal, I'll propose another fix, but I don't know what pychecker would say. ;-) I'd like to move the definition of usage() into the if __name__ == "__main__" block, in which case the problem goes away since there wouldn't be a usage() unless sys has been imported. I'll check it in that way. ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From jack@oratrix.nl Wed Dec 5 15:59:39 2001 From: jack@oratrix.nl (Jack Jansen) Date: Wed, 05 Dec 2001 16:59:39 +0100 Subject: [Python-Dev] Mac OS X sys.platform value Message-ID: <20011205155939.8936A303183@snelboot.oratrix.nl> Folks, Apple "wisely" changed their uname version numbering scheme with a micro-release (10.1.1). Of course, nobody cares about this, except Python, which stuffs the version into sys.platform, from where it is used by a gazillion things. Anyway, where sys.platform used to be "darwin1" upto and including 10.1.0 it is now "darwin5" for 10.1.1 (and it will go up to "darwin6" for 10.2, etc). I am of a mind to take the "1" out of sys.platform, so that it becomes "darwin". Good idea or not? -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.cwi.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++ From guido@python.org Wed Dec 5 16:07:19 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 05 Dec 2001 11:07:19 -0500 Subject: [Python-Dev] Mac OS X sys.platform value In-Reply-To: Your message of "Wed, 05 Dec 2001 16:59:39 +0100." <20011205155939.8936A303183@snelboot.oratrix.nl> References: <20011205155939.8936A303183@snelboot.oratrix.nl> Message-ID: <200112051607.fB5G7Jw28901@odiug.zope.com> > Anyway, where sys.platform used to be "darwin1" upto and including 10.1.0 it > is now "darwin5" for 10.1.1 (and it will go up to "darwin6" for 10.2, etc). > > I am of a mind to take the "1" out of sys.platform, so that it becomes > "darwin". > > Good idea or not? Fine with me as long as it doesn't affect other platforms. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@rahul.net Wed Dec 5 17:43:38 2001 From: aahz@rahul.net (Aahz Maruch) Date: Wed, 5 Dec 2001 09:43:38 -0800 (PST) Subject: [Python-Dev] python2.1.2 plans, plus the curse of PEP-006. In-Reply-To: <200112050705.fB575UN09520@mbuna.arbhome.com.au> from "Anthony Baxter" at Dec 05, 2001 06:05:29 PM Message-ID: <20011205174339.3EC2CE8C1@waltz.rahul.net> Anthony Baxter wrote: > > (I'm not sure if it's just me, but it's possible the reason no-one's > done two bugfix releases in a row is that PEP-006 is accursed, and > brings down the wrath of gods on anyone foolish enough to get involved. > I'm hoping that the last few weeks horror is over and done now...) Hmmmm.... I think the fact that three different people have stepped up to the plate to handle bugfix releases is a *feature*. In addition, three releases in six months does not a long-term trend make. As Barry said, if there's a problem with PEP6, I'd love to know about it; other than that, there haven't IMO been enough Python releases and patches to warrant more bugfix releases than have occurred or are on the plate. Good work so far! -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista We must not let the evil of a few trample the freedoms of the many. From Anthony Baxter Wed Dec 5 22:09:40 2001 From: Anthony Baxter (Anthony Baxter) Date: Thu, 06 Dec 2001 09:09:40 +1100 Subject: [Python-Dev] python2.1.2 plans, plus the curse of PEP-006. In-Reply-To: Message from barry@zope.com (Barry A. Warsaw) of "Wed, 05 Dec 2001 09:49:04 CDT." <15374.13280.512488.799375@anthem.wooz.org> Message-ID: <200112052209.fB5M9eZ14675@mbuna.arbhome.com.au> >>> Barry A. Warsaw wrote > Please do suggest changes! If you want to co-own it so you can update > it yourself, we can do that too. > Patch releases should be easy(er). I had a think about this, and I think the major issue is that there was quite a gap to backfill between 2.1.1 and me starting 2.1.2. There was a bit of backporting (thank you twouters!), but not that much - and not that much done as far as marking bugfixes in the changelogs. Given the rate of change in the python core, I'd say that it's more likely that people are going to want the ability to switch versions when it suits them, and thus keeping up the version before the current bleeding edge is appropriate. I'd say that something like the following would make life easier The branch for 2.N-maint should be maintained until 2.(N+1) gets a first patch release. That is, 2.1-maint gets bugfixes until 2.2.1 gets out the door. Alternately, waiting X months after 2.(N+1).0 would be another approach. In any case, marking bugfix candidates in CVS logs should be a standard thing to do. In terms of concrete things I'm looking for right now: . someone who's either a HP/UX victim^W user who's prepared to try and work out what bits need backporting for the threading fix, or someone who's got a HP/UX box and is prepared to let me ssh into the box and try and do it myself. . people to test the 2.1.2 release candidate when it's ready on a bunch of different platforms - I only have access to solaris, linux, and whatever's in the sf compile farm. Anthony From martin@v.loewis.de Wed Dec 5 20:00:18 2001 From: martin@v.loewis.de (Martin v. Loewis) Date: Wed, 5 Dec 2001 21:00:18 +0100 Subject: [Python-Dev] Mac OS X sys.platform value In-Reply-To: <20011205155939.8936A303183@snelboot.oratrix.nl> (message from Jack Jansen on Wed, 05 Dec 2001 16:59:39 +0100) References: <20011205155939.8936A303183@snelboot.oratrix.nl> Message-ID: <200112052000.fB5K0Ip01526@mira.informatik.hu-berlin.de> > Anyway, where sys.platform used to be "darwin1" upto and including 10.1.0 it > is now "darwin5" for 10.1.1 (and it will go up to "darwin6" for 10.2, etc). > > I am of a mind to take the "1" out of sys.platform, so that it becomes > "darwin". > > Good idea or not? Bad idea. Anybody comparing to "darwin1" will have to update the code. When they do so, they could just as well write sys.platform.startswith("darwin"). The same happens on Linux, which happens to be "linux2" since Linux kernel version 2.x is in use. That says little about the system, though, since glibc version or name of the distribution may be relevant. On OS X, it seems sys.platform could be used to distinguish minor differences. We should not take away that opportunity. Regards, Martin From aahz@rahul.net Wed Dec 5 22:43:00 2001 From: aahz@rahul.net (Aahz Maruch) Date: Wed, 5 Dec 2001 14:43:00 -0800 (PST) Subject: [Python-Dev] python2.1.2 plans, plus the curse of PEP-006. In-Reply-To: <200112052209.fB5M9eZ14675@mbuna.arbhome.com.au> from "Anthony Baxter" at Dec 06, 2001 09:09:40 AM Message-ID: <20011205224300.9EFFEE8C7@waltz.rahul.net> Anthony Baxter wrote: > > I'd say that something like the following would make life easier > > The branch for 2.N-maint should be maintained until 2.(N+1) gets > a first patch release. That is, 2.1-maint gets bugfixes until 2.2.1 > gets out the door. Alternately, waiting X months after 2.(N+1).0 > would be another approach. What does "should be maintained" mean? With CVS, IIUC, there's no reason not to keep branches around forever. If instead what you're talking about is policy for work to be assigned, I won't let that into PEP6; the entire point of PEP6 is that it only prescribes procedure for someone who is already working, it does not specify any work to be accomplished in the abstract. The original form of PEP6 was looser in this respect (in terms of specifying when patch releases would occur and so on), but I was convinced by Guido et al to stick strictly to "how" and not "when" or "what". At the very least, I think we should stick with the current base plan for one or two more release cycles before making any structured attempts to integrate bugfixes into the standard Python release cycle. > In any case, marking bugfix candidates in CVS logs should be a standard > thing to do. The problem lies in creating the social structure to make that happen. People who monitor CVS need to get into the habit of reminding each other to mark bugfix candidates. But that's beyond the scope of PEP6. -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista We must not let the evil of a few trample the freedoms of the many. From neal@metaslash.com Thu Dec 6 01:28:43 2001 From: neal@metaslash.com (Neal Norwitz) Date: Wed, 05 Dec 2001 20:28:43 -0500 Subject: [Python-Dev] pychecker on 2.2 current References: <3C0C3806.B7325138@metaslash.com> <15374.19715.930571.37939@grendel.zope.com> Message-ID: <3C0EC9CB.48BBD0F0@metaslash.com> "Fred L. Drake, Jr." wrote: > > Neal Norwitz writes: > > mimetypes.py:405: No global (sys) found > > > > This is inside usage() which presumably should only be used internally. > > sys is imported if __name__ == '__main__'. > > So the problem can only be triggered if usage() is called > > by external source. > > > > Possible fixes are: > > sys could be locally imported in usage() > > sys could be imported at the module scope > > usage could be made _usage() to denote it's private > > I'll propose another fix, but I don't know what pychecker would > say. ;-) I'd like to move the definition of usage() into the if > __name__ == "__main__" block, in which case the problem goes away > since there wouldn't be a usage() unless sys has been imported. > I'll check it in that way. ;-) I like your fix better than my suggestions. :-) I thought pychecker would complain, but it's happy too. :-) Neal From guido@python.org Thu Dec 6 02:15:15 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 05 Dec 2001 21:15:15 -0500 Subject: [Python-Dev] pychecker on 2.2 current In-Reply-To: Your message of "Wed, 05 Dec 2001 20:28:43 EST." <3C0EC9CB.48BBD0F0@metaslash.com> References: <3C0C3806.B7325138@metaslash.com> <15374.19715.930571.37939@grendel.zope.com> <3C0EC9CB.48BBD0F0@metaslash.com> Message-ID: <200112060215.VAA18132@cj20424-a.reston1.va.home.com> > I thought pychecker would complain, but it's happy too. :-) But isn't that because the new code is all at the module-global level, which pychecker doesn't "see"? Or have you fixed that yet? --Guido van Rossum (home page: http://www.python.org/~guido/) From neal@metaslash.com Thu Dec 6 02:26:15 2001 From: neal@metaslash.com (Neal Norwitz) Date: Wed, 05 Dec 2001 21:26:15 -0500 Subject: [Python-Dev] pychecker on 2.2 current References: <3C0C3806.B7325138@metaslash.com> <15374.19715.930571.37939@grendel.zope.com> <3C0EC9CB.48BBD0F0@metaslash.com> <200112060215.VAA18132@cj20424-a.reston1.va.home.com> Message-ID: <3C0ED747.8E6E519C@metaslash.com> Guido van Rossum wrote: > > > I thought pychecker would complain, but it's happy too. :-) > > But isn't that because the new code is all at the module-global level, > which pychecker doesn't "see"? Or have you fixed that yet? pychecker does "see" all code at the module level, but it isn't perfect. If there is code like: import sys if sys.platform == 'win32': def x(a, b): pass else: def x(a): pass The following incorrect warning is produced: t2.py:5: Redefining attribute (x) original line (3) Because both versions of the function x() are seen. Neal From guido@python.org Thu Dec 6 02:35:25 2001 From: guido@python.org (Guido van Rossum) Date: Wed, 05 Dec 2001 21:35:25 -0500 Subject: [Python-Dev] pychecker on 2.2 current In-Reply-To: Your message of "Wed, 05 Dec 2001 21:26:15 EST." <3C0ED747.8E6E519C@metaslash.com> References: <3C0C3806.B7325138@metaslash.com> <15374.19715.930571.37939@grendel.zope.com> <3C0EC9CB.48BBD0F0@metaslash.com> <200112060215.VAA18132@cj20424-a.reston1.va.home.com> <3C0ED747.8E6E519C@metaslash.com> Message-ID: <200112060235.VAA18752@cj20424-a.reston1.va.home.com> > > But isn't that because the new code is all at the module-global level, > > which pychecker doesn't "see"? Or have you fixed that yet? > > pychecker does "see" all code at the module level, I stand corrected. I think this was the case for an older version? > but it isn't perfect. If there is code like: > > import sys > if sys.platform == 'win32': > def x(a, b): pass > else: > def x(a): pass > > The following incorrect warning is produced: > t2.py:5: Redefining attribute (x) original line (3) > > Because both versions of the function x() are seen. Hm, I would call that warning correct. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From neal@metaslash.com Thu Dec 6 02:50:24 2001 From: neal@metaslash.com (Neal Norwitz) Date: Wed, 05 Dec 2001 21:50:24 -0500 Subject: [Python-Dev] pychecker on 2.2 current References: <3C0C3806.B7325138@metaslash.com> <15374.19715.930571.37939@grendel.zope.com> <3C0EC9CB.48BBD0F0@metaslash.com> <200112060215.VAA18132@cj20424-a.reston1.va.home.com> <3C0ED747.8E6E519C@metaslash.com> <200112060235.VAA18752@cj20424-a.reston1.va.home.com> Message-ID: <3C0EDCF0.94AB3B88@metaslash.com> Guido van Rossum wrote: > > > > But isn't that because the new code is all at the module-global level, > > > which pychecker doesn't "see"? Or have you fixed that yet? > > > > pychecker does "see" all code at the module level, > > I stand corrected. I think this was the case for an older version? Yes, you are correct it was the case a while ago. BTW, same is true for code at the class scope, ie, it used to be missed, but it is checked now. > > but it isn't perfect. If there is code like: > > > > import sys > > if sys.platform == 'win32': > > def x(a, b): pass > > else: > > def x(a): pass > > > > The following incorrect warning is produced: > > t2.py:5: Redefining attribute (x) original line (3) > > > > Because both versions of the function x() are seen. > > Hm, I would call that warning correct. :-) :-) From neal@metaslash.com Thu Dec 6 02:50:38 2001 From: neal@metaslash.com (Neal Norwitz) Date: Wed, 05 Dec 2001 21:50:38 -0500 Subject: [Python-Dev] memory leaks in 2.2 Message-ID: <3C0EDCFE.D01E6EA3@metaslash.com> Here is a list of all the outstanding memory leaks from the regression tests. All of these leaks have a bug report on SF. test_descr: 117 bytes (0.0407%) test_email: 30 bytes (0.0149%) test_re: 30 bytes (0.0149%) test_richcmp: 16 bytes (0.0106%) test_scope: 39 bytes (0.031%) test_sre: 188 bytes (0.137%) test_symtable: 12 bytes (0.00991%) test_unicode: 66 bytes (0.0224%) syntax error in interpreter: 38 bytes (0.05%) From Anthony Baxter Thu Dec 6 02:59:11 2001 From: Anthony Baxter (Anthony Baxter) Date: Thu, 06 Dec 2001 13:59:11 +1100 Subject: [Python-Dev] python2.1.2 plans, plus the curse of PEP-006. In-Reply-To: Message from aahz@rahul.net (Aahz Maruch) of "Wed, 05 Dec 2001 14:43:00 -0800." <20011205224300.9EFFEE8C7@waltz.rahul.net> Message-ID: <200112060259.fB62xBi16480@mbuna.arbhome.com.au> >>> Aahz Maruch wrote > What does "should be maintained" mean? With CVS, IIUC, there's no > reason not to keep branches around forever. If instead what you're I mean "should continually have backported patches applied". > talking about is policy for work to be assigned, I won't let that into > PEP6; the entire point of PEP6 is that it only prescribes procedure for > someone who is already working, it does not specify any work to be > accomplished in the abstract. Oh, sure - I'm not talking about adding it to PEP006 - it's not appropriate for it. More talking about the abstract here. > The problem lies in creating the social structure to make that happen. > People who monitor CVS need to get into the habit of reminding each > other to mark bugfix candidates. But that's beyond the scope of PEP6. *nod* absolutely. From tim.one@home.com Thu Dec 6 04:14:00 2001 From: tim.one@home.com (Tim Peters) Date: Wed, 5 Dec 2001 23:14:00 -0500 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: <3C0EDCFE.D01E6EA3@metaslash.com> Message-ID: [Neal Norwitz] > Here is a list of all the outstanding memory leaks from the > regression tests. > All of these leaks have a bug report on SF. > > test_descr: 117 bytes (0.0407%) > test_email: 30 bytes (0.0149%) > test_re: 30 bytes (0.0149%) > test_richcmp: 16 bytes (0.0106%) > test_scope: 39 bytes (0.031%) > test_sre: 188 bytes (0.137%) > test_symtable: 12 bytes (0.00991%) > test_unicode: 66 bytes (0.0224%) > > syntax error in interpreter: 38 bytes (0.05%) Can anyone lend a hand here? Take a test, stick it in an infinite loop, watch your process size blow off the roof, then whittle the test down until the trigger becomes clear. We found a very serious 2.2 leak today doing this to test_binop.py, and not even Guido (let alone God ...) knows how many more may be lurking. It's not a Good Sign that test_binop has vanished from the list now but everything else is still there. have-some-fun-plug-a-leak-ly y'rs - tim From fdrake@acm.org Thu Dec 6 05:17:58 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 6 Dec 2001 00:17:58 -0500 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: References: <3C0EDCFE.D01E6EA3@metaslash.com> Message-ID: <15374.65414.640831.895781@grendel.zope.com> Tim Peters writes: > more may be lurking. It's not a Good Sign that test_binop has vanished from > the list now but everything else is still there. Err, test_parser is no longer on the list! -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From tim.one@home.com Thu Dec 6 04:36:51 2001 From: tim.one@home.com (Tim Peters) Date: Wed, 5 Dec 2001 23:36:51 -0500 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: <15374.65414.640831.895781@grendel.zope.com> Message-ID: [Tim] > more may be lurking. It's not a Good Sign that test_binop has > vanished from the list now but everything else is still there. [Fred L. Drake, Jr.] > Err, test_parser is no longer on the list! Which is bad news: it means that test_binop and test_parser had leaks for reasons independent of each other, and of all the others on the list: we're not making swift progress on these. You only looked at test_parser because Guido assigned it to you. I only looked at test_binop because I was coughing too hard to get any difficult work done . From fdrake@acm.org Thu Dec 6 06:58:38 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 6 Dec 2001 01:58:38 -0500 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: References: <15374.65414.640831.895781@grendel.zope.com> Message-ID: <15375.5918.776405.747630@grendel.zope.com> Tim Peters writes: > Which is bad news: it means that test_binop and test_parser had leaks for > reasons independent of each other, and of all the others on the list: we're > not making swift progress on these. You only looked at test_parser because > Guido assigned it to you. I only looked at test_binop because I was > coughing too hard to get any difficult work done . Aside from these specific memory-leak reports, we've generally fallen down on bug triage. I'm not sure what we can do about this in the short term, but I really think we're not doing as good a job at this as we should be. I wonder if we could take a lesson from the Mozilla community and organize Bug Days (group hack days where the goal is to resolve bugs) or have a "BugAThon" where we call for smaller/simpler test cases that exercise reported bugs. I suspect a page about how to narrow down a bug to a smaller test case would be very helpful, especially if it also discusses how to use the various available tools for things like pinpointing leaks, etc. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From tim.one@home.com Thu Dec 6 07:00:28 2001 From: tim.one@home.com (Tim Peters) Date: Thu, 6 Dec 2001 02:00:28 -0500 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: <15375.5918.776405.747630@grendel.zope.com> Message-ID: [Fred L. Drake, Jr.] > Aside from these specific memory-leak reports, we've generally > fallen down on bug triage. Look on the bright side: we're doing a much better job on bugs than on patch review, and our patch review work in turn is stellar compared to what we've accomplished in the PSF . > I'm not sure what we can do about this in the short term, but I really > think we're not doing as good a job at this as we should be. I think it's just one of many areas where we "should be" doing a better job. > I wonder if we could take a lesson from the Mozilla community and > organize Bug Days (group hack days where the goal is to resolve bugs) > or have a "BugAThon" where we call for smaller/simpler test cases that > exercise reported bugs. I suspect a page about how to narrow down a > bug to a smaller test case would be very helpful, especially if it > also discusses how to use the various available tools for things like > pinpointing leaks, etc. Frankly, I don't think it will help. If you look at the bug statistics, we've made steady and dramatic improvement in bug turnaround time over the last six months, and we can't sustain even that improvement. Fact is we have very few people fixing bugs or reviewing patches. It's not like the majority of Python developers work on this stuff in spurts, it's that most don't work on it at all, and almost nobody works on it routinely. We couldn't even meet our modest internal (admittedly imposed by me ) "close one bug or patch per day per person" goal last week (you'll never see that idea again either ...). Maybe we should resume offering a free copy of the next Python release to contributors working beyond the bounds of sanity? Na -- even Thomas eventually figured out he could get a free copy anyway -- let's just pray he doesn't tell Martin . better-at-eating-kung-pao-chicken-than-motivating-the-overburdened- to-spread-themselves-thinner-than-gas-ly y'rs - tim From DavidA@ActiveState.com Thu Dec 6 07:12:45 2001 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 05 Dec 2001 23:12:45 -0800 Subject: [Python-Dev] memory leaks in 2.2 References: <15374.65414.640831.895781@grendel.zope.com> <15375.5918.776405.747630@grendel.zope.com> Message-ID: <3C0F1A6D.417B603C@ActiveState.com> "Fred L. Drake, Jr." wrote: > Aside from these specific memory-leak reports, we've generally > fallen down on bug triage. I'm not sure what we can do about this in > the short term, but I really think we're not doing as good a job at > this as we should be. It's hard. Bugzilla has some good features there w.r.t. "unverified" vs. "new" vs. "accepted" and "fixed" vs "resolved", QA contacts, keyword handling for milestone/release targets, etc. I'm not sure how the SF bugtracker compares in practice but from a distance it seems a little weak there. [BTW -- I know this isn't the right channel, but we (the Komodo team) have found a couple of bugs in popen recently which y'all may not have seen yet and you might want to tackle for 2.2 -- we'll get them in SF in the next few days hopefully, but this is better than nothing: - popen on Windows XP seems problematic (needs a repro test there -- not analyzed yet) - popen on Posix has a nasty failure mode if the cmd argument is Unicode (see the first line in Popen3._run_child for a hint =)] > I wonder if we could take a lesson from the Mozilla community and > organize Bug Days (group hack days where the goal is to resolve bugs) > or have a "BugAThon" where we call for smaller/simpler test cases that > exercise reported bugs. I suspect a page about how to narrow down a > bug to a smaller test case would be very helpful, especially if it > also discusses how to use the various available tools for things like > pinpointing leaks, etc. One thing the Mozilla community has going for it is really dedicated QA folks. One in particular, Asa Dotzler, does an amazing job at bug triage, categorization, etc. He's now a Netscape employee, but used to do this for fun. It would be a good way for someone who'se 'on the periphery' to get more deeply involved and learn a lot. In general, that kind of QA job can be a gateway to fame and fortune much like documentation writing can be, and you really get to learn the product/technology. I don't have first-hand experience of the success of bug days, but they're doing more of them, so they must get some value out of them. --david guessing-from-the-"it hurts! then don't do that!"-corrolary ascher From martin@v.loewis.de Thu Dec 6 07:27:33 2001 From: martin@v.loewis.de (Martin v. Loewis) Date: Thu, 6 Dec 2001 08:27:33 +0100 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: References: Message-ID: <200112060727.fB67RXZ08417@mira.informatik.hu-berlin.de> > Maybe we should resume offering a free copy of the next Python release to > contributors working beyond the bounds of sanity? Na -- even Thomas > eventually figured out he could get a free copy anyway -- let's just pray he > doesn't tell Martin . :-) I personally don't see it as a problem that bug reports are piling up. The GCC GNATS has 1341 "open" bug reports (meaning either "open" or "analyzed") at the moment. bugzilla.mozilla.org reports 19611 bugs in the "open" categories. Microsoft reportedly has over 60000 open reports for Windows 2000. Neither of these groups despair :-) As for the things we should change: I believe it is pointless to sit on a report for a long time without doing anything about this. If nothing happens for a year, the person that was assigned this report likely won't do anything about it for the next months, either. Unassigning things you know you won't do is particularly important for patches, but to a degree also important for bug reports. It signals that somebody else may need to step forward, or that some deity should just decide on the fate of this report. Regards, Martin From DavidA@ActiveState.com Thu Dec 6 07:39:08 2001 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 05 Dec 2001 23:39:08 -0800 Subject: [Python-Dev] memory leaks in 2.2 References: Message-ID: <3C0F209C.9FAE0599@ActiveState.com> Tim Peters wrote: > > [Fred L. Drake, Jr.] > > Aside from these specific memory-leak reports, we've generally > > fallen down on bug triage. > > Look on the bright side: we're doing a much better job on bugs than on > patch review, and our patch review work in turn is stellar compared to what > we've accomplished in the PSF . I'll pipe in with our experience with Mozilla & Komodo, which have related but different processes. Mozilla has a _very strict_ review process. Up to two reviews and one super-review for each checkin (http://www.mozilla.org/hacking/reviewers.html). 99% of all communication is done through comments & patches attached to bugs. It works for them, in that the quality of the code checkins tends to be high, but it's a fairly imposing process (which is limiting the number of contributors, I expect -- not because the process is _that_ hard to follow, but it's hard to discover and learn and adopt, especially as part of a side project). In the Komodo team, the number and type of reviews we require depends on the stage of the product (how close we are to release). We used to have periods when no review was needed, but I think we're going to keep the minimal one-review requirement on indefinitely. At critical times we've even had a "change review board" process in place. Having tools that generate good patches with 'proposed changelogs' and apply them safely, good integration between the source code control system and the bug tracking system are all good things which have made our code quality go up as well. Bugzilla is a good patch manager as well. It takes time to learn/teach how to be a good reviewer -- it's not the same kind of skill as writing a patch or logging a good bug. Review processes also affect 'superficial development speed' -- the patch author has to sit on a patch until the process is complete, and that can be frustrating. Also, to the extent that patches uncover other bugs, it can slow down the bug discovery process. Of course we wouldn't do it if we didn't feel it increased the long-term effective rate of quality increase. Note that in both of the above cases, _everyone_ must go through the review process, including the top dogs, for any change to the code base, build system, etc. (the documentation and test suite subtree are excluded usually). Note also that I don't mean to imply that the above process is the right one for Python. I'm just speaking on the processes I know something about =). --david From martin@v.loewis.de Thu Dec 6 07:40:22 2001 From: martin@v.loewis.de (Martin v. Loewis) Date: Thu, 6 Dec 2001 08:40:22 +0100 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: <3C0F1A6D.417B603C@ActiveState.com> (message from David Ascher on Wed, 05 Dec 2001 23:12:45 -0800) References: <15374.65414.640831.895781@grendel.zope.com> <15375.5918.776405.747630@grendel.zope.com> <3C0F1A6D.417B603C@ActiveState.com> Message-ID: <200112060740.fB67eMV08466@mira.informatik.hu-berlin.de> > It's hard. Bugzilla has some good features there w.r.t. "unverified" > vs. "new" vs. "accepted" and "fixed" vs "resolved", QA contacts, keyword > handling for milestone/release targets, etc. I'm not sure how the SF > bugtracker compares in practice but from a distance it seems a little > weak there. SF recommends to use the Group for "unsupported", "unverified". Unfortunately, the filtering capabilities leave a lot of wishes. > - popen on Posix has a nasty failure mode if the cmd argument is > Unicode (see the first line in Popen3._run_child for a hint =)] You mean, def _run_child(self, cmd): if isinstance(cmd, types.StringTypes): cmd = ['/bin/sh', '-c', cmd] :-? Fixed since 2001/12/02. > It would be a good way for someone who'se 'on the periphery' to get > more deeply involved and learn a lot. This is free software. Most contributors work on it for fun, or because they need it for their job. Posting "volunteers needed" may be a good idea, but then, perhaps not so shortly before 2.2. Regards, Martin From DavidA@ActiveState.com Thu Dec 6 07:49:39 2001 From: DavidA@ActiveState.com (David Ascher) Date: Wed, 05 Dec 2001 23:49:39 -0800 Subject: [Python-Dev] memory leaks in 2.2 References: <15374.65414.640831.895781@grendel.zope.com> <15375.5918.776405.747630@grendel.zope.com> <3C0F1A6D.417B603C@ActiveState.com> <200112060740.fB67eMV08466@mira.informatik.hu-berlin.de> Message-ID: <3C0F2313.C9B7824F@ActiveState.com> > if isinstance(cmd, types.StringTypes): > cmd = ['/bin/sh', '-c', cmd] > > :-? Fixed since 2001/12/02. Ha! Good. One more resolved bug then =). > > It would be a good way for someone who'se 'on the periphery' to get > > more deeply involved and learn a lot. > > This is free software. Most contributors work on it for fun, or > because they need it for their job. Sure. Are you disagreeing with something I said? --david From anthony@interlink.com.au Thu Dec 6 07:45:04 2001 From: anthony@interlink.com.au (Anthony Baxter) Date: Thu, 06 Dec 2001 18:45:04 +1100 Subject: [Python-Dev] on the subject of SF... Message-ID: <200112060745.fB67j4e18189@mbuna.arbhome.com.au> can someone with the appropriate level of magickness add a 'Python 2.1.2' group to the bug tracker and patch tracker? I've been meaning to get this done for a while... ta. Anthony From martin@v.loewis.de Thu Dec 6 07:47:31 2001 From: martin@v.loewis.de (Martin v. Loewis) Date: Thu, 6 Dec 2001 08:47:31 +0100 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: <3C0F209C.9FAE0599@ActiveState.com> (message from David Ascher on Wed, 05 Dec 2001 23:39:08 -0800) References: <3C0F209C.9FAE0599@ActiveState.com> Message-ID: <200112060747.fB67lV108506@mira.informatik.hu-berlin.de> > Note that in both of the above cases, _everyone_ must go through the > review process, including the top dogs, for any change to the code base, > build system, etc. (the documentation and test suite subtree are > excluded usually). Note also that I don't mean to imply that the above > process is the right one for Python. I'm just speaking on the processes > I know something about =). To report on the processes of another project; GCC organizes people in "blanket write priviledge", "maintainers" (CPU port, OS, misc), and "write after approval". The blanket priviledge allows people to commit contributions they have reviewed from contributors without any write access; those are used sparingly. Maintainers can change "their" files any time they like, except after a freeze. Everybody else must wait for a patch review. Regards, Martin From barry@zope.com Thu Dec 6 07:49:02 2001 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 6 Dec 2001 02:49:02 -0500 Subject: [Python-Dev] on the subject of SF... References: <200112060745.fB67j4e18189@mbuna.arbhome.com.au> Message-ID: <15375.8942.722195.533597@anthem.wooz.org> >>>>> "AB" == Anthony Baxter writes: AB> can someone with the appropriate level of magickness add a AB> 'Python 2.1.2' group to the bug tracker and patch tracker? AB> I've been meaning to get this done for a while... ta. Anthony Done! -Barry From martin@v.loewis.de Thu Dec 6 07:52:38 2001 From: martin@v.loewis.de (Martin v. Loewis) Date: Thu, 6 Dec 2001 08:52:38 +0100 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: <3C0F2313.C9B7824F@ActiveState.com> (message from David Ascher on Wed, 05 Dec 2001 23:49:39 -0800) References: <15374.65414.640831.895781@grendel.zope.com> <15375.5918.776405.747630@grendel.zope.com> <3C0F1A6D.417B603C@ActiveState.com> <200112060740.fB67eMV08466@mira.informatik.hu-berlin.de> <3C0F2313.C9B7824F@ActiveState.com> Message-ID: <200112060752.fB67qcw08535@mira.informatik.hu-berlin.de> > > > It would be a good way for someone who'se 'on the periphery' to get > > > more deeply involved and learn a lot. > > > > This is free software. Most contributors work on it for fun, or > > because they need it for their job. > > Sure. Are you disagreeing with something I said? No. I just think this won't help much. Just because it would be good for somebody doesn't mean that this somebody will actually invest time into the project. People normally get involved by other means than starting as a bug/patch reviewer. Typically, they get involved because they need something fixed that apparently nobody else has fixed for them while they were waiting for it (atleast that's how I got involved in all free software projects I was working on). Regards, Martin From mal@lemburg.com Thu Dec 6 08:40:00 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 06 Dec 2001 09:40:00 +0100 Subject: [Python-Dev] on the subject of SF... References: <200112060745.fB67j4e18189@mbuna.arbhome.com.au> <15375.8942.722195.533597@anthem.wooz.org> Message-ID: <3C0F2EE0.BE1D2980@lemburg.com> "Barry A. Warsaw" wrote: > > >>>>> "AB" == Anthony Baxter writes: > > AB> can someone with the appropriate level of magickness add a > AB> 'Python 2.1.2' group to the bug tracker and patch tracker? > AB> I've been meaning to get this done for a while... ta. Anthony > > Done! And please also a Group "Python 2.3" so that feature requests for that version can be assigned to it ?! -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From mal@lemburg.com Thu Dec 6 09:08:59 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 06 Dec 2001 10:08:59 +0100 Subject: [Python-Dev] Mac OS X sys.platform value References: <20011205155939.8936A303183@snelboot.oratrix.nl> <200112052000.fB5K0Ip01526@mira.informatik.hu-berlin.de> Message-ID: <3C0F35AB.2CF9D067@lemburg.com> "Martin v. Loewis" wrote: > > > Anyway, where sys.platform used to be "darwin1" upto and including 10.1.0 it > > is now "darwin5" for 10.1.1 (and it will go up to "darwin6" for 10.2, etc). > > > > I am of a mind to take the "1" out of sys.platform, so that it becomes > > "darwin". > > > > Good idea or not? > > Bad idea. Anybody comparing to "darwin1" will have to update the > code. When they do so, they could just as well write > sys.platform.startswith("darwin"). > > The same happens on Linux, which happens to be "linux2" since Linux > kernel version 2.x is in use. That says little about the system, > though, since glibc version or name of the distribution may be > relevant. Exactly and that's why I think that version numbers in sys.platform are plain wrong. There are better tools for finding out details about the system, e.g. platform.py which you can download from my Python pages. > On OS X, it seems sys.platform could be used to distinguish minor > differences. We should not take away that opportunity. As you already noted, the information given in the current version number is too inaccurate to be of any real value. Why not add platform.py to the std lib in Python 2.3 and remove all version info from sys.platform ?! -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From ping@lfw.org Thu Dec 6 09:57:41 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Thu, 6 Dec 2001 01:57:41 -0800 (PST) Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: <3C0EDCFE.D01E6EA3@metaslash.com> Message-ID: On Wed, 5 Dec 2001, Neal Norwitz wrote: > Here is a list of all the outstanding memory leaks from the regression tests. > All of these leaks have a bug report on SF. > > test_descr: 117 bytes (0.0407%) Okay, i have tested this case and narrowed it down to: while 1: class C: __metaclass__ = type Try it on a Python interpreter near you. With mine, it grows without bound. As far as i can tell, of all the calls in test_descr.test_main, only metaclass() leaks memory. -- ?!ng From mal@lemburg.com Thu Dec 6 12:12:46 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 06 Dec 2001 13:12:46 +0100 Subject: [Python-Dev] Comparing sequences ?! Message-ID: <3C0F60BE.5616F544@lemburg.com> I just tried (1,2,3) == [1,2,3] and found that the compare returns false. Is that intended ? Background: 1.0 == 1 == 1L also works, so it seems natural that comparing sequences of different types should work in the same way. I would even expect a two iterators to compare equal if they always return the same values (this could cause an endless loop for some endless iterators ;-). -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From fdrake@acm.org Thu Dec 6 13:25:15 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 6 Dec 2001 08:25:15 -0500 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: References: <15375.5918.776405.747630@grendel.zope.com> Message-ID: <15375.29115.974845.732239@grendel.zope.com> Tim Peters writes: > better-at-eating-kung-pao-chicken-than-motivating-the-overburdened- > to-spread-themselves-thinner-than-gas-ly y'rs - tim Ah, but gas can be spread infinately thin -- something that can't be said of most developers! ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From guido@python.org Thu Dec 6 13:30:58 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 06 Dec 2001 08:30:58 -0500 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: Your message of "Wed, 05 Dec 2001 23:14:00 EST." References: Message-ID: <200112061330.IAA21262@cj20424-a.reston1.va.home.com> > Can anyone lend a hand here? Take a test, stick it in an infinite loop, > watch your process size blow off the roof, then whittle the test down until > the trigger becomes clear. We found a very serious 2.2 leak today doing > this to test_binop.py, and not even Guido (let alone God ...) knows how many > more may be lurking. It's not a Good Sign that test_binop has vanished from > the list now but everything else is still there. I'll take test_descr.py right now. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Dec 6 13:38:54 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 06 Dec 2001 08:38:54 -0500 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: Your message of "Wed, 05 Dec 2001 23:36:51 EST." References: Message-ID: <200112061338.IAA21671@cj20424-a.reston1.va.home.com> I can't find a leak in test_descr.py when I simply call test_main() in an infinite loop. (I can when I use reload() in the loop, but that's an import issue I believe.) Neil, did you test this before or after I fixed the __slots__ leak? --Guido van Rossum (home page: http://www.python.org/~guido/) From mwh@python.net Thu Dec 6 13:42:23 2001 From: mwh@python.net (Michael Hudson) Date: 06 Dec 2001 08:42:23 -0500 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: "Martin v. Loewis"'s message of "Thu, 6 Dec 2001 08:40:22 +0100" References: <15374.65414.640831.895781@grendel.zope.com> <15375.5918.776405.747630@grendel.zope.com> <3C0F1A6D.417B603C@ActiveState.com> <200112060740.fB67eMV08466@mira.informatik.hu-berlin.de> Message-ID: <2my9kgcvnk.fsf@starship.python.net> "Martin v. Loewis" writes: > > It's hard. Bugzilla has some good features there w.r.t. "unverified" > > vs. "new" vs. "accepted" and "fixed" vs "resolved", QA contacts, keyword > > handling for milestone/release targets, etc. I'm not sure how the SF > > bugtracker compares in practice but from a distance it seems a little > > weak there. > > SF recommends to use the Group for "unsupported", > "unverified". Unfortunately, the filtering capabilities leave a lot of > wishes. Is roundup usable yet? I played with it a bit yesterday, but I'm not really sure what's needed. It would have the advantage that if it didn't do something we needed, we could probably club it into doing it easily enough. Cheers, M. -- I never disputed the Perl hacking skill of the Slashdot creators. My objections are to the editors' taste, the site's ugly visual design, and the Slashdot community's raging stupidity. -- http://www.cs.washington.edu/homes/klee/misc/slashdot.html#faq From guido@python.org Thu Dec 6 13:48:25 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 06 Dec 2001 08:48:25 -0500 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: Your message of "Thu, 06 Dec 2001 08:27:33 +0100." <200112060727.fB67RXZ08417@mira.informatik.hu-berlin.de> References: <200112060727.fB67RXZ08417@mira.informatik.hu-berlin.de> Message-ID: <200112061348.IAA22096@cj20424-a.reston1.va.home.com> > As for the things we should change: I believe it is pointless to sit > on a report for a long time without doing anything about this. If > nothing happens for a year, the person that was assigned this report > likely won't do anything about it for the next months, either. > Unassigning things you know you won't do is particularly important for > patches, but to a degree also important for bug reports. It signals > that somebody else may need to step forward, or that some deity should > just decide on the fate of this report. Yup. We've missed important bugs in the past because they happened to be assigned to someone who didn't care. (I noticed that Jack recommended some bug reporter to assign bugs to him rather than let them sit anonymously. That may work in his case, but in general it's the wrong thing to do.) --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik@pythonware.com Thu Dec 6 13:50:51 2001 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 6 Dec 2001 14:50:51 +0100 Subject: [Python-Dev] memory leaks in 2.2 References: <15374.65414.640831.895781@grendel.zope.com> <15375.5918.776405.747630@grendel.zope.com> <3C0F1A6D.417B603C@ActiveState.com> <200112060740.fB67eMV08466@mira.informatik.hu-berlin.de> <2my9kgcvnk.fsf@starship.python.net> Message-ID: <01ad01c17e5d$051fe840$0900a8c0@spiff> Michael Hudson wrote: > Is roundup usable yet? I played with it a bit yesterday, but I'm not > really sure what's needed. It would have the advantage that if it > didn't do something we needed, we could probably club it into doing it > easily enough. the drawback with "easily enough" is that you cannot resist modifying it -- so instead of learning how to use the tool you have in the best way, you end up hacking on the tool instead of doing more important work. (fwiw, we're using a very slightly modified version of Ping's original one-day hack. took a couple of months to find the right "use patterns", but now that we know how to use it in a way that fits our development process, I doubt you can find a "more usable" tool...) From guido@python.org Thu Dec 6 13:59:53 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 06 Dec 2001 08:59:53 -0500 Subject: [Python-Dev] Comparing sequences ?! In-Reply-To: Your message of "Thu, 06 Dec 2001 13:12:46 +0100." <3C0F60BE.5616F544@lemburg.com> References: <3C0F60BE.5616F544@lemburg.com> Message-ID: <200112061359.IAA22524@cj20424-a.reston1.va.home.com> > I just tried (1,2,3) == [1,2,3] and found that the compare > returns false. Is that intended ? Yes. --Guido van Rossum (home page: http://www.python.org/~guido/) From neal@metaslash.com Thu Dec 6 14:01:16 2001 From: neal@metaslash.com (Neal Norwitz) Date: Thu, 06 Dec 2001 09:01:16 -0500 Subject: [Python-Dev] memory leaks in 2.2 References: <200112061338.IAA21671@cj20424-a.reston1.va.home.com> Message-ID: <3C0F7A2C.1B416FF0@metaslash.com> Guido van Rossum wrote: > > I can't find a leak in test_descr.py when I simply call test_main() in > an infinite loop. (I can when I use reload() in the loop, but that's > an import issue I believe.) > > Neil, did you test this before or after I fixed the __slots__ leak? I believe I tested after your initial checkin, but before the most recent. I just updated and re-ran. 117 bytes were still leaked. I'll try to narrow down the test to get a smaller test case. Neal From guido@python.org Thu Dec 6 14:01:06 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 06 Dec 2001 09:01:06 -0500 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: Your message of "Thu, 06 Dec 2001 08:38:54 EST." <200112061338.IAA21671@cj20424-a.reston1.va.home.com> References: <200112061338.IAA21671@cj20424-a.reston1.va.home.com> Message-ID: <200112061401.JAA22566@cj20424-a.reston1.va.home.com> > I can't find a leak in test_descr.py when I simply call test_main() in > an infinite loop. (I can when I use reload() in the loop, but that's > an import issue I believe.) > > Neil, did you test this before or after I fixed the __slots__ leak? Never mind, with Ping's hint I see a leak. I'll fixit momentarily. --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@zope.com Thu Dec 6 14:15:34 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Thu, 6 Dec 2001 09:15:34 -0500 (EST) Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: <3C0EDCFE.D01E6EA3@metaslash.com> References: <3C0EDCFE.D01E6EA3@metaslash.com> Message-ID: <15375.32134.907220.808367@slothrop.digicool.com> >>>>> "NN" == Neal Norwitz writes: NN> Here is a list of all the outstanding memory leaks from the NN> regression tests. All of these leaks have a bug report on SF. I don't see an SF bug report for test_scope. If you have any data from Purify, that would be helpful. It certainly looks like it's leaking. Jeremy From neal@metaslash.com Thu Dec 6 14:18:47 2001 From: neal@metaslash.com (Neal Norwitz) Date: Thu, 06 Dec 2001 09:18:47 -0500 Subject: [Python-Dev] memory leaks in 2.2 References: <3C0EDCFE.D01E6EA3@metaslash.com> <15375.32134.907220.808367@slothrop.digicool.com> Message-ID: <3C0F7E47.BF969740@metaslash.com> Jeremy Hylton wrote: > > >>>>> "NN" == Neal Norwitz writes: > > NN> Here is a list of all the outstanding memory leaks from the > NN> regression tests. All of these leaks have a bug report on SF. > > I don't see an SF bug report for test_scope. If you have any data > from Purify, that would be helpful. It certainly looks like it's > leaking. https://sourceforge.net/tracker/index.php?func=detail&aid=485152&group_id=5470&atid=105470 From neal@metaslash.com Thu Dec 6 14:24:19 2001 From: neal@metaslash.com (Neal Norwitz) Date: Thu, 06 Dec 2001 09:24:19 -0500 Subject: [Python-Dev] memory leaks in 2.2 References: <200112061338.IAA21671@cj20424-a.reston1.va.home.com> <200112061401.JAA22566@cj20424-a.reston1.va.home.com> Message-ID: <3C0F7F93.5555623B@metaslash.com> Guido van Rossum wrote: > > > I can't find a leak in test_descr.py when I simply call test_main() in > > an infinite loop. (I can when I use reload() in the loop, but that's > > an import issue I believe.) > > > > Neil, did you test this before or after I fixed the __slots__ leak? > > Never mind, with Ping's hint I see a leak. I'll fixit momentarily. In test_descr, buffer_inherit() leaks 35 bytes which may indicate the problem. Neal From mal@lemburg.com Thu Dec 6 14:26:28 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 06 Dec 2001 15:26:28 +0100 Subject: [Python-Dev] Comparing sequences ?! References: <3C0F60BE.5616F544@lemburg.com> <200112061359.IAA22524@cj20424-a.reston1.va.home.com> Message-ID: <3C0F8014.FA2ECF58@lemburg.com> Guido van Rossum wrote: > > > I just tried (1,2,3) == [1,2,3] and found that the compare > > returns false. Is that intended ? > > Yes. I presume that means "...and we're not going to change it." ;-) -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From barry@zope.com Thu Dec 6 14:36:11 2001 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 6 Dec 2001 09:36:11 -0500 Subject: [Python-Dev] on the subject of SF... References: <200112060745.fB67j4e18189@mbuna.arbhome.com.au> <15375.8942.722195.533597@anthem.wooz.org> <3C0F2EE0.BE1D2980@lemburg.com> Message-ID: <15375.33371.540885.525907@anthem.wooz.org> >>>>> "M" == M writes: M> And please also a Group "Python 2.3" so that feature requests M> for that version can be assigned to it ?! So far we have no groups defined for the feature request tracker. I wonder if that's intentional. -Barry From guido@python.org Thu Dec 6 14:47:18 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 06 Dec 2001 09:47:18 -0500 Subject: [Python-Dev] Comparing sequences ?! In-Reply-To: Your message of "Thu, 06 Dec 2001 15:26:28 +0100." <3C0F8014.FA2ECF58@lemburg.com> References: <3C0F60BE.5616F544@lemburg.com> <200112061359.IAA22524@cj20424-a.reston1.va.home.com> <3C0F8014.FA2ECF58@lemburg.com> Message-ID: <200112061447.JAA23518@cj20424-a.reston1.va.home.com> > > > I just tried (1,2,3) == [1,2,3] and found that the compare > > > returns false. Is that intended ? > > > > Yes. > > I presume that means "...and we're not going to change it." ;-) Not any time soon. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Thu Dec 6 21:52:16 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 06 Dec 2001 16:52:16 -0500 Subject: [Python-Dev] Need help with distutils Message-ID: <200112062152.fB6LqGT11265@odiug.zope.com> Andrew Kuchling just unassigned from himself a whole slew of distutils-related bug reports. It appears that neither Andrew nor Greg Ward (the original maintainer) has the time to maintain this code any more. No-one at PythonLabs understands it or has the time to learn about it. So it's effectively orphaned. Does anybody want to volunteer? --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Thu Dec 6 22:39:05 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 06 Dec 2001 23:39:05 +0100 Subject: [Python-Dev] Need help with distutils References: <200112062152.fB6LqGT11265@odiug.zope.com> Message-ID: <3C0FF389.9C5FF5A5@lemburg.com> Guido van Rossum wrote: > > Andrew Kuchling just unassigned from himself a whole slew of > distutils-related bug reports. It appears that neither Andrew nor > Greg Ward (the original maintainer) has the time to maintain this code > any more. No-one at PythonLabs understands it or has the time to > learn about it. So it's effectively orphaned. > > Does anybody want to volunteer? The distutils list can take over, I guess. I will certainly want to see distutils have a bright future and so do many other users like e.g. Thomas Heller, Jack Jansen, etc. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From DavidA@ActiveState.com Fri Dec 7 00:56:46 2001 From: DavidA@ActiveState.com (David Ascher) Date: Thu, 06 Dec 2001 16:56:46 -0800 Subject: [Python-Dev] memory leaks in 2.2 References: <15374.65414.640831.895781@grendel.zope.com> <15375.5918.776405.747630@grendel.zope.com> <3C0F1A6D.417B603C@ActiveState.com> <200112060740.fB67eMV08466@mira.informatik.hu-berlin.de> <2my9kgcvnk.fsf@starship.python.net> <01ad01c17e5d$051fe840$0900a8c0@spiff> Message-ID: <3C1013CE.B1AE8D82@ActiveState.com> Fredrik Lundh wrote: > > the drawback with "easily enough" is that you cannot resist > modifying it -- so instead of learning how to use the tool you > have in the best way, you end up hacking on the tool instead > of doing more important work. Indeed =) > (fwiw, we're using a very slightly modified version of Ping's > original one-day hack. took a couple of months to find the > right "use patterns", but now that we know how to use it in > a way that fits our development process, I doubt you can > find a "more usable" tool...) FWIW, we used to use a fairly seriously modified version of Ping's original one-day hack, and gave it up. It was generating too much email and the 'nosylist' concept wasn't working for us. The more explicit list management that bugzilla uses seems to work better for us. YMMV. --david From Keila - Curitiba" Ol=E1! Veja meu site pessoal. Basta clicar no endere=E7o abaixo. GARANTO SER SUI-GENERIS - CLIQUE ABAIXO: http://www.pastorinha.atfreeweb.com Mais de 162.000 internautas visitaram a PG., existe 6 =C1lbuns: Se voc=EA quiser, por favor, indique minha Home Page, a outros Internautas. Mais detalhes, se comunique, passe um e-mail, que responderei brevemente. Dentro da Home Page, ao lado das fotos, voc=EA poder=E1 saber muito mais sobre mim! Obrigada. e-mail: arosadesaron@zipmail.com.br Beijos:- Keila - Curitiba - Pr - Podes falar comigo, direto dela. Brevemente uma Carta Aberta. - Embora derrubem a p=E1gina eu a subo em 3 horas novamente. "Esta mensagem =E9 enviada com a complac=EAncia da nova legisla=E7=E3o=20 sobre correio eletr=F4nico, Se=E7=E3o 301, Par=E1grafo (a) (2) (c) Decreto S. 1= 618, T=EDtulo Terceiro aprovado pelo "105=BA Congresso Base das Normativas Internacionais sobre o SPAM". Este E-mail n=E3o poder=E1 ser considerado SPAM quando incluir uma forma de ser removido. Para ser removido de futuros correios, simplesmente responda indicando no Assunto: REMOVER" From gward@python.net Fri Dec 7 02:27:57 2001 From: gward@python.net (Greg Ward) Date: Thu, 6 Dec 2001 21:27:57 -0500 Subject: [Python-Dev] Mac OS X sys.platform value In-Reply-To: <20011205155939.8936A303183@snelboot.oratrix.nl> References: <20011205155939.8936A303183@snelboot.oratrix.nl> Message-ID: <20011206212757.A9167@gerg.ca> On 05 December 2001, Jack Jansen said: > Anyway, where sys.platform used to be "darwin1" upto and including 10.1.0 it > is now "darwin5" for 10.1.1 (and it will go up to "darwin6" for 10.2, etc). > > I am of a mind to take the "1" out of sys.platform, so that it becomes > "darwin". +1 from me. If we could turn the clock back 5 or 6 years, I would also do the same on Linux. Note that I share Marc-André's dim view of tacking the first digit of the kernel version number (or whatever) onto sys.platform -- the Distutils has Yet Another way of creating platform descriptions. It's more complex than sys.platform, but quite a lot less fancy than Marc-André's platform.py. See distutils.util.get_platform(), if you're curious. Greg -- Greg Ward - Linux weenie gward@python.net http://starship.python.net/~gward/ Never put off till tomorrow what you can put off till the day after tomorrow. From anthony@interlink.com.au Fri Dec 7 03:58:41 2001 From: anthony@interlink.com.au (Anthony Baxter) Date: Fri, 07 Dec 2001 14:58:41 +1100 Subject: [Python-Dev] dumbdbm.py Message-ID: <200112070358.fB73wf425864@mbuna.arbhome.com.au> just had reason to be looking at dumbdbm.py - perhaps "terminallybrokendbm.py" would be a better name. There's a whole host of "oh. my. god." type things in there that are just broken, with massive dataloss problems if you end up (innocently) getting it from anydbm.py. I'm going to fix it either this arvo/evening or tomorrow, but it really shouldn't go out as part of 2.2 in it's current form. Is it too late to consider either fixing it or removing it from anydbm's list? Anthony From guido@python.org Fri Dec 7 04:03:29 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 06 Dec 2001 23:03:29 -0500 Subject: [Python-Dev] dumbdbm.py In-Reply-To: Your message of "Fri, 07 Dec 2001 14:58:41 +1100." <200112070358.fB73wf425864@mbuna.arbhome.com.au> References: <200112070358.fB73wf425864@mbuna.arbhome.com.au> Message-ID: <200112070403.XAA07071@cj20424-a.reston1.va.home.com> > just had reason to be looking at dumbdbm.py - perhaps > "terminallybrokendbm.py" would be a better name. There's a whole host of > "oh. my. god." type things in there that are just broken, with massive > dataloss problems if you end up (innocently) getting it from anydbm.py. > > I'm going to fix it either this arvo/evening or tomorrow, but it really > shouldn't go out as part of 2.2 in it's current form. Is it too late to > consider either fixing it or removing it from anydbm's list? Please fix it. It was never intended to be production quality. But it has found more use than it should. Then again, dbhash.py isn't much better...[*] :-( --Guido van Rossum (home page: http://www.python.org/~guido/) [*] http://www.sleepycat.com/historic.html From mwh@python.net Fri Dec 7 15:59:13 2001 From: mwh@python.net (Michael Hudson) Date: 07 Dec 2001 10:59:13 -0500 Subject: [Python-Dev] on the subject of SF... In-Reply-To: barry@zope.com's message of "Thu, 6 Dec 2001 09:36:11 -0500" References: <200112060745.fB67j4e18189@mbuna.arbhome.com.au> <15375.8942.722195.533597@anthem.wooz.org> <3C0F2EE0.BE1D2980@lemburg.com> <15375.33371.540885.525907@anthem.wooz.org> Message-ID: <2mu1v33tta.fsf@starship.python.net> barry@zope.com (Barry A. Warsaw) writes: > >>>>> "M" == M writes: > > M> And please also a Group "Python 2.3" so that feature requests > M> for that version can be assigned to it ?! > > So far we have no groups defined for the feature request tracker. I > wonder if that's intentional. Does anybody look at the feature request tracker ever? I notice I'm one of four possible assignees (presumably 'cause I'm new). Does email from it go anywhere? Cheers, M. -- Slim Shady is fed up with your shit, and he's going to kill you. -- Eminem, "Public Service Announcement 2000" From guido@python.org Fri Dec 7 16:13:49 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 07 Dec 2001 11:13:49 -0500 Subject: [Python-Dev] on the subject of SF... In-Reply-To: Your message of "07 Dec 2001 10:59:13 EST." <2mu1v33tta.fsf@starship.python.net> References: <200112060745.fB67j4e18189@mbuna.arbhome.com.au> <15375.8942.722195.533597@anthem.wooz.org> <3C0F2EE0.BE1D2980@lemburg.com> <15375.33371.540885.525907@anthem.wooz.org> <2mu1v33tta.fsf@starship.python.net> Message-ID: <200112071613.LAA09963@cj20424-a.reston1.va.home.com> > Does anybody look at the feature request tracker ever? I notice I'm > one of four possible assignees (presumably 'cause I'm new). Does > email from it go anywhere? New features were supposed to be sent to python-bugs-list@python.org, but somehow that wasn't set, so no, nobody has looked at this in a long time... :-( I'll have a look now, and I've set the email address. --Guido van Rossum (home page: http://www.python.org/~guido/) From neal@metaslash.com Fri Dec 7 16:15:24 2001 From: neal@metaslash.com (Neal Norwitz) Date: Fri, 07 Dec 2001 11:15:24 -0500 Subject: [Python-Dev] update on memory leaks in 2.2 Message-ID: <3C10EB1C.9302CA8@metaslash.com> Command line: (longexp crashes purify for some reason) ./python -E -tt ./Lib/test/regrtest.py -u network -x test_longexp.py Results: 168 tests OK. 1 test failed: test_socket_ssl 15 tests skipped: test_al test_bsddb test_cd test_cl test_curses test_dl test_gl test_imgfile test_linuxaudiodev test_nis test_ntpath test_openpty test_sundry test_winreg test_winsound test test_socket_ssl crashed -- socket.sslerror: SSL_CTX_new error When running test_curses: test test_curses crashed -- _curses.error: curs_set() returned ERR Uninitialized Memory Reads (don't think these are python's fault): gdbm_open() gdbmmodule.c:68 getaddrinfo() socketmodule.c:636 Memory Leaks: NONE Potential Memory Leaks: 1012 from 4 different allocating points -- all in WeakRef Things look great! Let me know if any of the skipped tests ought to have been run. Most likely possibility seems to be bsddb, sundry also needs bsddb. Neal From guido@python.org Fri Dec 7 16:26:58 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 07 Dec 2001 11:26:58 -0500 Subject: [Python-Dev] tracker admin Message-ID: <200112071626.LAA09993@cj20424-a.reston1.va.home.com> I just discovered a few new (?) features of the bug/patch tracker admin page. - You can disable anonymous submission. I've done this for all our trackers (bugs, patches, features), since we have way too many problems with anonymous submissions where the user doesn't provide enough info and we have no way to get back to them. - You can provide custom text for the "submit new" page. I've added a warning about the annoying upload checkbox, and some additional suggestions for patches. Please feel free to provide additional suggestions. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Dec 7 16:38:35 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 07 Dec 2001 11:38:35 -0500 Subject: [Python-Dev] update on memory leaks in 2.2 In-Reply-To: Your message of "Fri, 07 Dec 2001 11:15:24 EST." <3C10EB1C.9302CA8@metaslash.com> References: <3C10EB1C.9302CA8@metaslash.com> Message-ID: <200112071638.LAA12876@cj20424-a.reston1.va.home.com> > Command line: (longexp crashes purify for some reason) > ./python -E -tt ./Lib/test/regrtest.py -u network -x test_longexp.py > > Results: > 168 tests OK. > 1 test failed: > test_socket_ssl > 15 tests skipped: > test_al test_bsddb test_cd test_cl test_curses test_dl test_gl > test_imgfile test_linuxaudiodev test_nis test_ntpath test_openpty > test_sundry test_winreg test_winsound A few of these skips weren't expected: test_sundry, test_openpty, test_bsddb. What's missing to run those? > test test_socket_ssl crashed -- socket.sslerror: SSL_CTX_new error Works for me. Probably your platform. The SSL code sucks anyway. > When running test_curses: > > test test_curses crashed -- _curses.error: curs_set() returned ERR Works for me. > > Uninitialized Memory Reads (don't think these are python's fault): > > gdbm_open() gdbmmodule.c:68 > getaddrinfo() socketmodule.c:636 > > Memory Leaks: > > NONE Woohoo! So I'll close the remaining leak report! > Potential Memory Leaks: > > 1012 from 4 different allocating points -- all in WeakRef Any more info on these, or are these just red herrings? > Things look great! Agreed. > Let me know if any of the skipped tests ought to have been run. > Most likely possibility seems to be bsddb, sundry also needs bsddb. Ah, that's why sundry fails. I think it shouldn't depend on bsddb; I'll look into that. --Guido van Rossum (home page: http://www.python.org/~guido/) From neal@metaslash.com Fri Dec 7 16:54:54 2001 From: neal@metaslash.com (Neal Norwitz) Date: Fri, 07 Dec 2001 11:54:54 -0500 Subject: [Python-Dev] update on memory leaks in 2.2 References: <3C10EB1C.9302CA8@metaslash.com> <200112071638.LAA12876@cj20424-a.reston1.va.home.com> Message-ID: <3C10F45E.6476A376@metaslash.com> Guido van Rossum wrote: > > > Command line: (longexp crashes purify for some reason) > > ./python -E -tt ./Lib/test/regrtest.py -u network -x test_longexp.py > > > > Results: > > 168 tests OK. > > 1 test failed: > > test_socket_ssl > > 15 tests skipped: > > test_al test_bsddb test_cd test_cl test_curses test_dl test_gl > > test_imgfile test_linuxaudiodev test_nis test_ntpath test_openpty > > test_sundry test_winreg test_winsound > > A few of these skips weren't expected: test_sundry, test_openpty, > test_bsddb. What's missing to run those? The platform is Sun Solaris 8 (2.8). Purify 2002. For bsddb, do I need to install something that isn't "standard"? test test_openpty skipped -- No openpty() available. > > Potential Memory Leaks: > > > > 1012 from 4 different allocating points -- all in WeakRef > > Any more info on these, or are these just red herrings? The details are below. Neal -- PLK: 704 bytes potentially leaked in 16 blocks This memory was allocated from: malloc [rtlib.o] _PyObject_GC_New [gcmodule.c:832] PyWeakref_NewRef [weakrefobject.c:37] weakref_ref [_weakref.c:73] PyCFunction_Call [methodobject.c:101] eval_frame [ceval.c:1989] PyEval_EvalCodeEx [ceval.c:2570] function_call [funcobject.c:374] PyObject_Call [abstract.c:1665] instancemethod_call [classobject.c:2276] PyObject_Call [abstract.c:1665] PyEval_CallObjectWithKeywords [ceval.c:3034] Block of 44 bytes (16 times); last block at 0x18bf0a8 -- PLK: 220 bytes potentially leaked in 5 blocks This memory was allocated from: malloc [rtlib.o] _PyObject_GC_New [gcmodule.c:832] PyWeakref_NewRef [weakrefobject.c:37] add_subclass [typeobject.c:2054] PyType_Ready [typeobject.c:2026] type_new [typeobject.c:1142] type_call [typeobject.c:148] PyObject_Call [abstract.c:1665] PyEval_CallObjectWithKeywords [ceval.c:3034] PyObject_CallFunction [abstract.c:1656] build_class [ceval.c:3583] eval_frame [ceval.c:1507] Block of 44 bytes (5 times); last block at 0xced9b8 -- PLK: 44 bytes potentially leaked at 0xcb2ba8 This memory was allocated from: malloc [rtlib.o] _PyObject_GC_New [gcmodule.c:832] PyWeakref_NewRef [weakrefobject.c:37] add_subclass [typeobject.c:2054] PyType_Ready [typeobject.c:2026] type_new [typeobject.c:1142] tp_new_wrapper [typeobject.c:2586] PyCFunction_Call [methodobject.c:69] PyObject_Call [abstract.c:1665] do_call [ceval.c:3247] eval_frame [ceval.c:1984] PyEval_EvalCodeEx [ceval.c:2570] -- PLK: 44 bytes potentially leaked at 0xcb3470 This memory was allocated from: malloc [rtlib.o] _PyObject_GC_New [gcmodule.c:832] PyWeakref_NewRef [weakrefobject.c:37] add_subclass [typeobject.c:2054] PyType_Ready [typeobject.c:2026] type_new [typeobject.c:1142] tp_new_wrapper [typeobject.c:2586] PyCFunction_Call [methodobject.c:69] PyObject_Call [abstract.c:1665] slot_tp_new [typeobject.c:3355] type_call [typeobject.c:148] PyObject_Call [abstract.c:1665] From neal@metaslash.com Fri Dec 7 16:58:53 2001 From: neal@metaslash.com (Neal Norwitz) Date: Fri, 07 Dec 2001 11:58:53 -0500 Subject: [Python-Dev] update on memory leaks in 2.2 References: <3C10EB1C.9302CA8@metaslash.com> <200112071638.LAA12876@cj20424-a.reston1.va.home.com> Message-ID: <3C10F54D.F20C2C71@metaslash.com> Guido van Rossum wrote: > Ah, that's why sundry fails. I think it shouldn't depend on bsddb; > I'll look into that. test_sundry passes with no leaks now. Neal From fdrake@acm.org Fri Dec 7 18:01:36 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 7 Dec 2001 13:01:36 -0500 Subject: [Python-Dev] update on memory leaks in 2.2 In-Reply-To: <3C10F45E.6476A376@metaslash.com> References: <3C10EB1C.9302CA8@metaslash.com> <200112071638.LAA12876@cj20424-a.reston1.va.home.com> <3C10F45E.6476A376@metaslash.com> Message-ID: <15377.1024.829156.22274@grendel.zope.com> Neal Norwitz's Purify writes: > PLK: 704 bytes potentially leaked in 16 blocks > This memory was allocated from: > malloc [rtlib.o] > _PyObject_GC_New [gcmodule.c:832] > PyWeakref_NewRef [weakrefobject.c:37] > weakref_ref [_weakref.c:73] > PyCFunction_Call [methodobject.c:101] > eval_frame [ceval.c:1989] > PyEval_EvalCodeEx [ceval.c:2570] > function_call [funcobject.c:374] > PyObject_Call [abstract.c:1665] > instancemethod_call [classobject.c:2276] > PyObject_Call [abstract.c:1665] > PyEval_CallObjectWithKeywords [ceval.c:3034] > Block of 44 bytes (16 times); last block at 0x18bf0a8 Free'd weakref objects are stored in a free list, so it makes sense that these are labelled potential leaks. We could either give up using a free list for these, or we could allocate blocks of these rather than allocating them individually. The latter would allow still better performance and would reduce the malloc overhead. That change would be a higher risk than tossing the free list, and tossing it would be a higher risk than keeping it this close to the release. But we're probably far enough away that any of the options (no free list, blocking allocation, and leaving it alone) are manageable. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From barry@zope.com Fri Dec 7 17:20:22 2001 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 7 Dec 2001 12:20:22 -0500 Subject: [Python-Dev] update on memory leaks in 2.2 References: <3C10EB1C.9302CA8@metaslash.com> <200112071638.LAA12876@cj20424-a.reston1.va.home.com> <3C10F45E.6476A376@metaslash.com> <15377.1024.829156.22274@grendel.zope.com> Message-ID: <15376.64086.678564.584731@anthem.wooz.org> Note that there are tons of memory that is validly still alive when Python exists. As we've seen, trying to clean all that up /can/ have a big negative impact on shutdown. So I wouldn't worry about them. Insure labels them as (paraphrasing) "memory still allocated at exit", distinct from memory leaked. I think this is equivalent to Purify's "potentially leaked" and "leaked" memory respectively. Note that at least stringobject.c has a hook for clearing out all those interned strings at program shutdown time, which I added to clean up Insure's output. When INTERN_STRINGS is defined, the extra function _Py_ReleaseInternedStrings() is defined. Then, in Modules/main.c, in Py_Main(), if __INSURE__ is defined that function is called, so this memory doesn't show up in the report. It may be worthwhile generalizing this approach, and adding it to other static long-lived data structures, simply as an aid to memory debugging. As long as it isn't enabled by default, that should be fine. -Barry From guido@python.org Fri Dec 7 17:28:15 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 07 Dec 2001 12:28:15 -0500 Subject: [Python-Dev] update on memory leaks in 2.2 In-Reply-To: Your message of "Fri, 07 Dec 2001 13:01:36 EST." <15377.1024.829156.22274@grendel.zope.com> References: <3C10EB1C.9302CA8@metaslash.com> <200112071638.LAA12876@cj20424-a.reston1.va.home.com> <3C10F45E.6476A376@metaslash.com> <15377.1024.829156.22274@grendel.zope.com> Message-ID: <200112071728.MAA13868@cj20424-a.reston1.va.home.com> > Neal Norwitz's Purify writes: > > PLK: 704 bytes potentially leaked in 16 blocks > > This memory was allocated from: > > malloc [rtlib.o] > > _PyObject_GC_New [gcmodule.c:832] > > PyWeakref_NewRef [weakrefobject.c:37] > > weakref_ref [_weakref.c:73] > > PyCFunction_Call [methodobject.c:101] > > eval_frame [ceval.c:1989] > > PyEval_EvalCodeEx [ceval.c:2570] > > function_call [funcobject.c:374] > > PyObject_Call [abstract.c:1665] > > instancemethod_call [classobject.c:2276] > > PyObject_Call [abstract.c:1665] > > PyEval_CallObjectWithKeywords [ceval.c:3034] > > Block of 44 bytes (16 times); last block at 0x18bf0a8 > > Free'd weakref objects are stored in a free list, so it makes sense > that these are labelled potential leaks. > We could either give up using a free list for these, or we could > allocate blocks of these rather than allocating them individually. > The latter would allow still better performance and would reduce the > malloc overhead. That change would be a higher risk than tossing the > free list, and tossing it would be a higher risk than keeping it this > close to the release. But we're probably far enough away that any of > the options (no free list, blocking allocation, and leaving it alone) > are manageable. Thanks for the explanation! The simplest way to avoid the Purify "potential leak" complaints then might be to add a way to explicitly zap this free list at Py_Finalize() time, like we do with other custom free lists. That should be a post-2.2 feature. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@home.com Fri Dec 7 17:35:21 2001 From: tim.one@home.com (Tim Peters) Date: Fri, 7 Dec 2001 12:35:21 -0500 Subject: [Python-Dev] update on memory leaks in 2.2 In-Reply-To: <15377.1024.829156.22274@grendel.zope.com> Message-ID: [Fred L. Drake, Jr.] > Free'd weakref objects are stored in a free list, so it makes sense > that these are labelled potential leaks. > We could either give up using a free list for these, or we could > allocate blocks of these rather than allocating them individually. > The latter would allow still better performance and would reduce the > malloc overhead. -0. When we turn on pymalloc in 2.3, blocks will get allocated by magic, and more space-efficiently than you're going to do by hand (it bites one malloc overhead per 256KB). pymalloc will likely be a little slower than a wholly customized free list, but it will get all the memory benefit, and unlike type-specific free lists allows memory to be recycled across types as programs progress. > That change would be a higher risk than tossing the free list, and > tossing it would be a higher risk than keeping it this close to the > release. But we're probably far enough away that any of the options > (no free list, blocking allocation, and leaving it alone) are > manageable. There's another possibility: as float objects do, add a module xxx_fini() routine to clear the free list at shutdown. From fdrake@acm.org Fri Dec 7 18:33:06 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 7 Dec 2001 13:33:06 -0500 Subject: [Python-Dev] update on memory leaks in 2.2 In-Reply-To: References: <15377.1024.829156.22274@grendel.zope.com> Message-ID: <15377.2914.145010.889503@grendel.zope.com> Tim Peters writes: > -0. When we turn on pymalloc in 2.3, blocks will get allocated by magic, > and more space-efficiently than you're going to do by hand (it bites one > malloc overhead per 256KB). pymalloc will likely be a little slower than a That's good enough for me! I'll look forward to ripping the free list out completely. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From guido@python.org Fri Dec 7 17:44:04 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 07 Dec 2001 12:44:04 -0500 Subject: [Python-Dev] update on memory leaks in 2.2 In-Reply-To: Your message of "Fri, 07 Dec 2001 12:20:22 EST." <15376.64086.678564.584731@anthem.wooz.org> References: <3C10EB1C.9302CA8@metaslash.com> <200112071638.LAA12876@cj20424-a.reston1.va.home.com> <3C10F45E.6476A376@metaslash.com> <15377.1024.829156.22274@grendel.zope.com> <15376.64086.678564.584731@anthem.wooz.org> Message-ID: <200112071744.MAA13971@cj20424-a.reston1.va.home.com> > Insure labels them as (paraphrasing) "memory still allocated at exit", > distinct from memory leaked. I think this is equivalent to Purify's > "potentially leaked" and "leaked" memory respectively. I don't think so. Insure (uselessly) lists *all* memory that's still allocated and reachable at exit. Purify only reports certain blocks (in this case they were a bunch of weak refs) as potentially leaked. I'm not sure what makes them potentially leaked, but it must be a stronger condition than "still exists at exit". As Fred mentioned, this may have to do with the weaklist free list implementation. > Note that at least stringobject.c has a hook for clearing out all > those interned strings at program shutdown time, which I added to > clean up Insure's output. When INTERN_STRINGS is defined, the extra > function _Py_ReleaseInternedStrings() is defined. Then, in > Modules/main.c, in Py_Main(), if __INSURE__ is defined that function > is called, so this memory doesn't show up in the report. > > It may be worthwhile generalizing this approach, and adding it to > other static long-lived data structures, simply as an aid to memory > debugging. As long as it isn't enabled by default, that should be > fine. Most types that have a custom free list already have a cleanup routine that's always called by Py_Finalize(); the weakref free list is lacking one. The interned list is special because it's generally unsafe to clear it out when Py_Finalize() might be followed by Py_Initialize() again -- that's why it's only done when running Insure++. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Fri Dec 7 18:02:30 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 07 Dec 2001 13:02:30 -0500 Subject: [Python-Dev] update on memory leaks in 2.2 In-Reply-To: Your message of "Fri, 07 Dec 2001 12:56:51 EST." <3C1102E3.447B6D8B@metaslash.com> References: <3C10EB1C.9302CA8@metaslash.com> <200112071638.LAA12876@cj20424-a.reston1.va.home.com> <3C10F45E.6476A376@metaslash.com> <200112071713.MAA13808@cj20424-a.reston1.va.home.com> <3C1102E3.447B6D8B@metaslash.com> Message-ID: <200112071802.NAA14180@cj20424-a.reston1.va.home.com> > Guido van Rossum wrote: > > > What's a potential leak? Could it be that these are in cycles? The > > reported blocks are all weak reference objects; most of then are > > allocated by add_subclass(), which uses weak refs for the list of > > subclasses of a given (new-style) class. I suppose most weak refs are > > in cycles (else there would be no need for weak refs). Neal replied (in private email): > From page 157 of this doc: > > http://www.rational.com/support/documentation/manuals/unixsuites/pdf/purify/purify.pdf > > A PLK message describes heap memory that you might have > leaked. You have pointers only to the middle of the region. PLK is a > warning message. > > Memory in use can sometimes appear as a PLK if the pointer > returned by malloc is offset. A common cause is referencing a > substring within a large string. Another example is when a > pointer to a C++ object is cast to the second or later base class of a > multiply-inherited object. It is offset past the other base class > objects. > > Truly leaked memory can sometimes appear as a PLK, if some > non-pointer integer within the program space, when interpreted > as a pointer, points within an otherwise leaked block of memory. > This is rather rare. Inspect the code to differentiate between these > causes of PLK reports. Thanks, that's very helpful! Here's how I now understand what's happened. Weak references are garbage collected, which means that the malloc pointer points 12 bytes *before* the start of the object as the rest of Python knows about it. Objects in the free list are linked together via their object pointer, so they are only known via a pointer into their "middle". We don't see this warning for other GC'ed objects, because they are in one of the collector's linked lists, which point to the start of the GC header which is the malloc'ed pointer. But freed weak references are not known to the collector! So I'm confident that these are not leaks. --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Sat Dec 8 06:30:53 2001 From: fdrake@acm.org (Fred L. Drake) Date: Sat, 8 Dec 2001 01:30:53 -0500 (EST) Subject: [Python-Dev] [development doc updates] Message-ID: <20011208063053.2367D286BC@beowolf.digicool.com> The development version of the documentation has been updated: http://python.sourceforge.net/devel-docs/ Another day of updates; various fixes and clarifications. From fredrik@pythonware.com Sun Dec 9 10:04:17 2001 From: fredrik@pythonware.com (Fredrik Lundh) Date: Sun, 9 Dec 2001 11:04:17 +0100 Subject: [Python-Dev] update on memory leaks in 2.2 References: <3C10EB1C.9302CA8@metaslash.com> <200112071638.LAA12876@cj20424-a.reston1.va.home.com> <3C10F45E.6476A376@metaslash.com> <15377.1024.829156.22274@grendel.zope.com> <15376.64086.678564.584731@anthem.wooz.org> <200112071744.MAA13971@cj20424-a.reston1.va.home.com> Message-ID: <05ab01c18098$e2380130$ced241d5@hagrid> guido wrote: > I don't think so. Insure (uselessly) lists *all* memory that's still > allocated and reachable at exit. Purify only reports certain blocks > (in this case they were a bunch of weak refs) as potentially leaked. > I'm not sure what makes them potentially leaked, but it must be a > stronger condition than "still exists at exit". purify runs a garbage collection algorithm on your program to figure out if something has leaked. iirc, a potential memory leak is when you no longer have a pointer to the beginning of an allocated block, but there's a valid pointer to somewhere inside the block. a true leak is when you no longer point to the block (i.e when even a conservative garbage collector could safely destroy it). From tim.one@home.com Mon Dec 10 00:06:13 2001 From: tim.one@home.com (Tim Peters) Date: Sun, 9 Dec 2001 19:06:13 -0500 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: <3C0F1A6D.417B603C@ActiveState.com> Message-ID: [David Ascher] > - popen on Windows XP seems problematic (needs a repro test there -- > not analyzed yet) We (PythonLabs) don't have XP, so you're on your own for tracking it down. popen() fails in more than one bizarre way on Win9X, and always has, and likely always will. From tim.one@home.com Mon Dec 10 00:11:43 2001 From: tim.one@home.com (Tim Peters) Date: Sun, 9 Dec 2001 19:11:43 -0500 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: <200112060727.fB67RXZ08417@mira.informatik.hu-berlin.de> Message-ID: [Martin v. Loewis] > :-) I personally don't see it as a problem that bug reports are piling > up. The GCC GNATS has 1341 "open" bug reports (meaning either "open" > or "analyzed") at the moment. bugzilla.mozilla.org reports 19611 bugs > in the "open" categories. Microsoft reportedly has over 60000 open > reports for Windows 2000. Neither of these groups despair :-) If any language I worked on previously had a few hundred open bug reports (not counting feature requests), we would have shut the group down and pursued a new implementation. If you can get the open bugs to fit on one screen, it tends to stay that way; once it spills over a screen, it appears there's no bound on how many additional screens are tolerated (0, 1, infinity). I'm doing my part by reducing my font size and buying a much larger monitor . From jason@crash.org Mon Dec 10 00:30:00 2001 From: jason@crash.org (Jason L. Asbahr) Date: Sun, 9 Dec 2001 18:30:00 -0600 Subject: [Python-Dev] win32all Message-ID: Any thoughts on making this package more prominent on the Python.org download pages? Like under the primary Python 2.1.1 and Python 2.2 download links? ______________________________________________________________________ Jason Asbahr jason@asbahr.com From guido@python.org Mon Dec 10 01:14:26 2001 From: guido@python.org (Guido van Rossum) Date: Sun, 09 Dec 2001 20:14:26 -0500 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: Your message of "Sun, 09 Dec 2001 19:11:43 EST." References: Message-ID: <200112100114.UAA11756@cj20424-a.reston1.va.home.com> > [Martin v. Loewis] > > :-) I personally don't see it as a problem that bug reports are piling > > up. The GCC GNATS has 1341 "open" bug reports (meaning either "open" > > or "analyzed") at the moment. bugzilla.mozilla.org reports 19611 bugs > > in the "open" categories. Microsoft reportedly has over 60000 open > > reports for Windows 2000. Neither of these groups despair :-) [Tim] > If any language I worked on previously had a few hundred open bug reports > (not counting feature requests), we would have shut the group down and > pursued a new implementation. It depends on what the bugs are. Many bugs in our tracker are intractable because we don't have the platform where it occurs or the user didn't supply enough information to reproduce it. I imagine the GCC bugs are much of the same, just more (because GCC is more complex and wider used than Python -- most GCC users are even more clueless about GCC's implementation than Python users are about Python's implementation). I try to close bugs from clueless users if there's no response to a request for more info, but often something that might be a bug but isn't easily reproduced stays open for many many months. So I'm not sure that it's realistic to keep all bugs on a page. I did my part against bugs without someone to ask for more info by forbidding anonymous bugs, recently. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Dec 10 03:05:03 2001 From: guido@python.org (Guido van Rossum) Date: Sun, 09 Dec 2001 22:05:03 -0500 Subject: [Python-Dev] win32all In-Reply-To: Your message of "Sun, 09 Dec 2001 18:30:00 CST." References: Message-ID: <200112100305.WAA12125@cj20424-a.reston1.va.home.com> > Any thoughts on making this package more prominent on the Python.org > download > pages? Like under the primary Python 2.1.1 and Python 2.2 download links? Good idea. Done for 1.5.2, 2.0.1 and 2.1.1. Not for 2.2 yet, since there's no version of win32all for 2.2 available AFAIK. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@home.com Mon Dec 10 03:44:31 2001 From: tim.one@home.com (Tim Peters) Date: Sun, 9 Dec 2001 22:44:31 -0500 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: <200112100114.UAA11756@cj20424-a.reston1.va.home.com> Message-ID: [Tim] > If any language I worked on previously had a few hundred open > bug reports (not counting feature requests), we would have shut the > group down and pursued a new implementation. [Guido] > It depends on what the bugs are. Actually, it would not have: a backlog of that many bugs (regardless of nature) would mean they'd never get fixed -- it's hopeless. Time then to cut losses. The compilers I worked on previously sold multi-million dollar machines, though, and had to work only with our own OS(es) and libraries. Python's life is more complicated than that -- and Python doesn't have a few hundred open bugs anyway (there are only 183 right now ). The Python bug base also has gripes about everything from docs to python-mode.el in it. > Many bugs in our tracker are intractable because we don't have the > platform where it occurs or the user didn't supply enough information > to reproduce it. Those should be closed after a reasonable time: nobody is going to work on them. OTOH, I don't think there are many *open* bugs of this ilk (I routinely close such after a month, with "closed for lack of requested followup" -- and nobody has complained about that). > I imagine the GCC bugs are much of the same, just more (because GCC is > more complex and wider used than Python -- most GCC users are even more > clueless about GCC's implementation than Python users are about Python's > implementation). My only interaction with the gcc bug base was helping to whittle down oodles of bad codegen cases on new ports. I have no idea what else may be in there. > I try to close bugs from clueless users if there's no response to a > request for more info, but often something that might be a bug but > isn't easily reproduced stays open for many many months. I don't think there are many open bugs of that ilk either. > So I'm not sure that it's realistic to keep all bugs on a page. We were close once. Now the open patches don't even fit on a page. > I did my part against bugs without someone to ask for more info by > forbidding anonymous bugs, recently. Appreciated, but those were the easiest to close <0.9 wink>. the-things-that-get-done-are-the-things-that-get-worked-on-ly y'rs - tim From tim.one@home.com Mon Dec 10 05:22:15 2001 From: tim.one@home.com (Tim Peters) Date: Mon, 10 Dec 2001 00:22:15 -0500 Subject: [Python-Dev] Subclassing varying length types (What's a PyStructSequence ?) In-Reply-To: <3C0B4152.3739F4EA@lemburg.com> Message-ID: [MAL] > Have you tried disabling all free list and using pymalloc > instead ? No, but I haven't tried anything -- it's a 2.3 issue. > If this pays off, I agree, we should get rid off all of them. When I do try it , it will be slower but more memory-efficient (both data and code) than the type-specific free lists, and faster and much more memory-efficient than using malloc(). > ... > I would consider moving from 8-bit strings to Unicode an > improvement in flexibility. Sure. Moving from one malloc to two is orthogonal. > It also results in better algroithms (== simpler, less error-prone, > etc. in this case). Unclear what "it" means; assuming it means using two mallocs instead of one for a Unicode string object, the 8-bit string algorithms haven't been a particular source of bugs. People mutating strings at the C level has been. > As I said, it's a tradeoff flexibility vs. memory consumption. > Whether it pays off depends on your application environment. It > certainly does for companies like Micron and pays off stock-wise > for a lot of people... uhm, getting off-topic here :-) I've got nothing against Unicode (apart from the larger issue that the whole world would obviously be a lot better off if they switched to American English ). >> Subclassing seems easy enough to me from the Python level; I >> don't have time to revisit C-level subclasssing here (and I don't >> know that it's hackish there either, but do think it's in need of >> docs). > It is beautifully easy for non-varying-length types. Unfortunately, > it happens that some of the basic types which would be attractive > for subclassing are varying length types (such as string and > tuples). It's easy to subclass from str and tuple in Python -- even to add your own instance data. > In my case, I'm looking for away to subclass strings, but I haven't > yet found an elegant solution to the problem of adding extra > data to the instances. It's easy if you're willing to use a dict: class STR(str): def __new__(cls, strguts, n): self = str.__new__(cls, strguts) self.n = n return self s = STR('abc', 42) print s # abc print s.n # 42 __slots__ doesn't work here, though. I admit I personally don't see much attraction to subclassing from str and tuple, apart from adding additional *methods*. I suppose someone could code up two-malloc variants ... From fredrik@pythonware.com Mon Dec 10 08:26:52 2001 From: fredrik@pythonware.com (Fredrik Lundh) Date: Mon, 10 Dec 2001 09:26:52 +0100 Subject: [Python-Dev] memory leaks in 2.2 References: Message-ID: <00b701c18155$9b4bbd80$ced241d5@hagrid> tim wrote: > If any language I worked on previously had a few hundred open bug reports > (not counting feature requests). btw, what's the right way to flag something as a feature request? both the data type and the group fields can be set to "feature request", but I'm not allowed to change the former. (and all my feature requests are listed under "Python - Bugs") From martin@v.loewis.de Mon Dec 10 08:24:29 2001 From: martin@v.loewis.de (Martin v. Loewis) Date: Mon, 10 Dec 2001 09:24:29 +0100 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: References: Message-ID: <200112100824.fBA8OTa01801@mira.informatik.hu-berlin.de> > My only interaction with the gcc bug base was helping to whittle down oodles > of bad codegen cases on new ports. I have no idea what else may be in > there. Many of them are of this kind. As a C++ person, I notice also (with regrets) that a significant portion is of the kind "C++ compiler rejects valid code, or crashes". They are easy to reproduce, so they probably would have caused your former companies to throw away the C++ front-end and rewrite it from scratch. In g++, this won't happen: nobody has the resources to rewrite it from scratch. And even with these bugs, g++ still is much better than the competition :-) > > So I'm not sure that it's realistic to keep all bugs on a page. > > We were close once. Now the open patches don't even fit on a page. I think the only time we were close was when the database was new. Regards, Martin From martin@v.loewis.de Mon Dec 10 08:34:19 2001 From: martin@v.loewis.de (Martin v. Loewis) Date: Mon, 10 Dec 2001 09:34:19 +0100 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: <00b701c18155$9b4bbd80$ced241d5@hagrid> (fredrik@pythonware.com) References: <00b701c18155$9b4bbd80$ced241d5@hagrid> Message-ID: <200112100834.fBA8YJ901851@mira.informatik.hu-berlin.de> > btw, what's the right way to flag something as a feature > request? I'd prefer to see them in the Feature Requests tracker. The Feature Requests group predates the separate feature requests tracker; if they are in a separate tracker, people won't have to go over them again and again > both the data type and the group fields can be set to "feature > request", but I'm not allowed to change the former. At the moment, it might be fastest if you post the numbers, so I can reclassify them. Alternatively, someone with appropriate powers could give you write access to the feature requests tracker, and the data type field (I think you need to be tracker admin for that). Regards, Martin From mal@lemburg.com Mon Dec 10 09:42:59 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 10 Dec 2001 10:42:59 +0100 Subject: [Python-Dev] on the subject of SF... References: <200112060745.fB67j4e18189@mbuna.arbhome.com.au> <15375.8942.722195.533597@anthem.wooz.org> <3C0F2EE0.BE1D2980@lemburg.com> <15375.33371.540885.525907@anthem.wooz.org> <2mu1v33tta.fsf@starship.python.net> Message-ID: <3C1483A3.AF10032F@lemburg.com> Michael Hudson wrote: > > barry@zope.com (Barry A. Warsaw) writes: > > > >>>>> "M" == M writes: > > > > M> And please also a Group "Python 2.3" so that feature requests > > M> for that version can be assigned to it ?! > > > > So far we have no groups defined for the feature request tracker. I > > wonder if that's intentional. > > Does anybody look at the feature request tracker ever? I notice I'm > one of four possible assignees (presumably 'cause I'm new). Does > email from it go anywhere? I was actually thinking of the bug tracker. I saw two group entries there for specific Python versions and thought it would be nice to use the group as feature request target specifier. Nevermind, -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From mal@lemburg.com Mon Dec 10 09:44:50 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 10 Dec 2001 10:44:50 +0100 Subject: [Python-Dev] tracker admin References: <200112071626.LAA09993@cj20424-a.reston1.va.home.com> Message-ID: <3C148412.3FA26450@lemburg.com> Guido van Rossum wrote: > > I just discovered a few new (?) features of the bug/patch tracker > admin page. > > - You can disable anonymous submission. I've done this for all our > trackers (bugs, patches, features), since we have way too many > problems with anonymous submissions where the user doesn't provide > enough info and we have no way to get back to them. > > - You can provide custom text for the "submit new" page. I've added a > warning about the annoying upload checkbox, and some additional > suggestions for patches. Please feel free to provide additional > suggestions. That's great news ! Both of these will help a lot with the PSF discussions we recently had about how to legally deal with submissions. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From fredrik@pythonware.com Mon Dec 10 10:16:38 2001 From: fredrik@pythonware.com (Fredrik Lundh) Date: Mon, 10 Dec 2001 11:16:38 +0100 Subject: [Python-Dev] memory leaks in 2.2 References: <200112100824.fBA8OTa01801@mira.informatik.hu-berlin.de> Message-ID: <011e01c18163$d61f3780$ced241d5@hagrid> martin wrote: > ...rewrite it from scratch. more likely, buy one from EDG: http://www.edg.com/customers.html From mal@lemburg.com Mon Dec 10 10:57:10 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 10 Dec 2001 11:57:10 +0100 Subject: [Python-Dev] Subclassing varying length types (What's a PyStructSequence ?) References: Message-ID: <3C149506.A5689065@lemburg.com> Tim Peters wrote: > > [MAL] > > Have you tried disabling all free list and using pymalloc > > instead ? > > No, but I haven't tried anything -- it's a 2.3 issue. > > > If this pays off, I agree, we should get rid off all of them. > > When I do try it , it will be slower but more memory-efficient (both > data and code) than the type-specific free lists, and faster and much more > memory-efficient than using malloc(). Well, let's do some pybench runs next year and see what the results look like. > > ... > > I would consider moving from 8-bit strings to Unicode an > > improvement in flexibility. > > Sure. Moving from one malloc to two is orthogonal. You know that I know that you knew what I was talking about :-) > > It also results in better algroithms (== simpler, less error-prone, > > etc. in this case). > > Unclear what "it" means; assuming it means using two mallocs instead of one > for a Unicode string object, the 8-bit string algorithms haven't been a > particular source of bugs. People mutating strings at the C level has been. If you ever try to support more than ASCII text in a user program, you'll find that having to deal with only one encoding safes you a whole lot of trouble. I won't even start talking about variable length encodings, encodings with builtin shift state and other goodies which are a complete nightmare to handle (e.g. various character properties such as title case, upper/lower mappings, different ways to encode a single character, collation,...). > > As I said, it's a tradeoff flexibility vs. memory consumption. > > Whether it pays off depends on your application environment. It > > certainly does for companies like Micron and pays off stock-wise > > for a lot of people... uhm, getting off-topic here :-) > > I've got nothing against Unicode (apart from the larger issue that the whole > world would obviously be a lot better off if they switched to American > English ). I suppose Mandarin would reach a larger share in world population ... and they *need* Unicode :-) > >> Subclassing seems easy enough to me from the Python level; I > >> don't have time to revisit C-level subclasssing here (and I don't > >> know that it's hackish there either, but do think it's in need of > >> docs). > > > It is beautifully easy for non-varying-length types. Unfortunately, > > it happens that some of the basic types which would be attractive > > for subclassing are varying length types (such as string and > > tuples). > > It's easy to subclass from str and tuple in Python -- even to add your own > instance data. Yeah, but that's not the point. I want to do this in C... > > In my case, I'm looking for away to subclass strings, but I haven't > > yet found an elegant solution to the problem of adding extra > > data to the instances. > > It's easy if you're willing to use a dict: I would be willing to use a dictionary. It's only that even the dictionary trick doesn't seem to work at C level. > class STR(str): > def __new__(cls, strguts, n): > self = str.__new__(cls, strguts) > self.n = n > return self > > s = STR('abc', 42) > print s # abc > print s.n # 42 > > __slots__ doesn't work here, though. > > I admit I personally don't see much attraction to subclassing from str and > tuple, apart from adding additional *methods*. I suppose someone could code > up two-malloc variants ... If you look at mxURL you'll find an extension type which tries to play nice with strings -- it would be a good candidate for a string subtype. A string type which carries along an encoding attribute would be another good candidate for a string subtype. Both need extra attributes/data fields. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From fdrake@acm.org Mon Dec 10 15:14:27 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 10 Dec 2001 10:14:27 -0500 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: <200112100824.fBA8OTa01801@mira.informatik.hu-berlin.de> References: <200112100824.fBA8OTa01801@mira.informatik.hu-berlin.de> Message-ID: <15380.53587.201337.1803@grendel.zope.com> Martin v. Loewis writes: > I think the only time we were close was when the database was new. No; when the DB was new (but old enough to notice), it contained an import of all the old JiterBug reports. I think we had it pretty well whittled down around the 2.0 release. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From jeremy@zope.com Mon Dec 10 10:20:01 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 10 Dec 2001 05:20:01 -0500 (EST) Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: <15380.53587.201337.1803@grendel.zope.com> References: <200112100824.fBA8OTa01801@mira.informatik.hu-berlin.de> <15380.53587.201337.1803@grendel.zope.com> Message-ID: <15380.35921.588329.420402@slothrop.digicool.com> >>>>> "FLD" == Fred L Drake, writes: FLD> Martin v. Loewis writes: >> I think the only time we were close was when the database was >> new. FLD> No; when the DB was new (but old enough to notice), it FLD> contained an FLD> import of all the old JiterBug reports. I think we had it FLD> pretty well whittled down around the 2.0 release. That's right. We spent a lot of time fixing bugs prior to the 2.0 release. Jeremy From tim.one@home.com Mon Dec 10 17:30:37 2001 From: tim.one@home.com (Tim Peters) Date: Mon, 10 Dec 2001 12:30:37 -0500 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: <00b701c18155$9b4bbd80$ced241d5@hagrid> Message-ID: [Fredrik Lundh] > btw, what's the right way to flag something as a feature > request? > > both the data type and the group fields can be set to "feature > request", but I'm not allowed to change the former. After literally hundreds of mouse clicks, just about everyone should have permission to do that now. The "data type" field determines the tracker a thing lives in, so that's the more important of the two to change. Unfortunately, when you move a report, old URLs to the report become invalid, and bring up a bizarre "invalide artifact ID" (something like that) page. > (and all my feature requests are listed under "Python - Bugs") Then I'm going to raise them all to priority 9 . From tim.one@home.com Mon Dec 10 17:44:22 2001 From: tim.one@home.com (Tim Peters) Date: Mon, 10 Dec 2001 12:44:22 -0500 Subject: Feature Requests (RE: [Python-Dev] memory leaks in 2.2) In-Reply-To: Message-ID: The SF Feature Request tracker didn't have any Categories defined. I've now added the same Categories as are available in the Bug tracker: Build Demos and Tools Distutils Documentation Extension Modules IDLE Installation Macintosh Parser/Compiler Python Interpreter Core Python Library Regular Expressions Threads Tkinter Type/class unification Unicode Windows XML Caution: it *could* be that when you change trackers, the Category is lost; you may have to set it again after it's moved (this was certainly the case when the FR tracker didn't have any Categories). From tim.one@home.com Tue Dec 11 04:31:39 2001 From: tim.one@home.com (Tim Peters) Date: Mon, 10 Dec 2001 23:31:39 -0500 Subject: [Python-Dev] memory leaks in 2.2 In-Reply-To: <200112100824.fBA8OTa01801@mira.informatik.hu-berlin.de> Message-ID: [Tim] >> My only interaction with the gcc bug base was helping to >> whittle down oodles of bad codegen cases on new ports. ... [Martin v. Loewis] > Many of them are of this kind. As a C++ person, I notice also (with > regrets) that a significant portion is of the kind "C++ compiler > rejects valid code, or crashes". They are easy to reproduce, so they > probably would have caused your former companies to throw away the C++ > front-end and rewrite it from scratch. As Fredrik suggested, we bought front ends whenever possible. When a single machine sells for millions, and you can buy a highly regarded front end for a few percent of that, you'd be crazy to write your own. > In g++, this won't happen: nobody has the resources to rewrite it from > scratch. And even with these bugs, g++ still is much better than the > competition :-) I'd be more upset if anyone had proved that C++ is compilable . [Guido] >>> So I'm not sure that it's realistic to keep all bugs on a page. >> We were close once. Now the open patches don't even fit on a page. > I think the only time we were close was when the database was new. As Fred and Jeremy said, when we (PythonLabs) worked intensely on closing bugs for 1.6 and 2.0, we were close to being in phenomenal shape here. We don't have a fraction of the internal resource to throw at that anymore, though, and volunteers haven't taken up the slack. So I figure the only reason Python stays in exceptionally good bug shape is that we also ignore the contributed patches <0.9 wink>. From mal@lemburg.com Tue Dec 11 10:04:55 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 11 Dec 2001 11:04:55 +0100 Subject: Feature Requests (RE: [Python-Dev] memory leaks in 2.2) References: Message-ID: <3C15DA47.FA5C39DA@lemburg.com> Tim Peters wrote: > > The SF Feature Request tracker didn't have any Categories defined. I've now > added the same Categories as are available in the Bug tracker: > > Build > Demos and Tools > Distutils > Documentation > Extension Modules > IDLE > Installation > Macintosh > Parser/Compiler > Python Interpreter Core > Python Library > Regular Expressions > Threads > Tkinter > Type/class unification > Unicode > Windows > XML Would be nice if we could also have Groups "Python 2.3", "Python 3k", etc. > Caution: it *could* be that when you change trackers, the Category is lost; > you may have to set it again after it's moved (this was certainly the case > when the FR tracker didn't have any Categories). Changing the data type pretty much reset all fields for me (including the assigned field which is particularly annoying) and the URL becomes invalid too. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From martin@v.loewis.de Tue Dec 11 10:12:00 2001 From: martin@v.loewis.de (Martin v. Loewis) Date: Tue, 11 Dec 2001 11:12:00 +0100 Subject: Feature Requests (RE: [Python-Dev] memory leaks in 2.2) In-Reply-To: <3C15DA47.FA5C39DA@lemburg.com> (mal@lemburg.com) References: <3C15DA47.FA5C39DA@lemburg.com> Message-ID: <200112111012.fBBAC0I01773@mira.informatik.hu-berlin.de> > Would be nice if we could also have Groups "Python 2.3", "Python 3k", > etc. I'm not sure. Feature requests are supposed to stay open until they are either implemented, withdrawn, or rejected for good. If you want to express that some requests are more important than others (so that they ought to go into 2.3, instead of P3k), you should give them a higher priority. Regards, Martin From mal@lemburg.com Tue Dec 11 11:00:14 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 11 Dec 2001 12:00:14 +0100 Subject: Feature Requests (RE: [Python-Dev] memory leaks in 2.2) References: <3C15DA47.FA5C39DA@lemburg.com> <200112111012.fBBAC0I01773@mira.informatik.hu-berlin.de> Message-ID: <3C15E73E.985DB93@lemburg.com> "Martin v. Loewis" wrote: > > > Would be nice if we could also have Groups "Python 2.3", "Python 3k", > > etc. > > I'm not sure. Feature requests are supposed to stay open until they > are either implemented, withdrawn, or rejected for good. If you want > to express that some requests are more important than others (so that > they ought to go into 2.3, instead of P3k), you should give them a > higher priority. If you look at it this way, yes. It's just that setting the Group to e.g. P3k would tell the submitter that the request is being considered, but won't get implemented until a few releases from now or that a specific request won't make it into e.g. Python 2.2 because of a feature freeze. Anyway, it's just an idea. BTW, I also think that we could use the feature request tracker to track all the PEP 42 requests and redirect PEP 42 to the tracker. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/ From fdrake@acm.org Tue Dec 11 22:16:01 2001 From: fdrake@acm.org (Fred L. Drake) Date: Tue, 11 Dec 2001 17:16:01 -0500 (EST) Subject: [Python-Dev] [development doc updates] Message-ID: <20011211221601.DB780286BC@cj42289-a.reston1.va.home.com> The development version of the documentation has been updated: http://python.sourceforge.net/devel-docs/ Many various updates, including documentation for generators and extended call syntax. From skip@pobox.com (Skip Montanaro) Tue Dec 11 23:48:58 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Tue, 11 Dec 2001 17:48:58 -0600 Subject: [Python-Dev] new pubkey for SF Message-ID: <15382.39786.533044.951881@beluga.mojam.com> Folks, Where do I send a new SSH pubkey to regain non-anonymous access to the CVS repository? Thx, Skip From tim.one@home.com Wed Dec 12 00:31:57 2001 From: tim.one@home.com (Tim Peters) Date: Tue, 11 Dec 2001 19:31:57 -0500 Subject: [Python-Dev] new pubkey for SF In-Reply-To: <15382.39786.533044.951881@beluga.mojam.com> Message-ID: > Where do I send a new SSH pubkey to regain non-anonymous access to the > CVS repository? Log in to SF. From your "personal page", click on Account Options, then down to Shell Account Information, then click Edit Keys. It should be obvious from there. Beware the 2 (4? 8? 512?) hour synchronization delay. From bizopp2002us@yahoo.com Wed Dec 12 20:52:46 2001 From: bizopp2002us@yahoo.com (John McCuullen) Date: Wed, 12 Dec 2001 21:52:46 +0100 Subject: [Python-Dev] The Secret to Supercharge your MLM! Message-ID: <2191817-220011231220524660@yahoo.com> Discover "The Secret to Supercharge your MLM!"=20 A must read=2E=2E=2E=20 Hi there,=20 The secret is out!=20 Here's the mail we've all been waiting for! Read carefully and take immediate action on it!=20 I've discovered an amazing formula that will give your MLM an enormous enrollers explosion=2E You'll benefit hugely if you use it with YOUR primary MLM!=20 -THE ULTIMATE RECRUITMENT FORMULA-=20 So if you are already in MLM or about to get into one this will be the most important mail you will ever read=2E=20 Right now I know people who enroll dozens in their downlines WEEKLY using the formula I'm about to reveal=2E=20 Fred Stege, Founder of the company enrolled 127 people within 7 day's after launching! One of his enrollees from this batch has become top recruiter in his company within 90 days after using this incredible easy-to-use concept=2E=20 The NEW Formula is meant as a Front End Program to enhance "YOUR" MLM=2E I'm NOT pitching you in any MLM=2E ____________________________________________ In fact you may try it for absolutely FREE and you'll get an electrifying stack of tools straight away! ____________________________________________=20 We've discovered that the majority of Qualified MLM Leads (Prospects) all have 1 thing in common=2E=2E=2E=20 They all are looking for 3 things they desperately want and can't get in any one 100% generic System=2E=2E=2E=20 I just know that you'll agree with these 3 things and they are all to your advantage!=20 Let me reveal you the 3 most sought after topics first:=20 =B7 Cutting Edge MLM Information=2E =B7 A Legitimate Proven Formula=2E =B7 Pre Qualified Leads=2E (And how to get them for FREE)=20 Isn't that what everybody is looking for?=2E=2E Will these topics ever go out of style?=2E=2E Doesn't THAT Attract the Masses?=2E=2E=20 Does 'OLD SCHOOL MLM' attract the masses?=2E=2E=20 Discover "How you can EARN RESIDUAL INCOME While Recruiting on Auto Pilot whether they join your MLM or not!"=20 Welcome to 21st Century Massive Recruitment=2E=20 I encourage you to check the details and sign up immediately=2E That means NOW!=20 For more info please mail to mccuullen47@zonnet=2Enl and write your First Name + 'MORE INFO' in the subject line=2E You'll receive further instructions immediately=2E=20 Sincerely,=20 John McCuullen P=2ES=2E Don't forget to write your FIRST NAME + 'MORE INFO' In Subject line!=20 P=2EP=2ES=2E On the given site you'll find a secret that every PRO in our industry knows and uses=2E=2E=2E It's called: S-L-L-G: Self Liquidating Lead Generation=2E It will Create a Huge, Electrifying Surge in Your Cash Flow=2E=20 P=2EP=2EP=2ES=2E Find out the NEW 3 foot rule in MLM=2E "Why Old School MLM is nearly dead=2E"=20 From fredrik@pythonware.com Thu Dec 13 02:23:56 2001 From: fredrik@pythonware.com (Fredrik Lundh) Date: Thu, 13 Dec 2001 03:23:56 +0100 Subject: [Python-Dev] what's the current plan for 2.2c1? Message-ID: <005101c1837d$3d0fc980$ced241d5@hagrid> the pep (and python.org) says december 12. just curious /F From fdrake@acm.org Thu Dec 13 19:08:41 2001 From: fdrake@acm.org (Fred L. Drake) Date: Thu, 13 Dec 2001 14:08:41 -0500 (EST) Subject: [Python-Dev] [development doc updates] Message-ID: <20011213190841.2EF95286BC@cj42289-a.reston1.va.home.com> The development version of the documentation has been updated: http://python.sourceforge.net/devel-docs/ Updated the build instructions for extensions on Windows. From barry@zope.com Thu Dec 13 20:01:36 2001 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 13 Dec 2001 15:01:36 -0500 Subject: [Python-Dev] Python 2.2rc1 - release branch created Message-ID: <15385.2336.521466.6255@anthem.wooz.org> I've created the Python 2.2rc1 release branch. The CVS tag is `r22rc1-branch'. There's also a tag `r22rc1-fork' -- that's the point on the trunk where the branch forks off. So far the only change in the branch is the version number. As usual, no one except Guido and myself (or our authorized bots) should make checkins to the branch. Checkins to the trunk are okay but discouraged, and generally will /not/ be merged into the release branch unless you notify us (a comment in the checkin message would suffice). I'll be cutting the release tomorrow morning local time (Friday December 14). After the release is done, the branch will be merged back into the trunk. Please note that as this is a release candidate, only essential or showstopper bugs should be fixed. Definitely, no new features should be added. According to PEP 251, Python 2.2 final is currently scheduled for 19-Dec-2001. If you're unsure whether to make a change or not, please contact us first. Enjoy, -Barry From fdrake@acm.org Thu Dec 13 20:02:45 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 13 Dec 2001 15:02:45 -0500 Subject: [Python-Dev] Doc/ tree closed for release Message-ID: <15385.2405.102160.574585@grendel.zope.com> The Doc/ tree is now closed so I can prepare for the release. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From fdrake@acm.org Fri Dec 14 04:25:14 2001 From: fdrake@acm.org (Fred L. Drake) Date: Thu, 13 Dec 2001 23:25:14 -0500 (EST) Subject: [Python-Dev] [development doc updates] Message-ID: <20011214042514.2204B286BC@beowolf.digicool.com> The development version of the documentation has been updated: http://python.sourceforge.net/devel-docs/ Final 2.2c1 documentation. From barry@zope.com Fri Dec 14 20:06:13 2001 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 14 Dec 2001 15:06:13 -0500 Subject: [Python-Dev] RELEASED Python 2.2 release candidate 1 is out! Message-ID: <15386.23477.137871.205510@anthem.wooz.org> Today we release Python 2.2c1, the first and hopefully only release candidate for Python 2.2, for your extravagation, eyestrain, and elucidation. Our thanks to everyone who helped test Python 2.2 during its alpha and beta phases, and to everyone who contributed patches, feedback, and suggestions! Barring any showstopper bugs, we plan on releasing Python 2.2 final in one week. Please continue to report any bugs you find to the bug tracker: http://sourceforge.net/bugs/?group_id=5470 Highlights of what's new for this release are outlined below. For a more complete list, please see: http://sf.net/project/shownotes.php?release_id=65583 Andrew Kuchling is writing a gentle introduction to the most important changes, titled "What's New in Python 2.2": http://www.amk.ca/python/2.2/ Guido's written his own introduction to the type/class unification at: http://www.python.org/2.2/descrintro.html Enjoy, -Barry -------------------- snip snip -------------------- What's New in Python 2.2c1? Release date: 14-Dec-2001 - The "delete attribute" method of descriptor objects is called __delete__, not __del__. - Some subtle issues with the super built-in were fixed. - hash() of an instance of a subclass of mutable type now properly raises TypeError. - New style objects now support deleting their __dict__. - -Qnew now works as documented in PEP 238. - complex() now only allows the first argument to be a string argument. - gc.get_referents was renamed to gc.get_referrers. - It is no longer necessary to use --with-suffix when building on a case-insensitive file system (such as Mac OS X HFS+). - In unix-Python on Mac OS X (and darwin) sys.platform is now "darwin", without any trailing digits. From barry@zope.com Fri Dec 14 20:19:56 2001 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 14 Dec 2001 15:19:56 -0500 Subject: [Python-Dev] RELEASED Python 2.2 release candidate 1 is out! References: <15386.23477.137871.205510@anthem.wooz.org> Message-ID: <15386.24300.351531.84724@anthem.wooz.org> Sigh. A cut and paste error omitted the download link from the announcement. As usual, it is http://www.python.org/2.2/ Apologies, and thanks MAL! -Barry From guido@python.org Sat Dec 15 16:23:44 2001 From: guido@python.org (Guido van Rossum) Date: Sat, 15 Dec 2001 11:23:44 -0500 Subject: [Python-Dev] mpz module broken with GMP 4.0 Message-ID: <200112151623.LAA32158@cj20424-a.reston1.va.home.com> See SF bug #493631: http://sourceforge.net/tracker/index.php?func=detail&aid=493631&group_id=5470&atid=105470 The mpz modules doesn't work with GMP 4.0 (which is brand new -- http://www.swox.com/gmp/ lists Dec 1, 2001 as the release date). It looks like the mpz module is a mess. The setup.py file contains a reference to pympz.sourceforge.net, but that doesn't exist. There is, however, a SF project gmpy, which seems to be a Python wrapper for GMP 3.0 (does it work with GMP 4.0?). I'm cc'ing the project admins, maybe they can shed light on this. Does anybody know how to fix mpzmodule.c? Or should we just get rid of it? Should we at least fix the reference in setup.py to point to gmpy? --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Sat Dec 15 16:43:45 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Sat, 15 Dec 2001 11:43:45 -0500 Subject: [Python-Dev] regression tests with regrtest.py -r Message-ID: <15387.32193.854351.323371@grendel.zope.com> On a lark, I decided to try running the regression tests with -r, which tells regrtest.py to run the tests in random order. With this, I get the following warnings from test_long.py: ../Lib/test/test_long.py:266: OverflowWarning: integer addition special = [0, 1, 2, 3, sys.maxint-1, sys.maxint, sys.maxint+1] ../Lib/test/test_long.py:287: OverflowWarning: integer addition got = x + y ../Lib/test/test_long.py:291: OverflowWarning: integer subtraction got = x - y ../Lib/test/test_long.py:295: OverflowWarning: integer multiplication got = x * y ../Lib/test/test_long.py:313: OverflowWarning: integer exponentiation got = x ** y ../Lib/test/test_long.py:320: OverflowWarning: integer exponentiation got = pow(x, y, z) ../Lib/test/test_long.py:377: OverflowWarning: integer exponentiation value = 10 ** exp This was with the CVS trunk, which is still virtually identical to the 2.2c1 release. I won't have time to look at them this weekend, but this should be fixed. It may be possible there are other order-dependent behaviors, so it would be really good if people who have time can run the tests with -r on whatever platforms are available and report any other anomolies. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From guido@python.org Sat Dec 15 18:01:07 2001 From: guido@python.org (Guido van Rossum) Date: Sat, 15 Dec 2001 13:01:07 -0500 Subject: [Python-Dev] regression tests with regrtest.py -r In-Reply-To: Your message of "Sat, 15 Dec 2001 11:43:45 EST." <15387.32193.854351.323371@grendel.zope.com> References: <15387.32193.854351.323371@grendel.zope.com> Message-ID: <200112151801.NAA02181@cj20424-a.reston1.va.home.com> > On a lark, I decided to try running the regression tests with -r, Good idea! > which tells regrtest.py to run the tests in random order. With this, > I get the following warnings from test_long.py: > > ../Lib/test/test_long.py:266: OverflowWarning: integer addition > special = [0, 1, 2, 3, sys.maxint-1, sys.maxint, sys.maxint+1] [...] > > This was with the CVS trunk, which is still virtually identical to > the 2.2c1 release. > I won't have time to look at them this weekend, but this should be > fixed. It may be possible there are other order-dependent behaviors, > so it would be really good if people who have time can run the tests > with -r on whatever platforms are available and report any other > anomolies. I've found the cause: test_scope.py contains a call to warnings.resetwarnings(), which removes *all* warning filters, not just those made by the calling module. Because of the way OverflowWarning is suppressed, this removes the "preset" filter that suppresses OverflowWarning. Since an API redesign before the 2.2 release is out of the question, I'm fixing this by removing the resetwarnings() call from test_scope.py and making the filterwarnings() call there a little more specific. --Guido van Rossum (home page: http://www.python.org/~guido/) From aleax@aleax.it Sat Dec 15 18:23:26 2001 From: aleax@aleax.it (Alex Martelli) Date: Sat, 15 Dec 2001 19:23:26 +0100 Subject: [Python-Dev] Re: mpz module broken with GMP 4.0 In-Reply-To: <200112151623.LAA32158@cj20424-a.reston1.va.home.com> References: <200112151623.LAA32158@cj20424-a.reston1.va.home.com> Message-ID: <0112151923261C.01556@arthur> On Saturday 15 December 2001 17:23, Guido van Rossum wrote: > See SF bug #493631: > > http://sourceforge.net/tracker/index.php?func=detail&aid=493631&group_id= >5470&atid=105470 > > The mpz modules doesn't work with GMP 4.0 (which is brand new -- > http://www.swox.com/gmp/ lists Dec 1, 2001 as the release date). > > It looks like the mpz module is a mess. The setup.py file contains a > reference to pympz.sourceforge.net, but that doesn't exist. > > There is, however, a SF project gmpy, which seems to be a Python > wrapper for GMP 3.0 (does it work with GMP 4.0?). I'm cc'ing the > project admins, maybe they can shed light on this. Thanks for the indication -- I was not aware of the new GMP 4.0. A rapid test shows the current gmpy.c, release 0.8, breaks in exactly one place (structure gmpstate_t used to have a member named seed, now renamed to _mp_seed). Once that single line is fixed, gmpy 0.8 builds fine (on a Linux box) with the new GMP 4.0 and passes all of its 973 unit-tests (with Python 2.2b1). I think we should rapidly release a 0.8.1 with the one-line fix (arranged so it keeps building with GMP 3.1 while also building with 4.0, of course), and plan the modest gmpy extensions needed to exploit 4.0's novelties later (it's likely that little or no work is needed for that, since 4.0 seems to have mostly relaxed previous argument constraints, and gmpy generally doesn't duplicate GMP's own argument validity tests). Having gmpy fully exploit Python 2.2 is a longer-term objective (expose all types and make them inheritable). Alex From guido@python.org Sat Dec 15 18:31:45 2001 From: guido@python.org (Guido van Rossum) Date: Sat, 15 Dec 2001 13:31:45 -0500 Subject: [Python-Dev] Re: mpz module broken with GMP 4.0 In-Reply-To: Your message of "Sat, 15 Dec 2001 19:23:26 +0100." <0112151923261C.01556@arthur> References: <200112151623.LAA32158@cj20424-a.reston1.va.home.com> <0112151923261C.01556@arthur> Message-ID: <200112151831.NAA02998@cj20424-a.reston1.va.home.com> Alex, do you have a suggestion for what to do with mpzmodule.c in the 2.2 distribution? --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Sat Dec 15 18:34:26 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Sat, 15 Dec 2001 13:34:26 -0500 Subject: [Python-Dev] Re: mpz module broken with GMP 4.0 In-Reply-To: <0112151923261C.01556@arthur> References: <200112151623.LAA32158@cj20424-a.reston1.va.home.com> <0112151923261C.01556@arthur> Message-ID: <15387.38834.552687.60342@grendel.zope.com> Alex Martelli writes: > Thanks for the indication -- I was not aware of the new GMP 4.0. A rapid > test shows the current gmpy.c, release 0.8, breaks in exactly one place I'm adding a link to the gmpy website from the mpz module documentation. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From tim.one@home.com Sat Dec 15 18:43:26 2001 From: tim.one@home.com (Tim Peters) Date: Sat, 15 Dec 2001 13:43:26 -0500 Subject: [Python-Dev] regression tests with regrtest.py -r In-Reply-To: <15387.32193.854351.323371@grendel.zope.com> Message-ID: [Fred L. Drake, Jr.] > ... > It may be possible there are other order-dependent behaviors, There are, but ... > so it would be really good if people who have time can run the tests > with -r on whatever platforms are available and report any other > anomolies. By far the most frequent cause of errors when using -r is bogus module objects left behind in sys.modules on non-Unix platforms that can't import some Unix-specific things. A later attempt to import the same module again doesn't fail, but since the module object is incomplete it fails for some other reason when a test attempts to use the module. I spent countless hours worming around this in the past, but lost heart for it; on my Win9X box, the failing test has usually scrolled off the (50-line) DOS box before regrtest completes, and so also the history of the test ordering that caused it. Heh -- -r failed in test___all__ while I was typing this, but I've no idea why. I'll pursue it until I get sick to death of it . From fdrake@acm.org Sat Dec 15 18:46:16 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Sat, 15 Dec 2001 13:46:16 -0500 Subject: [Python-Dev] regression tests with regrtest.py -r In-Reply-To: <200112151801.NAA02181@cj20424-a.reston1.va.home.com> References: <15387.32193.854351.323371@grendel.zope.com> <200112151801.NAA02181@cj20424-a.reston1.va.home.com> Message-ID: <15387.39544.377965.395044@grendel.zope.com> Guido van Rossum writes: > I've found the cause: test_scope.py contains a call to > warnings.resetwarnings(), which removes *all* warning filters, not > just those made by the calling module. Because of the way This is the danger of running the tests the way we do (all in one process), but that also makes us be more careful with the tests. I'll have to run with -r more often! -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From mal@lemburg.com Sat Dec 15 19:03:21 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Sat, 15 Dec 2001 20:03:21 +0100 Subject: [Python-Dev] Re: mpz module broken with GMP 4.0 References: <200112151623.LAA32158@cj20424-a.reston1.va.home.com> <0112151923261C.01556@arthur> <15387.38834.552687.60342@grendel.zope.com> Message-ID: <3C1B9E79.1020202@lemburg.com> Fred L. Drake, Jr. wrote: > Alex Martelli writes: > > Thanks for the indication -- I was not aware of the new GMP 4.0. A rapid > > test shows the current gmpy.c, release 0.8, breaks in exactly one place > > I'm adding a link to the gmpy website from the mpz module > documentation. You might also want to add a link to mxNumber which is another wrapper for GMP. Haven't tried it with GMP 4.0 yet, but it works great with 3.1.1 -- even on Windows. http://www.egenix.com/files/python/mxNumber.html -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From tim.one@home.com Sat Dec 15 20:29:15 2001 From: tim.one@home.com (Tim Peters) Date: Sat, 15 Dec 2001 15:29:15 -0500 Subject: [Python-Dev] regression tests with regrtest.py -r In-Reply-To: Message-ID: Here's one cause of current -r failures on Windows: C:\Code\python\PCbuild>python ../lib/test/regrtest.py \ test_pty test___all__ test_pty test test_pty skipped -- No module named termios test___all__ test test___all__ failed -- pty has no __all__ attribute 1 test failed: test___all__ 1 test skipped: test_pty Those skips are all expected on win32. It's "the usual": importing pty fails on Windows, but test_pty then leaves behind a bogus pty module object in sys.modules, which fools test___all__. Note that pty.py already has this atrocity from a previous round of -r wrestling: """ # Absurd: import termios and then delete it. This is to force an attempt # to import pty to raise an ImportError on platforms that lack termios. # Without this explicit import of termios here, some other module may # import tty first, which in turn imports termios and dies with an # ImportError then. But since tty *does* exist across platforms, that # leaves a damaged module object for tty in sys.modules, and the import # of tty here then appears to work despite that the tty imported is junk. import termios del termios import tty """ test___all__ also tries to compensate for pty (among others) insanity: """ def check_all(modname): names = {} try: exec "import %s" % modname in names except ImportError: # Silent fail here seems the best route since some modules # may not be available in all environments. # Since an ImportError may leave a partial module object in # sys.modules, get rid of that first. Here's what happens if # you don't: importing pty fails on Windows because pty tries to # import FCNTL, which doesn't exist. That raises an ImportError, # caught here. It also leaves a partial pty module in sys.modules. # So when test_pty is called later, the import of pty succeeds, # but shouldn't. As a result, test_pty crashes with an # AttributeError instead of an ImportError, and regrtest interprets # the latter as a test failure (ImportError is treated as "test # skipped" -- which is what test_pty should say on Windows). try: del sys.modules[modname] except KeyError: pass return """ I've added another hack to regrtest.py to worm around the test_pty followed by test___all__ case (more generally, any test_xxx followed by test___all__ case, when test_xxx fails due to leaving behind a damaged module object for xxx). However, that still doesn't fix test_rlcompleter followed by test___all__ on Windows. In that case, test_rlcompleter leaves behind a damaged rlcompleter module, because rlcompleter.py does exist on Windows (it fails due to the lack of the readline module). leaving-behind-incomplete-module-objects-still-seems-like-a-bad-idea- to-me-ly y'rs - tim From fdrake@acm.org Sat Dec 15 20:36:42 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Sat, 15 Dec 2001 15:36:42 -0500 Subject: [Python-Dev] Re: mpz module broken with GMP 4.0 In-Reply-To: <3C1B9E79.1020202@lemburg.com> References: <200112151623.LAA32158@cj20424-a.reston1.va.home.com> <0112151923261C.01556@arthur> <15387.38834.552687.60342@grendel.zope.com> <3C1B9E79.1020202@lemburg.com> Message-ID: <15387.46170.620220.286507@grendel.zope.com> M.-A. Lemburg writes: > You might also want to add a link to mxNumber which is another > wrapper for GMP. Haven't tried it with GMP 4.0 yet, but it works > great with 3.1.1 -- even on Windows. I'll have this checked in shortly -- thanks! -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From fdrake@beowolf.digicool.com Sat Dec 15 20:44:16 2001 From: fdrake@beowolf.digicool.com (Fred Drake) Date: Sat, 15 Dec 2001 15:44:16 -0500 (EST) Subject: [Python-Dev] [maintenance doc updates] Message-ID: <20011215204416.B421F286BC@beowolf.digicool.com> The development version of the documentation has been updated: http://python.sourceforge.net/maint-docs/ Miscellaneous minor updates/fixes. From fdrake@acm.org Sat Dec 15 20:46:40 2001 From: fdrake@acm.org (Fred L. Drake) Date: Sat, 15 Dec 2001 15:46:40 -0500 (EST) Subject: [Python-Dev] [development doc updates] Message-ID: <20011215204640.8FF5E286BC@beowolf.digicool.com> The development version of the documentation has been updated: http://python.sourceforge.net/devel-docs/ Miscellaneous minor updates/fixes post 2.2c1. From aleax@aleax.it Sat Dec 15 21:17:12 2001 From: aleax@aleax.it (Alex Martelli) Date: Sat, 15 Dec 2001 22:17:12 +0100 Subject: [Python-Dev] Re: mpz module broken with GMP 4.0 In-Reply-To: <200112151831.NAA02998@cj20424-a.reston1.va.home.com> References: <200112151623.LAA32158@cj20424-a.reston1.va.home.com> <0112151923261C.01556@arthur> <200112151831.NAA02998@cj20424-a.reston1.va.home.com> Message-ID: <0112152217121F.01556@arthur> On Saturday 15 December 2001 19:31, Guido van Rossum wrote: > Alex, do you have a suggestion for what to do with mpzmodule.c in the > 2.2 distribution? Remove it? It appears to me that there's no maintenance of it, and either Lemburg's module, or gmpy, would be a better bet for Pythonistas needing GMP access, anyway. Alex From tim.one@home.com Sat Dec 15 21:20:32 2001 From: tim.one@home.com (Tim Peters) Date: Sat, 15 Dec 2001 16:20:32 -0500 Subject: [Python-Dev] regrtest.py mystery Message-ID: We do this after running every test: # Unload the newly imported modules (best effort finalization) for module in sys.modules.keys(): if module not in save_modules and module.startswith("test."): test_support.unload(module) Unfortunately, that doesn't nuke whatever damaged non-test module objects may have been left behind. If I change it to make a truly best effort: if module not in save_modules: test_support.unload(module) then about a dozen std tests fail in mysterious ways, but only when running more than one test. Like just running test_email test_string causes test_string to die like so: python ../lib/test/regrtest.py -v test_email test_string ... test test_string crashed -- exceptions.AttributeError: 'NoneType' object has no attribute 'get' Traceback (most recent call last): File "../lib/test/regrtest.py", line 305, in runtest the_module = __import__(test, globals(), locals(), []) File "../lib/test\test_string.py", line 34, in ? string_tests.run_method_tests(test) File "../lib/test\string_tests.py", line 233, in run_method_tests verify('hello world'.encode('zlib') == data) File "C:\CODE\PYTHON\lib\encodings\__init__.py", line 43, in search_function entry = _cache.get(encoding,_unknown) AttributeError: 'NoneType' object has no attribute 'get' It seems generally the case that these failures are due to various module-level names getting rebound to None. A simpler example: python ../lib/test/regrtest.py -v test_string test_string ... test test_string crashed -- exceptions.AttributeError: 'NoneType' object has no attribute 'compress' Traceback (most recent call last): File "../lib/test/regrtest.py", line 305, in runtest the_module = __import__(test, globals(), locals(), []) File "../lib/test\test_string.py", line 34, in ? string_tests.run_method_tests(test) File "../lib/test\string_tests.py", line 233, in run_method_tests verify('hello world'.encode('zlib') == data) File "C:\CODE\PYTHON\lib\encodings\zlib_codec.py", line 25, in zlib_encode output = zlib.compress(input) AttributeError: 'NoneType' object has no attribute 'compress' What's up with that? So far, they all seem to involve the encodings directory ... From martin@v.loewis.de Sat Dec 15 22:36:39 2001 From: martin@v.loewis.de (Martin v. Loewis) Date: Sat, 15 Dec 2001 23:36:39 +0100 Subject: [Python-Dev] Re: mpz module broken with GMP 4.0 In-Reply-To: <0112152217121F.01556@arthur> (message from Alex Martelli on Sat, 15 Dec 2001 22:17:12 +0100) References: <200112151623.LAA32158@cj20424-a.reston1.va.home.com> <0112151923261C.01556@arthur> <200112151831.NAA02998@cj20424-a.reston1.va.home.com> <0112152217121F.01556@arthur> Message-ID: <200112152236.fBFMadK05615@mira.informatik.hu-berlin.de> > Remove it? I think this is unacceptable for 2.2, now that 2.2c1 carries the module. It seems that it still works fine as before with older versions of GMP, and it also seems that there are a few users out there of the module. If users care enough, somebody will contribute a patch. If they don't care, we can deprecate it for 2.3, and remove it in 2.4 (following the guidelines for language evolution). Regards, Martin From martin@v.loewis.de Sat Dec 15 22:52:17 2001 From: martin@v.loewis.de (Martin v. Loewis) Date: Sat, 15 Dec 2001 23:52:17 +0100 Subject: [Python-Dev] regrtest.py mystery In-Reply-To: References: Message-ID: <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> > What's up with that? So far, they all seem to involve the encodings > directory ... Python/codecs.c holds search functions in _PyCodec_SearchPath, among them encodings.__init__.search_function. It also imports "encodings" the first time somebody invokes .encode/.decode on some kind of string, but doesn't hold onto the module. When the last reference to encodings.__init__ goes away, Python will clear all globals - but codecs still holds a reference to search_function, so that won't go away. Invoking search_function later will cause the problem you see. One solution would be to bind all globals used inside encodings.__init__.search_function to the search_function, e.g. by means of a class: class GlobalSearchFunction: _cache = _unknown = class CodecRegistryError(...): def search_function(self,encoding): codecs.register(GlobalSearchFunction().search_function) Of course, if this gets cleaned from sys.modules, nobody will be able to catch CodecRegistryError anymore (since re-importing the module will produce a different class). Regards, Martin From fdrake@acm.org Sat Dec 15 23:53:57 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Sat, 15 Dec 2001 18:53:57 -0500 Subject: [Python-Dev] regrtest.py mystery In-Reply-To: <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> References: <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> Message-ID: <15387.58005.329451.71260@grendel.zope.com> Martin v. Loewis writes: > Python/codecs.c holds search functions in _PyCodec_SearchPath, among > them encodings.__init__.search_function. It also imports "encodings" > the first time somebody invokes .encode/.decode on some kind of > string, but doesn't hold onto the module. Perhaps it should hold onto the module as well? That would avoid it getting cleaned up before possibly dependent modules. For the test case problem, perhaps there should be a list of exceptional modules that don't get cleaned up by regrtest once they appear. encodings and warnings would probably both need to be on the list. This list should probably be in the sys module; it would also be useful for the RollbackImporter in the unittestgui.py script that comes with the separate PyUnit package. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From guido@python.org Sun Dec 16 01:01:35 2001 From: guido@python.org (Guido van Rossum) Date: Sat, 15 Dec 2001 20:01:35 -0500 Subject: [Python-Dev] Re: mpz module broken with GMP 4.0 In-Reply-To: Your message of "Sat, 15 Dec 2001 23:36:39 +0100." <200112152236.fBFMadK05615@mira.informatik.hu-berlin.de> References: <200112151623.LAA32158@cj20424-a.reston1.va.home.com> <0112151923261C.01556@arthur> <200112151831.NAA02998@cj20424-a.reston1.va.home.com> <0112152217121F.01556@arthur> <200112152236.fBFMadK05615@mira.informatik.hu-berlin.de> Message-ID: <200112160101.UAA03490@cj20424-a.reston1.va.home.com> > > Remove it? > > I think this is unacceptable for 2.2, now that 2.2c1 carries the > module. It seems that it still works fine as before with older > versions of GMP, and it also seems that there are a few users out > there of the module. > > If users care enough, somebody will contribute a patch. If they don't > care, we can deprecate it for 2.3, and remove it in 2.4 (following the > guidelines for language evolution). Let's mark it as deprecated in 2.2, and remove it in 2.3. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@home.com Sun Dec 16 05:23:43 2001 From: tim.one@home.com (Tim Peters) Date: Sun, 16 Dec 2001 00:23:43 -0500 Subject: [Python-Dev] regrtest.py mystery In-Reply-To: <15387.58005.329451.71260@grendel.zope.com> Message-ID: [Fred] > Perhaps it should hold onto the module as well? That would avoid it > getting cleaned up before possibly dependent modules. > For the test case problem, perhaps there should be a list of > exceptional modules that don't get cleaned up by regrtest once they > appear. encodings and warnings would probably both need to be on the > list. This list should probably be in the sys module; it would also > be useful for the RollbackImporter in the unittestgui.py script that > comes with the separate PyUnit package. I expect the problem is intractable. Here's a revealing experiment: add this block to the start of test___all__.check_all(): if modname in sys.modules: del sys.modules[modname] That is, just prior to test___all__ trying to import a module, remove *only* that module from sys.modules if it's already there. Sounds pretty harmless, right? But on Windows, it causes 28 standard tests to fail (possibly more on Linux): 28 tests failed: test___all__ test_difflib test_email test_fileinput test_fnmatch test_future test_gettext test_glob test_global test_htmllib test_htmlparser test_inspect test_mimetypes test_mmap test_os test_pyclbr test_re test_regex test_repr test_scope test_sre test_strftime test_strop test_sundry test_tokenize test_urllib2 test_xmllib test_xmlrpc Trying to figure which modules "hurt" turns out to be a nightmare. Here's the minimal set that allows the test suite to pass again on Windows, using the default test order: if modname in sys.modules: if modname not in ( "copy_reg", "os", "sgmllib", "sre", "tempfile", "tokenize", "warnings", ): del sys.modules[modname] Am I gonna check that in? Fat chance . From tim.one@home.com Sun Dec 16 05:42:33 2001 From: tim.one@home.com (Tim Peters) Date: Sun, 16 Dec 2001 00:42:33 -0500 Subject: [Python-Dev] regression tests with regrtest.py -r In-Reply-To: Message-ID: [Tim] > ordering that caused it. Heh -- -r failed in test___all__ while I was > typing this, but I've no idea why. I'll pursue it until I get > sick to death of it . In case anyone's waiting, stop -- I got sick to death of it, and won't be checking in anything related. (BTW, there is no test_rlcompleter, of course; it's the combo of test_sundry and test___all__ that screws up (both of those import rlcompleter)) From martin@v.loewis.de Sun Dec 16 09:38:34 2001 From: martin@v.loewis.de (Martin v. Loewis) Date: Sun, 16 Dec 2001 10:38:34 +0100 Subject: [Python-Dev] regrtest.py mystery In-Reply-To: <15387.58005.329451.71260@grendel.zope.com> (fdrake@acm.org) References: <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> <15387.58005.329451.71260@grendel.zope.com> Message-ID: <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de> > > Python/codecs.c holds search functions in _PyCodec_SearchPath, among > > them encodings.__init__.search_function. It also imports "encodings" > > the first time somebody invokes .encode/.decode on some kind of > > string, but doesn't hold onto the module. > > Perhaps it should hold onto the module as well? That would avoid it > getting cleaned up before possibly dependent modules. Re-importing encodings would still do funny things, then: It would register itself once more, resulting in two search functions being registered. codecs would hold one of the encodings modules, but not the other. So if the second search function is ever invoked, you get the same error. Regards, Martin From skip@pobox.com (Skip Montanaro) Sun Dec 16 15:11:31 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Sun, 16 Dec 2001 09:11:31 -0600 Subject: [Python-Dev] regression tests with regrtest.py -r In-Reply-To: References: Message-ID: <15388.47523.871195.694764@12-248-41-177.client.attbi.com> Tim> In case anyone's waiting, stop -- I got sick to death of it, and Tim> won't be checking in anything related. (BTW, there is no Tim> test_rlcompleter, of course; it's the combo of test_sundry and Tim> test___all__ that screws up (both of those import rlcompleter)) A couple wild-ass-guess ideas: * Perhaps more test___all__ smarts are needed? You added a "known skips" list to regrtest.py. test___all__ could perhaps avoid importing any module that's not supposed to work on the current platform. * How about comparing sys.modules.keys() before and after a test and then remove any that were added that are Python modules? (It's not safe to remove extension modules, right?) calling-it-intractable-means-someone-will-solve-it-ly, y'rs, Skip From simon@simon-cozens.org Sun Dec 16 21:14:05 2001 From: simon@simon-cozens.org (Simon Cozens) Date: Sun, 16 Dec 2001 21:14:05 +0000 Subject: [Python-Dev] First cut of pyc2pasm released Message-ID: <20011216211405.GA16141@netthink.co.uk> I hope you do not mind another brief Parroty interlude. If you're sick of that kind of thing, I apologise, I'm not trying to poach your time, so excuse me. But! With the 0.0.3 release of Parrot list week, we've got sufficiently powerful that it's sensible to think about running other languages' bytecode on the Parrot VM. (Converting the bytecode alone saves the hassle of writing a compiler, which is the difficult bit.) So naturally, I started with Perl. I think that lasted for about half an hour before I ran away screaming. Don't get me wrong, I like Perl, I just wouldn't like to turn its op tree into anything sensible. So I thought I'd show the buggers. Python's bytecode is nice and straightforward, and dis.py turns it into relatively simple assembly. The operations that it needs, Parrot provides, even things like set_lineno and rot_four. It was only a matter of time before I could disassemble Python bytecode and put it together into Parrot assembly. http://archive.develooper.com/perl6-internals@perl.org/msg07040.html is about as far as I got. It works. It runs "Hello World" and things slightly more complex, and it groks simple variable manipulation. It wouldn't be much hassle to add control structures. As a slight disclaimer, the variable classes [1] it uses - PythonInteger and so on - don't actually exist, but you can fake it by changing the outputted assembler code to use the ones that Parrot provides. I'm going to work on providing Python types in the core Real Soon Now, tuit permitting. Anyway, there it is. Enjoy it. If you want to work on it, feel free; if you want to take it and rewrite it in Python, then I'd be honored. Whatever, it's out there, and we're serious about making Parrot work for Python. (Or is it the other way around..? ;) Simon [1] One of Parrot's internal data types is the Parrot Magic Cookie, something which is not a million miles distant from a PyObject. You define PMCs by filling in a standard structure of manipulator methods, just like in Objects/*object.c. -- Hubris is when you really do have it, enough so only the gods slap you down. Pretentiousness is when you don't have it, and everyone slaps you down. Arrogance is somewhere in between. - Thorfinn From mal@lemburg.com Mon Dec 17 10:56:14 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 17 Dec 2001 11:56:14 +0100 Subject: [Python-Dev] regrtest.py mystery References: <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> <15387.58005.329451.71260@grendel.zope.com> <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de> Message-ID: <3C1DCF4E.193D27AB@lemburg.com> "Martin v. Loewis" wrote: > > > > Python/codecs.c holds search functions in _PyCodec_SearchPath, among > > > them encodings.__init__.search_function. It also imports "encodings" > > > the first time somebody invokes .encode/.decode on some kind of > > > string, but doesn't hold onto the module. > > > > Perhaps it should hold onto the module as well? That would avoid it > > getting cleaned up before possibly dependent modules. > > Re-importing encodings would still do funny things, then: It would > register itself once more, resulting in two search functions being > registered. codecs would hold one of the encodings modules, but not > the other. So if the second search function is ever invoked, you get > the same error. Rather than "fixing" the encodings package, I'd rather suggest to fix the unload procedure in the test suite. Some modules simply should not be unloaded because it is known that the interpreter needs them for internal purposes. In the case of the encodings package it is not only the search function that gets kept alive, it's also the codecs themselves which are cached by the implementation in _codecs.c. How about following Fred's suggestion to run each test in a separate process (possibly as an option to regrtest.py) ?! -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From guido@python.org Mon Dec 17 14:08:31 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 17 Dec 2001 09:08:31 -0500 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Modules xxsubtype.c,2.9,2.10 In-Reply-To: Your message of "Mon, 17 Dec 2001 10:39:31 +0100." <3C1DBD53.2672A1C@lemburg.com> References: <3C1DBD53.2672A1C@lemburg.com> Message-ID: <200112171408.JAA16669@cj20424-a.reston1.va.home.com> [Tim] > > Log Message: > > David Abrahams tried to compile this as a separate DLL under MSVC, and > > got a barrage of compile errors that didn't make sense to the C++ brain: > > MSVC does not allow C (but does allow C++) initializers to contain > > data addresses supplied by other DLLs. So changed the initializers here > > to use dummy nulls, and changed module init to plug in the foreign > > addresses at runtime (manually simulating what C++ does by magic). Tested > > on Windows, and Guido tested on Linux (thanks!). BTW, the *point* is that > > people are going to use this module as a template for writing their own > > subtypes, and it's unusual for extension authors to build their extensions > > into Python directly (separate DLLs are the norm on Windows); so it's > > better if we give them a template that works . [MAL] > This is a common problem when compiling Windows extensions. Perhaps > we should provide some more generic way to help the extension > writers here ?! FWIW, I'm using this macro for doing the deferred > type init: > > #define PyType_Init(x) \ > { \ > x.ob_type = &PyType_Type; \ > } Hm, alternatively, we could put a statement in PyType_Ready() that initializes ob_type if it's NULL -- then we'd encourage people to call PyType_Ready(), which is a good idea anyway. --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas.heller@ion-tof.com Mon Dec 17 14:23:06 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Mon, 17 Dec 2001 15:23:06 +0100 Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Modules xxsubtype.c,2.9,2.10 References: <3C1DBD53.2672A1C@lemburg.com> <200112171408.JAA16669@cj20424-a.reston1.va.home.com> Message-ID: <021401c18706$6aacaf20$e000a8c0@thomasnotebook> > Hm, alternatively, we could put a statement in PyType_Ready() that > initializes ob_type if it's NULL -- then we'd encourage people to call > PyType_Ready(), which is a good idea anyway. > > --Guido van Rossum (home page: http://www.python.org/~guido/) +2 ;-) Thomas From fdrake@acm.org Mon Dec 17 14:26:49 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 17 Dec 2001 09:26:49 -0500 (EST) Subject: [Python-Dev] Re: [Python-checkins] CVS: python/dist/src/Modules xxsubtype.c,2.9,2.10 In-Reply-To: <200112171408.JAA16669@cj20424-a.reston1.va.home.com> References: <3C1DBD53.2672A1C@lemburg.com> <200112171408.JAA16669@cj20424-a.reston1.va.home.com> Message-ID: <15390.169.62909.589215@cj42289-a.reston1.va.home.com> Guido van Rossum writes: > Hm, alternatively, we could put a statement in PyType_Ready() that > initializes ob_type if it's NULL -- then we'd encourage people to call > PyType_Ready(), which is a good idea anyway. This would be very good. I'll second Thomas's +2. ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From fdrake@acm.org Mon Dec 17 14:32:00 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 17 Dec 2001 09:32:00 -0500 (EST) Subject: [Python-Dev] regrtest.py mystery In-Reply-To: <3C1DCF4E.193D27AB@lemburg.com> References: <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> <15387.58005.329451.71260@grendel.zope.com> <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de> <3C1DCF4E.193D27AB@lemburg.com> Message-ID: <15390.480.513981.132090@cj42289-a.reston1.va.home.com> M.-A. Lemburg writes: > How about following Fred's suggestion to run each test in a separate > process (possibly as an option to regrtest.py) ?! I wasn't really suggesting it, but it would solve this particular problem. Note that the effectiveness of this to solve this particular problem depends on the set of modules regrtest itself imports; if it imports something and catches an ImportError, using a simple fork() to create the child is not sufficient. Using fork()/exec() would be extremely slow and should be avoided if we can. Frankly, I like that we find interactions between the tests, so I'm not convinced that there are any real problems with the test framework. I do use to a C unit test package that uses fork() to make sure individual tests can't core the framework, but that's not what we're protecting against here. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From mwh@python.net Mon Dec 17 14:36:59 2001 From: mwh@python.net (Michael Hudson) Date: 17 Dec 2001 14:36:59 +0000 Subject: [Python-Dev] regrtest.py mystery In-Reply-To: "Fred L. Drake, Jr."'s message of "Mon, 17 Dec 2001 09:32:00 -0500 (EST)" References: <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> <15387.58005.329451.71260@grendel.zope.com> <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de> <3C1DCF4E.193D27AB@lemburg.com> <15390.480.513981.132090@cj42289-a.reston1.va.home.com> Message-ID: <2mbsgxlxpw.fsf@starship.python.net> "Fred L. Drake, Jr." writes: > M.-A. Lemburg writes: > > How about following Fred's suggestion to run each test in a separate > > process (possibly as an option to regrtest.py) ?! > > I wasn't really suggesting it, but it would solve this particular > problem. Note that the effectiveness of this to solve this particular > problem depends on the set of modules regrtest itself imports; if it > imports something and catches an ImportError, using a simple fork() to > create the child is not sufficient. Using fork()/exec() would be > extremely slow and should be avoided if we can. And I think you'd have fun getting anything like that to run on MacOS before OSX, and probably other oddball platforms (Windows at least has the os.spawn* functions). Making the test suite less portable is /unlikely/ to be a win, I'd have thought. Cheers, M. -- People think I'm a nice guy, and the fact is that I'm a scheming, conniving bastard who doesn't care for any hurt feelings or lost hours of work if it just results in what I consider to be a better system. -- Linus Torvalds From barry@zope.com Mon Dec 17 15:45:36 2001 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 17 Dec 2001 10:45:36 -0500 Subject: [Python-Dev] regrtest.py mystery References: <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> <15387.58005.329451.71260@grendel.zope.com> <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de> <3C1DCF4E.193D27AB@lemburg.com> Message-ID: <15390.4896.825987.759927@anthem.wooz.org> >>>>> "M" == M writes: M> Rather than "fixing" the encodings package, I'd rather suggest M> to fix the unload procedure in the test suite. Featurette request for Python 2.3: a builtin unload() which Does The Right Thing to unload a module, or raise an exception if the module cannot be unloaded. Lots and lots of applications would find this useful. -Barry From fdrake@acm.org Mon Dec 17 15:54:51 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 17 Dec 2001 10:54:51 -0500 (EST) Subject: [Python-Dev] regrtest.py mystery In-Reply-To: <15390.4896.825987.759927@anthem.wooz.org> References: <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> <15387.58005.329451.71260@grendel.zope.com> <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de> <3C1DCF4E.193D27AB@lemburg.com> <15390.4896.825987.759927@anthem.wooz.org> Message-ID: <15390.5451.781522.718432@cj42289-a.reston1.va.home.com> Barry A. Warsaw writes: > Featurette request for Python 2.3: a builtin unload() which Does The > Right Thing to unload a module, or raise an exception if the module > cannot be unloaded. Lots and lots of applications would find this > useful. Useful or not, isn't the basic problem that we haven't been able to define The Right Thing? I'd certainly expect a module to be able to declare in some way that it could not be unloaded, or control how it gets unloaded. Perhaps allow Python modules to define an __unload__() function; when unload() is called on a module, the __unload__() is called; if that raises an exception, it gets propogated, otherwise the unload continues if the module still exists in sys.modules. Not clear that this would be useful for extensions. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From paul@pfdubois.com Mon Dec 17 16:09:59 2001 From: paul@pfdubois.com (Paul F. Dubois) Date: Mon, 17 Dec 2001 08:09:59 -0800 Subject: [Python-Dev] RE: [Numpy-discussion] Python 2.2 In-Reply-To: <006201c186e7$da0d32f0$df13000a@RDREWXP> Message-ID: <000001c18715$477816c0$0c01a8c0@freedom> I downloaded and installed 2.2c1 and built the Numeric installers. However, when I run them they all fail when the installation begins (after one clicks the final click to install) with an access violation. I removed any previous Numeric and it still happened. Building and installing with setup.py install works ok. -----Original Message----- From: numpy-discussion-admin@lists.sourceforge.net [mailto:numpy-discussion-admin@lists.sourceforge.net] On Behalf Of Ray Drew Sent: Monday, December 17, 2001 2:45 AM To: numpy-discussion@lists.sourceforge.net Subject: [Numpy-discussion] Python 2.2 Does anyone know when a release for Python 2.2 on Windows will be available? regards, Ray Drew _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From guido@python.org Mon Dec 17 16:12:54 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 17 Dec 2001 11:12:54 -0500 Subject: [Python-Dev] regrtest.py mystery In-Reply-To: Your message of "Mon, 17 Dec 2001 10:45:36 EST." <15390.4896.825987.759927@anthem.wooz.org> References: <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> <15387.58005.329451.71260@grendel.zope.com> <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de> <3C1DCF4E.193D27AB@lemburg.com> <15390.4896.825987.759927@anthem.wooz.org> Message-ID: <200112171612.LAA01013@cj20424-a.reston1.va.home.com> > Featurette request for Python 2.3: a builtin unload() which Does The > Right Thing to unload a module, or raise an exception if the module > cannot be unloaded. Lots and lots of applications would find this > useful. I haven't been following this discussion. Can you give a clear definition of The Right Thing? --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Mon Dec 17 16:18:42 2001 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 17 Dec 2001 11:18:42 -0500 Subject: [Python-Dev] regrtest.py mystery References: <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> <15387.58005.329451.71260@grendel.zope.com> <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de> <3C1DCF4E.193D27AB@lemburg.com> <15390.4896.825987.759927@anthem.wooz.org> <15390.5451.781522.718432@cj42289-a.reston1.va.home.com> Message-ID: <15390.6882.514689.285195@anthem.wooz.org> >>>>> "Fred" == Fred L Drake, Jr writes: Fred> Useful or not, isn't the basic problem that we haven't Fred> been able to define The Right Thing? Probably so. Fred> I'd certainly expect a module to be able to declare in some Fred> way that it could not be unloaded, or control how it gets Fred> unloaded. Perhaps allow Python modules to define an Fred> __unload__() function; when unload() is called on a module, Fred> the __unload__() is called; if that raises an exception, it Fred> gets propogated, otherwise the unload continues if the Fred> module still exists in sys.modules. Not clear that this Fred> would be useful for extensions. The "unload protocol" would be I think. I.e. extension modules ought to be unloadable if they can cooperate. From the Linux dl*() manpages: dlclose decrements the reference count on the dynamic library handle handle. If the reference count drops to zero and no other loaded libraries use symbols in it, then the dynamic library is unloaded. If the dynamic library exports a routine named _fini, then that routine is called just before the library is unloaded. season-to-tasste-on-your-favorite-OS-bread-l y'rs, -Barry From guido@python.org Mon Dec 17 16:16:27 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 17 Dec 2001 11:16:27 -0500 Subject: [Python-Dev] RE: [Numpy-discussion] Python 2.2 In-Reply-To: Your message of "Mon, 17 Dec 2001 08:09:59 PST." <000001c18715$477816c0$0c01a8c0@freedom> References: <000001c18715$477816c0$0c01a8c0@freedom> Message-ID: <200112171616.LAA04194@cj20424-a.reston1.va.home.com> > I downloaded and installed 2.2c1 and built the Numeric installers. > However, when I run them they all fail when the installation begins > (after one clicks the final click to install) with an access violation. > I removed any previous Numeric and it still happened. > > Building and installing with setup.py install works ok. I don't know anything about your installers, but could it be that you were trying to install without Administrator permissions? That used to crash the previous Python installer too. (The new one doesn't, but it's a commercial product so we can't share it.) --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Mon Dec 17 16:21:41 2001 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 17 Dec 2001 11:21:41 -0500 Subject: [Python-Dev] regrtest.py mystery References: <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> <15387.58005.329451.71260@grendel.zope.com> <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de> <3C1DCF4E.193D27AB@lemburg.com> <15390.4896.825987.759927@anthem.wooz.org> <200112171612.LAA01013@cj20424-a.reston1.va.home.com> Message-ID: <15390.7061.122528.163470@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> I haven't been following this discussion. Can you give a GvR> clear definition of The Right Thing? I think we need to define the goals of unloading. From mwh@python.net Mon Dec 17 16:25:54 2001 From: mwh@python.net (Michael Hudson) Date: 17 Dec 2001 16:25:54 +0000 Subject: [Python-Dev] regrtest.py mystery In-Reply-To: barry@zope.com's message of "Mon, 17 Dec 2001 11:21:41 -0500" References: <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> <15387.58005.329451.71260@grendel.zope.com> <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de> <3C1DCF4E.193D27AB@lemburg.com> <15390.4896.825987.759927@anthem.wooz.org> <200112171612.LAA01013@cj20424-a.reston1.va.home.com> <15390.7061.122528.163470@anthem.wooz.org> Message-ID: <2mheqpke3x.fsf@starship.python.net> barry@zope.com (Barry A. Warsaw) writes: > >>>>> "GvR" == Guido van Rossum writes: > > GvR> I haven't been following this discussion. Can you give a > GvR> clear definition of The Right Thing? > > I think we need to define the goals of unloading. Doesn't that make it into a solution in search of a problem? When is unloading useful? I'm not being facetious, I've just never been hit by the need for it. Cheers, M. -- Q: Isn't it okay to just read Slashdot for the links? A: No. Reading Slashdot for the links is like having "just one hit" off the crack pipe. -- http://www.cs.washington.edu/homes/klee/misc/slashdot.html#faq From guido@python.org Mon Dec 17 16:31:17 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 17 Dec 2001 11:31:17 -0500 Subject: [Python-Dev] regrtest.py mystery In-Reply-To: Your message of "Mon, 17 Dec 2001 11:21:41 EST." <15390.7061.122528.163470@anthem.wooz.org> References: <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> <15387.58005.329451.71260@grendel.zope.com> <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de> <3C1DCF4E.193D27AB@lemburg.com> <15390.4896.825987.759927@anthem.wooz.org> <200112171612.LAA01013@cj20424-a.reston1.va.home.com> <15390.7061.122528.163470@anthem.wooz.org> Message-ID: <200112171631.LAA15524@cj20424-a.reston1.va.home.com> > >>>>> "GvR" == Guido van Rossum writes: > > GvR> I haven't been following this discussion. Can you give a > GvR> clear definition of The Right Thing? > > I think we need to define the goals of unloading. Then how can you call it a featurette? Who knows if it's even a tractable problem? --Guido van Rossum (home page: http://www.python.org/~guido/) From barry@zope.com Mon Dec 17 16:46:02 2001 From: barry@zope.com (Barry A. Warsaw) Date: Mon, 17 Dec 2001 11:46:02 -0500 Subject: [Python-Dev] regrtest.py mystery References: <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> <15387.58005.329451.71260@grendel.zope.com> <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de> <3C1DCF4E.193D27AB@lemburg.com> <15390.4896.825987.759927@anthem.wooz.org> <200112171612.LAA01013@cj20424-a.reston1.va.home.com> <15390.7061.122528.163470@anthem.wooz.org> <200112171631.LAA15524@cj20424-a.reston1.va.home.com> Message-ID: <15390.8522.971549.671038@anthem.wooz.org> >>>>> "GvR" == Guido van Rossum writes: GvR> Then how can you call it a featurette? Who knows if it's GvR> even a tractable problem? Okay, so it's a PEPable feature. -Barry From thomas.heller@ion-tof.com Mon Dec 17 16:49:54 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Mon, 17 Dec 2001 17:49:54 +0100 Subject: [Python-Dev] RE: [Numpy-discussion] Python 2.2 References: <000001c18715$477816c0$0c01a8c0@freedom> Message-ID: <028501c1871a$ec399260$e000a8c0@thomasnotebook> > I downloaded and installed 2.2c1 and built the Numeric installers. > However, when I run them they all fail when the installation begins > (after one clicks the final click to install) with an access violation. > I removed any previous Numeric and it still happened. > > Building and installing with setup.py install works ok. > Are you talking about bdist_wininst installers? I just could reproduce a problem with them (after renaming away all my zip.exe programs in the PATH). Thomas From guido@python.org Mon Dec 17 16:52:21 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 17 Dec 2001 11:52:21 -0500 Subject: [Python-Dev] regrtest.py mystery In-Reply-To: Your message of "Mon, 17 Dec 2001 11:46:02 EST." <15390.8522.971549.671038@anthem.wooz.org> References: <200112152252.fBFMqH205644@mira.informatik.hu-berlin.de> <15387.58005.329451.71260@grendel.zope.com> <200112160938.fBG9cYV01682@mira.informatik.hu-berlin.de> <3C1DCF4E.193D27AB@lemburg.com> <15390.4896.825987.759927@anthem.wooz.org> <200112171612.LAA01013@cj20424-a.reston1.va.home.com> <15390.7061.122528.163470@anthem.wooz.org> <200112171631.LAA15524@cj20424-a.reston1.va.home.com> <15390.8522.971549.671038@anthem.wooz.org> Message-ID: <200112171652.LAA15707@cj20424-a.reston1.va.home.com> > Okay, so it's a PEPable feature. Not until you've thought about what it should do. I bet you lunch that you can't come up with a working definition. --Guido van Rossum (home page: http://www.python.org/~guido/) From paul@pfdubois.com Mon Dec 17 17:24:14 2001 From: paul@pfdubois.com (Paul F. Dubois) Date: Mon, 17 Dec 2001 09:24:14 -0800 Subject: [Python-Dev] RE: [Numpy-discussion] Python 2.2 In-Reply-To: <200112171616.LAA04194@cj20424-a.reston1.va.home.com> Message-ID: <000201c1871f$a6f514e0$0c01a8c0@freedom> I am on Windows Me, so there is no concept of administrator. -----Original Message----- From: numpy-discussion-admin@lists.sourceforge.net [mailto:numpy-discussion-admin@lists.sourceforge.net] On Behalf Of Guido van Rossum Sent: Monday, December 17, 2001 8:16 AM To: Paul F. Dubois Cc: 'Ray Drew'; numpy-discussion@lists.sourceforge.net; python-dev@python.org Subject: Re: [Python-Dev] RE: [Numpy-discussion] Python 2.2 > I downloaded and installed 2.2c1 and built the Numeric installers. > However, when I run them they all fail when the installation begins > (after one clicks the final click to install) with an access > violation. I removed any previous Numeric and it still happened. > > Building and installing with setup.py install works ok. I don't know anything about your installers, but could it be that you were trying to install without Administrator permissions? That used to crash the previous Python installer too. (The new one doesn't, but it's a commercial product so we can't share it.) --Guido van Rossum (home page: http://www.python.org/~guido/) _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From jeremy@zope.com Mon Dec 17 18:24:26 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 17 Dec 2001 13:24:26 -0500 (EST) Subject: [Python-Dev] compiler package udpate Message-ID: <15390.14426.296516.102494@slothrop.digicool.com> I'd like to backport changes from the 2.2 compiler package (Lib/compiler) to the 2.1.2 version (Tools/compiler/compiler). The changes are bug fixes, but include some non-trivial changes because the old code has some non-trivial bugs. The chief problem is that it underestimates the stack depth, which leads to core dumps. Any objections to me making these changes today? Jeremy From tim.one@home.com Mon Dec 17 19:38:48 2001 From: tim.one@home.com (Tim Peters) Date: Mon, 17 Dec 2001 14:38:48 -0500 Subject: [Python-Dev] regrtest.py mystery In-Reply-To: <3C1DCF4E.193D27AB@lemburg.com> Message-ID: Without diving into the unloading pit, the endless problems with -r on non-Unix boxes would go away if a module that failed to import didn't leave behind a damaged module object in sys.modules ("damaged" == it may not fulfill some or any of the module's contracts, and it's impossible to guess which ones it won't, or to guess how the unfufilled ones may fail). I'm tempted to add a pile of Windows-specific modules that import each other in incestuous ways, so everyone can enjoy this chronic pain . "A fix" seems to amount to treating import M as if it were try: import M except: if M in sys.modules: del sys.modules[M] raise How would that hurt? From ping@lfw.org Mon Dec 17 21:05:12 2001 From: ping@lfw.org (Ka-Ping Yee) Date: Mon, 17 Dec 2001 15:05:12 -0600 (CST) Subject: [Python-Dev] regrtest.py mystery In-Reply-To: Message-ID: On Mon, 17 Dec 2001, Tim Peters wrote: > "A fix" seems to amount to treating > > import M > > as if it were > > try: > import M > except: > if M in sys.modules: > del sys.modules[M] > raise I remember suggesting exactly that some time ago (i think it was motivated at the time by the extreme pain that broken modules were causing for webserver-mode pydoc in its attempt to update loaded modules if the source files had changed on disk). Guido rejected it because you can't guarantee that the refcount on M is 1 at the point where you attempt to 'del sys.modules[M]' above. (For example, some other module imported by M could have imported M again, and so hold a reference to it while M is running its startup commands. This is why the entry is added to sys.modules before the body of M starts to run.) He deemed the situation where M is loaded-but-missing-from-sys.modules to be even worse than for M to be left loaded-but-broken therein. If you allow M to stay in sys.modules, then you can at least maintain the guarantee that there is a one-to-one association between loaded module names and loaded module objects. If you remove M from sys.modules but it lingers in memory, referenced elsewhere, you lose even that -- there can be many modules loaded all with the same name and it's a nightmare. The argument is compelling and i am forced to agree, but i still think that we should look for a better solution. I'll ponder this and post any ideas i come up with. -- ?!ng From mal@lemburg.com Mon Dec 17 21:25:56 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 17 Dec 2001 22:25:56 +0100 Subject: [Python-Dev] Unloading modules (regrtest.py mystery) References: Message-ID: <3C1E62E4.C4E9A7A0@lemburg.com> Ka-Ping Yee wrote: > > On Mon, 17 Dec 2001, Tim Peters wrote: > > "A fix" seems to amount to treating > > > > import M > > > > as if it were > > > > try: > > import M > > except: > > if M in sys.modules: > > del sys.modules[M] > > raise > > I remember suggesting exactly that some time ago (i think it was > motivated at the time by the extreme pain that broken modules were > causing for webserver-mode pydoc in its attempt to update loaded > modules if the source files had changed on disk). > > Guido rejected it because you can't guarantee that the refcount on M > is 1 at the point where you attempt to 'del sys.modules[M]' above. > (For example, some other module imported by M could have imported M > again, and so hold a reference to it while M is running its startup > commands. This is why the entry is added to sys.modules before the > body of M starts to run.) > > He deemed the situation where M is loaded-but-missing-from-sys.modules > to be even worse than for M to be left loaded-but-broken therein. If > you allow M to stay in sys.modules, then you can at least maintain the > guarantee that there is a one-to-one association between loaded module > names and loaded module objects. If you remove M from sys.modules but > it lingers in memory, referenced elsewhere, you lose even that -- there > can be many modules loaded all with the same name and it's a nightmare. > > The argument is compelling and i am forced to agree, but i still think > that we should look for a better solution. I'll ponder this and post > any ideas i come up with. How about only deleting it from sys.modules iff the ref count is 1 ?! Note that you cannot currently remove extension modules that have failed to import correctly. Would be nice if we could come up with a reasonable way of unloading modules, but I guess this requires some hard PEP work... -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From sdm7g@Virginia.EDU Mon Dec 17 21:40:01 2001 From: sdm7g@Virginia.EDU (Steven Majewski) Date: Mon, 17 Dec 2001 16:40:01 -0500 (EST) Subject: [Python-Dev] regrtest.py mystery In-Reply-To: <2mheqpke3x.fsf@starship.python.net> Message-ID: On 17 Dec 2001, Michael Hudson wrote: > barry@zope.com (Barry A. Warsaw) writes: > > > >>>>> "GvR" == Guido van Rossum writes: > > > > GvR> I haven't been following this discussion. Can you give a > > GvR> clear definition of The Right Thing? > > > > I think we need to define the goals of unloading. > > Doesn't that make it into a solution in search of a problem? > > When is unloading useful? I'm not being facetious, I've just never > been hit by the need for it. Shared libraries in XlispStat support unloading. But all of the times I've ever used it, I've used it as a 'reload' : usually while debugging shared libs -- it's handy to keep the interpreter window running while you fix and recompile a bug and load it again. The only case I can thing of where I might use it in production code would also be for a reload, where you might have several alternate 'plugin' modules. When you do an unload, references to objects that were in the shared library are replaced by references to a stub that generates an error/exception. ( That stub routine is part of the builtin dynamic load "module" . ) It doesn't involve traceing all the references -- the pointers are unchanged, but the object is changed. I have no idea what 'the Right Thing' for Python would be here -- I'm not quite clear on what you're trying to fix. I'm just volunteering an example. -- Steve Majewski From sdm7g@Virginia.EDU Mon Dec 17 22:19:13 2001 From: sdm7g@Virginia.EDU (Steven Majewski) Date: Mon, 17 Dec 2001 17:19:13 -0500 (EST) Subject: [Python-Dev] regrtest.py mystery In-Reply-To: Message-ID: On Mon, 17 Dec 2001, Steven Majewski wrote: > I have no idea what 'the Right Thing' for Python would be here -- > I'm not quite clear on what you're trying to fix. I'm just > volunteering an example. However, I will add that if I *did* need an unload for anything, I would think the XlispStat semantics: unloading the module immediately and making any dangling references into exceptions, or replacing them with references to a reloaded module, would seem to be more useful than not really unloading it until all of the references are decremented (if ever) and then unloading. I can't think of a case where I would want to unload a module, where I would not want a dangling reference to something in that module to generate an exception. -- Steve From guido@python.org Mon Dec 17 22:24:31 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 17 Dec 2001 17:24:31 -0500 Subject: [Python-Dev] regrtest.py mystery In-Reply-To: Your message of "Mon, 17 Dec 2001 17:19:13 EST." References: Message-ID: <200112172224.RAA19426@cj20424-a.reston1.va.home.com> > However, I will add that if I *did* need an unload for anything, > I would think the XlispStat semantics: unloading the module > immediately and making any dangling references into exceptions, > or replacing them with references to a reloaded module, > would seem to be more useful than not really unloading it until > all of the references are decremented (if ever) and then > unloading. > > I can't think of a case where I would want to unload a module, > where I would not want a dangling reference to something in that > module to generate an exception. Python doesn't let you replace a reference to an object with something else -- except in special cases (e.g. with explicit weak refs) there's no way to know where references to an object might exist. So that's not a useful idea, alas. --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Mon Dec 17 22:37:53 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 17 Dec 2001 17:37:53 -0500 (EST) Subject: [Python-Dev] regrtest.py mystery In-Reply-To: <200112172224.RAA19426@cj20424-a.reston1.va.home.com> References: <200112172224.RAA19426@cj20424-a.reston1.va.home.com> Message-ID: <15390.29633.527008.217751@cj42289-a.reston1.va.home.com> Guido van Rossum writes: > So that's not a useful idea, alas. Well, it may be a useful idea, but difficult & impractical to implement in Python. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From greg@cosc.canterbury.ac.nz Mon Dec 17 22:47:06 2001 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 18 Dec 2001 11:47:06 +1300 (NZDT) Subject: [Python-Dev] regrtest.py mystery In-Reply-To: Message-ID: <200112172247.LAA17572@s454.cosc.canterbury.ac.nz> Ka-Ping Yee : > He deemed the situation where M is loaded-but-missing-from-sys.modules > to be even worse than for M to be left loaded-but-broken therein. Maybe modules should have a this-module-is-broken flag which is set when the module fails to import properly, and which causes an exception on any further attempt to reference a name in it. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From fredrik@pythonware.com Mon Dec 17 23:14:43 2001 From: fredrik@pythonware.com (Fredrik Lundh) Date: Tue, 18 Dec 2001 00:14:43 +0100 Subject: [Python-Dev] regrtest.py mystery References: <200112172247.LAA17572@s454.cosc.canterbury.ac.nz> Message-ID: <164201c18750$9ef89830$ced241d5@hagrid> greg wrote: > > > He deemed the situation where M is loaded-but-missing-from-sys.modules > > to be even worse than for M to be left loaded-but-broken therein. > > Maybe modules should have a this-module-is-broken flag which > is set when the module fails to import properly, and which > causes an exception on any further attempt to reference a > name in it. how do you detect that a module is successfully imported? the only way I can think of is "execution reached the end of the module", which would probably break considerable amounts of code that rely on "temporarily recursive" imports. From greg@cosc.canterbury.ac.nz Mon Dec 17 23:23:53 2001 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 18 Dec 2001 12:23:53 +1300 (NZDT) Subject: [Python-Dev] regrtest.py mystery In-Reply-To: <164201c18750$9ef89830$ced241d5@hagrid> Message-ID: <200112172323.MAA17577@s454.cosc.canterbury.ac.nz> Fredrik Lundh : > how do you detect that a module is successfully imported? > > the only way I can think of is "execution reached the end of > the module", which would probably break considerable amounts > of code that rely on "temporarily recursive" imports. If an exception propagates back through an import statement, the module that it's importing is marked as broken. This may result in a single error causing more than one module to be marked as broken, but I think that's the right thing to do anyway. E.g. if you import module A which imports module B which fails, you can't be sure that either A or B is fit to use, so both should be marked as broken. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim.one@home.com Mon Dec 17 23:36:22 2001 From: tim.one@home.com (Tim Peters) Date: Mon, 17 Dec 2001 18:36:22 -0500 Subject: [Python-Dev] regrtest.py mystery In-Reply-To: <164201c18750$9ef89830$ced241d5@hagrid> Message-ID: [/F] > how do you detect that a module is successfully imported? > > the only way I can think of is "execution reached the end of > the module", which would probably break considerable amounts > of code that rely on "temporarily recursive" imports. Presumably Greg's flag would be initialized to "not broken", and set to "broken" when and only when an uncaught exception is raised. So in import A A imports B which imports A back to B B completes back to A A blows up before completing A's "broken" flag isn't set until the end, so B's recursive import of A isn't harmed. From sdm7g@Virginia.EDU Mon Dec 17 23:43:48 2001 From: sdm7g@Virginia.EDU (Steven Majewski) Date: Mon, 17 Dec 2001 18:43:48 -0500 (EST) Subject: [Python-Dev] regrtest.py mystery In-Reply-To: <200112172224.RAA19426@cj20424-a.reston1.va.home.com> Message-ID: On Mon, 17 Dec 2001, Guido van Rossum wrote: > Python doesn't let you replace a reference to an object with something > else -- except in special cases (e.g. with explicit weak refs) there's > no way to know where references to an object might exist. I was too fuzzy in my choice of words. What I meant was not finding and replacing the references, but replacing the object to which all of the 'dangling references' point. But that won't work either: Python objects share the same head, but total size of the object structs are too variable, and I don't know it's possible to make the minimal size object show the required behaviour. But even if that is possible, another problem with implementing it in a Python extension module is that Python objects are not (except for maybe functions and things that drag along a reference to global module namespace) "owned" by their modules in anything like the association between Lisp symbols and their packages: python modules can import other modules or can dynamically add things to their namespaces. We can't just iterate through a module dict and 'zero' out all of the objects there. > So that's not a useful idea, alas. If the problem was important enough, I think a way could be found around those problems above, but I'm still not sure that module unloading isn't a solution looking for a problem! -- Steve From martin@v.loewis.de Mon Dec 17 23:47:04 2001 From: martin@v.loewis.de (Martin v. Loewis) Date: Tue, 18 Dec 2001 00:47:04 +0100 Subject: [Python-Dev] regrtest.py mystery In-Reply-To: <164201c18750$9ef89830$ced241d5@hagrid> (fredrik@pythonware.com) References: <200112172247.LAA17572@s454.cosc.canterbury.ac.nz> <164201c18750$9ef89830$ced241d5@hagrid> Message-ID: <200112172347.fBHNl4m11257@mira.informatik.hu-berlin.de> > the only way I can think of is "execution reached the end of > the module", which would probably break considerable amounts > of code that rely on "temporarily recursive" imports. I would have said "does not throw ImportError". Regards, Martin From jeremy@zope.com Tue Dec 18 00:06:52 2001 From: jeremy@zope.com (Jeremy Hylton) Date: Mon, 17 Dec 2001 19:06:52 -0500 (EST) Subject: [Python-Dev] Re: compiler package udpate In-Reply-To: <200112172209.fBHM9Xx02804@mbuna.arbhome.com.au> References: <15390.14426.296516.102494@slothrop.digicool.com> <200112172209.fBHM9Xx02804@mbuna.arbhome.com.au> Message-ID: <15390.34972.767112.259073@slothrop.digicool.com> I've made some good progress on it-- backported all the obvious stuff. I need to downgrade some of my testing tools so that they work with the old interface and re-run them. Once I've finished with these tests, I'll be satisfied. Jeremy From greg@cosc.canterbury.ac.nz Tue Dec 18 00:13:57 2001 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Tue, 18 Dec 2001 13:13:57 +1300 (NZDT) Subject: Broken module detection (Re: [Python-Dev] regrtest.py mystery) In-Reply-To: <200112172347.fBHNl4m11257@mira.informatik.hu-berlin.de> Message-ID: <200112180013.NAA17623@s454.cosc.canterbury.ac.nz> "Martin v. Loewis" : > I would have said "does not throw ImportError". Or any other error. If a module fails to complete all its initialisation code for any reason, it can't be trusted. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From guido@python.org Tue Dec 18 00:15:59 2001 From: guido@python.org (Guido van Rossum) Date: Mon, 17 Dec 2001 19:15:59 -0500 Subject: [Python-Dev] regrtest.py mystery In-Reply-To: Your message of "Mon, 17 Dec 2001 18:43:48 EST." References: Message-ID: <200112180015.TAA19857@cj20424-a.reston1.va.home.com> > > Python doesn't let you replace a reference to an object with something > > else -- except in special cases (e.g. with explicit weak refs) there's > > no way to know where references to an object might exist. > > I was too fuzzy in my choice of words. What I meant was not > finding and replacing the references, but replacing the object to > which all of the 'dangling references' point. > > But that won't work either: Python objects share the same head, but > total size of the object structs are too variable, and I don't know > it's possible to make the minimal size object show the required > behaviour. In general it's impossible. You can't even replace the type pointer with a pointer to a dummy type, because it would leak any objects referenced from the original object, and it would break if the original object lived in a special free list. > If the problem was important enough, I think a way could be found > around those problems above, but I'm still not sure that module > unloading isn't a solution looking for a problem! Well, there are a bunch of people who want to undo a partial import, and that boils down to about the same thing. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one@home.com Tue Dec 18 02:41:23 2001 From: tim.one@home.com (Tim Peters) Date: Mon, 17 Dec 2001 21:41:23 -0500 Subject: [Python-Dev] regrtest.py mystery In-Reply-To: Message-ID: [Ka-Ping Yee] Hi, Ping! If we don't hear from you again, Merry Christmas too . > I remember suggesting exactly that some time ago (i think it was > motivated at the time by the extreme pain that broken modules were > causing for webserver-mode pydoc in its attempt to update loaded > modules if the source files had changed on disk). > > Guido rejected it because you can't guarantee that the refcount on M > is 1 at the point where you attempt to 'del sys.modules[M]' above. If somebody wants to hold on to a reference to a damaged module object, that's OK by me. That's an unusual case, though, so shouldn't wag the dog. > (For example, some other module imported by M could have imported M > again, and so hold a reference to it while M is running its startup > commands. This is why the entry is added to sys.modules before the > body of M starts to run.) Understood, but it's rare. > He deemed the situation where M is loaded-but-missing-from-sys.modules > to be even worse than for M to be left loaded-but-broken therein. Why? The module is broken no mater what. I'm much keener on seeing that the *next* attempt to import the module doesn't falsely appear to succeed. > If you allow M to stay in sys.modules, then you can at least maintain the > guarantee that there is a one-to-one association between loaded module > names and loaded module objects. If you remove M from sys.modules but > it lingers in memory, referenced elsewhere, you lose even that -- there > can be many modules loaded all with the same name and it's a nightmare. Whether there's one copy or a million, it's broken. Given that, I don't see value in maintaining the one_name<->one_module invariant: of what practical use is it if the module is an insane state regardless? If there is some value to it I'm missing, it would be easy to give it a unique random (re)name . > The argument is compelling and i am forced to agree, but i still think > that we should look for a better solution. I'll ponder this and post > any ideas i come up with. Greg's "broken module" flag would do it for me too, if an attempt to import a known-to-be-broken module raised ImportError (in addition to Greg's suggestion that attribute references complain). That scheme has holes too, but it gets increasingly harder to fall into them. The current scheme has so many pits it's proved impossible not to fall into them when using regrtest -r, just doing utterly vanilla non-recursive imports. compelling-my-foot -ly y'rs - tim From tim.one@home.com Tue Dec 18 02:42:51 2001 From: tim.one@home.com (Tim Peters) Date: Mon, 17 Dec 2001 21:42:51 -0500 Subject: [Python-Dev] Unloading modules (regrtest.py mystery) In-Reply-To: <3C1E62E4.C4E9A7A0@lemburg.com> Message-ID: [MAL] > ... > Would be nice if we could come up with a reasonable way of unloading > modules, but I guess this requires some hard PEP work... Yes, I believe it would. There are different kinds of unloading too: a module may well want to do different things depending on whether, e.g., it's really a reload, or it's because Py_Finalize was called ... hmm! That reminds me: Guido sent me this msg in March; it's still sitting in my inbox waiting for a rainy day: """ OK, so we need to define a way for modules to specify a function to be called at cleanup time. I propose that we bring this out in the open, at least in python-dev. Note that there are three or four potential cleanup times: - when the module object is deleted from sys.modules - when reload() is used on an extension - when an interpreter created with PyInterpreter_New() is finalized - when Py_Finalize() is called - when Python exits - when the Python DLL is unloaded (Windows only) """ There. Somebody double the word count and call it a PEP . Note that this is orthogonal to regrtest -r's woes. From mal@lemburg.com Tue Dec 18 09:22:25 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 18 Dec 2001 10:22:25 +0100 Subject: [Python-Dev] Unloading modules (regrtest.py mystery) References: Message-ID: <3C1F0AD1.8D580A4@lemburg.com> Tim Peters wrote: > > [MAL] > > ... > > Would be nice if we could come up with a reasonable way of unloading > > modules, but I guess this requires some hard PEP work... > > Yes, I believe it would. There are different kinds of unloading too: a > module may well want to do different things depending on whether, e.g., it's > really a reload, or it's because Py_Finalize was called ... hmm! That > reminds me: Guido sent me this msg in March; it's still sitting in my inbox > waiting for a rainy day: > > """ > OK, so we need to define a way for modules to specify a function to be > called at cleanup time. I propose that we bring this out in the open, > at least in python-dev. Note that there are three or four potential > cleanup times: > > - when the module object is deleted from sys.modules > > - when reload() is used on an extension > > - when an interpreter created with PyInterpreter_New() is finalized > > - when Py_Finalize() is called I think we already have this one: I usually register a Py_AtExit (or whatever the name is) function with Python to implement this. > - when Python exits > > - when the Python DLL is unloaded (Windows only) Why Windows only ? I believe we could support this on other platforms as well (e.g. Linux). > """ > > There. Somebody double the word count and call it a PEP . Since Barry brought this up, I guess we already have a volunteer ;-) > Note that this is orthogonal to regrtest -r's woes. True and it won't help the suite with the encodings package either (how would you know what to cleanup given the fact that the codec search functions decide how encoding names are mapped to codecs ?). -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From niemeyer@conectiva.com Tue Dec 18 11:50:04 2001 From: niemeyer@conectiva.com (Gustavo Niemeyer) Date: Tue, 18 Dec 2001 09:50:04 -0200 Subject: [Python-Dev] Re: mpz module broken with GMP 4.0 In-Reply-To: <0112151923261C.01556@arthur> References: <200112151623.LAA32158@cj20424-a.reston1.va.home.com> <0112151923261C.01556@arthur> Message-ID: <20011218095004.B1599@ibook.distro.conectiva> --0eh6TmSyL6TZE2Uz Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable > Thanks for the indication -- I was not aware of the new GMP 4.0. A rapid= =20 > test shows the current gmpy.c, release 0.8, breaks in exactly one place= =20 > (structure gmpstate_t used to have a member named seed, now renamed to=20 > _mp_seed). Once that single line is fixed, gmpy 0.8 builds fine (on a=20 Have you commited to the CVS? It'd be nice to have it compiling with both, the new and the old gmp. > Linux box) with the new GMP 4.0 and passes all of its 973 unit-tests (wit= h=20 > Python 2.2b1). I think we should rapidly release a 0.8.1 with the one-li= ne=20 > fix (arranged so it keeps building with GMP 3.1 while also building with= =20 > 4.0, of course), and plan the modest gmpy extensions needed to exploit=20 Ohh.. that's it! Nice.. :-) I'll commit the multiprecision pi code, so the next version already includes it. Thanks!! --=20 Gustavo Niemeyer [ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ] --0eh6TmSyL6TZE2Uz Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE8Hy1sIlOymmZkOgwRAs8dAJ9UgFEYGeLQY/gjz1OMM4FGCNcPFQCg0uuv Uvq90IZdQTWRWUXTJjjgtn0= =KsOJ -----END PGP SIGNATURE----- --0eh6TmSyL6TZE2Uz-- From barry@zope.com Tue Dec 18 14:40:55 2001 From: barry@zope.com (Barry A. Warsaw) Date: Tue, 18 Dec 2001 09:40:55 -0500 Subject: [Python-Dev] Unloading modules (regrtest.py mystery) References: <3C1F0AD1.8D580A4@lemburg.com> Message-ID: <15391.21879.762287.430662@anthem.wooz.org> >>>>> "M" == M writes: >> """ There. Somebody double the word count and call it a PEP >> . M> Since Barry brought this up, I guess we already have a M> volunteer ;-) Sure MAL, I'll assign you a PEP number! -Barry From fdrake@acm.org Tue Dec 18 17:26:36 2001 From: fdrake@acm.org (Fred L. Drake) Date: Tue, 18 Dec 2001 12:26:36 -0500 (EST) Subject: [Python-Dev] [development doc updates] Message-ID: <20011218172636.9E823286BC@cj42289-a.reston1.va.home.com> The development version of the documentation has been updated: http://python.sourceforge.net/devel-docs/ Added documentation for pydoc. Misc. other fixes. From thomas.heller@ion-tof.com Tue Dec 18 21:25:06 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Tue, 18 Dec 2001 22:25:06 +0100 Subject: Need help with bdist_wininst, was Re: [Python-Dev] RE: [Numpy-discussion] Python 2.2 References: <000001c18715$477816c0$0c01a8c0@freedom> <028501c1871a$ec399260$e000a8c0@thomasnotebook> Message-ID: <09bc01c1880a$8f0a4530$e000a8c0@thomasnotebook> I've just fixed the bdist_wininst command in CVS. There is not much time left before the release of Python 2.2 (how much exactly?). To make sure that the current version works correctly, I would kindly request some testers to build windows installers from some of their package distributions, and check them to make sure they work correctly. The full story is in bug [#483982]: Python 2.2b2 bdist_wininst crashes: http://sourceforge.net/tracker/index.php?func=detail&aid=483982&group_id=5470&atid=105470 To test the new version, you only have to replace the file Lib/distutils/command/bdist_wininst.py in your Python 2.2 distribution with the new one from CVS (rev 1.27): http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Lib/distutils/command/bdist_wininst.py Thanks, Thomas From mal@lemburg.com Tue Dec 18 21:47:24 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 18 Dec 2001 22:47:24 +0100 Subject: Need help with bdist_wininst, was Re: [Python-Dev] RE: [Numpy-discussion] Python 2.2 References: <000001c18715$477816c0$0c01a8c0@freedom> <028501c1871a$ec399260$e000a8c0@thomasnotebook> <09bc01c1880a$8f0a4530$e000a8c0@thomasnotebook> Message-ID: <3C1FB96C.E3EE8DCB@lemburg.com> Thomas Heller wrote: > > I've just fixed the bdist_wininst command in CVS. > There is not much time left before the release of Python 2.2 (how much exactly?). > > To make sure that the current version works correctly, I would > kindly request some testers to build windows installers from some > of their package distributions, and check them to make sure they work > correctly. > > The full story is in bug [#483982]: Python 2.2b2 bdist_wininst crashes: > http://sourceforge.net/tracker/index.php?func=detail&aid=483982&group_id=5470&atid=105470 > > To test the new version, you only have to replace the file > Lib/distutils/command/bdist_wininst.py in your Python 2.2 distribution > with the new one from CVS (rev 1.27): > > http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Lib/distutils/command/bdist_wininst.py FYI, I just tested it with egenix-mx-base and egenix-mx-commercial and both seem to work just fine. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From tim@zope.com Tue Dec 18 22:17:22 2001 From: tim@zope.com (Tim Peters) Date: Tue, 18 Dec 2001 17:17:22 -0500 Subject: Need help with bdist_wininst, was Re: [Python-Dev] RE: [Numpy-discussion] Python 2.2 In-Reply-To: <09bc01c1880a$8f0a4530$e000a8c0@thomasnotebook> Message-ID: [Thomas Heller] > I've just fixed the bdist_wininst command in CVS. Thank you! > There is not much time left before the release of Python 2.2 (how > much exactly?). 2.2 final should be released this Friday. From barry@zope.com Tue Dec 18 22:19:03 2001 From: barry@zope.com (Barry A. Warsaw) Date: Tue, 18 Dec 2001 17:19:03 -0500 Subject: Need help with bdist_wininst, was Re: [Python-Dev] RE: [Numpy-discussion] Python 2.2 References: <000001c18715$477816c0$0c01a8c0@freedom> <028501c1871a$ec399260$e000a8c0@thomasnotebook> <09bc01c1880a$8f0a4530$e000a8c0@thomasnotebook> Message-ID: <15391.49367.712590.185830@anthem.wooz.org> >>>>> "TH" == Thomas Heller writes: TH> I've just fixed the bdist_wininst command in CVS. There is TH> not much time left before the release of Python 2.2 (how much TH> exactly?). About 3 days. :) -Barry From paul@pfdubois.com Wed Dec 19 00:06:57 2001 From: paul@pfdubois.com (Paul F. Dubois) Date: Tue, 18 Dec 2001 16:06:57 -0800 Subject: Need help with bdist_wininst, was Re: [Python-Dev] RE: [Numpy-discussion] Python 2.2 In-Reply-To: <09bc01c1880a$8f0a4530$e000a8c0@thomasnotebook> Message-ID: <000001c18821$20455ec0$0c01a8c0@freedom> Our hero! Numeric installs ok now. -----Original Message----- From: numpy-discussion-admin@lists.sourceforge.net [mailto:numpy-discussion-admin@lists.sourceforge.net] On Behalf Of Thomas Heller Sent: Tuesday, December 18, 2001 1:25 PM To: Paul F. Dubois; numpy-discussion@lists.sourceforge.net Cc: python-dev@python.org Subject: Need help with bdist_wininst, was Re: [Python-Dev] RE: [Numpy-discussion] Python 2.2 I've just fixed the bdist_wininst command in CVS. There is not much time left before the release of Python 2.2 (how much exactly?). To make sure that the current version works correctly, I would kindly request some testers to build windows installers from some of their package distributions, and check them to make sure they work correctly. The full story is in bug [#483982]: Python 2.2b2 bdist_wininst crashes: http://sourceforge.net/tracker/index.php?func=detail&aid=483982&group_id =5470&atid=105470 To test the new version, you only have to replace the file Lib/distutils/command/bdist_wininst.py in your Python 2.2 distribution with the new one from CVS (rev 1.27): http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Li b/distutils/command/bdist_wininst.py Thanks, Thomas _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion From mhammond@skippinet.com.au Wed Dec 19 01:57:31 2001 From: mhammond@skippinet.com.au (Mark Hammond) Date: Wed, 19 Dec 2001 12:57:31 +1100 Subject: [Python-Dev] No longer at ActiveState. Message-ID: FYI, I am no longer emplyed by ActiveState (Merry Christmas! :) ActiveState have pretty much stopped all Komodo and Python work for the forseeable future. This left me without an immediate project and therefore without an immediate job. It was very dissapointing to hear this just after slaving to get the 1.2 release of Komodo out the door and just a few days before Christmas. So, please don't use my ActiveState.com email address. Further, I will probably be moving back to being self-employed (in the short term anyway), so if anyone hears of suitable opportunities, then feel free to pass them my way :) On the bright side, it will free my time to finish up some pending Python work I have been meaning to do. Happy holidays, Mark. From neal@metaslash.com Wed Dec 19 02:45:03 2001 From: neal@metaslash.com (Neal Norwitz) Date: Tue, 18 Dec 2001 21:45:03 -0500 Subject: [Python-Dev] test coverage Message-ID: <3C1FFF2F.A69CDCE@metaslash.com> I've been doing a bunch of testing using the regression tests and have tried to improve the test coverage. I've made some progress. Overall, the regression tests do a pretty good job. Below are all the functions that are not currently covered by the regression tests in Objects/ Parser/ and Python/ (Modules/ has too many at this point). This list may not be perfect, but should be pretty close (some tests failed: test_new, test_funcattrs, and test_symtable). I don't know how many of these can be reached from within the python (interpreter), some may only be callable from C code which embeds python. After improving test coverage, I'll go back and run purify to see if there are any new memory leaks. Sorry, the list is kinda long. Let me know if there's a better way to distribute this info. Neal -- Objects/abstract.c PyObject_CallMethodObjArgs PyObject_CallFunctionObjArgs PyObject_Type PyObject_Length PyObject_AsWriteBuffer PyNumber_InPlaceTrueDivide PySequence_Length PySequence_In PySequence_InPlaceConcat PySequence_InPlaceRepeat PyMapping_Length PyMapping_HasKeyString PyMapping_HasKey PyMapping_GetItemString PyMapping_SetItemString null_error sliceobj_from_intint objargs_mktuple Objects/bufferobject.c PyBuffer_FromReadWriteObject PyBuffer_FromReadWriteMemory PyBuffer_New buffer_compare buffer_getwritebuf Objects/cellobject.c cell_compare cell_repr Objects/classobject.c instance_ass_item PyMethod_Function PyMethod_Self PyMethod_Class set_slot set_attr_slots set_dict set_bases set_name instancemethod_hash Objects/cobject.c PyCObject_FromVoidPtrAndDesc PyCObject_GetDesc PyCObject_Import Objects/complexobject.c PyComplex_FromDoubles complex_print complex_div Objects/descrobject.c member_repr getset_repr wrapper_repr method_get_doc wrapper_get_doc proxy_len proxy_has_key proxy_get proxy_values proxy_items proxy_copy proxy_getiter proxy_str proxy_traverse wrapper_name Objects/dictobject.c PyDict_Values PyDict_Items dictiter_next Objects/fileobject.c PyFile_FromString PyFile_Name Objects/floatobject.c PyFloat_AsString PyFloat_AsStringEx float_print Objects/funcobject.c PyFunction_GetCode PyFunction_GetGlobals PyFunction_GetDefaults PyFunction_GetClosure PyClassMethod_New func_set_code func_set_defaults Objects/iterobject.c iter_traverse iter_next calliter_traverse calliter_next Objects/listobject.c list_print immutable_list_op immutable_list_ass Objects/longobject.c PyLong_AsVoidPtr Objects/methodobject.c listmethodchain PyCFunction_GetFunction PyCFunction_GetSelf PyCFunction_GetFlags meth_traverse meth_get__self__ meth_compare Objects/object.c _PyObject_Dump PyObject_Not PyObject_Malloc PyObject_Realloc PyObject_Free none_dealloc NotImplemented_repr Objects/rangeobject.c range_compare range_slice range_tolist range_getattr Objects/sliceobject.c PySlice_GetIndices slice_compare Objects/stringobject.c _Py_ReleaseInternedStrings PyString_Decode PyString_Encode PyString_AsEncodedString PyString_AsDecodedString string_getbuffer string_buffer_getwritebuf Objects/structseq.c structseq_repr structseq_concat structseq_repeat structseq_contains structseq_hash Objects/tupleobject.c tupleprint Objects/typeobject.c slot_nb_coerce type_set_module type_subclasses wrap_binaryfunc_r wrap_coercefunc wrap_ternaryfunc_r wrap_sq_delitem wrap_delslice wrap_delitem wrap_hashfunc slot_sq_length slot_sq_ass_item slot_tp_iter slot_tp_iternext super_repr Objects/unicodeobject.c PyUnicodeUCS2_Replace PyUnicodeUCS2_AsUnicode PyUnicodeUCS2_GetSize PyUnicodeUCS2_FromWideChar PyUnicodeUCS2_AsWideChar PyUnicodeUCS2_SetDefaultEncoding PyUnicodeUCS2_Encode PyUnicodeUCS2_AsUTF16String PyUnicodeUCS2_AsUnicodeEscapeString PyUnicodeUCS2_AsRawUnicodeEscapeString PyUnicodeUCS2_AsCharmapString PyUnicodeUCS2_Translate PyUnicodeUCS2_Tailmatch PyUnicodeUCS2_Find utf8_decoding_error utf16_decoding_error latin1_encoding_error charmap_encoding_error unicode_index unicode_rindex unicode_buffer_getreadbuf unicode_buffer_getwritebuf Objects/weakrefobject.c PyWeakref_GetObject gc_clear weakref_repr proxy_print proxy_repr proxy_compare proxy_nonzero proxy_slice proxy_ass_slice proxy_contains proxy_length proxy_setitem Parser/bitset.c _Py_delbitset Parser/grammar1.c PyGrammar_LabelRepr Parser/listnode.c list1node PyNode_ListTree listnode Parser/myreadline.c PyOS_StdioReadline PyOS_Readline my_fgets Parser/parsetok.c PyParser_ParseString Parser/pgen.c dumpstate dumpnfa printssdfa Parser/pgenmain.c PySys_WriteStderr Py_FatalError PyOS_Readline Parser/tokenizer.c indenterror Python/ceval.c call_exc_trace PyEval_CallObject Py_SetRecursionLimit Py_GetRecursionLimit PyEval_AcquireLock PyEval_ReleaseThread loop_subscript gen_traverse gen_next Python/codecs.c PyCodec_StreamReader PyCodec_StreamWriter build_stream_codec Python/compile.c PyCode_Addr2Line PyNode_CompileSymtable code_repr do_pad dump Python/errors.c PyErr_WriteUnraisable PyErr_BadArgument PyErr_NoMemory PyErr_BadInternalCall _PyErr_BadInternalCall Python/frozenmain.c Py_FrozenMain Python/getargs.c PyArg_VaParse Python/import.c PyImport_ExtendInittab PyImport_ExecCodeModule PyImport_AppendInittab get_frozen_object imp_init_builtin imp_init_frozen imp_get_frozen_object imp_is_builtin imp_is_frozen imp_load_compiled imp_load_dynamic imp_load_module imp_load_package Python/marshal.c PyMarshal_ReadShortFromFile PyMarshal_ReadObjectFromFile w_more marshal_dump marshal_load Python/modsupport.c PyEval_CallFunction _ustrlen Python/pyfpe.c PyFPE_dummy Python/pystate.c PyInterpreterState_Head PyInterpreterState_Next PyInterpreterState_ThreadHead PyThreadState_Next Python/pythonrun.c Py_NewInterpreter PyRun_InteractiveLoopFlags PyRun_InteractiveOneFlags parse_syntax_error print_error_text PyErr_Display Py_FatalError Py_SetPythonHome Py_EndInterpreter PyRun_AnyFile PyRun_AnyFileEx PyRun_AnyFileFlags PyRun_SimpleString PyRun_SimpleStringFlags PyRun_SimpleFile PyRun_SimpleFileEx PyRun_InteractiveOne PyRun_InteractiveLoop PyRun_File PyRun_FileFlags Py_CompileString Py_SymtableString PyErr_Print Py_AtExit run_pyc_file add_flag Python/structmember.c PyMember_Set listmembers Python/symtable.c ste_repr Python/sysmodule.c mywrite PySys_WriteStdout PySys_WriteStderr PySys_GetFile PySys_AddWarnOption sys_excepthook sys_getdefaultencoding sys_setdefaultencoding sys_setcheckinterval sys_setrecursionlimit sys_getrecursionlimit sys_setdlopenflags sys_getdlopenflags Python/thread.c PyThread_allocate_sema PyThread_free_sema PyThread_down_sema PyThread_up_sema PyThread__exit_thread Python/traceback.c tb_displayline PyTraceBack_Print tb_clear tb_printinternal From guido@python.org Wed Dec 19 02:51:25 2001 From: guido@python.org (Guido van Rossum) Date: Tue, 18 Dec 2001 21:51:25 -0500 Subject: [Python-Dev] No longer at ActiveState. In-Reply-To: Your message of "Wed, 19 Dec 2001 12:57:31 +1100." References: Message-ID: <200112190251.VAA23676@cj20424-a.reston1.va.home.com> > FYI, I am no longer emplyed by ActiveState (Merry Christmas! :) Au, that hurts. :( > ActiveState have pretty much stopped all Komodo and Python work for > the forseeable future. This left me without an immediate project > and therefore without an immediate job. It was very dissapointing > to hear this just after slaving to get the 1.2 release of Komodo out > the door and just a few days before Christmas. I'll say. I'm guessing we might as well drop ActiveState from the list of PSF sponsors now. > So, please don't use my ActiveState.com email address. Further, I > will probably be moving back to being self-employed (in the short > term anyway), so if anyone hears of suitable opportunities, then > feel free to pass them my way :) Hmm... The economy is going through a rough time on this side of the pond / border as well. :( > On the bright side, it will free my time to finish up some pending > Python work I have been meaning to do. Yay! Will we see you at the next (last) Python conference? --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz@rahul.net Wed Dec 19 03:38:52 2001 From: aahz@rahul.net (Aahz Maruch) Date: Tue, 18 Dec 2001 19:38:52 -0800 (PST) Subject: [Python-Dev] IPC10 In-Reply-To: <200112190251.VAA23676@cj20424-a.reston1.va.home.com> from "Guido van Rossum" at Dec 18, 2001 09:51:25 PM Message-ID: <20011219033853.67370E8C3@waltz.rahul.net> Guido van Rossum wrote: > > Will we see you at the next (last) Python conference? Am I misreading this, or is IPC10 going to be the final Python conference? (I'm still way behind on c.l.py, but I figure something like this ought to have been posted to c.l.py.announce if true.) -- --- Aahz (@pobox.com) Hugs and backrubs -- I break Rule 6 <*> http://www.rahul.net/aahz/ Androgynous poly kinky vanilla queer het Pythonista We must not let the evil of a few trample the freedoms of the many. From tim.one@home.com Wed Dec 19 06:13:44 2001 From: tim.one@home.com (Tim Peters) Date: Wed, 19 Dec 2001 01:13:44 -0500 Subject: [Python-Dev] No longer at ActiveState. In-Reply-To: Message-ID: [Mark Hammond] > FYI, I am no longer emplyed by ActiveState (Merry Christmas! :) Indeed! So you'll be using the PythonLabs installer again? Pay close attention to its "Installation Finished" screen -- I changed that months ago in anticipation of your coming home to us . > ActiveState have pretty much stopped all Komodo and Python work for the > forseeable future. This left me without an immediate project and > therefore without an immediate job. Bummer. It's times like this I'm glad Aussies don't have feelings. > It was very dissapointing to hear this just after slaving to get the 1.2 > release of Komodo out the door and just a few days before Christmas. Yup. If it's any consolation, BeOpen.com stopped paying us the week before Python 2.0 was scheduled for release. We all wanted to go on a month-long bender, but Guido didn't let us leave his office until the release was finished. See? You *could* have been working for Guido, and *then* what a living hell your life would have been ... > So, please don't use my ActiveState.com email address. Further, I will > probably be moving back to being self-employed (in the short term anyway), > so if anyone hears of suitable opportunities, then feel free to > pass them my way :) Will do. > On the bright side, it will free my time to finish up some pending Python > work I have been meaning to do. If you find a way to move Armin Rigo's Psyco along while getting paid for it, I'll join you. Else I'll just reassign every bug and patch with the string "win" in it to you -- heh heh. Have a relaxed holiday, Mark! We'll still be here when you get out of detox. or-at-least-guido-will-be-and-that's-enough-ly y'rs - tim From tim.one@home.com Wed Dec 19 06:20:09 2001 From: tim.one@home.com (Tim Peters) Date: Wed, 19 Dec 2001 01:20:09 -0500 Subject: [Python-Dev] IPC10 In-Reply-To: <20011219033853.67370E8C3@waltz.rahul.net> Message-ID: [Guido] > Will we see you at the next (last) Python conference? [Aahz] > Am I misreading this, or is IPC10 going to be the final Python > conference? I expect Guido was confusing tenses due to time machine travel to fix the worst 2.2 bugs before they got reported -- they were (or would have been -- or will be had they not been fixed beforehand in the past now) really embarrassing. Whether there will be another Python conference is always a matter for keen speculation. The organizers claim to lose money on it, despite the craters it leaves in attendees' wallets. Ask again about a year from now ... From thomas.heller@ion-tof.com Wed Dec 19 07:48:23 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Wed, 19 Dec 2001 08:48:23 +0100 Subject: Need help with bdist_wininst, was Re: [Python-Dev] RE: [Numpy-discussion] Python 2.2 References: <000001c18715$477816c0$0c01a8c0@freedom> <028501c1871a$ec399260$e000a8c0@thomasnotebook> <09bc01c1880a$8f0a4530$e000a8c0@thomasnotebook> Message-ID: <0c8101c18861$a101a790$e000a8c0@thomasnotebook> > I've just fixed the bdist_wininst command in CVS. > There is not much time left before the release of Python 2.2 (how much exactly?). > > To make sure that the current version works correctly, I would > kindly request some testers to build windows installers from some > of their package distributions, and check them to make sure they work > correctly. Thanks for the tests, Paul and Marc-Andre. Thomas From fdrake@acm.org Wed Dec 19 16:52:38 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 19 Dec 2001 11:52:38 -0500 (EST) Subject: [Python-Dev] test coverage In-Reply-To: <3C1FFF2F.A69CDCE@metaslash.com> References: <3C1FFF2F.A69CDCE@metaslash.com> Message-ID: <15392.50646.688242.432369@cj42289-a.reston1.va.home.com> Neal Norwitz writes: > I don't know how many of these can be reached from within > the python (interpreter), some may only be callable from C code > which embeds python. I've added a few tests for the weakref support that should at least improve the coverage there. I don't think I'll have any more time to help in this effort before 2.2 goes out, but I think this is really important. > After improving test coverage, I'll go back and run purify > to see if there are any new memory leaks. > > Sorry, the list is kinda long. Let me know if there's a better way > to distribute this info. I can live with this distribution method; if it gets us to actually improve coverage, we all get to see the benefit of a shorter list. ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From mhammond@skippinet.com.au Thu Dec 20 06:06:04 2001 From: mhammond@skippinet.com.au (Mark Hammond) Date: Thu, 20 Dec 2001 17:06:04 +1100 Subject: [Python-Dev] No longer at ActiveState. In-Reply-To: <200112190251.VAA23676@cj20424-a.reston1.va.home.com> Message-ID: > > ActiveState have pretty much stopped all Komodo and Python work for > > the forseeable future. This left me without an immediate project > > and therefore without an immediate job. It was very dissapointing > > to hear this just after slaving to get the 1.2 release of Komodo out > > the door and just a few days before Christmas. > > I'll say. I'm guessing we might as well drop ActiveState from the > list of PSF sponsors now. I obviously can't speak for ActiveState ;-) I really have no idea exactly what "pretty much stopped ... Python work" means in reality, nor what the longer term plans are. > > On the bright side, it will free my time to finish up some pending > > Python work I have been meaning to do. > > Yay! Indeed :) > Will we see you at the next (last) Python conference? Unfortunately not :( It would be irresponsible of me to commit to that much money without knowing where my next dollar was coming from :) And if you had ever met my girlfriend, you would know that I am very aware exactly where my next dollar is *going* :) Mark. From barry@zope.com Thu Dec 20 06:18:02 2001 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 20 Dec 2001 01:18:02 -0500 Subject: [Python-Dev] No longer at ActiveState. References: <200112190251.VAA23676@cj20424-a.reston1.va.home.com> Message-ID: <15393.33434.171378.132917@anthem.wooz.org> >>>>> "MH" == Mark Hammond writes: MH> Unfortunately not :( It would be irresponsible of me to commit MH> to that much money without knowing where my next dollar was MH> coming from :) And if you had ever met my girlfriend, you MH> would know that I am very aware exactly where my next dollar MH> is *going* :) LOL, COL. We'll raise a pint for ya Mark. purple-ly y'rs, -Barry From barry@zope.com Thu Dec 20 19:28:32 2001 From: barry@zope.com (Barry A. Warsaw) Date: Thu, 20 Dec 2001 14:28:32 -0500 Subject: [Python-Dev] Python 2.2 final - release branch created Message-ID: <15394.15328.321075.820271@anthem.wooz.org> I've created the Python 2.2 final release branch. The CVS tag is different than with the alpha and beta release branches. Here, it is called `release22-branch'. There's also a tag `release22-fork' -- that's the point on the trunk where the branch forks off. So far the only change in the branch is the version number. As usual, no one except Guido and myself (or our authorized bots) should make checkins to the branch. Checkins to the trunk and branch are discouraged unless they fix showstopper bugs in Python 2.2. Nothing will be merged from the trunk to the branch unless you notify us, e.g. with a comment in the checkin message. I'm planning to cut the release tomorrow morning local time, Friday December 21. Enjoy, -Barry From skip@pobox.com (Skip Montanaro) Thu Dec 20 21:02:01 2001 From: skip@pobox.com (Skip Montanaro) (Skip Montanaro) Date: Thu, 20 Dec 2001 15:02:01 -0600 Subject: [Python-Dev] Can we checkin on the main trunk? Message-ID: <15394.20937.842175.101270@beluga.mojam.com> I have a trivial change for asynchat to shut up pychecker: /usr/local/lib/python2.2/asynchat.py:83: Local variable (why) not used /usr/local/lib/python2.2/asynchat.py:217: Local variable (why) not used (delete the "why" variable in both cases). Can I check in on the main trunk and have it go into 2.3 without disturbing the 2.2 release? I'm never sure during these alpha/beta/release candidate situations what is and isn't allowed (other than that Barry's new branch is clearly off-limits). This is such a trivial change that it seems like overkill to me to submit a bug report or a patch, have it assigned to someone who two weeks later says, "sure, looks fine to me - check it in". (Hint: For those of us with rotting neurons, whenever development enters a release phase such as we are in right now, it would be helpful to remind us what we can do as well as what we can't.) Skip From jack@oratrix.nl Thu Dec 20 23:20:48 2001 From: jack@oratrix.nl (Jack Jansen) Date: Fri, 21 Dec 2001 00:20:48 +0100 Subject: [Python-Dev] test_cpickle failing in MacPython - ideas needed Message-ID: <20011220232048.430FDE8453@oratrix.oratrix.nl> Folks, test_cpickle suddenly started failing (often with a hang) in MacPython (CVS tree, main branch). As I'm going to be offline all day tomorrow and I'd like to get MacPython out the door simultaneously with unix and windows-Python: does anyone know what could have changed over the last couple of days to cause this? -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.cwi.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++ From guido@python.org Thu Dec 20 23:19:42 2001 From: guido@python.org (Guido van Rossum) Date: Thu, 20 Dec 2001 18:19:42 -0500 Subject: [Python-Dev] Can we checkin on the main trunk? In-Reply-To: Your message of "Thu, 20 Dec 2001 15:02:01 CST." <15394.20937.842175.101270@beluga.mojam.com> References: <15394.20937.842175.101270@beluga.mojam.com> Message-ID: <200112202319.fBKNJjR24505@odiug.zope.com> > I have a trivial change for asynchat to shut up pychecker: > > /usr/local/lib/python2.2/asynchat.py:83: Local variable (why) not used > /usr/local/lib/python2.2/asynchat.py:217: Local variable (why) not used > > (delete the "why" variable in both cases). > > Can I check in on the main trunk and have it go into 2.3 without disturbing > the 2.2 release? Yes, the release branch has been made. The trunk is all yours. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim@zope.com Fri Dec 21 00:19:22 2001 From: tim@zope.com (Tim Peters) Date: Thu, 20 Dec 2001 19:19:22 -0500 Subject: [Python-Dev] test_cpickle failing in MacPython - ideas needed In-Reply-To: <20011220232048.430FDE8453@oratrix.oratrix.nl> Message-ID: [Jack Jansen] > test_cpickle suddenly started failing (often with a hang) in MacPython > (CVS tree, main branch). Ouch! > As I'm going to be offline all day tomorrow and I'd like to get MacPython > out the door simultaneously with unix and windows-Python: does anyone > know what could have changed over the last couple of days to cause this? Two very recent changes: 1. Low probability. Guido changed the way pickle works for instances of new-style classes with custom metatypes. 2. High probability. Since 2001/10/15, test_pickle and test_cpickle actually didn't run any tests at all. Guido discovered this when working on #1 and not seeing an expected error. When they were converted to unittests, an inappropriate piece of if __name__ == '__main__':' boilerplate prevented them from running any tests unless run directly (when run via regrtest.py, they didn't do anything). Fred fixed that about 24 hours ago. Unfortunately, #2 means "the real" problem may have been introduced anytime within the last 2 months. Please try to whittle the failure down -- I'll be happy to help, but there are no failures here. It would also help to know once again *exactly* how you run the test suite (previous problems unique to you have been pinned on something unique about the way you run tests, no matter how much you protest in advance that it isn't possible ). From tim.one@home.com Fri Dec 21 02:10:47 2001 From: tim.one@home.com (Tim Peters) Date: Thu, 20 Dec 2001 21:10:47 -0500 Subject: [Python-Dev] Can we checkin on the main trunk? In-Reply-To: <15394.20937.842175.101270@beluga.mojam.com> Message-ID: [Skip Montanaro] > ... > (Hint: For those of us with rotting neurons, whenever development enters a > release phase such as we are in right now, it would be helpful to remind us > what we can do as well as what we can't.) The things you can do are those you don't get publicly humiliated for doing. Sure, it's a two-pass algorithm, and even then it makes the set of things you can do at best recursively enumerable (there's no bound on how long it may take before you're attacked, so you can never know for sure that you're safe, only that if you're damned you'll *eventually* find out) -- but in the absence of a PEP, that's the best we can do for you . To a good first approximation, don't check in anything on the release *branch* without Barry's explicit advance blessing, and I expect you'll live to be appreciated in 2002. From tim.one@home.com Fri Dec 21 02:48:11 2001 From: tim.one@home.com (Tim Peters) Date: Thu, 20 Dec 2001 21:48:11 -0500 Subject: [Python-Dev] For Jack Jansen Message-ID: Mail to you from me bounces, Jack: Reporting-MTA: dns; femail4.home.com Arrival-Date: Thu, 20 Dec 2001 18:43:54 -0800 Received-From-MTA: dns; CJ569191B (65.1.136.53) Final-Recipient: RFC822; Action: failed Status: 5.1.1 Remote-MTA: dns; hera.cwi.nl (192.16.191.8) Diagnostic-Code: smtp; 550 ... Blocked relay femail4.sdc1.sfba.home.com I'm not sure CWI wants to cut off the 4 million remaining @Home customers via one dumb spam trick -- although I'm sure it's an effective one . From akuchlin@mems-exchange.org Fri Dec 21 04:53:04 2001 From: akuchlin@mems-exchange.org (akuchlin@mems-exchange.org) Date: Thu, 20 Dec 2001 23:53:04 -0500 Subject: [Python-Dev] 2.2 "What's New" article finished Message-ID: <20011220235304.A18909@mozart.mems-exchange.org> I've made a few last updates and called it complete for 2.2final. (Bug reports and suggestions are still welcome, though.) Note for the PythonLabs gang: if you want to make a copy on python.org or zope.com (for press release purposes, consistent page design, or whatever), feel free to do so; the LaTeX source is in the pyhowto CVS tree at sourceforge.net. Oh, yeah, the URL: still http://www.amk.ca/python/2.2/ . --amk (www.amk.ca) The Sony Walkman will become obsolete thanks to the Sony rnman. The instructions will be on a separate device, the Sony manman. -- Kibo, in the Happynet Manifesto From anthony@interlink.com.au Fri Dec 21 05:57:14 2001 From: anthony@interlink.com.au (Anthony Baxter) Date: Fri, 21 Dec 2001 16:57:14 +1100 Subject: [Python-Dev] 2.1.2 status. Message-ID: <200112210557.fBL5vE908871@mbuna.arbhome.com.au> I'm churning through the list of "things that I want to get fixed" for 2.1.2. I've had to abandon my initial plan of examining pretty much all patches for suitability - the backlog since 2.1.1 is just too huge. If there's something you know of that you think really should be in there, and you're not sure if it is actually there yet, please let me know ASAP! Anthony. From barry@zope.com Fri Dec 21 06:05:23 2001 From: barry@zope.com (Barry A. Warsaw) Date: Fri, 21 Dec 2001 01:05:23 -0500 Subject: [Python-Dev] Can we checkin on the main trunk? References: <15394.20937.842175.101270@beluga.mojam.com> Message-ID: <15394.53539.912400.804116@anthem.wooz.org> >>>>> "SM" == Skip Montanaro writes: SM> Can I check in on the main trunk and have it go into 2.3 SM> without disturbing the 2.2 release? I'm never sure during SM> these alpha/beta/release candidate situations what is and SM> isn't allowed (other than that Barry's new branch is clearly SM> off-limits). Trunk checkins are always safe. We create a release branch specifically so that no strict freeze on checkins need be imposed on CVS committers. The release branch is always off-limits to everyone except those of us who are making the releases (or our designated bots), as per PEP 101. This way we can ensure a clean, controlled release branch but not hold up ongoing Python development and maintenance on the trunk. BTW, this policy is fairly new with Python 2.2 and, speaking as Release Manager, I think it has worked exceedingly well. We actually got the idea from the way Zope's CVS is managed, though we've tweaked the process a bit to suite Python's differing needs. aIMO, the most important adjustment was in finding the right balance between the need for a branch and the reality that the inevitable merge from branch back to trunk can be a royal PITA (ask Tim some time about is descr-branch merge experience. ;) Currently we try to branch at about noon (local time 'natch) the day before the release. This seems like a pretty good balance between the conflicting goals of release isolation and a pain-free trunk-merge. I know my announcement earlier today discouraged trunk checkins until we spin the Python 2.2 final release. I think that makes sense given that this is a "final" release, so there are a few special considerations. I'm willing to accept that my discouraging trunk checkins is just paranoia on my part ;). -Barry From davida@ActiveState.com Fri Dec 21 18:06:25 2001 From: davida@ActiveState.com (David Ascher) Date: Fri, 21 Dec 2001 10:06:25 -0800 (PST) Subject: [Python-Dev] ActiveState & Python Message-ID: As Mark mentioned in his email to python-dev, he is no longer employed by ActiveState. While I'm dealing with my grief about that unfortunate event in private, I think it's worth clarifying a few things in public: 1) Over the last two years, Mark has become my personal god of programming. ActiveState as a whole has benefited tremendously from having Mark on staff, I have learned a tremendous amount from him, and the entire ActiveState team hope to be able to bring him back on board as soon as possible. 2) The reason for Mark's departure is simple -- ActiveState has had to make some painful cuts in staff in order to ensure our continued survival in these difficult economic times. We hope these cuts will be short-lived and that we will be able to bring back the excellent personel that had to be let go soon. 3) ActiveState is still committed to supporting its Python initiatives and current product line. For example, a release of ActivePython based on Python 2.2 is forthcoming. As regards Komodo, we do have plans for future releases. The features we are looking at adding in future releases include: * Integrated version control support (right now it's done through the "external command" interface) * Perl autocompletion (not sure that python-dev'ers care about that one =) * Improved web services consumption (1.2 has support for this already) * Macro support * Key-mappings (like emacs and vi key modes) As usual, we look forward to feedback from customers as to what the next version of Komodo should look like. If you have any questions about any of the above, feel free to contact me. I'm on vacation but am reading email periodically. --David Ascher Friend of Mark's Director, Programming Tools ActiveState From guido@python.org Fri Dec 21 19:54:36 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 21 Dec 2001 14:54:36 -0500 Subject: [Python-Dev] RELEASED - Python 2.2 final Message-ID: <200112211954.OAA16972@cj20424-a.reston1.va.home.com> On December 21, just in time to be placed under the Christmas tree, we're issuing the final release of Python 2.2. We're proud of this release, and expect that you'll like it. Please check it out here: http://www.python.org/2.2/ Our thanks to everyone who helped test Python 2.2 during the alpha, beta and release candidate phases, and to everyone who contributed patches, feedback, and suggestions! Please continue to report any bugs you find to the bug tracker: http://sourceforge.net/bugs/?group_id=5470 Highlights of what's new for this release are outlined below. For a more complete list, please see: http://sf.net/project/shownotes.php?release_id=?????? What's new since 2.1 -------------------- In brief (for PEPs, see http://www.python.org/peps/): - type/class unification (PEP 252 and 253) - iterators (PEP 234) and generators (PEP 255) - nested scopes standard (PEP 227) - int overflows return longs (PEP 237) - new operator // to future-proof int division (PEP 238) Andrew Kuchling has written a gentle introduction to the most important changes, titled "What's New in Python 2.2": http://www.amk.ca/python/2.2/ A thorough introduction to the type/class unification is at: http://www.python.org/2.2/descrintro.html Status of the type/class unification ------------------------------------ With a set of changes as large as this, it's unavoidable that after the release we'll find that there are certain things that would work better if we did them a little differently. Also, it is clear that some of the new features (e.g. class and static methods, properties, super, and slots, to mention a few) could use additional syntactic sugar to make their use easier and more intuitive. This will almost certainly require us to make changes in future releases. However, with the release of these features as part of the Python 2.2 release, we are committed to the same standard of release-to-release compatibility that we have used in the past, as explained in PEP 5. When we feel the need to change things, we'll use the PEP system to propose changes, and we'll provide warnings and backwards compatiblity for at least a year -- or more in cases where a year is deemed too short by the user community. In the meantime, we'll work on completing the documentation for these features (both in Python and in the C API) as they are found in Python 2.2. This documentation will clarify the status of individual features. What's new since 2.2c1 ---------------------- (Adapted from the Misc/NEWS file.) Type/class unification and new-style classes - pickle.py, cPickle: allow pickling instances of new-style classes with a custom metaclass. Core and builtins - weakref proxy object: when comparing, unwrap both arguments if both are proxies. Extension modules - binascii.b2a_base64(): fix a potential buffer overrun when encoding very short strings. - cPickle: the obscure "fast" mode caused stack overflows on the Mac. We fixed this by setting the recursion limit much smaller. If the limit is too low (it only affects performance), you can change it by defining PY_CPICKLE_FAST_LIMIT when compiling cPickle.c (or in pyconfig.h). Library - dumbdbm.py: fixed a dumb old bug (the file didn't get synched at close or delete time). - rfc822.py: fixed a bug where the address '<>' was converted to None instead of an empty string (also fixes the email.Utils module). - xmlrpclib.py: version 1.0.0; uses precision for doubles. - test suite: the pickle and cPickle tests were not executing any code when run from the standard regresssion test. Windows - distutils package: fixed broken Windows installers (bdist_wininst). - tempfile.py: prevent mysterious warnings when TemporaryFileWrapper instances are deleted at process exit time. - socket.py: prevent mysterious warnings when socket instances are deleted at process exit time. - posixmodule.c: fix a Windows crash with stat() of a filename ending in backslash. Mac - The Carbon toolbox modules have been upgraded to Universal Headers 3.4, and experimental CoreGraphics and CarbonEvents modules have been added. All only for framework-enabled MacOSX. --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik@pythonware.com Fri Dec 21 20:24:11 2001 From: fredrik@pythonware.com (Fredrik Lundh) Date: Fri, 21 Dec 2001 21:24:11 +0100 Subject: [Python-Dev] RELEASED - Python 2.2 final References: <200112211954.OAA16972@cj20424-a.reston1.va.home.com> Message-ID: <001401c18a5d$746ebad0$ced241d5@hagrid> > On December 21, just in time to be placed under the Christmas tree, > we're issuing the final release of Python 2.2. hurray for us! see y'all after the holidays. From fdrake@acm.org Fri Dec 21 21:12:12 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 21 Dec 2001 16:12:12 -0500 (EST) Subject: [Python-Dev] Python 2.2.1 Message-ID: <15395.42412.811106.303578@cj42289-a.reston1.va.home.com> I've created the branch release22-maint as the Python 2.2 maintenance branch. We don't yet have a Patch Czar for 2.2.1, though. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From mwh@python.net Fri Dec 21 21:27:43 2001 From: mwh@python.net (Michael Hudson) Date: 21 Dec 2001 21:27:43 +0000 Subject: [Python-Dev] RELEASED - Python 2.2 final In-Reply-To: Guido van Rossum's message of "Fri, 21 Dec 2001 14:54:36 -0500" References: <200112211954.OAA16972@cj20424-a.reston1.va.home.com> Message-ID: <2m8zbwjmb4.fsf@starship.python.net> Guido van Rossum writes: > On December 21, just in time to be placed under the Christmas tree, > we're issuing the final release of Python 2.2. Hurrah! > http://sf.net/project/shownotes.php?release_id=?????? Oops. Cheers, M. -- ARTHUR: Why should a rock hum? FORD: Maybe it feels good about being a rock. -- The Hitch-Hikers Guide to the Galaxy, Episode 8 From guido@python.org Fri Dec 21 21:33:15 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 21 Dec 2001 16:33:15 -0500 Subject: [Python-Dev] RELEASED - Python 2.2 final In-Reply-To: Your message of "21 Dec 2001 21:27:43 GMT." <2m8zbwjmb4.fsf@starship.python.net> References: <200112211954.OAA16972@cj20424-a.reston1.va.home.com> <2m8zbwjmb4.fsf@starship.python.net> Message-ID: <200112212133.QAA18024@cj20424-a.reston1.va.home.com> > > http://sf.net/project/shownotes.php?release_id=?????? Oops, sorry. The correct URL is: http://sourceforge.net/project/shownotes.php?release_id=66610 --Guido van Rossum (home page: http://www.python.org/~guido/) From anthony@interlink.com.au Sun Dec 23 06:16:28 2001 From: anthony@interlink.com.au (Anthony Baxter) Date: Sun, 23 Dec 2001 17:16:28 +1100 Subject: [Python-Dev] 2.1.2rc is ready to go. Message-ID: <200112230616.fBN6GSS07067@mbuna.arbhome.com.au> the 2.1.2 is ready for release candidate - I've yet to go through the hoops to cut the rc1, but please please feel free to pull the CVS version and try it out. Anthony. From guido@python.org Sun Dec 23 14:01:48 2001 From: guido@python.org (Guido van Rossum) Date: Sun, 23 Dec 2001 09:01:48 -0500 Subject: [Python-Dev] 2.1.2rc is ready to go. In-Reply-To: Your message of "Sun, 23 Dec 2001 17:16:28 +1100." <200112230616.fBN6GSS07067@mbuna.arbhome.com.au> References: <200112230616.fBN6GSS07067@mbuna.arbhome.com.au> Message-ID: <200112231401.JAA08934@cj20424-a.reston1.va.home.com> > the 2.1.2 is ready for release candidate - I've yet to go through the hoops > to cut the rc1, but please please feel free to pull the CVS version and try > it out. Excellent! When do you plan to release it? What kind of help do you want from us? We'd be happy to follow PEP 101 but it might have to wait until 2002. --Guido van Rossum (home page: http://www.python.org/~guido/) From mwh@python.net Sun Dec 23 18:08:44 2001 From: mwh@python.net (Michael Hudson) Date: 23 Dec 2001 18:08:44 +0000 Subject: [Python-Dev] Python 2.2.1 In-Reply-To: "Fred L. Drake, Jr."'s message of "Fri, 21 Dec 2001 16:12:12 -0500 (EST)" References: <15395.42412.811106.303578@cj42289-a.reston1.va.home.com> Message-ID: <2mzo49yfkj.fsf@starship.python.net> "Fred L. Drake, Jr." writes: > I've created the branch release22-maint as the Python 2.2 > maintenance branch. > We don't yet have a Patch Czar for 2.2.1, though. I've just spent half an hour fiddling with gnus to the point that it should be easy for me to keep a log of checkins that should be put on the release22-maint branch. This isn't me volunteering to be Patch Czar, by the way... I don't know if I'll have time for that, but I hope that doing this will help whoever it ends up being. Cheers, M. -- About the use of language: it is impossible to sharpen a pencil with a blunt axe. It is equally vain to try to do it with ten blunt axes instead. -- E.W.Dijkstra, 18th June 1975. Perl did not exist at the time. From jack@oratrix.nl Sun Dec 23 22:22:04 2001 From: jack@oratrix.nl (Jack Jansen) Date: Sun, 23 Dec 2001 23:22:04 +0100 Subject: [Python-Dev] Python 2.2.1 In-Reply-To: Message by Michael Hudson , 23 Dec 2001 18:08:44 +0000 , <2mzo49yfkj.fsf@starship.python.net> Message-ID: <20011223222209.C9CE5E8453@oratrix.oratrix.nl> Two questions wrt 2.2.1: - Is there a timeline for it? I couldn't find one in PEP251 (which stops at 2.2 final). The reason I ask is that there's a serious flaw in MacPython (has been there since day one) that renders it useless on multiprocessor Mac OS X machines. I'm waiting for an external party to fix this (as the problem is in an external library), but I could put a bit of pressure on if 2.2.1 is due reasonably soon (as I hope). - The final bits of the Mac tree havne't been checked in yet, should we move the release22-maint branch when that's done? Can you move a branch? How? -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.cwi.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From Anthony Baxter Sun Dec 23 22:25:02 2001 From: Anthony Baxter (Anthony Baxter) Date: Mon, 24 Dec 2001 09:25:02 +1100 Subject: [Python-Dev] Python 2.2.1 In-Reply-To: Message from Michael Hudson of "23 Dec 2001 18:08:44 -0000." <2mzo49yfkj.fsf@starship.python.net> Message-ID: <200112232225.fBNMP2910673@mbuna.arbhome.com.au> >>> Michael Hudson wrote > I've just spent half an hour fiddling with gnus to the point that it > should be easy for me to keep a log of checkins that should be put on > the release22-maint branch. > > This isn't me volunteering to be Patch Czar, by the way... I don't > know if I'll have time for that, but I hope that doing this will help > whoever it ends up being. I'm happy enough to continue doing this - now that it's starting from a caught-up position, it's not that hard. The problem with 2.1.2 was trawling back through 6 months of checkins... Anthony -- Anthony Baxter It's never too late to have a happy childhood. From fdrake@acm.org Sun Dec 23 22:29:56 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Sun, 23 Dec 2001 17:29:56 -0500 (EST) Subject: [Python-Dev] Python 2.2.1 In-Reply-To: <20011223222209.C9CE5E8453@oratrix.oratrix.nl> References: <2mzo49yfkj.fsf@starship.python.net> <20011223222209.C9CE5E8453@oratrix.oratrix.nl> Message-ID: <15398.23268.960166.935728@cj42289-a.reston1.va.home.com> Jack Jansen writes: > - The final bits of the Mac tree havne't been checked in yet, should > we move the release22-maint branch when that's done? Can you move a > branch? How? Yes, it should be moved as necessary to reflect the MacPython release; this can be done using these commands from the dist/src/ directory: cvs tag release22-mac cvs tag -r release22-mac -b -F release22-maint -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From guido@python.org Sun Dec 23 22:31:52 2001 From: guido@python.org (Guido van Rossum) Date: Sun, 23 Dec 2001 17:31:52 -0500 Subject: [Python-Dev] Python 2.2.1 In-Reply-To: Your message of "Sun, 23 Dec 2001 23:22:04 +0100." <20011223222209.C9CE5E8453@oratrix.oratrix.nl> References: <20011223222209.C9CE5E8453@oratrix.oratrix.nl> Message-ID: <200112232231.RAA15943@cj20424-a.reston1.va.home.com> > Two questions wrt 2.2.1: > > - Is there a timeline for it? I couldn't find one in PEP251 (which > stops at 2.2 final). The reason I ask is that there's a serious flaw > in MacPython (has been there since day one) that renders it useless on > multiprocessor Mac OS X machines. I'm waiting for an external party to > fix this (as the problem is in an external library), but I could put a > bit of pressure on if 2.2.1 is due reasonably soon (as I hope). No timeline -- that sort of depends on how many really bad bugs are reported, and whether someone volunteers as a release mananger. > - The final bits of the Mac tree havne't been checked in yet, should > we move the release22-maint branch when that's done? Can you move a > branch? How? Fred anticipated that; he said he would just move the tags when necessary. Don't worry about it. --Guido van Rossum (home page: http://www.python.org/~guido/) From Anthony Baxter Sun Dec 23 22:34:32 2001 From: Anthony Baxter (Anthony Baxter) Date: Mon, 24 Dec 2001 09:34:32 +1100 Subject: [Python-Dev] Python 2.2.1 In-Reply-To: Message from Jack Jansen of "Sun, 23 Dec 2001 23:22:04 BST." <20011223222209.C9CE5E8453@oratrix.oratrix.nl> Message-ID: <200112232234.fBNMYWc10781@mbuna.arbhome.com.au> >>> Jack Jansen wrote > - The final bits of the Mac tree havne't been checked in yet, should > we move the release22-maint branch when that's done? Can you move a > branch? How? D'oh! Hm, I'm not sure how easily that can be done. In theory you should be able to "cvs tag -d release22-maint" in the Mac-specific subdirectory, then do the checkins, then "cvs tag -b release22-maint" in the Mac directory again. So long as there's not been any checkins on the branch in the directories that you're doing the tag shifting, you should be fine. Probably. Although checking on a little toy CVS root is probably a great idea :) In fact, I just did this on a local toy project, and it worked fine. I'd strongly recommend using cvs tag -d followed by retagging, rather than using tag -F to try and move it - I've had bad experiences in the past with -F -- Anthony Baxter It's never too late to have a happy childhood. From martin@v.loewis.de Mon Dec 24 08:32:13 2001 From: martin@v.loewis.de (Martin v. Loewis) Date: Mon, 24 Dec 2001 09:32:13 +0100 Subject: [Python-Dev] Python 2.2.1 In-Reply-To: <15398.23268.960166.935728@cj42289-a.reston1.va.home.com> (fdrake@acm.org) References: <2mzo49yfkj.fsf@starship.python.net> <20011223222209.C9CE5E8453@oratrix.oratrix.nl> <15398.23268.960166.935728@cj42289-a.reston1.va.home.com> Message-ID: <200112240832.fBO8WDO01324@mira.informatik.hu-berlin.de> > > - The final bits of the Mac tree havne't been checked in yet, should > > we move the release22-maint branch when that's done? Can you move a > > branch? How? > > Yes, it should be moved as necessary to reflect the MacPython > release; this can be done using these commands from the dist/src/ > directory: > > cvs tag release22-mac > cvs tag -r release22-mac -b -F release22-maint Maybe I'm misunderstanding. It seems pointless to move a branch: Just check in to it. In terms of CVS commands cvs update -rrelease22-maint # get a sandbox of the branch # make modifications cvs commit # make modifications to the branch cvs update -A # return to mainline What am I missing? Regards, Martin From fdrake@acm.org Tue Dec 25 16:49:23 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 25 Dec 2001 11:49:23 -0500 (EST) Subject: [Python-Dev] Python 2.2.1 In-Reply-To: <200112240832.fBO8WDO01324@mira.informatik.hu-berlin.de> References: <2mzo49yfkj.fsf@starship.python.net> <20011223222209.C9CE5E8453@oratrix.oratrix.nl> <15398.23268.960166.935728@cj42289-a.reston1.va.home.com> <200112240832.fBO8WDO01324@mira.informatik.hu-berlin.de> Message-ID: <15400.44563.436248.81801@cj42289-a.reston1.va.home.com> Martin v. Loewis writes: > Maybe I'm misunderstanding. It seems pointless to move a branch: Just > check in to it. In terms of CVS commands That would work too; I'm not going to be picky about the process. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From fdrake@acm.org Wed Dec 26 17:15:40 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 26 Dec 2001 12:15:40 -0500 (EST) Subject: [Python-Dev] Python 2.2.1 In-Reply-To: <200112232234.fBNMYWc10781@mbuna.arbhome.com.au> References: <20011223222209.C9CE5E8453@oratrix.oratrix.nl> <200112232234.fBNMYWc10781@mbuna.arbhome.com.au> Message-ID: <15402.1468.577251.936398@cj42289-a.reston1.va.home.com> Has anything been done to get the release22-maint branch to see the changes made for MacPython yet? We should do this before we lose track of it. Jack, any preference for how this is done, or did you already do it? -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From jack@oratrix.nl Wed Dec 26 22:37:26 2001 From: jack@oratrix.nl (Jack Jansen) Date: Wed, 26 Dec 2001 23:37:26 +0100 Subject: [Python-Dev] Python 2.2.1 In-Reply-To: Message by "Fred L. Drake, Jr." , Wed, 26 Dec 2001 12:15:40 -0500 (EST) , <15402.1468.577251.936398@cj42289-a.reston1.va.home.com> Message-ID: <20011226223731.262A1E8448@oratrix.oratrix.nl> Recently, "Fred L. Drake, Jr." said: > > Has anything been done to get the release22-maint branch to see the > changes made for MacPython yet? We should do this before we lose > track of it. > Jack, any preference for how this is done, or did you already do it? They haven\t been checked in yet. There was a snag in the Mac distriubution that has just been solved, the checkins will follow shortly. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.cwi.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From fdrake@acm.org Wed Dec 26 22:39:17 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 26 Dec 2001 17:39:17 -0500 (EST) Subject: [Python-Dev] Python 2.2.1 In-Reply-To: <20011226223731.262A1E8448@oratrix.oratrix.nl> References: <15402.1468.577251.936398@cj42289-a.reston1.va.home.com> <20011226223731.262A1E8448@oratrix.oratrix.nl> Message-ID: <15402.20885.998208.923051@cj42289-a.reston1.va.home.com> Jack Jansen writes: > They haven\t been checked in yet. There was a snag in the Mac > distriubution that has just been solved, the checkins will follow shortly. OK, I'll watch for them. You can make sure the changes hit both the branch and the trunk yourself, or check into the trunk and we can play with CVS to move the changes onto the branch for the Mac tree. If you want me to do anything to help out with this, I should be able to lend a hand later this evening or in the morning. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From fdrake@acm.org Fri Dec 28 04:56:31 2001 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 27 Dec 2001 23:56:31 -0500 (EST) Subject: [Python-Dev] Documentation for Python 2.2.1 Message-ID: <15403.64383.728258.799543@cj42289-a.reston1.va.home.com> I've brought the docs in Python's release22-maint branch up to date with the corrections that have been made on the trunk. I'll be checking in relevant additions to this branch as well as the trunk when I make the trunk changes. When anyone else makes a change on the trunk which should also be made on the 2.2 maintenance branch, feel free to check it in in both places, or add a note to the checkin message noting that it should be made on the branch as well, and I'll try to make sure that gets done. The 2.2 maintenance branch has replaced the 2.1 maintenance branch as the "maintenance version" at http://python.sourceforge.net/; if anyone thinks we should continue to offer the 2.1 maintenance docs there, please let me know. I don't think we have a Patch Czar for 2.2.1 yet; if anyone would like to take that duty for the documentation tree, I'd certainly appreciate the help! Thanks! -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From fdrake@acm.org Fri Dec 28 04:59:57 2001 From: fdrake@acm.org (Fred L. Drake) Date: Thu, 27 Dec 2001 23:59:57 -0500 (EST) Subject: [Python-Dev] [development doc updates] Message-ID: <20011228045957.6E9FD286BB@beowolf.digicool.com> The development version of the documentation has been updated: http://python.sourceforge.net/maint-docs/ The "maintenance version" of the Python documentation on the Python development site at SourceForge has been updated to show the docs being developed for Python 2.2.1. From fdrake@acm.org Fri Dec 28 05:03:42 2001 From: fdrake@acm.org (Fred L. Drake) Date: Fri, 28 Dec 2001 00:03:42 -0500 (EST) Subject: [Python-Dev] [development doc updates] Message-ID: <20011228050342.CB129286BB@beowolf.digicool.com> The development version of the documentation has been updated: http://python.sourceforge.net/devel-docs/ A variety of updates made since Python 2.2 was released. From martin@v.loewis.de Fri Dec 28 10:13:17 2001 From: martin@v.loewis.de (Martin v. Loewis) Date: Fri, 28 Dec 2001 11:13:17 +0100 Subject: [Python-Dev] Re: [XML-SIG] printing Unicode xml to StringIO In-Reply-To: <3C2C3406.CB6D49DB@lemburg.com> (mal@lemburg.com) References: <200112271506.fBRF6uF26242@spinaker.ins.cwi.nl> <200112271736.fBRHaom02848@mira.informatik.hu-berlin.de> <3C2C3406.CB6D49DB@lemburg.com> Message-ID: <200112281013.fBSADHu01819@mira.informatik.hu-berlin.de> [Over to python-dev. Jaco noticed that writing Unicode objects to a StringIO object stopped working in 2.2, see http://mail.python.org/pipermail/xml-sig/2001-December/006891.html ] Marc-Andre writes > Actually, I think that this is a bug in the documentation, not the > code. StringIO and cStringIO were never meant to work on anything but > strings and memory buffers. IMO, "strings" should include both byte strings and Unicode strings. Mixing them may not be allowed, but that is a different story. In fact, there is an open bug (#216388) that cStringIO rejects Unicode objects. If that gets fixed, we get the funny scenario that StringIO rejects Unicode object, whereas cStringIO accepts them. > The note that Fred added to the docs about StringIO's capability of > storing Unicode in it's buffer list is simply an artifact of the > implementation. There are many developers who take this note literally. Claiming that this was not intentional is a mistake. > Please use the .encode() method on Unicode objects before writing > them to a StringIO object. If you want to end up with a byte string, this is a good idea. But I think it is pointless to require encoding them when you want to end up with a Unicode string; you'd have to invoke unicode() on the result, for no apparent reason but a bug in the StringIO implementation. Regards, Martin From mal@lemburg.com Fri Dec 28 10:50:25 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 28 Dec 2001 11:50:25 +0100 Subject: [Python-Dev] Re: [XML-SIG] printing Unicode xml to StringIO References: <200112271506.fBRF6uF26242@spinaker.ins.cwi.nl> <200112271736.fBRHaom02848@mira.informatik.hu-berlin.de> <3C2C3406.CB6D49DB@lemburg.com> <200112281013.fBSADHu01819@mira.informatik.hu-berlin.de> Message-ID: <3C2C4E71.2BA58BF3@lemburg.com> "Martin v. Loewis" wrote: > > [Over to python-dev. Jaco noticed that writing Unicode objects to > a StringIO object stopped working in 2.2, see > http://mail.python.org/pipermail/xml-sig/2001-December/006891.html > ] > > Marc-Andre writes > > Actually, I think that this is a bug in the documentation, not the > > code. StringIO and cStringIO were never meant to work on anything but > > strings and memory buffers. > > IMO, "strings" should include both byte strings and Unicode strings. > Mixing them may not be allowed, but that is a different story. > > In fact, there is an open bug (#216388) that cStringIO rejects Unicode > objects. If that gets fixed, we get the funny scenario that StringIO > rejects Unicode object, whereas cStringIO accepts them. StringIO and cStringIO use different methods for storing the snippets: StringIO makes use of a buffer list which gets compressed every now and then, while cStringIO uses a raw memory buffer for this purpose. Both of these implementation are targetted at providing a file IO like interface to in-memory "files". Since Python file object don't magically support Unicode, I wonder where the idea came from that StringIO/cStringIO should. That patch I applied to StringIO/cStringIO for 2.2 was aimed at making these two more compatible to the standard Python file object. The latter uses the "s#" parser marker for .write() and thus can also accept memory buffers. This was previously not possible with either of the two StringIO implementation (StringIO.py failed when trying to join different buffer compatible objects, cStringIO only accepted real string objects). > > The note that Fred added to the docs about StringIO's capability of > > storing Unicode in it's buffer list is simply an artifact of the > > implementation. > > There are many developers who take this note literally. Claiming that > this was not intentional is a mistake. > > > Please use the .encode() method on Unicode objects before writing > > them to a StringIO object. > > If you want to end up with a byte string, this is a good idea. That's the idea behind StringIO objects... they are in-memory file object emulators. > But I > think it is pointless to require encoding them when you want to end up > with a Unicode string; you'd have to invoke unicode() on the result, > for no apparent reason but a bug in the StringIO implementation. This is a different application. It should be easy enough to subclass StringIO as UnicodeIO class and then have this class implement fast Unicode snippet joining. I'm not sure whether the same can be done with cStringIO's type. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From martin@v.loewis.de Fri Dec 28 11:04:30 2001 From: martin@v.loewis.de (Martin v. Loewis) Date: Fri, 28 Dec 2001 12:04:30 +0100 Subject: [Python-Dev] Re: [XML-SIG] printing Unicode xml to StringIO In-Reply-To: <3C2C4E71.2BA58BF3@lemburg.com> (mal@lemburg.com) References: <200112271506.fBRF6uF26242@spinaker.ins.cwi.nl> <200112271736.fBRHaom02848@mira.informatik.hu-berlin.de> <3C2C3406.CB6D49DB@lemburg.com> <200112281013.fBSADHu01819@mira.informatik.hu-berlin.de> <3C2C4E71.2BA58BF3@lemburg.com> Message-ID: <200112281104.fBSB4UI02359@mira.informatik.hu-berlin.de> From martin@v.loewis.de Fri Dec 28 11:25:46 2001 From: martin@v.loewis.de (Martin v. Loewis) Date: Fri, 28 Dec 2001 12:25:46 +0100 Subject: [Python-Dev] Re: [XML-SIG] printing Unicode xml to StringIO In-Reply-To: <3C2C4E71.2BA58BF3@lemburg.com> (mal@lemburg.com) References: <200112271506.fBRF6uF26242@spinaker.ins.cwi.nl> <200112271736.fBRHaom02848@mira.informatik.hu-berlin.de> <3C2C3406.CB6D49DB@lemburg.com> <200112281013.fBSADHu01819@mira.informatik.hu-berlin.de> <3C2C4E71.2BA58BF3@lemburg.com> Message-ID: <200112281125.fBSBPkC02403@mira.informatik.hu-berlin.de> > Both of these implementation are targetted at providing > a file IO like interface to in-memory "files". Since Python > file object don't magically support Unicode, I wonder where the > idea came from that StringIO/cStringIO should. The exact source of this idea is unknown. However, there are many early references to it: - codecs.open returns an "encoded file" which "will only accept ... Unicode objects". That is perhaps the earliest precedent of a file object supporting Unicode. - At some point in time, you said that it is a bug that cStringIO does not support Unicode strings, see http://mail.python.org/pipermail/i18n-sig/2000-November/000550.html - the documentation of StringIO suggests that they should accept Unicode. So I would not blame the users for adopting far-off ideas, when the Python core itself suggests that these ideas are Pythonic. > That patch I applied to StringIO/cStringIO for 2.2 was > aimed at making these two more compatible to the standard > Python file object. The latter uses the "s#" parser > marker for .write() and thus can also accept memory > buffers. This was previously not possible with either > of the two StringIO implementation (StringIO.py failed > when trying to join different buffer compatible objects, > cStringIO only accepted real string objects). There is nothing wrong with that. The patch should just have special-cased Unicode objects (and that bug can still be corrected). Regards, Martin From guido@python.org Fri Dec 28 14:20:18 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 28 Dec 2001 09:20:18 -0500 Subject: [Python-Dev] Re: [XML-SIG] printing Unicode xml to StringIO In-Reply-To: Your message of "Fri, 28 Dec 2001 12:25:46 +0100." <200112281125.fBSBPkC02403@mira.informatik.hu-berlin.de> References: <200112271506.fBRF6uF26242@spinaker.ins.cwi.nl> <200112271736.fBRHaom02848@mira.informatik.hu-berlin.de> <3C2C3406.CB6D49DB@lemburg.com> <200112281013.fBSADHu01819@mira.informatik.hu-berlin.de> <3C2C4E71.2BA58BF3@lemburg.com> <200112281125.fBSBPkC02403@mira.informatik.hu-berlin.de> Message-ID: <200112281420.JAA22961@cj20424-a.reston1.va.home.com> Whoa! - Since we added a note to the docs that StringIO supports Unicode, we clearly should continue to support that, and it's a bug if it doesn't. - OTOH, Unicode for cStringIO should be considered at best a feature request. I don't mind if cStringIO doesn't support Unicode -- it never has, AFAIK, so it won't break much code. I don't believe it's much faster than StringIO, unless you use the C API (like cPickle does). - Of course, when Unicode is supported, mixing ASCII and Unicode should be supported too. (But not necessarily mixing 8-bit strings containing characters in the range \200-\377, since there's no default encoding for this range.) - Since this changed from 2.1 to 2.2, we should restore this capability in 2.2.1; I would say that 2.2.1 can't go out until this is fixed. --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Fri Dec 28 15:30:34 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 28 Dec 2001 16:30:34 +0100 Subject: [Python-Dev] Re: [XML-SIG] printing Unicode xml to StringIO References: <200112271506.fBRF6uF26242@spinaker.ins.cwi.nl> <200112271736.fBRHaom02848@mira.informatik.hu-berlin.de> <3C2C3406.CB6D49DB@lemburg.com> <200112281013.fBSADHu01819@mira.informatik.hu-berlin.de> <3C2C4E71.2BA58BF3@lemburg.com> <200112281125.fBSBPkC02403@mira.informatik.hu-berlin.de> <200112281420.JAA22961@cj20424-a.reston1.va.home.com> Message-ID: <3C2C901A.D61001D9@lemburg.com> Guido van Rossum wrote: > > Whoa! > > - Since we added a note to the docs that StringIO supports Unicode, we > clearly should continue to support that, and it's a bug if it > doesn't. I still believe that the docs are wrong, but nevermind. I'll fix StringIO.py to continue to support Unicode in addition to strings and buffer objects. It's basically only about special casing Unicode in the .write() method. BTW, I was never aware of the doc changes in this area and the test suite didn't bring up the issues either. > - OTOH, Unicode for cStringIO should be considered at best a feature > request. I don't mind if cStringIO doesn't support Unicode -- it > never has, AFAIK, so it won't break much code. I don't believe it's > much faster than StringIO, unless you use the C API (like cPickle > does). Unicode support in cStringIO would require a new implementation since the machinery uses raw byte buffers. > - Of course, when Unicode is supported, mixing ASCII and Unicode > should be supported too. (But not necessarily mixing 8-bit strings > containing characters in the range \200-\377, since there's no > default encoding for this range.) In StringIO.py this is not much of a problem since it uses a list of snippets. Note that this is also why StringIO.py "supported" Unicode in the first place (and that's why I think it was more an artifact of the implementation than true intent). > - Since this changed from 2.1 to 2.2, we should restore this > capability in 2.2.1; I would say that 2.2.1 can't go out until this > is fixed. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From guido@python.org Fri Dec 28 15:27:02 2001 From: guido@python.org (Guido van Rossum) Date: Fri, 28 Dec 2001 10:27:02 -0500 Subject: [Python-Dev] Re: [XML-SIG] printing Unicode xml to StringIO In-Reply-To: Your message of "Fri, 28 Dec 2001 16:30:34 +0100." <3C2C901A.D61001D9@lemburg.com> References: <200112271506.fBRF6uF26242@spinaker.ins.cwi.nl> <200112271736.fBRHaom02848@mira.informatik.hu-berlin.de> <3C2C3406.CB6D49DB@lemburg.com> <200112281013.fBSADHu01819@mira.informatik.hu-berlin.de> <3C2C4E71.2BA58BF3@lemburg.com> <200112281125.fBSBPkC02403@mira.informatik.hu-berlin.de> <200112281420.JAA22961@cj20424-a.reston1.va.home.com> <3C2C901A.D61001D9@lemburg.com> Message-ID: <200112281527.KAA23737@cj20424-a.reston1.va.home.com> > > - Since we added a note to the docs that StringIO supports Unicode, we > > clearly should continue to support that, and it's a bug if it > > doesn't. > > I still believe that the docs are wrong, but nevermind. I'll fix > StringIO.py to continue to support Unicode in addition to strings > and buffer objects. It's basically only about special casing > Unicode in the .write() method. Thanks. > BTW, I was never aware of the doc changes in this area and the > test suite didn't bring up the issues either. Can you please add something to the test suite that makes sure this feature works? > > - OTOH, Unicode for cStringIO should be considered at best a feature > > request. I don't mind if cStringIO doesn't support Unicode -- it > > never has, AFAIK, so it won't break much code. I don't believe it's > > much faster than StringIO, unless you use the C API (like cPickle > > does). > > Unicode support in cStringIO would require a new implementation > since the machinery uses raw byte buffers. That's why I don't care much about it. :-) > > - Of course, when Unicode is supported, mixing ASCII and Unicode > > should be supported too. (But not necessarily mixing 8-bit strings > > containing characters in the range \200-\377, since there's no > > default encoding for this range.) > > In StringIO.py this is not much of a problem since it uses > a list of snippets. Note that this is also why StringIO.py "supported" > Unicode in the first place (and that's why I think it was more an > artifact of the implementation than true intent). But it was useful! :-) > > - Since this changed from 2.1 to 2.2, we should restore this > > capability in 2.2.1; I would say that 2.2.1 can't go out until this > > is fixed. Try to mark the checkin messages as "2.2.1 bugfix", for the 2.2.1 patch czar. --Guido van Rossum (home page: http://www.python.org/~guido/) From martin@v.loewis.de Fri Dec 28 17:02:17 2001 From: martin@v.loewis.de (Martin v. Loewis) Date: Fri, 28 Dec 2001 18:02:17 +0100 Subject: [Python-Dev] Re: [XML-SIG] printing Unicode xml to StringIO In-Reply-To: <3C2C901A.D61001D9@lemburg.com> (mal@lemburg.com) References: <200112271506.fBRF6uF26242@spinaker.ins.cwi.nl> <200112271736.fBRHaom02848@mira.informatik.hu-berlin.de> <3C2C3406.CB6D49DB@lemburg.com> <200112281013.fBSADHu01819@mira.informatik.hu-berlin.de> <3C2C4E71.2BA58BF3@lemburg.com> <200112281125.fBSBPkC02403@mira.informatik.hu-berlin.de> <200112281420.JAA22961@cj20424-a.reston1.va.home.com> <3C2C901A.D61001D9@lemburg.com> Message-ID: <200112281702.fBSH2Ha04205@mira.informatik.hu-berlin.de> > Unicode support in cStringIO would require a new implementation > since the machinery uses raw byte buffers. Not necessarily. You could add a flag saying that whether those bytes represent Unicode strings or not. If they do, you pick an encoding of your choice (perhaps unicode-internal), and convert the existing bytes to that encoding on first sighting of a Unicode string (assuming that everything so far is in the system encoding). When returning the bytes to the user, you build a Unicode object if the flag is set. Of course, you'd still have to touch every method, to analyse the flag... Regards, Martin From Jack.Jansen@cwi.nl Mon Dec 31 13:39:09 2001 From: Jack.Jansen@cwi.nl (Jack Jansen) Date: Mon, 31 Dec 2001 14:39:09 +0100 Subject: [Python-Dev] Offline for a while Message-ID: <200112311339.fBVDd9O16531@spruit.ins.cwi.nl> Folks, I will probably be unreacable by mail for the next few days, the backup we put in for our move seems to be non-functional. So don't worry about bounceds to the oratrix.nl address, all should be well again on wednseday or thusrday (and backspace also doesn't work on this machine:-) JackRetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4