Showing content from https://mail.python.org/pipermail/python-dev/2008-August.txt below:
References: <20080823202744.367931E4003@bag.python.org> <1afaf6160808231350y44b9d53i4e4e534ede049260@mail.gmail.com> Message-ID: <48B0D1EE.6050903@gmail.com> Brett Cannon wrote: > On Sat, Aug 23, 2008 at 1:50 PM, Benjamin Peterson > wrote: >> On Sat, Aug 23, 2008 at 3:48 PM, Brett Cannon wrote: >>> I don't think Victor has commit privileges, which would mean you >>> should have had another committer review this first before committing. >> Ah. Ok. >> >> Can we make a wiki page for rules like this? I feel like I'm poking >> around it a dark room which is actually a monster's mouth. :) > > This commit review thing is new to all of us; never been done while I > have been around. It's just that committing a non-committers patch > without a second opinion is just like any other external patch we > accept, and thus lacks the extra protection that having a second > committer sign off it gives us during the rc phase. > > And yes, I will write up a doc on all of this after we hit final. My > current plan is that once we release final to spark a conversation > about our workflow and then write up a doc that lists what is expected > for each step in the life of an issue. In the meantime, I put some (very) rough notes based on what we did for the betas and are doing for the release candidates up on the wiki: http://wiki.python.org/moin/PythonCommitterGuidelines Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From nnorwitz at gmail.com Sun Aug 24 07:12:49 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sat, 23 Aug 2008 22:12:49 -0700 Subject: [Python-Dev] current problems that should be fixed before release (2.6) Message-ID: Below are the problems I found that have not been fixed at r65995 on trunk (2.6). There will be a separate mail for the 3.0 problems. I've done the following: * built in debug and opt mode (gcc 4.1.2) fixing the important warnings * run all the tests in both modes * run all the tests (except test_logging and test_ssl) under valgrind 3.3.1 fixing the problems * run failmalloc (1.0) on startup for the first 4000 allocations fixing most problems * run pychecker 0.8.18 over the stdlib (Lib/*.py, Lib/[bcdelhjmswx]*/*.py) Can someone run purify on windows over 2.6? Can someone (else) compare performance of 2.5, 2.6, and 3.0? pychecker reported problems: Lib/gzip.py:114: Using property (filename) in classic class GzipFile may not work Lib/io.py:1394: No module attribute (device_encoding) found - device_encoding access is protected, but I don't see where it can be set Lib/sched.py:122: Using property (queue) in classic class scheduler may not work Lib/tempfile.py:535: Using property (closed) in classic class SpooledTemporaryFile may not work Lib/tempfile.py:539: Using property (encoding) in classic class SpooledTemporaryFile may not work Lib/tempfile.py:553: Using property (mode) in classic class SpooledTemporaryFile may not work Lib/tempfile.py:557: Using property (name) in classic class SpooledTemporaryFile may not work Lib/tempfile.py:561: Using property (newlines) in classic class SpooledTemporaryFile may not work Lib/tempfile.py:580: Using property (softspace) in classic class SpooledTemporaryFile may not work Lib/multiprocessing/connection.py:95: No global (PipeListener) found Lib/multiprocessing/connection.py:132: No global (PipeClient) found - both of the Pipe classes are defined only for windows. I don't see a similar def for Unix. Lib/multiprocessing/connection.py:378: No global (AuthenticationError) found I haven't looked into the property complaint to see how bad it is. test_bsddb3 is failing on several 2.6 buildbots. test_bsddb3 leaks 80 references. Results of valgrind 3.3.1 on Ubuntu 6.10 amd64, gcc 4.1.2 (prerelease): Uninitialized memory reads: No problems in python code. Though it's possible there are errors in ctypes. I'm ignoring those reports due to a problem in dlopen. Wild memory writes: No problems in python code. Memory leaks: No major problems. However, it seems that there are a bunch of little leaks when forking a process. I haven't determined if these are real problems or not. There is one memory leak in ctypes (sorry no real useful information): 44 bytes in 1 blocks are definitely lost in loss record 43 of 191 malloc (vg_replace_malloc.c:207) resize (callproc.c:1733) It seems to always leak 44 bytes n From amauryfa at gmail.com Sun Aug 24 11:40:51 2008 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Sun, 24 Aug 2008 11:40:51 +0200 Subject: [Python-Dev] "error: None" when building extensions under 2.6 In-Reply-To: References: Message-ID: Hello, 2008/8/23 Fredrik Lundh : > when I'm trying to build extensions under Python 2.6 on Windows XP, the > build process terminates with single line that says: > > error: None > > which is about as useless as an error message can be. Googling for this > brings up a few hits which all seem to involve setuptools (and none of the > hits contain any answers), but I'm using straightforward distutils stuff > that has worked without a glitch since 1.5.2. Has anyone else seen this for > 2.6? I tried to rebuild some extensions, and it either succeeds, or fails with a meaningful (for an expert) error message. Which extensions did you try? is there some output before the error? -- Amaury Forgeot d'Arc From fredrik at pythonware.com Sun Aug 24 11:49:26 2008 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun, 24 Aug 2008 11:49:26 +0200 Subject: [Python-Dev] "error: None" when building extensions under 2.6 In-Reply-To: References: Message-ID: Amaury Forgeot d'Arc wrote: >> when I'm trying to build extensions under Python 2.6 on Windows XP, the >> build process terminates with single line that says: >> >> error: None >> >> which is about as useless as an error message can be. Googling for this >> brings up a few hits which all seem to involve setuptools (and none of the >> hits contain any answers), but I'm using straightforward distutils stuff >> that has worked without a glitch since 1.5.2. Has anyone else seen this for >> 2.6? > > I tried to rebuild some extensions, and it either succeeds, > or fails with a meaningful (for an expert) error message. > > Which extensions did you try? is there some output before the error? pick any: http://effbot.org/downloads/ ;-) I'm beginning to suspect that I have a botched VS install on this machine, though. I'll investigate. From theller at ctypes.org Sun Aug 24 13:03:54 2008 From: theller at ctypes.org (Thomas Heller) Date: Sun, 24 Aug 2008 13:03:54 +0200 Subject: [Python-Dev] "error: None" when building extensions under 2.6 In-Reply-To: References: Message-ID: Fredrik Lundh schrieb: > Amaury Forgeot d'Arc wrote: > >>> when I'm trying to build extensions under Python 2.6 on Windows XP, the >>> build process terminates with single line that says: >>> >>> error: None >>> >>> which is about as useless as an error message can be. Googling for this >>> brings up a few hits which all seem to involve setuptools (and none of the >>> hits contain any answers), but I'm using straightforward distutils stuff >>> that has worked without a glitch since 1.5.2. Has anyone else seen this for >>> 2.6? >> >> I tried to rebuild some extensions, and it either succeeds, >> or fails with a meaningful (for an expert) error message. >> >> Which extensions did you try? is there some output before the error? > > pick any: > > http://effbot.org/downloads/ > > ;-) > > I'm beginning to suspect that I have a botched VS install on this > machine, though. I'll investigate. > Do you get a traceback when you set the DISTUTILS_DEBUG environment variable? Thomas From skip at pobox.com Sun Aug 24 14:16:56 2008 From: skip at pobox.com (skip at pobox.com) Date: Sun, 24 Aug 2008 07:16:56 -0500 Subject: [Python-Dev] current problems that should be fixed before release (2.6) In-Reply-To: References: Message-ID: <18609.20792.53909.137357@montanaro-dyndns-org.local> Neal> Lib/gzip.py:114: Using property (filename) in classic class GzipFile Neal> may not work Seems shallow. Just inherit from object. Neal> Lib/sched.py:122: Using property (queue) in classic class scheduler Neal> may not work Ditto. Neal> Lib/tempfile.py:535: Using property (closed) in classic class Neal> SpooledTemporaryFile may not work ... Ditto. Neal> Lib/multiprocessing/connection.py:378: No global (AuthenticationError) found This is defined in the __init__ module of the package. Is it accessible from connection.py without further import? (My guess would be no.) A slight code rearrangement or adding imports at the point of the raise statements probably fixes that. Neal> I haven't looked into the property complaint to see how bad it is. I'm testing out the inherit-from-object fixes now. Will report back later. Skip From solipsis at pitrou.net Sun Aug 24 14:58:31 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 24 Aug 2008 12:58:31 +0000 (UTC) Subject: [Python-Dev] performance References: Message-ID: Neal Norwitz gmail.com> writes: > Can someone (else) compare performance of 2.5, 2.6, and 3.0? Tests done on a 32-bit Linux installation on an Athlon 3600+ X2. Compiled with gcc in UCS2 mode. pystone ------- - 2.5: 43859.6 pystones/second - 2.6: 42016.8 pystones/second - 3.0: 38759.7 pystones/second richards.py ----------- (roughly, richards is an object-oriented method-calling benchmark) - 2.5: 770.54 ms per iteration - 2.6: 572.84 ms per iteration - 3.0: 566.69 ms per iteration stringbench ----------- - 2.5: unicode: 265.84 s / bytes: 180.50 s - 2.6: unicode: 256.22 s / bytes: 184.45 s - 3.0: unicode: 248.07 s / bytes: not tested pybench: 2.6 vs. 2.5 -------------------- "this" is 2.6, "other" is 2.5. Test minimum run-time average run-time this other diff this other diff ------------------------------------------------------------------------------- BuiltinFunctionCalls: 177ms 181ms -2.2% 178ms 182ms -2.1% BuiltinMethodLookup: 157ms 177ms -11.1% 158ms 177ms -11.0% CompareFloats: 171ms 171ms -0.3% 171ms 171ms -0.3% CompareFloatsIntegers: 158ms 162ms -2.1% 160ms 167ms -4.8% CompareIntegers: 208ms 208ms -0.0% 209ms 209ms -0.3% CompareInternedStrings: 189ms 183ms +3.4% 189ms 184ms +2.7% CompareLongs: 154ms 154ms -0.2% 154ms 154ms -0.2% CompareStrings: 162ms 159ms +1.7% 163ms 159ms +2.5% CompareUnicode: 144ms 149ms -3.6% 144ms 152ms -5.0% ComplexPythonFunctionCalls: 173ms 242ms -28.5% 176ms 243ms -27.6% ConcatStrings: 196ms 198ms -0.9% 203ms 200ms +1.4% ConcatUnicode: 153ms 151ms +1.5% 155ms 157ms -1.5% CreateInstances: 172ms 169ms +1.5% 173ms 170ms +1.4% CreateNewInstances: 131ms 148ms -11.6% 132ms 151ms -12.2% CreateStringsWithConcat: 209ms 206ms +1.4% 209ms 208ms +0.8% CreateUnicodeWithConcat: 128ms 124ms +3.7% 129ms 124ms +3.4% DictCreation: 115ms 149ms -22.8% 116ms 150ms -22.8% DictWithFloatKeys: 208ms 207ms +0.4% 208ms 208ms +0.0% DictWithIntegerKeys: 173ms 173ms -0.0% 174ms 173ms +0.5% DictWithStringKeys: 162ms 162ms -0.1% 162ms 162ms +0.1% ForLoops: 181ms 181ms -0.2% 181ms 182ms -0.2% IfThenElse: 169ms 168ms +0.2% 169ms 169ms -0.2% ListSlicing: 109ms 108ms +0.2% 109ms 109ms +0.1% NestedForLoops: 198ms 197ms +0.2% 198ms 197ms +0.1% NormalClassAttribute: 176ms 172ms +2.0% 176ms 173ms +1.7% NormalInstanceAttribute: 162ms 161ms +1.0% 163ms 161ms +1.1% PythonFunctionCalls: 161ms 151ms +6.6% 162ms 153ms +6.0% PythonMethodCalls: 188ms 189ms -0.7% 189ms 193ms -2.0% Recursion: 235ms 230ms +2.2% 236ms 233ms +1.2% SecondImport: 113ms 114ms -1.0% 115ms 115ms +0.0% SecondPackageImport: 120ms 116ms +3.6% 120ms 117ms +2.9% SecondSubmoduleImport: 154ms 146ms +6.0% 156ms 148ms +5.7% SimpleComplexArithmetic: 163ms 151ms +8.3% 163ms 151ms +8.6% SimpleDictManipulation: 177ms 173ms +2.6% 180ms 174ms +3.2% SimpleFloatArithmetic: 169ms 164ms +3.0% 169ms 166ms +2.2% SimpleIntFloatArithmetic: 153ms 152ms +1.2% 156ms 153ms +1.8% SimpleIntegerArithmetic: 156ms 152ms +2.5% 156ms 153ms +2.0% SimpleListManipulation: 156ms 158ms -1.0% 157ms 158ms -0.5% SimpleLongArithmetic: 159ms 150ms +6.1% 161ms 151ms +6.3% SmallLists: 156ms 154ms +1.1% 160ms 155ms +3.0% SmallTuples: 156ms 155ms +0.4% 157ms 156ms +0.8% SpecialClassAttribute: 173ms 172ms +0.6% 173ms 172ms +0.7% SpecialInstanceAttribute: 202ms 198ms +2.4% 203ms 199ms +2.2% StringMappings: 164ms 170ms -3.2% 165ms 171ms -3.9% StringPredicates: 160ms 185ms -13.6% 160ms 186ms -13.9% StringSlicing: 169ms 178ms -5.3% 174ms 180ms -3.4% TryExcept: 181ms 184ms -1.5% 181ms 184ms -1.5% TryFinally: 157ms 158ms -0.3% 159ms 161ms -1.2% TryRaiseExcept: 183ms 122ms +49.6% 184ms 124ms +48.2% TupleSlicing: 142ms 140ms +1.5% 144ms 141ms +2.0% UnicodeMappings: 198ms 190ms +4.1% 198ms 190ms +4.1% UnicodePredicates: 157ms 175ms -10.2% 157ms 176ms -10.6% UnicodeProperties: 161ms 161ms +0.1% 162ms 170ms -5.0% UnicodeSlicing: 148ms 153ms -3.3% 151ms 155ms -2.6% WithFinally: 191ms 203ms -5.9% 193ms 208ms -7.5% WithRaiseExcept: 151ms 160ms -5.8% 152ms 167ms -9.0% ------------------------------------------------------------------------------- Totals: 9287ms 9363ms -0.8% 9351ms 9455ms -1.1% pybench: 3.0 ------------ Not entirely comparable with the above since some tests disappeared. Test minimum average operation overhead ------------------------------------------------------------------------------- BuiltinFunctionCalls: 134ms 134ms 0.26us 0.631ms BuiltinMethodLookup: 127ms 127ms 0.12us 0.738ms CompareFloats: 174ms 174ms 0.14us 0.845ms CompareFloatsIntegers: 274ms 274ms 0.30us 0.630ms CompareIntegers: 277ms 277ms 0.15us 1.272ms CompareInternedStrings: 261ms 261ms 0.17us 3.201ms CompareLongs: 162ms 162ms 0.15us 0.736ms CompareStrings: 171ms 173ms 0.17us 2.188ms ComplexPythonFunctionCalls: 172ms 178ms 0.89us 1.073ms ConcatStrings: 260ms 273ms 0.55us 1.222ms CreateInstances: 178ms 178ms 1.59us 0.944ms CreateNewInstances: 133ms 135ms 1.61us 0.759ms CreateStringsWithConcat: 253ms 256ms 0.26us 2.127ms DictCreation: 118ms 118ms 0.29us 0.845ms DictWithFloatKeys: 207ms 215ms 0.24us 1.593ms DictWithIntegerKeys: 174ms 176ms 0.15us 2.132ms DictWithStringKeys: 158ms 158ms 0.13us 2.127ms ForLoops: 188ms 188ms 7.51us 0.097ms IfThenElse: 212ms 212ms 0.16us 1.597ms ListSlicing: 111ms 111ms 7.96us 0.165ms NestedForLoops: 218ms 220ms 0.15us 0.001ms NormalClassAttribute: 339ms 340ms 0.28us 1.111ms NormalInstanceAttribute: 186ms 186ms 0.16us 1.116ms PythonFunctionCalls: 156ms 158ms 0.48us 0.633ms PythonMethodCalls: 200ms 201ms 0.89us 0.374ms Recursion: 259ms 260ms 5.21us 1.059ms SecondImport: 128ms 128ms 1.28us 0.417ms SecondPackageImport: 140ms 143ms 1.43us 0.417ms SecondSubmoduleImport: 200ms 200ms 2.00us 0.417ms SimpleComplexArithmetic: 142ms 144ms 0.16us 0.844ms SimpleDictManipulation: 280ms 285ms 0.24us 1.059ms SimpleFloatArithmetic: 161ms 161ms 0.12us 1.275ms SimpleIntFloatArithmetic: 199ms 213ms 0.16us 1.272ms SimpleIntegerArithmetic: 200ms 209ms 0.16us 1.301ms SimpleListManipulation: 165ms 166ms 0.14us 1.379ms SimpleLongArithmetic: 132ms 137ms 0.21us 0.630ms SmallLists: 188ms 191ms 0.28us 0.844ms SmallTuples: 197ms 199ms 0.37us 0.951ms SpecialClassAttribute: 534ms 535ms 0.45us 1.121ms SpecialInstanceAttribute: 185ms 186ms 0.15us 1.125ms StringMappings: 471ms 471ms 1.87us 0.884ms StringPredicates: 190ms 191ms 0.27us 4.315ms StringSlicing: 285ms 290ms 0.52us 1.881ms TryExcept: 182ms 182ms 0.08us 1.663ms TryFinally: 134ms 136ms 0.85us 0.882ms TryRaiseExcept: 137ms 138ms 2.15us 0.885ms TupleSlicing: 202ms 204ms 0.78us 0.110ms WithFinally: 185ms 188ms 1.18us 0.881ms WithRaiseExcept: 274ms 275ms 3.44us 1.105ms ------------------------------------------------------------------------------- Totals: 10011ms 10120ms From skip at pobox.com Sun Aug 24 15:12:06 2008 From: skip at pobox.com (skip at pobox.com) Date: Sun, 24 Aug 2008 08:12:06 -0500 Subject: [Python-Dev] current problems that should be fixed before release (2.6) In-Reply-To: <18609.20792.53909.137357@montanaro-dyndns-org.local> References: <18609.20792.53909.137357@montanaro-dyndns-org.local> Message-ID: <18609.24102.380986.616063@montanaro-dyndns-org.local> me> Seems shallow. Just inherit from object. ... me> I'm testing out the inherit-from-object fixes now. They all passed. Here's the patch: http://bugs.python.org/issue3658 In addition, I get these two test failures on trunk: % ./python.exe ../Lib/test/regrtest.py test_distutils test_multiprocessing test_distutils test test_distutils failed -- Traceback (most recent call last): File "/Users/skip/src/python/trunk/Lib/distutils/tests/test_build_ext.py", line 23, in setUp shutil.copy(xx_c, self.tmp_dir) File "/Users/skip/src/python/trunk/Lib/shutil.py", line 88, in copy copyfile(src, dst) File "/Users/skip/src/python/trunk/Lib/shutil.py", line 52, in copyfile fsrc = open(src, 'rb') IOError: [Errno 2] No such file or directory: '/Users/skip/src/python/trunk/regular/Modules/xxmodule.c' test_multiprocessing test test_multiprocessing failed -- Traceback (most recent call last): File "/Users/skip/src/python/trunk/Lib/test/test_multiprocessing.py", line 1223, in test_connection res = conn.recv_bytes_into(buffer) TypeError: argument 1 must be pinned buffer, not bytearray 2 tests failed: test_distutils test_multiprocessing The former is almost certainly because I'm building in a subdirectory of my trunk sandbox. (Doesn't anybody else build that way these days? I routinely encounter problems with this setup.) Looks like distutils.sysconfig.project_base is set incorrectly: % ./python.exe Python 2.6b3+ (trunk:66009M, Aug 24 2008, 07:17:12) [GCC 4.0.1 (Apple Inc. build 5465)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> import sys >>> project_base = os.path.dirname(os.path.abspath(sys.executable)) >>> project_base '/Users/skip/src/python/trunk/regular' I think project_base should be /Users/skip/src/python/trunk, not .../trunk/regular. Skip From michele.simionato at gmail.com Sun Aug 24 12:19:54 2008 From: michele.simionato at gmail.com (michele.simionato at gmail.com) Date: Sun, 24 Aug 2008 03:19:54 -0700 (PDT) Subject: [Python-Dev] Things to Know About Super Message-ID: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> Recently I have opened a blog on Artima and I am publishing a few Python-related essays I had in store. In particular a trilogy of papers about "super". From the foreword: """ In 2004 I decided to write a comprehensive paper documenting ``super`` pitfalls and traps, with the goal of publishing it on the Python web site, just as I had published my essay on multiple inheritance and the `Method Resolution Order`_. With time the paper grew longer and longer but I never had the feeling that I had covered everything I needed to say: moreover I have a full time job, so I never had the time to fully revise the paper as a whole. As a consequence, four years have passed and the paper is still in draft status. This is a pity, since it documents issues that people encounter and that regularly come out on the Python newsgroups and forums. Keeping the draft sitting on my hard disk is doing a disservice to the community. Still, I lack to time to finish it properly. To come out from the impasse, I decided to split the long paper in a series of short blog posts, which I do have the time to review properly. Moreover people are free to post comments and corrections in case I am making mistakes (speaking about ``super`` this is always possible). Once I finish the series, I may integrate the corrections, put it together again and possibly publish it as whole on the Python website. In other words, in order to finish the task, I am trying the strategies of *divide et conquer* and *release early, release often*. We will see how it goes. """ It seems the strategy worked since I have finished the paper, by splitting it in three post. Actually, it it not really finished because it does not say anything about the new super in Python 3.0, but that could do in a separate essay. Here I am asking for feedback/corrections. Moreover if people think it is worthy idea, I can re-assemble the paper again, donate it to the PSF and publish it on the Python website. Michele Simionato From fredrik at pythonware.com Sun Aug 24 15:26:45 2008 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun, 24 Aug 2008 15:26:45 +0200 Subject: [Python-Dev] "error: None" when building extensions under 2.6 In-Reply-To: References: Message-ID: Thomas Heller wrote: >> I'm beginning to suspect that I have a botched VS install on this >> machine, though. I'll investigate. > > Do you get a traceback when you set the DISTUTILS_DEBUG environment > variable? Indeed I do: ... File "c:\python26\lib\distutils\msvc9compiler.py", line 436, in compile self.initialize() File "c:\python26\lib\distutils\msvc9compiler.py", line 347, in initialize vc_env = query_vcvarsall(VERSION, plat_spec) File "c:\python26\lib\distutils\msvc9compiler.py", line 239, in query_vcvarsall raise IOError("Unable to find vcvarsall.bat") Oops. Guess that's what you get for having too many Windows development boxes ;-) (a nicer error message in non-debug mode would be a good thing, I think.) PS. Can any resident Microsoft compiler expert perhaps summarize the differences between the Express Edition and the "real" editions wrt. generated code? Are there any differences at all? From lists at cheimes.de Sun Aug 24 15:36:38 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 24 Aug 2008 15:36:38 +0200 Subject: [Python-Dev] "error: None" when building extensions under 2.6 In-Reply-To: References: Message-ID: <48B163E6.5090005@cheimes.de> Fredrik Lundh wrote: > PS. Can any resident Microsoft compiler expert perhaps summarize the > differences between the Express Edition and the "real" editions wrt. > generated code? Are there any differences at all? The express edition does neither support PGO (profile guided optimization) nor cross compilation of 64bit binaries on 32bit platforms. The other missing features are mostly uninteresting from the PoV of a extension developer. From wikipedia: Visual C++ 2008 Express does not include OpenMP support, 64-bit compilers, or a resource editor. The higher-end commercial editions of Visual Studio, specifically the Professional and Team Suite editions, have these features. From MS: http://msdn.microsoft.com/en-us/library/hs24szh9.aspx Christian From matt.giuca at gmail.com Sun Aug 24 15:43:01 2008 From: matt.giuca at gmail.com (Matt Giuca) Date: Sun, 24 Aug 2008 23:43:01 +1000 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> Message-ID: Hi Michele, Do you have a URL for this blog? Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt.giuca at gmail.com Sun Aug 24 17:17:16 2008 From: matt.giuca at gmail.com (Matt Giuca) Date: Mon, 25 Aug 2008 01:17:16 +1000 Subject: [Python-Dev] Fwd: Things to Know About Super In-Reply-To: References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <6cd7c525-dd2e-4389-b5ec-73617bfefbf1@m45g2000hsb.googlegroups.com> Message-ID: Had a brief offline discussion with Michele - forwarding. ---------- Forwarded message ---------- From: michele.simionato at gmail.com Date: Mon, Aug 25, 2008 at 12:13 AM On Aug 24, 3:43 pm, "Matt Giuca" wrote: > Hi Michele, > > Do you have a URL for this blog? Sorry, here it is: http://www.artima.com/weblogs/index.jsp?blogger=micheles ---------- Forwarded message ---------- From: Matt Giuca Date: Mon, Aug 25, 2008 at 1:15 AM I skimmed (will read in detail later). As an "intermediate" (I'll describe myself as) Python developer, I tend not to use/understand super (I just call baseclassname.methodname(self,...) directly, so I guess I'm the target audience of this article. It's good - very informative and thorough. It's a bit too informal, personal, and opinionative to be used as "documentation" IMHO but it could certainly be cleaned up without being rewritten. Of interest though, is this: "The first sentence is just plain wrong: super does not return the superclass." >From what I remember of using super, this statement is true, and the documentation is wrong (or at least over-simplifies things). http://docs.python.org/dev/library/functions.html#super http://docs.python.org/dev/3.0/library/functions.html#super Perhaps this should be amended? (A brief statement to the effect of super creating a proxy object which can call the methods of any base class). It actually mentions the "super object" later, even though it claims to be returning the superclass. Also Michele, looks as if super in Python 3 works about the same but has the additional feature of supporting 0 arguments, in which case it defaults to super(this_class, first_arg). (Does not create unbound super objects). Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From nnorwitz at gmail.com Sun Aug 24 19:25:55 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 24 Aug 2008 10:25:55 -0700 Subject: [Python-Dev] performance In-Reply-To: References: Message-ID: Thanks Antoine! On Sun, Aug 24, 2008 at 5:58 AM, Antoine Pitrou wrote: > Neal Norwitz gmail.com> writes: >> Can someone (else) compare performance of 2.5, 2.6, and 3.0? > > Tests done on a 32-bit Linux installation on an Athlon 3600+ X2. Compiled with > gcc in UCS2 mode. > > pystone > ------- > > - 2.5: 43859.6 pystones/second > - 2.6: 42016.8 pystones/second > - 3.0: 38759.7 pystones/second So 3.0 is about 10% slower than 2.x. Given all the changes, that doesn't seem too bad. Though, see below. It looks like at least class attribute lookup is much, much slower. Can you investigate some of these and file bugs as appropriate? > richards.py > ----------- > > (roughly, richards is an object-oriented method-calling benchmark) > > - 2.5: 770.54 ms per iteration > - 2.6: 572.84 ms per iteration > - 3.0: 566.69 ms per iteration I'm a little concerned about why the big change here. Though if I'm reading this right it's a speed up...or am I just being optimistic? The only things I remember that changed between 2.5 and 2.6 that might affect this (without looking at any code) were: 1) Armin's method caching, and 2) the slowdown to isinstance/issubclass which has a release blocker: http://bugs.python.org/issue2534 Can you dig into this and see what caused the slowdown? > > stringbench > ----------- > > - 2.5: unicode: 265.84 s / bytes: 180.50 s > - 2.6: unicode: 256.22 s / bytes: 184.45 s > - 3.0: unicode: 248.07 s / bytes: not tested > > pybench: 2.6 vs. 2.5 > -------------------- > > "this" is 2.6, "other" is 2.5. Hopefully anything >10% is a real change and not just noise. > Test minimum run-time average run-time > this other diff this other diff > ------------------------------------------------------------------------------- > BuiltinFunctionCalls: 177ms 181ms -2.2% 178ms 182ms -2.1% > BuiltinMethodLookup: 157ms 177ms -11.1% 158ms 177ms -11.0% Maybe explained by Armin's patch. > CompareFloats: 171ms 171ms -0.3% 171ms 171ms -0.3% > CompareFloatsIntegers: 158ms 162ms -2.1% 160ms 167ms -4.8% > CompareIntegers: 208ms 208ms -0.0% 209ms 209ms -0.3% > CompareInternedStrings: 189ms 183ms +3.4% 189ms 184ms +2.7% > CompareLongs: 154ms 154ms -0.2% 154ms 154ms -0.2% > CompareStrings: 162ms 159ms +1.7% 163ms 159ms +2.5% > CompareUnicode: 144ms 149ms -3.6% 144ms 152ms -5.0% > ComplexPythonFunctionCalls: 173ms 242ms -28.5% 176ms 243ms -27.6% Maybe explained by Armin's patch. > ConcatStrings: 196ms 198ms -0.9% 203ms 200ms +1.4% > ConcatUnicode: 153ms 151ms +1.5% 155ms 157ms -1.5% > CreateInstances: 172ms 169ms +1.5% 173ms 170ms +1.4% > CreateNewInstances: 131ms 148ms -11.6% 132ms 151ms -12.2% Maybe explained by Armin's patch. > CreateStringsWithConcat: 209ms 206ms +1.4% 209ms 208ms +0.8% > CreateUnicodeWithConcat: 128ms 124ms +3.7% 129ms 124ms +3.4% > DictCreation: 115ms 149ms -22.8% 116ms 150ms -22.8% Why? What changed? > DictWithFloatKeys: 208ms 207ms +0.4% 208ms 208ms +0.0% > DictWithIntegerKeys: 173ms 173ms -0.0% 174ms 173ms +0.5% > DictWithStringKeys: 162ms 162ms -0.1% 162ms 162ms +0.1% > ForLoops: 181ms 181ms -0.2% 181ms 182ms -0.2% > IfThenElse: 169ms 168ms +0.2% 169ms 169ms -0.2% > ListSlicing: 109ms 108ms +0.2% 109ms 109ms +0.1% > NestedForLoops: 198ms 197ms +0.2% 198ms 197ms +0.1% > NormalClassAttribute: 176ms 172ms +2.0% 176ms 173ms +1.7% > NormalInstanceAttribute: 162ms 161ms +1.0% 163ms 161ms +1.1% > PythonFunctionCalls: 161ms 151ms +6.6% 162ms 153ms +6.0% > PythonMethodCalls: 188ms 189ms -0.7% 189ms 193ms -2.0% > Recursion: 235ms 230ms +2.2% 236ms 233ms +1.2% > SecondImport: 113ms 114ms -1.0% 115ms 115ms +0.0% > SecondPackageImport: 120ms 116ms +3.6% 120ms 117ms +2.9% > SecondSubmoduleImport: 154ms 146ms +6.0% 156ms 148ms +5.7% > SimpleComplexArithmetic: 163ms 151ms +8.3% 163ms 151ms +8.6% > SimpleDictManipulation: 177ms 173ms +2.6% 180ms 174ms +3.2% > SimpleFloatArithmetic: 169ms 164ms +3.0% 169ms 166ms +2.2% > SimpleIntFloatArithmetic: 153ms 152ms +1.2% 156ms 153ms +1.8% > SimpleIntegerArithmetic: 156ms 152ms +2.5% 156ms 153ms +2.0% > SimpleListManipulation: 156ms 158ms -1.0% 157ms 158ms -0.5% > SimpleLongArithmetic: 159ms 150ms +6.1% 161ms 151ms +6.3% > SmallLists: 156ms 154ms +1.1% 160ms 155ms +3.0% > SmallTuples: 156ms 155ms +0.4% 157ms 156ms +0.8% > SpecialClassAttribute: 173ms 172ms +0.6% 173ms 172ms +0.7% > SpecialInstanceAttribute: 202ms 198ms +2.4% 203ms 199ms +2.2% > StringMappings: 164ms 170ms -3.2% 165ms 171ms -3.9% > StringPredicates: 160ms 185ms -13.6% 160ms 186ms -13.9% Maybe explained by Armin's patch. > StringSlicing: 169ms 178ms -5.3% 174ms 180ms -3.4% > TryExcept: 181ms 184ms -1.5% 181ms 184ms -1.5% > TryFinally: 157ms 158ms -0.3% 159ms 161ms -1.2% > TryRaiseExcept: 183ms 122ms +49.6% 184ms 124ms +48.2% Whoa, that's a big slowdown. I wonder if it's consistent? > TupleSlicing: 142ms 140ms +1.5% 144ms 141ms +2.0% > UnicodeMappings: 198ms 190ms +4.1% 198ms 190ms +4.1% > UnicodePredicates: 157ms 175ms -10.2% 157ms 176ms -10.6% ? > UnicodeProperties: 161ms 161ms +0.1% 162ms 170ms -5.0% > UnicodeSlicing: 148ms 153ms -3.3% 151ms 155ms -2.6% > WithFinally: 191ms 203ms -5.9% 193ms 208ms -7.5% > WithRaiseExcept: 151ms 160ms -5.8% 152ms 167ms -9.0% > ------------------------------------------------------------------------------- > Totals: 9287ms 9363ms -0.8% 9351ms 9455ms -1.1% > > > pybench: 3.0 > ------------ > > Not entirely comparable with the above since some tests disappeared. > > Test minimum average operation overhead > ------------------------------------------------------------------------------- > BuiltinFunctionCalls: 134ms 134ms 0.26us 0.631ms > BuiltinMethodLookup: 127ms 127ms 0.12us 0.738ms > CompareFloats: 174ms 174ms 0.14us 0.845ms > CompareFloatsIntegers: 274ms 274ms 0.30us 0.630ms Much slower, but probably due to switch from int -> long. There could be potential for optimizing this case. > CompareIntegers: 277ms 277ms 0.15us 1.272ms That's really slow! > CompareInternedStrings: 261ms 261ms 0.17us 3.201ms Much slower, but probably str -> unicode conversion. > CompareLongs: 162ms 162ms 0.15us 0.736ms But this is comparable to 2.x. I don't understand that? > CompareStrings: 171ms 173ms 0.17us 2.188ms > ComplexPythonFunctionCalls: 172ms 178ms 0.89us 1.073ms > ConcatStrings: 260ms 273ms 0.55us 1.222ms This is much slower in 3.0, even taking into account str -> unicode? > CreateInstances: 178ms 178ms 1.59us 0.944ms > CreateNewInstances: 133ms 135ms 1.61us 0.759ms > CreateStringsWithConcat: 253ms 256ms 0.26us 2.127ms > DictCreation: 118ms 118ms 0.29us 0.845ms > DictWithFloatKeys: 207ms 215ms 0.24us 1.593ms > DictWithIntegerKeys: 174ms 176ms 0.15us 2.132ms > DictWithStringKeys: 158ms 158ms 0.13us 2.127ms > ForLoops: 188ms 188ms 7.51us 0.097ms > IfThenElse: 212ms 212ms 0.16us 1.597ms Slower? > ListSlicing: 111ms 111ms 7.96us 0.165ms > NestedForLoops: 218ms 220ms 0.15us 0.001ms > NormalClassAttribute: 339ms 340ms 0.28us 1.111ms Over twice as slow? > NormalInstanceAttribute: 186ms 186ms 0.16us 1.116ms > PythonFunctionCalls: 156ms 158ms 0.48us 0.633ms > PythonMethodCalls: 200ms 201ms 0.89us 0.374ms > Recursion: 259ms 260ms 5.21us 1.059ms > SecondImport: 128ms 128ms 1.28us 0.417ms > SecondPackageImport: 140ms 143ms 1.43us 0.417ms > SecondSubmoduleImport: 200ms 200ms 2.00us 0.417ms > SimpleComplexArithmetic: 142ms 144ms 0.16us 0.844ms > SimpleDictManipulation: 280ms 285ms 0.24us 1.059ms Slower. str -> unicode? > SimpleFloatArithmetic: 161ms 161ms 0.12us 1.275ms > SimpleIntFloatArithmetic: 199ms 213ms 0.16us 1.272ms > SimpleIntegerArithmetic: 200ms 209ms 0.16us 1.301ms > SimpleListManipulation: 165ms 166ms 0.14us 1.379ms > SimpleLongArithmetic: 132ms 137ms 0.21us 0.630ms > SmallLists: 188ms 191ms 0.28us 0.844ms > SmallTuples: 197ms 199ms 0.37us 0.951ms > SpecialClassAttribute: 534ms 535ms 0.45us 1.121ms ~4x slower! > SpecialInstanceAttribute: 185ms 186ms 0.15us 1.125ms > StringMappings: 471ms 471ms 1.87us 0.884ms ~3X slower. > StringPredicates: 190ms 191ms 0.27us 4.315ms > StringSlicing: 285ms 290ms 0.52us 1.881ms > TryExcept: 182ms 182ms 0.08us 1.663ms > TryFinally: 134ms 136ms 0.85us 0.882ms > TryRaiseExcept: 137ms 138ms 2.15us 0.885ms > TupleSlicing: 202ms 204ms 0.78us 0.110ms > WithFinally: 185ms 188ms 1.18us 0.881ms > WithRaiseExcept: 274ms 275ms 3.44us 1.105ms Slower. > ------------------------------------------------------------------------------- > Totals: 10011ms 10120ms > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > From solipsis at pitrou.net Sun Aug 24 21:04:16 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 24 Aug 2008 21:04:16 +0200 Subject: [Python-Dev] performance In-Reply-To: References: Message-ID: <1219604656.5635.2.camel@fsol> Hi, > So 3.0 is about 10% slower than 2.x. Given all the changes, that > doesn't seem too bad. Yes, I think it's rather good. > > - 2.5: 770.54 ms per iteration > > - 2.6: 572.84 ms per iteration > > - 3.0: 566.69 ms per iteration > > I'm a little concerned about why the big change here. Though if I'm > reading this right it's a speed up...or am I just being optimistic? Yes, it is a speed up. It is thanks to Armin's method cache. > > ComplexPythonFunctionCalls: 173ms 242ms -28.5% 176ms 243ms -27.6% > > Maybe explained by Armin's patch. Actually it is I and Raymond's patch speeding up function calls with named parameters. > > DictCreation: 115ms 149ms -22.8% 116ms 150ms -22.8% > > Why? What changed? The opcode sequence for creation of dict literals has been optimized. > > StringPredicates: 160ms 185ms -13.6% 160ms 186ms -13.9% > > Maybe explained by Armin's patch. I don't think so. > > TryRaiseExcept: 183ms 122ms +49.6% 184ms 124ms +48.2% > > Whoa, that's a big slowdown. I wonder if it's consistent? Yes, I can definitely reproduce it. > > UnicodePredicates: 157ms 175ms -10.2% 157ms 176ms -10.6% > > ? Probably thanks to the speedup in whitespace detection. > > CompareFloatsIntegers: 274ms 274ms 0.30us 0.630ms > > Much slower, but probably due to switch from int -> long. There could > be potential for optimizing this case. Well honestly you don't often compare different types. I think the most common exception to this rule would be None vs. non-None. > > CompareIntegers: 277ms 277ms 0.15us 1.272ms > > That's really slow! > > > CompareInternedStrings: 261ms 261ms 0.17us 3.201ms > > Much slower, but probably str -> unicode conversion. > > > CompareLongs: 162ms 162ms 0.15us 0.736ms > > But this is comparable to 2.x. I don't understand that? I think CompareIntegers and CompareLongs aren't the same tests exactly, although they test the same operations. I've already proposed a patch to speedup comparisons in py3k: http://bugs.python.org/issue3106 > > ConcatStrings: 260ms 273ms 0.55us 1.222ms > > This is much slower in 3.0, even taking into account str -> unicode? The number of rounds is not the same (60000 for 2.6 with unicode, 100000 for 3.0 with str). It should probably be fixed. > > IfThenElse: 212ms 212ms 0.16us 1.597ms > > Slower? It is based on integer comparisons though. > > NormalClassAttribute: 339ms 340ms 0.28us 1.111ms > > Over twice as slow? Yes, should be investigated. > > SimpleDictManipulation: 280ms 285ms 0.24us 1.059ms > > Slower. str -> unicode? No, there are just integers. > > SpecialClassAttribute: 534ms 535ms 0.45us 1.121ms > > ~4x slower! Should be investigated as well. > > StringMappings: 471ms 471ms 1.87us 0.884ms > > ~3X slower. It's normal. This test measures functions like upper(), etc., which are much expensive using the full unicode database than when doing a straight lookup in a 256-entry table. > > WithFinally: 185ms 188ms 1.18us 0.881ms > > WithRaiseExcept: 274ms 275ms 3.44us 1.105ms > > Slower. Yes, the new exception semantics have a cost. Regards Antoine. From martin at v.loewis.de Sun Aug 24 21:35:24 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 24 Aug 2008 21:35:24 +0200 Subject: [Python-Dev] Unicode 5.1.0 In-Reply-To: <368a5cd50808220347o403844d3m9dc856184ac7da79@mail.gmail.com> References: <48ADDD94.2080903@egenix.com> <368a5cd50808220347o403844d3m9dc856184ac7da79@mail.gmail.com> Message-ID: <48B1B7FC.60706@v.loewis.de> > is the suggestion to *replace* the 4.1.0 database with a 5.1.0 > database, or to add yet another database in that module? I would replace it. > (how's the 3.2/4.1 dual support implemented? The compiler needs data files for all supported versions, with old_versions listing the, well, old versions. It then computes deltas, expecting that they should mostly consist of new assignments (i.e. characters unassigned in 3.2 might be assigned in newer versions). It detects all differences, but might not be able to represent all changes. > do we have two distinct > datasets, or are the differences encoded in some clever way? The latter. It doesn't really need to be that clever: primarily just a compressed list of "new" characters is needed, per version. > would it > make sense to split the unicodedata module into three separate > modules, one for each major Unicode version?) You couldn't use the space savings then, I suppose. Regards, Martin From martin at v.loewis.de Sun Aug 24 21:40:29 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 24 Aug 2008 21:40:29 +0200 Subject: [Python-Dev] Unicode 5.1.0 In-Reply-To: References: <48ADDD94.2080903@egenix.com> <368a5cd50808220347o403844d3m9dc856184ac7da79@mail.gmail.com> Message-ID: <48B1B92D.3090204@v.loewis.de> > That's up to us. I don't know what the reason was for keeping the > 3.2.0 database around -- does anyone here recall ever using it? For > what? It's needed for IDNA. The IDNA RFC requires that Unicode 3.2 is used for performing IDNA (in particular, for determining what a valid domain name is). The IDNA people consider it security-relevant that it is really the 3.2 database, and would probably consider it a serious security bug if newer Python versions suddenly started to use newer Unicode databases for IDNA. At some point, IDNA might get updated to a newer version of the Unicode spec; we can then drop 3.2 (and stick with whatever the RFC then specifies). Regards, Martin From martin at v.loewis.de Sun Aug 24 21:44:04 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 24 Aug 2008 21:44:04 +0200 Subject: [Python-Dev] Unicode 5.1.0 In-Reply-To: <368a5cd50808220813r6e48c6a5w25413e1a1c17946a@mail.gmail.com> References: <48ADDD94.2080903@egenix.com> <368a5cd50808220347o403844d3m9dc856184ac7da79@mail.gmail.com> <368a5cd50808220813r6e48c6a5w25413e1a1c17946a@mail.gmail.com> Message-ID: <48B1BA04.40805@v.loewis.de> > I can tinker a little with this over the weekend, unless Martin tells > me not to ;-) Go ahead; I can't work on this at the moment, anyway. I would also be confident that a mere replacement of 4.1 with 5.1 should be easy, and I see no reason to keep the 4.1 version. Perhaps makeunicodedata should list *why* certain old versions remain supported; for 3.2, the use case is IDNA. Regards, Martin From solipsis at pitrou.net Sun Aug 24 21:46:13 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 24 Aug 2008 19:46:13 +0000 (UTC) Subject: [Python-Dev] IDNA References: <48ADDD94.2080903@egenix.com> <368a5cd50808220347o403844d3m9dc856184ac7da79@mail.gmail.com> <48B1B92D.3090204@v.loewis.de> Message-ID: Martin v. L?wis v.loewis.de> writes: > > It's needed for IDNA. The IDNA RFC requires that Unicode 3.2 is used > for performing IDNA (in particular, for determining what a valid domain > name is). Speaking of which, Martin, did you take a look at http://bugs.python.org/issue3232 ? I suppose the fix is trivial, but I don't know what it should be :-) Regards Antoine. From g.brandl at gmx.net Mon Aug 25 00:37:19 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 25 Aug 2008 00:37:19 +0200 Subject: [Python-Dev] performance In-Reply-To: <1219604656.5635.2.camel@fsol> References: <1219604656.5635.2.camel@fsol> Message-ID: Antoine Pitrou schrieb: > Hi, pystone ------- - 2.5: 43859.6 pystones/second - 2.6: 42016.8 pystones/second - 3.0: 38759.7 pystones/second >> So 3.0 is about 10% slower than 2.x. Given all the changes, that >> doesn't seem too bad. > > Yes, I think it's rather good. Well, pystone really doesn't test much of what causes the largest slowdowns in 3.0... Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From steve at pearwood.info Mon Aug 25 01:54:48 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 25 Aug 2008 09:54:48 +1000 Subject: [Python-Dev] performance In-Reply-To: <1219604656.5635.2.camel@fsol> References: <1219604656.5635.2.camel@fsol> Message-ID: <200808250954.49208.steve@pearwood.info> On Mon, 25 Aug 2008 05:04:16 am Antoine Pitrou wrote: > > > ? ? ? ? CompareFloatsIntegers: ? ?274ms ? ?274ms ? ?0.30us ? > > > ?0.630ms > > > > Much slower, but probably due to switch from int -> long. ?There > > could be potential for optimizing this case. > > Well honestly you don't often compare different types. I think the > most common exception to this rule would be None vs. non-None. Surely the most common None vs non-None comparison would be the "is" operator, which I hope remains fast. I don't know about other people, but I often compare floats with ints. Here's a contrived example: x = some_float() if x == -1: return -2 else: return (x**2-1)/(x+1) I suppose it's no hardship to start writing float literals instead of int literals, if needed. -- Steven From nnorwitz at gmail.com Mon Aug 25 03:15:06 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 24 Aug 2008 18:15:06 -0700 Subject: [Python-Dev] what version of bsddb to use/accept for 2.6/3.0 Message-ID: Many buildbots are running bsddb 4.7, particularly the debian/ubuntu ones (4.7.25 which seems to be the latest). Some of them are crashing, others are not. The max version we support in both 2.6 and 3.0 is 4.7. Should we allow this version or should we use a lower maximum that is more likely to work (ie, not crash)? It looks like the WIndows buildbots use 4.4.20. Unfortunately, the Windows bots aren't in great shape either. Additionally, there are reference leaks in both 2.6 and 3.0: test_bsddb3 leaked [80, 80] references, sum=160 (2.6) test_bsddb3 leaked [63, 63] references, sum=126 (3.0) It would be nice to get as many of these things fixed up before release. Jesus, Greg, Trent, can you make some time over the next week to stabilize bsddb support? Thanks, n PS. To change the max version of bsddb supported in Unix, modify max_db_ver in setup.py. From doko at ubuntu.com Mon Aug 25 03:21:02 2008 From: doko at ubuntu.com (Matthias Klose) Date: Mon, 25 Aug 2008 03:21:02 +0200 Subject: [Python-Dev] what version of bsddb to use/accept for 2.6/3.0 In-Reply-To: References: Message-ID: <48B208FE.8050600@ubuntu.com> [CCing Clint] 4.7.25 generally looks fine, except for sparc (afaics); I asked the Debian bsddb maintainer to have a look at the sparc problems. At least on sparc, there's no difference between 4.6 and 4.7. I usually try to run the buildbots with the recent software in the distribution, as this is what users will see with the next release. Matthias PS: currently travelling, won't be able to answer in time. Neal Norwitz schrieb: > Many buildbots are running bsddb 4.7, particularly the debian/ubuntu > ones (4.7.25 which seems to be the latest). Some of them are > crashing, others are not. The max version we support in both 2.6 and > 3.0 is 4.7. Should we allow this version or should we use a lower > maximum that is more likely to work (ie, not crash)? > > It looks like the WIndows buildbots use 4.4.20. Unfortunately, the > Windows bots aren't in great shape either. > > Additionally, there are reference leaks in both 2.6 and 3.0: > test_bsddb3 leaked [80, 80] references, sum=160 (2.6) > test_bsddb3 leaked [63, 63] references, sum=126 (3.0) > > It would be nice to get as many of these things fixed up before > release. Jesus, Greg, Trent, can you make some time over the next > week to stabilize bsddb support? > > Thanks, > n > PS. To change the max version of bsddb supported in Unix, modify > max_db_ver in setup.py. From michele.simionato at gmail.com Sun Aug 24 16:12:18 2008 From: michele.simionato at gmail.com (michele.simionato at gmail.com) Date: Sun, 24 Aug 2008 07:12:18 -0700 (PDT) Subject: [Python-Dev] Things to Know About Super In-Reply-To: References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> Message-ID: <922fecbe-e873-4df9-ada1-52730ba53388@79g2000hsk.googlegroups.com> On Aug 24, 3:43?pm, "Matt Giuca" wrote: > Hi Michele, > > Do you have a URL for this blog? > Sorry, here it is: http://www.artima.com/weblogs/index.jsp?blogger=micheles From robert.schuppenies at gmail.com Mon Aug 25 10:32:48 2008 From: robert.schuppenies at gmail.com (Robert Schuppenies) Date: Mon, 25 Aug 2008 10:32:48 +0200 Subject: [Python-Dev] no tp_traverse for dict iters Message-ID: <48B26E30.3020701@gmail.com> Hi. Could you please explain to me why some iterators have a tp_traverse implementation and others do not? For example tupleiterator has one, but none of the dict iterators. Same for set iterators (and possibly others). It shows in Python when you use the get_referents function. >>> t = (1,2,3) >>> gc.get_referents(iter(t)) [(1, 2, 3)] >>> s = set([1,2,3]) >>> gc.get_referents(iter(s)) [] >>> d = {1:1, 2:2} >>> gc.get_referents(iter(d)) [] And is it correct that you can rephrase the question to 'why some iterators are seen as container objects and others are not'? thanks, robert From barry at barrys-emacs.org Mon Aug 25 13:18:50 2008 From: barry at barrys-emacs.org (Barry Scott) Date: Mon, 25 Aug 2008 12:18:50 +0100 Subject: [Python-Dev] Should we help pythonmac.org? In-Reply-To: <6a36e7290808181406w24b8b836wb9619227c0e67333@mail.gmail.com> References: <59C86CC6-D154-4704-81B9-8AB3CB408E6F@python.org> <6a36e7290808181406w24b8b836wb9619227c0e67333@mail.gmail.com> Message-ID: <19CF9B30-6D42-4D88-AEC2-49DF0F1AB546@barrys-emacs.org> On Aug 18, 2008, at 22:06, Bob Ippolito wrote: > The major difference between the packages on macports and > pythonmac.org is that macports is their own distro of nearly > everything, akin to installing a copy of FreeBSD over top of Mac OS X. > pythonmac.org contains packages that are self-contained and don't have > a whole new set of libraries to install (in the cases where they do > require libraries, they link them in statically for the most part). > Theses comments are from my point of view as a developer for end users on the Mac. For my pysvn project I release kits for all the python versions * Apple python 2.3 powerpc * Apple python 2.4 intel * MacPython 2.4.x powerpc * MacPython 2.5.x powerpc and intel And in the future I intend to support: * 2.6b3 framework build * 3.0b3 framework build These are all framework builds and they are totally isolated from each other. And I release pysvn as DMG kits as well. For pysvn workbench I use the wxWidgets released DMG kits. And create a application using python module bundlebuilder. Whatever you do should point to a framework build including bundelbuilder which is installed using a Mac installer. Which is the status quo so far. Barry From amauryfa at gmail.com Mon Aug 25 13:22:10 2008 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Mon, 25 Aug 2008 13:22:10 +0200 Subject: [Python-Dev] no tp_traverse for dict iters In-Reply-To: <48B26E30.3020701@gmail.com> References: <48B26E30.3020701@gmail.com> Message-ID: Hello, 2008/8/25 Robert Schuppenies : > Hi. > > Could you please explain to me why some iterators have a tp_traverse > implementation and others do not? For example tupleiterator has one, > but none of the dict iterators. Same for set iterators (and possibly > others). It shows in Python when you use the get_referents function. > >>>> t = (1,2,3) >>>> gc.get_referents(iter(t)) > [(1, 2, 3)] >>>> s = set([1,2,3]) >>>> gc.get_referents(iter(s)) > [] >>>> d = {1:1, 2:2} >>>> gc.get_referents(iter(d)) > [] > > And is it correct that you can rephrase the question to 'why some > iterators are seen as container objects and others are not'? Yes, this can lead to some object cycle that are not collected. See the attached script: a cycle involving a list iterator is collected by the garbage collector, but a cycle with a dict iterator is not. This is worth a bug report IMO. -- Amaury Forgeot d'Arc -------------- next part -------------- A non-text attachment was scrubbed... Name: cycle.py Type: text/x-python Size: 740 bytes Desc: not available URL: From barry at python.org Mon Aug 25 14:50:01 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 25 Aug 2008 08:50:01 -0400 Subject: [Python-Dev] Unicode 5.1.0 In-Reply-To: References: Message-ID: <33B21715-8C49-4AE2-BB7D-8829264DCC64@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I was away for the weekend and am struggling to catch up on my email. Since I haven't digested this entire thread, I'll refrain for the moment from giving my opinion, however this comment jumped out to me. On Aug 22, 2008, at 9:42 AM, Facundo Batista wrote: > - We always could have a beta4 if it's necessary... I do not want to slip the schedule if at all possible. If serious security issues, performance problems, show stopper bugs crop up, then we will obviously slip so that we don't have to put a brown bag over our heads. Slipping to get yet one more feature in is not (IMO) acceptable. An incentive for keeping the schedule: If we hit our October 1st deadline, then 2.6 and 3.0 will almost certainly be included in some upcoming major new OS releases. If we slip, then it's unlikely to happen. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSLKqenEjvBPtnXfVAQJukAP+L93nxTP436Au9GkLZQUhy1Gbk8rDvq2K jZtJA5Rb9VKUr7TDoqZ2iFRRg9tsxwz+fLzZp0m00WWGRvKHdgqS+c6sHBaXazzk txFhyspkw0cndD7zsNoqThlY6Q1CkhK3BHYmRLWS+PVhfOm6bRgudL+ePcWneT2X 24pFB83GSjo= =/lq8 -----END PGP SIGNATURE----- From horvath.arpad at roik.bmf.hu Mon Aug 25 15:19:56 2008 From: horvath.arpad at roik.bmf.hu (harp) Date: Mon, 25 Aug 2008 06:19:56 -0700 (PDT) Subject: [Python-Dev] Doc translation -- from LaTeX to reStructuredText In-Reply-To: <1afaf6160808221118p3a6f032btf0e93183798362a7@mail.gmail.com> References: <1afaf6160808221118p3a6f032btf0e93183798362a7@mail.gmail.com> Message-ID: <06ddf912-ef51-47d4-bbd4-783b0da08ab8@a1g2000hsb.googlegroups.com> Not succeeded. I have done these steps: - I downloaded the tutorial in LaTeX. (http://www.python.org/doc/2.5.2/ download/ ) - Extracted. - Put the hungarian tut.tex into the tut directory - downloaded the files from http://svn.python.org/view/doctools/converter - Run the `convert.py . ../pydoc` in the /Python-Docs-2.5.2 directory - Because of some errors* I leave only the 'tut' item in the subdirs in converter/__init__.py (for subdir in...) - Run again. - In the pydoc/tutorial directory there is only an index.rst file. No appetite.rst and so on. And the index.rst is english as well. *The error message mentioned above: (...) lib/libal.tex lib/liballos.tex Traceback (most recent call last): File "./converter/convert.py", line 26, in convert_dir(destdir, *sys.argv[3:]) File "/home/ha/Munkaasztal/Python-Docs-2.5.2/convert/converter/ __init__.py", line 86, in convert_dir newname = fn_mapping[subdir][filename] KeyError: 'libamoeba' On aug. 22, 20:18, "Benjamin Peterson" wrote: > On Fri, Aug 22, 2008 at 9:03 AM, harp wrote: > > ?Dear Developers, > > > We have translated the tutorial into Hungarian earlier, and I do not > > know how to convert the LaTeX file into reStructuredText. Could > > somebody help us? > > Have you looked at the LaTeX to reST converter athttp://svn.python.org/view/doctools/converter? > > > > > HTML:http://pythonlib.pergamen.hu/html/tut/ > > LaTeX file:http://mail.roik.bmf.hu/linux/python/tutorial/tut.tex > > > There will be included translations of documentation on the python.org > > homepage? > > > ? Arpad .python.org/mailman/options/python-dev/musiccomposition%40... > > -- > Cheers, > Benjamin Peterson > "There's no place like 127.0.0.1." From ismail at namtrac.org Mon Aug 25 15:43:43 2008 From: ismail at namtrac.org (=?UTF-8?Q?=C4=B0smail_D=C3=B6nmez?=) Date: Mon, 25 Aug 2008 16:43:43 +0300 Subject: [Python-Dev] Unicode 5.1.0 In-Reply-To: References: Message-ID: <19e566510808250643v496a138dp64fa45f3eda9ee8d@mail.gmail.com> Hi, On Thu, Aug 21, 2008 at 23:35, Guido van Rossum wrote: > I was just paid a visit by my Google colleague Mark Davis, co-founder > of the Unicode project and the president of the Unicode Consortium. He > would like to see improved Unicode support for Python. (Well duh. :-) > On his list of top priorities are: > > 1. Upgrade the unicodata module to the Unicode 5.1.0 standard > 2. Extende the unicodedata module with some additional properties > 3. Add support for Unicode properties to the regex syntax, including > Boolean combinations Adding support for SpecialCasing rules[0] would be good for full Unicode support too. It would fix i/I problems that are currently going on with Turkish locale. [0] http://unicode.org/Public/UNIDATA/SpecialCasing.txt Regards, ismail -- Programmer Excuse #17: The processor stack spring has worn out. From mal at egenix.com Mon Aug 25 16:06:33 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 25 Aug 2008 16:06:33 +0200 Subject: [Python-Dev] try-except slower in 2.6 (was: performance) In-Reply-To: <1219604656.5635.2.camel@fsol> References: <1219604656.5635.2.camel@fsol> Message-ID: <48B2BC69.3080601@egenix.com> On 2008-08-24 21:04, Antoine Pitrou wrote: >>> TryRaiseExcept: 183ms 122ms +49.6% 184ms 124ms +48.2% >> Whoa, that's a big slowdown. I wonder if it's consistent? > > Yes, I can definitely reproduce it. That's a huge slow-down compared to 2.5. Are there any obvious reasons for this ? Has the exception handling mechanism changed that much between 2.5 and 2.6 ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 25 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From mal at egenix.com Mon Aug 25 16:49:45 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 25 Aug 2008 16:49:45 +0200 Subject: [Python-Dev] Unicode 5.1.0 In-Reply-To: References: <48ADDD94.2080903@egenix.com> Message-ID: <48B2C689.5030503@egenix.com> On 2008-08-22 03:25, Guido van Rossum wrote: > On Thu, Aug 21, 2008 at 2:26 PM, M.-A. Lemburg wrote: >> On 2008-08-21 22:35, Guido van Rossum wrote: >>> I was just paid a visit by my Google colleague Mark Davis, co-founder >>> of the Unicode project and the president of the Unicode Consortium. He >>> would like to see improved Unicode support for Python. (Well duh. :-) >>> On his list of top priorities are: >>> >>> 1. Upgrade the unicodata module to the Unicode 5.1.0 standard >>> 2. Extende the unicodedata module with some additional properties >>> 3. Add support for Unicode properties to the regex syntax, including >>> Boolean combinations >>> >>> I've tried to explain our release schedule and >>> no-new-features-in-point-releases policies to him, and he understands >>> that it's too late to add #2 or #3 to 2.6 and 3.0, and that these will >>> have to wait for 2.7 and 3.1, respectively. However, I've kept the >>> door sligthtly ajar for adding #1 -- it can't be too much work and it >>> can't have too much impact. Or can it? I don't actually know what the >>> impact would be, so I'd like some impact from developers who are >>> closer to the origins of the unicodedata module. >>> >>> The two, quite separate, questions, then, are (a) how much work would >>> it be to upgrade to version 5.1.0 of the database; and (b) would it be >>> acceptable to do this post-beta3 (but before rc1). If the answer to >>> (b) is positive, Google can help with (a). >>> >>> In general, Google has needs in this area that can't wait for 2.7/3.1, >>> so what we may end up doing is create internal implementations of all >>> three features (compatible with Python 2.4 and later), publish them as >>> open source on Google Code, and fold them into core Python at the >>> first opportunity, which would likely be 2.7 and 3.1. >>> >>> Comments? >> There are two things to consider: >> >> unicodedata is just an optimized database for accessing code >> point properties of a specific Unicode version (currently 4.1.0 >> and 3.2.0). Adding support for a new version needs some work on >> the generation script, perhaps keeping the 4.1.0 version of it >> like we did for 3.2.0, but that's about it. >> >> However, there are other implications to consider when moving to >> Unicode 5.1.0. >> >> Just see the top of http://www.unicode.org/versions/Unicode5.1.0/ >> for a summary of changes compared to 5.0, plus >> http://www.unicode.org/versions/Unicode5.0.0/ for changes between >> 4.1.0 and 5.0. >> >> So while we could say: "we provide access to the Unicode 5.1.0 >> database", we cannot say: "we support Unicode 5.1.0", simply because >> we have not reviewed the all the necessary changes and implications. > > Mark's response to this was: > > """ > I'd suspect that you'll be as conformant to U5.1.0 as you were to U4.1.0 ;-) > > More seriously, I don't think this is a roadblock -- I doubt that > there are real differences between U5.1.0 and U4.10 in terms of > conformance that would be touched by Python -- the conformance changes > tend to be either completely backward compatible or very esoteric. > What I can do is to review the Python support to see if and where > there are any problems, but I wouldn't anticipate any. > """ > > Which suggests that he believes that the differences in the database > are very minor, and that upgrading just the database would not cause > any problems for code that worked well with the 4.1.0 database. Fine with me. >> I think it's better to look through all the changes and then come >> up with proper support for 2.7/3.1. If Google wants to contribute >> to this, even better. To avoid duplication of work or heading in >> different directions, it may be a good idea to create a >> unicode-sig to discuss things. > > Not me. :-) I would really like to see more Unicode support in Python, e.g. for collation, compression, indexing based on graphemes and code points, better support for special casing situations (to cover e.g. the dotted vs. non-dotted i in the Turkish scripts), etc. There are also a few changes that we'd need to incorporate into the UTF codecs, e.g. warn about more ill-formed byte sequences. Would Google be willing to contribute such support or part of it ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 25 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From solipsis at pitrou.net Mon Aug 25 17:42:41 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 25 Aug 2008 15:42:41 +0000 (UTC) Subject: [Python-Dev] try-except slower in 2.6 (was: performance) References: <1219604656.5635.2.camel@fsol> <48B2BC69.3080601@egenix.com> Message-ID: M.-A. Lemburg egenix.com> writes: > > That's a huge slow-down compared to 2.5. > > Are there any obvious reasons for this ? Has the exception handling > mechanism changed that much between 2.5 and 2.6 ? I've looked at it a bit and I think it's because of the issubclass()/isinstance() slowdown (which itself is due to ABCs). If I apply the patch in http://bugs.python.org/issue2534, the exception handling slowdown seems to disappear. From guido at python.org Mon Aug 25 18:04:20 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 25 Aug 2008 09:04:20 -0700 Subject: [Python-Dev] Unicode 5.1.0 In-Reply-To: <48B2C689.5030503@egenix.com> References: <48ADDD94.2080903@egenix.com> <48B2C689.5030503@egenix.com> Message-ID: 2008/8/25 M.-A. Lemburg : > I would really like to see more Unicode support in Python, e.g. > for collation, compression, indexing based on graphemes and > code points, better support for special casing situations (to > cover e.g. the dotted vs. non-dotted i in the Turkish scripts), > etc. > > There are also a few changes that we'd need to incorporate into > the UTF codecs, e.g. warn about more ill-formed byte sequences. > > Would Google be willing to contribute such support or part > of it ? That depends purely on how much need Google itself has for these features. I'll ask around, but for now I wouldn't bet on anything beyond the three points I raised at the start of this thread: 1. Upgrade the unicodata module to the Unicode 5.1.0 standard 2. Extende the unicodedata module with some additional properties 3. Add support for Unicode properties to the regex syntax, including Boolean combinations -- --Guido van Rossum (home page: http://www.python.org/~guido/) -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Mon Aug 25 19:13:32 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 25 Aug 2008 10:13:32 -0700 Subject: [Python-Dev] subprocess insufficiently platform-independent? Message-ID: Several people at Google seem to have independently discovered that despite all of the platform-independent goodness in subprocess.py, you still need to be platform aware. One of my colleagues summarized it like this: """ Given a straightforward command list like: cmd = ['svn', 'ls', 'http://rietveld.googlecode.com/svn/trunk'] You apparently cannot pass this list to any subprocess function (subprocess.call() or otherwise) with a set of arguments that allow it to "just work" on both Windows and non-Windows systems. If you call: subprocess.call(cmd, shell=False) Then it works on Linux, but fails on Windows because it does not perform the Windows %PATHEXT% search that allows it to find that "svn.exe" is the actual executable to be invoked. If you call: subprocess.call(cmd, shell=True) Then it works on Windows (it finds the "svn.exe" executable), but it fails on Linux because it *only* executes the first argument in the list ("svn") and does not pass the remaining arguments in the list to the "svn" invocation. This forces you to code platform-dependent behavior in how you call subprocess.call() when using a list. For example, you can make the shell= argument depend on the platform you're running on, like: subprocess.call(cmd, shell=(sys.platform=='win32')) Or you could vary cmd[0] in your list based on the platform: if sys.platform == 'win32': svn = 'svn.exe' else: svn = 'svn' cmd = [svn, 'ls', 'http://rietveld.googlecode.com/svn/trunk'] But either way, this seems clearly broken (or at least sub-optimal) for a module that's supposed to abstract platform-specific execution issues from the user. """ -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Mon Aug 25 19:13:33 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 25 Aug 2008 13:13:33 -0400 Subject: [Python-Dev] Unicode 5.1.0 In-Reply-To: References: <48ADDD94.2080903@egenix.com> <48B2C689.5030503@egenix.com> Message-ID: Guido van Rossum wrote: > 2008/8/25 M.-A. Lemburg >: > > I would really like to see more Unicode support in Python, e.g. > > for collation, compression, indexing based on graphemes and > > code points, better support for special casing situations (to > > cover e.g. the dotted vs. non-dotted i in the Turkish scripts), > > etc. > > > > There are also a few changes that we'd need to incorporate into > > the UTF codecs, e.g. warn about more ill-formed byte sequences. > > > > Would Google be willing to contribute such support or part > > of it ? > > That depends purely on how much need Google itself has for these > features. I'll ask around, but for now I wouldn't bet on anything beyond > the three points I raised at the start of this thread: > > 1. Upgrade the unicodata module to the Unicode 5.1.0 standard > 2. Extende the unicodedata module with some additional properties > 3. Add support for Unicode properties to the regex syntax, including > Boolean combinations I think an Improve Unicode Support PEP would be a good idea to collect (and get approval or not for) various ideas from various people, even if Google only implements part of the PEP. From barry at python.org Mon Aug 25 19:30:58 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 25 Aug 2008 13:30:58 -0400 Subject: [Python-Dev] subprocess insufficiently platform-independent? In-Reply-To: References: Message-ID: <989B2B74-D5AB-4FAA-BE6D-8D409D194388@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 25, 2008, at 1:13 PM, Guido van Rossum wrote: > Several people at Google seem to have independently discovered that > despite all of the platform-independent goodness in subprocess.py, you > still need to be platform aware. One of my colleagues summarized it > like this: > > """ > Given a straightforward command list like: > > cmd = ['svn', 'ls', 'http://rietveld.googlecode.com/svn/trunk'] > > You apparently cannot pass this list to any subprocess function > (subprocess.call() or otherwise) with a set of arguments that allow it > to "just work" on both Windows and non-Windows systems. Unless I'm misremembering (I no longer have access to Windows), I believe that if you use ' '.join(cmd) as the first argument, it will work cross-platform. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSLLsU3EjvBPtnXfVAQJapQP+N4HY0I/uczbdQKB1bi6OV0BZgj5JS8an Tz4FEnaD9LDegTnV8fqAx5/blIidZEdPjVkdmW4m4bz8tRNIEdoZyghHUmKycgRj d65FU0e1tL40u0AoKl3ARO6WWkKKhaqn4R17065lh+V1ZNKutu2btiAso6VfWVW5 V7hvo/61ACM= =P0Nb -----END PGP SIGNATURE----- From phd at phd.pp.ru Mon Aug 25 19:33:46 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Mon, 25 Aug 2008 21:33:46 +0400 Subject: [Python-Dev] subprocess insufficiently platform-independent? In-Reply-To: <989B2B74-D5AB-4FAA-BE6D-8D409D194388@python.org> References: <989B2B74-D5AB-4FAA-BE6D-8D409D194388@python.org> Message-ID: <20080825173346.GA24017@phd.pp.ru> On Mon, Aug 25, 2008 at 01:30:58PM -0400, Barry Warsaw wrote: > Unless I'm misremembering (I no longer have access to Windows), I > believe that if you use ' '.join(cmd) as the first argument, it will > work cross-platform. What about arguments that contain spaces? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From guido at python.org Mon Aug 25 19:34:09 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 25 Aug 2008 10:34:09 -0700 Subject: [Python-Dev] subprocess insufficiently platform-independent? In-Reply-To: <989B2B74-D5AB-4FAA-BE6D-8D409D194388@python.org> References: <989B2B74-D5AB-4FAA-BE6D-8D409D194388@python.org> Message-ID: On Mon, Aug 25, 2008 at 10:30 AM, Barry Warsaw wrote: > Unless I'm misremembering (I no longer have access to Windows), I believe > that if you use ' '.join(cmd) as the first argument, it will work > cross-platform. That will mean something different if there are spaces or shell metacharacters in the arguments. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Mon Aug 25 19:34:39 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 25 Aug 2008 13:34:39 -0400 Subject: [Python-Dev] Unicode 5.1.0 In-Reply-To: References: Message-ID: <57CCEBDE-ABE7-46A9-AD3F-4843FF9E1F98@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 21, 2008, at 6:30 PM, Terry Reedy wrote: > > http://www.unicode.org/versions/Unicode5.1.0/ > "Unicode 5.1.0 contains over 100,000 characters, and provides > significant additions and improvements..." to existing features, > including new files and upgrades to existing files. Sounds close to > adding features ;-) I agree. This seriously feels like new, potentially high risk code to be adding this late in the game. The BDFL can always override, but unless someone is really convincing that this is low risk high benefit, I'd vote no for 2.6/3.0. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSLLtMnEjvBPtnXfVAQKg0wP+LJ1XYXhEQHUAvT3fPbPzStCN8Lb+D7XG hZOANnTCbPGaeCY19B8mYZbXkvjkCBptauKGB5yGOAnb1KCkSaQWx0wCInkeyIFE mVMupGZCUsdsO7KreEwvyhBpOJ/HNY0+eacv8GZKCwC9xW3WmhaOjry7sZFhjffw hAX1AuxaPWA= =2j8a -----END PGP SIGNATURE----- From barry at python.org Mon Aug 25 19:36:45 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 25 Aug 2008 13:36:45 -0400 Subject: [Python-Dev] subprocess insufficiently platform-independent? In-Reply-To: <20080825173346.GA24017@phd.pp.ru> References: <989B2B74-D5AB-4FAA-BE6D-8D409D194388@python.org> <20080825173346.GA24017@phd.pp.ru> Message-ID: <152426DD-3825-4DA6-B420-24A6CC319F13@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 25, 2008, at 1:33 PM, Oleg Broytmann wrote: > On Mon, Aug 25, 2008 at 01:30:58PM -0400, Barry Warsaw wrote: >> Unless I'm misremembering (I no longer have access to Windows), I >> believe that if you use ' '.join(cmd) as the first argument, it will >> work cross-platform. > > What about arguments that contain spaces? Oh sure, throw a monkey wrench into it . Guido's original problem statement is still correct though. It's advertised to take a sequence of arguments, so that should work. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSLLtr3EjvBPtnXfVAQKJpgP/TPmVgsRdRiAE+jvZOCl9unHWU6LtLSx/ uU3gIkQfvPYcyv9oUS0mcTwWsRDCeP42foQxP+MgX2Zx1ItQPi8/QDbW2bS809pF q8igObharc0kSokhTw2zrkNXsEx3S+epPJXaiueY2cs9jC4mCvXnlnt67ZtWEc8r r+eNLlI2/eo= =SfRl -----END PGP SIGNATURE----- From steven.bethard at gmail.com Mon Aug 25 19:39:22 2008 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 25 Aug 2008 11:39:22 -0600 Subject: [Python-Dev] subprocess insufficiently platform-independent? In-Reply-To: <989B2B74-D5AB-4FAA-BE6D-8D409D194388@python.org> References: <989B2B74-D5AB-4FAA-BE6D-8D409D194388@python.org> Message-ID: On Mon, Aug 25, 2008 at 11:30 AM, Barry Warsaw wrote: > On Aug 25, 2008, at 1:13 PM, Guido van Rossum wrote: > >> Several people at Google seem to have independently discovered that >> despite all of the platform-independent goodness in subprocess.py, you >> still need to be platform aware. One of my colleagues summarized it >> like this: >> >> """ >> Given a straightforward command list like: >> >> cmd = ['svn', 'ls', 'http://rietveld.googlecode.com/svn/trunk'] >> >> You apparently cannot pass this list to any subprocess function >> (subprocess.call() or otherwise) with a set of arguments that allow it >> to "just work" on both Windows and non-Windows systems. > > Unless I'm misremembering (I no longer have access to Windows), I believe > that if you use ' '.join(cmd) as the first argument, it will work > cross-platform. FWIW, I've also struggled with similar issues. For example, with no shell= argument, Windows typically opens up an extra window to run the command, while Unix doesn't. My most recent hack around this looked something like:: try: import win32con except ImportError: win32con = None kwargs = dict(...) if win32con is not None: kwargs['creationflags'] = win32con.CREATE_NO_WINDOW subprocess.Popen(cmd, **kwargs) I'd love to see subprocess become more consistent cross-platform. Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From fumanchu at aminus.org Mon Aug 25 19:32:36 2008 From: fumanchu at aminus.org (Robert Brewer) Date: Mon, 25 Aug 2008 10:32:36 -0700 Subject: [Python-Dev] subprocess insufficiently platform-independent? In-Reply-To: References: Message-ID: Guido van Rossum wrote: > Several people at Google seem to have independently discovered that > despite all of the platform-independent goodness in subprocess.py, you > still need to be platform aware. I can verify this. For CP we went back to using spawnl, but in an internal project we checked sys.platform and set shell=True for Windows. Robert Brewer fumanchu at aminus.org From exarkun at divmod.com Mon Aug 25 19:44:09 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Mon, 25 Aug 2008 13:44:09 -0400 Subject: [Python-Dev] subprocess insufficiently platform-independent? In-Reply-To: Message-ID: <20080825174409.29191.1018559950.divmod.quotient.18404@ohm> On Mon, 25 Aug 2008 10:13:32 -0700, Guido van Rossum wrote: >Several people at Google seem to have independently discovered that >despite all of the platform-independent goodness in subprocess.py, you >still need to be platform aware. One of my colleagues summarized it >like this: > >""" >Given a straightforward command list like: > > cmd = ['svn', 'ls', 'http://rietveld.googlecode.com/svn/trunk'] > Launching child processes on Windows is actually pretty hard to do. The idea of an array of arguments is more like a group hallucination than a reality there. If you assume certain things about how the launched process will interpret its command line (something each program gets to decide for itself on Windows), and many programs do actually use the same rules, then you can get something together that mostly works and is mostly general. Twisted's process support does this and presents a cross-platform API (at least as far as arguments are concerned). Perhaps the subprocess module should borrow that implementation? http://twistedmatrix.com/trac/browser/trunk/twisted/python/win32.py#L66 This doesn't handle all possible inputs correctly (you can read about its flaws in some detail at ) but it does handle *most* inputs. It also isn't appropriate for all programs, but for the programs which don't follow these quoting rules, it's probably a mistake to have a list-based API anyway: they should be invoked using one string giving the entire command line. Jean-Paul From musiccomposition at gmail.com Mon Aug 25 19:52:39 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 25 Aug 2008 12:52:39 -0500 Subject: [Python-Dev] Unicode 5.1.0 In-Reply-To: <57CCEBDE-ABE7-46A9-AD3F-4843FF9E1F98@python.org> References: <57CCEBDE-ABE7-46A9-AD3F-4843FF9E1F98@python.org> Message-ID: <1afaf6160808251052q78fea0c2u4ca97ac911bab7e7@mail.gmail.com> On Mon, Aug 25, 2008 at 12:34 PM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Aug 21, 2008, at 6:30 PM, Terry Reedy wrote: >> >> http://www.unicode.org/versions/Unicode5.1.0/ >> "Unicode 5.1.0 contains over 100,000 characters, and provides significant >> additions and improvements..." to existing features, including new files and >> upgrades to existing files. Sounds close to adding features ;-) > > I agree. This seriously feels like new, potentially high risk code to be > adding this late in the game. The BDFL can always override, but unless > someone is really convincing that this is low risk high benefit, I'd vote no > for 2.6/3.0. +1 Something I think we should also be considering is the 2.7/3.1 release cycle. I propose that we shorten it to ~1 year from 2.6/3.0's release with our main aim being binding 2.x and 3.x more closely. This would get the new unicode features out fairly quickly without having to wait another 2.5 years like 2.5 -> 2.6. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From fredrik at pythonware.com Mon Aug 25 19:53:07 2008 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon, 25 Aug 2008 19:53:07 +0200 Subject: [Python-Dev] Unicode 5.1.0 In-Reply-To: <57CCEBDE-ABE7-46A9-AD3F-4843FF9E1F98@python.org> References: <57CCEBDE-ABE7-46A9-AD3F-4843FF9E1F98@python.org> Message-ID: Barry Warsaw wrote: > I agree. This seriously feels like new, potentially high risk code to > be adding this late in the game. The BDFL can always override, but > unless someone is really convincing that this is low risk high benefit, > I'd vote no for 2.6/3.0. at least two Unicode experts have stated that they don't think the changes are that important. determining exactly what the changes to the *core* character database was the whole point of my offer to tinker with this. (I got distracted due to compiler issues and certain other things to be announced later, but I expect to have some results later this week). From guido at python.org Mon Aug 25 19:56:05 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 25 Aug 2008 10:56:05 -0700 Subject: [Python-Dev] Unicode 5.1.0 In-Reply-To: <1afaf6160808251052q78fea0c2u4ca97ac911bab7e7@mail.gmail.com> References: <57CCEBDE-ABE7-46A9-AD3F-4843FF9E1F98@python.org> <1afaf6160808251052q78fea0c2u4ca97ac911bab7e7@mail.gmail.com> Message-ID: On Mon, Aug 25, 2008 at 10:52 AM, Benjamin Peterson wrote: > On Mon, Aug 25, 2008 at 12:34 PM, Barry Warsaw wrote: >> On Aug 21, 2008, at 6:30 PM, Terry Reedy wrote: >>> >>> http://www.unicode.org/versions/Unicode5.1.0/ >>> "Unicode 5.1.0 contains over 100,000 characters, and provides significant >>> additions and improvements..." to existing features, including new files and >>> upgrades to existing files. Sounds close to adding features ;-) >> >> I agree. This seriously feels like new, potentially high risk code to be >> adding this late in the game. The BDFL can always override, but unless >> someone is really convincing that this is low risk high benefit, I'd vote no >> for 2.6/3.0. > > +1 > > Something I think we should also be considering is the 2.7/3.1 release > cycle. I propose that we shorten it to ~1 year from 2.6/3.0's release > with our main aim being binding 2.x and 3.x more closely. This would > get the new unicode features out fairly quickly without having to wait > another 2.5 years like 2.5 -> 2.6. I was never proposing to support any new features in 2.6/3.0. I was only proposing to update the data files that we already support to the versions provided by 5.1.0. Those data files should have the same format, just slightly improved content: some new characters, some corrected properties. Fredrik says it best: > at least two Unicode experts have stated that they don't think the changes > are that important. determining exactly what the changes to the *core* > character database was the whole point of my offer to tinker with this. > > (I got distracted due to compiler issues and certain other things to be > announced later, but I expect to have some results later this week). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From musiccomposition at gmail.com Mon Aug 25 20:05:30 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 25 Aug 2008 13:05:30 -0500 Subject: [Python-Dev] 2to3 patches and reviews Message-ID: <1afaf6160808251105xf189a01nf7978465ba3a408b@mail.gmail.com> I've been working on and reviewing some of the release blocking 2to3 patches and issues. Unfortunately, it seems like we don't have many core devs available for reviewing 2to3 changes. I have one patch in particular [1] that I would like to get in soon, since it changes the API a bit to make it more extensible. [1] http://bugs.python.org/3637 -- Thanks, Benjamin Peterson "There's no place like 127.0.0.1." From mal at egenix.com Mon Aug 25 20:15:36 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 25 Aug 2008 20:15:36 +0200 Subject: [Python-Dev] Unicode 5.1.0 In-Reply-To: <57CCEBDE-ABE7-46A9-AD3F-4843FF9E1F98@python.org> References: <57CCEBDE-ABE7-46A9-AD3F-4843FF9E1F98@python.org> Message-ID: <48B2F6C8.90800@egenix.com> On 2008-08-25 19:34, Barry Warsaw wrote: > On Aug 21, 2008, at 6:30 PM, Terry Reedy wrote: > >> http://www.unicode.org/versions/Unicode5.1.0/ >> "Unicode 5.1.0 contains over 100,000 characters, and provides >> significant additions and improvements..." to existing features, >> including new files and upgrades to existing files. Sounds close to >> adding features ;-) > > I agree. This seriously feels like new, potentially high risk code to > be adding this late in the game. The BDFL can always override, but > unless someone is really convincing that this is low risk high benefit, > I'd vote no for 2.6/3.0. The above quote from the Unicode site is misleading in this context. Guido's request was just for updating the Unicode database with the data from 5.1 - without adding new support for properties or changing the interfaces. See this page for a list of changes to the Unicode database: http://www.unicode.org/Public/UNIDATA/UCD.html The main file used for the unicodedata module is called "UnicodeData.txt". -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 25 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From brett at python.org Mon Aug 25 20:16:33 2008 From: brett at python.org (Brett Cannon) Date: Mon, 25 Aug 2008 11:16:33 -0700 Subject: [Python-Dev] Unicode 5.1.0 In-Reply-To: References: <57CCEBDE-ABE7-46A9-AD3F-4843FF9E1F98@python.org> <1afaf6160808251052q78fea0c2u4ca97ac911bab7e7@mail.gmail.com> Message-ID: On Mon, Aug 25, 2008 at 10:56 AM, Guido van Rossum wrote: > On Mon, Aug 25, 2008 at 10:52 AM, Benjamin Peterson > wrote: >> On Mon, Aug 25, 2008 at 12:34 PM, Barry Warsaw wrote: >>> On Aug 21, 2008, at 6:30 PM, Terry Reedy wrote: >>>> >>>> http://www.unicode.org/versions/Unicode5.1.0/ >>>> "Unicode 5.1.0 contains over 100,000 characters, and provides significant >>>> additions and improvements..." to existing features, including new files and >>>> upgrades to existing files. Sounds close to adding features ;-) >>> >>> I agree. This seriously feels like new, potentially high risk code to be >>> adding this late in the game. The BDFL can always override, but unless >>> someone is really convincing that this is low risk high benefit, I'd vote no >>> for 2.6/3.0. >> >> +1 >> >> Something I think we should also be considering is the 2.7/3.1 release >> cycle. I propose that we shorten it to ~1 year from 2.6/3.0's release >> with our main aim being binding 2.x and 3.x more closely. This would >> get the new unicode features out fairly quickly without having to wait >> another 2.5 years like 2.5 -> 2.6. > > I was never proposing to support any new features in 2.6/3.0. I was > only proposing to update the data files that we already support to the > versions provided by 5.1.0. Those data files should have the same > format, just slightly improved content: some new characters, some > corrected properties. Fredrik says it best: > >> at least two Unicode experts have stated that they don't think the changes >> are that important. determining exactly what the changes to the *core* >> character database was the whole point of my offer to tinker with this. >> >> (I got distracted due to compiler issues and certain other things to be >> announced later, but I expect to have some results later this week). > Plus the Europeans who probably use Unicode more than the dissenting Americans also seem to think it's a good idea. It's just a data table, and it's auto-generated, *and* one of the main guys from the Unicode Consortium is willing to help. I say let the change go in. -Brett From arfrever.fta at gmail.com Mon Aug 25 20:40:26 2008 From: arfrever.fta at gmail.com (Arfrever Frehtes Taifersar Arahesis) Date: Mon, 25 Aug 2008 20:40:26 +0200 Subject: [Python-Dev] [PATCH] Respect LDFLAGS Message-ID: <200808252040.34277.Arfrever.FTA@gmail.com> I'm attaching the patch which fixes respecting LDFLAGS when building libpython$(VERSION).so. -- Arfrever Frehtes Taifersar Arahesis -------------- next part -------------- A non-text attachment was scrubbed... Name: python-respect_LDFLAGS.patch Type: text/x-diff Size: 712 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part. URL: From barry at python.org Mon Aug 25 21:04:40 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 25 Aug 2008 15:04:40 -0400 Subject: [Python-Dev] Unicode 5.1.0 In-Reply-To: References: <57CCEBDE-ABE7-46A9-AD3F-4843FF9E1F98@python.org> Message-ID: <6DD88C6B-7586-43C9-A060-A1D8C5D3E4E3@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 25, 2008, at 1:53 PM, Fredrik Lundh wrote: > Barry Warsaw wrote: > >> I agree. This seriously feels like new, potentially high risk code >> to be adding this late in the game. The BDFL can always override, >> but unless someone is really convincing that this is low risk high >> benefit, I'd vote no for 2.6/3.0. > > at least two Unicode experts have stated that they don't think the > changes are that important. determining exactly what the changes to > the *core* character database was the whole point of my offer to > tinker with this. You don't mean the experts claimed they weren't important, right? Unimportant changes definitely don't need to go in now . - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSLMCSHEjvBPtnXfVAQKgLwP/YlrqcdlmoeBsK9JdJMnxkgN92L1K86cg lzvQT6bv8vda64Su8bV81UT+NdoB+/ZGpZ1t+Dn4Z0uvB0uaVrZZ7uOUoqQTkvG7 yrj/Clbedi2v35vYjudqAaZyBnJtz+V0rH8tdgpDVU5zILSK4gQm385nFuoUXQpC iJlqok3tjuU= =YfQR -----END PGP SIGNATURE----- From barry at python.org Mon Aug 25 21:07:52 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 25 Aug 2008 15:07:52 -0400 Subject: [Python-Dev] Unicode 5.1.0 In-Reply-To: <48B2F6C8.90800@egenix.com> References: <57CCEBDE-ABE7-46A9-AD3F-4843FF9E1F98@python.org> <48B2F6C8.90800@egenix.com> Message-ID: <41911284-196D-484A-97DD-A2AD8547D826@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 25, 2008, at 2:15 PM, M.-A. Lemburg wrote: > Guido's request was just for updating the Unicode database with > the data from 5.1 - without adding new support for properties or > changing the interfaces. > > See this page for a list of changes to the Unicode database: > > http://www.unicode.org/Public/UNIDATA/UCD.html > > The main file used for the unicodedata module is called > "UnicodeData.txt". That's much less scary. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSLMDCHEjvBPtnXfVAQIrYQQAoABjn/KWd8VdFeplR1i3Lqx5lNAWiJu9 6QWhE/4PPGpCBWhsUejnqHTdCOHmo7y6g3YWwPJ1hDZbl+oXrHd4/bcnHWMJgbOO BV7ACRdVcf9tuewiyNkWXGDn99WcXrGHSTXEnhQsQWL58PDbLxbbDgDUPdbXsGgC zCQbSykYv2E= =yheO -----END PGP SIGNATURE----- From fredrik at pythonware.com Mon Aug 25 21:17:21 2008 From: fredrik at pythonware.com (Fredrik Lundh) Date: Mon, 25 Aug 2008 21:17:21 +0200 Subject: [Python-Dev] Unicode 5.1.0 In-Reply-To: <6DD88C6B-7586-43C9-A060-A1D8C5D3E4E3@python.org> References: <57CCEBDE-ABE7-46A9-AD3F-4843FF9E1F98@python.org> <6DD88C6B-7586-43C9-A060-A1D8C5D3E4E3@python.org> Message-ID: Barry Warsaw wrote: > You don't mean the experts claimed they weren't important, right? > Unimportant changes definitely don't need to go in now . Well, at least Guido managed to figure out what I was trying to say ;-) From barry at python.org Mon Aug 25 21:36:11 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 25 Aug 2008 15:36:11 -0400 Subject: [Python-Dev] Unicode 5.1.0 In-Reply-To: References: <57CCEBDE-ABE7-46A9-AD3F-4843FF9E1F98@python.org> <6DD88C6B-7586-43C9-A060-A1D8C5D3E4E3@python.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 25, 2008, at 3:17 PM, Fredrik Lundh wrote: > Barry Warsaw wrote: > >> You don't mean the experts claimed they weren't important, right? >> Unimportant changes definitely don't need to go in now . > > Well, at least Guido managed to figure out what I was trying to > say ;-) Yeah, I was just being curmudgeonly. :) - -B -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSLMJq3EjvBPtnXfVAQKG6QQAl3MliLaqhaibf12INX6EUoeIiYBEJlhY IiFgdc6VXe7evMsxUj2xE+1S+rg9BEhwiY38NTZdaqDCRiHBUY6aKFYlnawuyaKf 8m+jIdkJyudBpT5dBvfBCvYlwmXl/RwKHzDRDCHUjKfBVgAo9elv+EDy2kFLUpM1 W8dVEYwo3dg= =fue6 -----END PGP SIGNATURE----- From robert.schuppenies at gmail.com Mon Aug 25 22:49:22 2008 From: robert.schuppenies at gmail.com (Robert Schuppenies) Date: Mon, 25 Aug 2008 22:49:22 +0200 Subject: [Python-Dev] no tp_traverse for dict iters In-Reply-To: References: <48B26E30.3020701@gmail.com> Message-ID: <48B31AD2.4070808@gmail.com> Created bug report at http://bugs.python.org/issue3546. Amaury Forgeot d'Arc wrote: > Yes, this can lead to some object cycle that are not collected. > See the attached script: a cycle involving a list iterator is > collected by the garbage collector, but a cycle with a dict iterator > is not. > This is worth a bug report IMO. From robert.schuppenies at gmail.com Mon Aug 25 23:09:34 2008 From: robert.schuppenies at gmail.com (Robert Schuppenies) Date: Mon, 25 Aug 2008 23:09:34 +0200 Subject: [Python-Dev] no tp_traverse for dict iters In-Reply-To: <48B31AD2.4070808@gmail.com> References: <48B26E30.3020701@gmail.com> <48B31AD2.4070808@gmail.com> Message-ID: <48B31F8E.80803@gmail.com> Sry, wrong link. I meant http://bugs.python.org/issue3680. Robert Schuppenies wrote: > Created bug report at http://bugs.python.org/issue3546. > > Amaury Forgeot d'Arc wrote: >> Yes, this can lead to some object cycle that are not collected. >> See the attached script: a cycle involving a list iterator is >> collected by the garbage collector, but a cycle with a dict iterator >> is not. >> This is worth a bug report IMO. From steve at holdenweb.com Tue Aug 26 00:55:36 2008 From: steve at holdenweb.com (Steve Holden) Date: Mon, 25 Aug 2008 18:55:36 -0400 Subject: [Python-Dev] Unicode 5.1.0 In-Reply-To: <57CCEBDE-ABE7-46A9-AD3F-4843FF9E1F98@python.org> References: <57CCEBDE-ABE7-46A9-AD3F-4843FF9E1F98@python.org> Message-ID: Barry Warsaw wrote: > On Aug 21, 2008, at 6:30 PM, Terry Reedy wrote: > >> http://www.unicode.org/versions/Unicode5.1.0/ >> "Unicode 5.1.0 contains over 100,000 characters, and provides >> significant additions and improvements..." to existing features, >> including new files and upgrades to existing files. Sounds close to >> adding features ;-) > > I agree. This seriously feels like new, potentially high risk code to > be adding this late in the game. The BDFL can always override, but > unless someone is really convincing that this is low risk high benefit, > I'd vote no for 2.6/3.0. But it's [the] wafer-thin [end of the wedge] ... The difficulties with subprocess suggest there's plenty to do without adding yet one more tiny little task. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From aahz at pythoncraft.com Tue Aug 26 03:11:27 2008 From: aahz at pythoncraft.com (Aahz) Date: Mon, 25 Aug 2008 18:11:27 -0700 Subject: [Python-Dev] [PATCH] Respect LDFLAGS In-Reply-To: <200808252040.34277.Arfrever.FTA@gmail.com> References: <200808252040.34277.Arfrever.FTA@gmail.com> Message-ID: <20080826011127.GA17310@panix.com> On Mon, Aug 25, 2008, Arfrever Frehtes Taifersar Arahesis wrote: > > I'm attaching the patch which fixes respecting LDFLAGS when > building libpython$(VERSION).so. Please post your patch to bugs.python.org -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Adopt A Process -- stop killing all your children! From greg.ewing at canterbury.ac.nz Tue Aug 26 03:52:16 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 26 Aug 2008 13:52:16 +1200 Subject: [Python-Dev] subprocess insufficiently platform-independent? In-Reply-To: References: Message-ID: <48B361D0.60402@canterbury.ac.nz> Guido van Rossum wrote: > If you call: > > subprocess.call(cmd, shell=False) > > Then it works on Linux, but fails on Windows because it does not > perform the Windows %PATHEXT% search that allows it to find that > "svn.exe" is the actual executable to be invoked. Maybe the Windows implementation should do its own %PATHEXT% lookup when the shell is not being used. -- Greg From foom at fuhm.net Tue Aug 26 04:17:32 2008 From: foom at fuhm.net (James Y Knight) Date: Mon, 25 Aug 2008 22:17:32 -0400 Subject: [Python-Dev] subprocess insufficiently platform-independent? In-Reply-To: <48B361D0.60402@canterbury.ac.nz> References: <48B361D0.60402@canterbury.ac.nz> Message-ID: <351E513C-4369-4426-BBF4-715250C2BB9B@fuhm.net> On Aug 25, 2008, at 9:52 PM, Greg Ewing wrote: > Guido van Rossum wrote: > >> If you call: >> subprocess.call(cmd, shell=False) >> Then it works on Linux, but fails on Windows because it does not >> perform the Windows %PATHEXT% search that allows it to find that >> "svn.exe" is the actual executable to be invoked. > > Maybe the Windows implementation should do its own > %PATHEXT% lookup when the shell is not being used. +1 to that. It's really nice to be able to *not* invoke a shell, and thus not have to worry about the shell doing nasty things to your process spawning. So, any solution that aids portability to windows without requiring the invocation of a shell seems like a good thing to me. James From ziade.tarek at gmail.com Tue Aug 26 10:54:24 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 26 Aug 2008 10:54:24 +0200 Subject: [Python-Dev] distutils small fix Message-ID: <94bdd2610808260154j5e726646gb39aceacaeee9dbc@mail.gmail.com> Hello Marc-Andr? and I are looking for one more reviewer for the patch http://bugs.python.org/issue2562 It makes it possible to use accented letters in distutils' metadata, like the author name, by switching those fields to Unicode. setuptools and PyPI are both fine with this, it is just a matter of making distutils writing the PKG_FILE correctly. Regards Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at rcn.com Tue Aug 26 17:32:23 2008 From: python at rcn.com (Raymond Hettinger) Date: Tue, 26 Aug 2008 12:32:23 -0300 Subject: [Python-Dev] Things to Know About Super References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> Message-ID: [Michele Simionato] > Recently I have opened a blog on Artima and I am publishing a > few Python-related essays I had in store. In particular a trilogy > of papers about "super". From the foreword: > > """ > In 2004 I decided to write a comprehensive paper documenting > ``super`` pitfalls and traps, ... Thanks for posting these blogs. I think they serve as a nice compilation of docs, original analysis, and various discussions on super(). What I would really like to see is a fourth blog entry that shows how to use super() reliably and correctly. In general, I opposed to creating documentation in the form of "danger, danger this could explode ...." IMO, there is not much point in dwelling on bugs that have already been fixed, nor is there an advantage to showing all the ways the tool can be misused. In applications without multiple inheritance, it is a straight-forward to use super() as a way to avoid directly naming an immediate parent class. This is useful in and of itself. For cooperative multiple inheritance, I take issue with the abstracted examples provided (i.e. inconsistent signatures). In a real app that actually needs cooperative multiple inheritance, it becomes self-evident what "cooperative" actually means -- the methods *have* to be designed to interoperate -- it is intrinsic to the problem at hand. Your reasoning is akin to saying that cooperative multitasking is intrinsically flawed because one of the tasks could be designed to not cooperate (never yield). Cooperative multiple inheritance is *not* about mixing two unrelated parents that just happen to use the same method name but have different semantics and were not designed to cooperate with each other. The A-B-C-D diagrams and foo/bar methods in the examples are deceptive because they silently drop the precondition of cooperation while attempting to demonstrate a supposed flaw in the tool. If I understand the problem correctly, in the rare cases where you do need cooperative multiple inheritance, then super() is the only workable solution short of designing some equivalent using composition instead of inheritance. I do agree with you that the docs can be improved. I'll work on patch that makes clear that super() returns a proxy-object (not an actual parent class) that dispatches explict method calls (not syntax driven) to one of multiple parents designed to interact cooperatively. Also, it may be controversial, but there may be some merit in de-documenting the "unbound" case. It seems to add more confusion than it's worth. Lastly, I take issue with one other part of the blogs. While they show a clear dislike for cooperative multiple inheritance, they take a potshot at multiple inheritance in general. I don't follow the logic here. IMO, mixin classes like DictMixin have proven themselves as being very useful. Plenty of frameworks share this approach. Likewise, the new ABCs offer mixin capabilities that are really nice. I think it is a non-sequiter to reason from "diamond diagrams are complicated" to "mixins should be disallowed". Instead, I think it better to simply recommend that a key to happiness is to keep various mixin classes completely orthogonal to one another (no overlapping method names). Lest I sound negative, let me thank you again for the blog entries and for compiling the most complete discussion of it in one place. Raymond From solipsis at pitrou.net Tue Aug 26 17:53:27 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 26 Aug 2008 15:53:27 +0000 (UTC) Subject: [Python-Dev] Mercurial mirrors Message-ID: Hello, Thanks to Neil Schemenauer, we now have some Mercurial mirrors hosted at http://code.python.org/hg/ Here are the URLs for each of the available repositories: - http://code.python.org/hg/trunk/ - http://code.python.org/hg/branches/py3k/ - http://code.python.org/hg/branches/release2.5-maint/ For Mercurial beginners, each of the URLs above references both a human-readable Web interface to browse the repositories (view changesets, subscribe to an RSS feed, annotate source files...), and a machine-queryable address to "clone" the repositories for local, offline use. That is, if you run "hg clone http://code.python.org/hg/trunk/", you'll get a "trunk" subdirectory containing the whole trunk history in a local repository, as well as a working copy. You can then do, e.g. "hg annotate setup.py" at light speed. The Mercurial mirrors are sync'ed with the SVN repo every 5 minutes. They are read-only, that is you cannot push to them. (technical note: the mirrors are done using my own "hgsvn", but an alternate strategy would be to use the builtin "hg convert" instead) Regards Antoine. From michele.simionato at gmail.com Tue Aug 26 18:13:03 2008 From: michele.simionato at gmail.com (Michele Simionato) Date: Tue, 26 Aug 2008 18:13:03 +0200 Subject: [Python-Dev] Things to Know About Super In-Reply-To: References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> Message-ID: <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> On Tue, Aug 26, 2008 at 5:32 PM, Raymond Hettinger wrote: > What I would really like to see is a fourth blog entry that shows > how to use super() reliably and correctly. That could be arranged. > In general, I opposed > to creating documentation in the form of "danger, danger this > could explode ...." IMO, there is not much point in dwelling on > bugs that have already been fixed, nor is there an advantage to > showing all the ways the tool can be misused. Yep. The parts about the bugs of super in 2.2 and 2.3 were written years ago, when they were relevant. Nowadays they are less relevant, but since they were already written and since there are still people using older versions of Python I decided to keep them. I would not keep them in a revised version intended as "semi-official" documentation of super. Still, I think they are fine as a blog post. > For cooperative multiple inheritance, I take issue with the abstracted > examples provided (i.e. inconsistent signatures). In a real app that > actually needs cooperative multiple inheritance, it becomes self-evident > what "cooperative" actually means -- the methods *have* to be > designed to interoperate -- it is intrinsic to the problem at hand. > Cooperative multiple inheritance is *not* about mixing two unrelated > parents that just happen to use the same method name but have > different semantics and were not designed to cooperate with each other. > > The A-B-C-D diagrams and foo/bar methods in the examples are > deceptive because they silently drop the precondition of cooperation > while attempting to demonstrate a supposed flaw in the tool. They just show that the tool is delicate and not easy to use. > If I understand the problem correctly, in the rare cases where you do > need cooperative multiple inheritance, then super() is the only workable > solution short of designing some equivalent using composition instead > of inheritance. In my experience, one can go a long way using composition instead of inheritance. I also think that Python would not lose much without cooperative multiple inheritance. This is however a personal opinion and in any case the point is moot because the language is the way it is. Still, in a blog post personal opinions and even rants have their place. That part could be removed in an "semi-official" document. > Also, it may be controversial, but there may be some merit in de-documenting > the "unbound" case. It seems to add more confusion than it's worth. Fine with me. > Lastly, I take issue with one other part of the blogs. While they show > a clear dislike for cooperative multiple inheritance, they take a potshot > at multiple inheritance in general. I don't follow the logic here. IMO, > mixin classes like DictMixin have proven themselves as being very > useful. Plenty of frameworks share this approach. Likewise, the > new ABCs offer mixin capabilities that are really nice. > > I think it is a non-sequiter to reason from "diamond diagrams are > complicated" to "mixins should be disallowed". Instead, I think it better > to simply recommend that a key to happiness is to keep various mixin classes > completely orthogonal to one another (no overlapping method names). I not completely against multiple inheritance. I am against multiple inheritance as it is now. A restricted form of multiple inheritance in which mixins classes are guaranteed to be orthogonal would be fine with me (provided it is not abused). This concept exists already in other languages, the orthogonal mixins are called "traits". I have written a trilogy of papers about mixins: if you read them, you will see where I come from (Zope, which I do not like) and you will also see that I like DictMixin instead. I will publish the papers in the blog soon or later, but you can find the Italian version here: http://stacktrace.it/articoli/2008/06/i-pericoli-della-programmazione-con-i-mixin1/ http://stacktrace.it/articoli/2008/07/i-pericoli-della-programmazione-con-i-mixin2/ http://stacktrace.it/articoli/2008/08/i-pericoli-della-programmazione-con-i-mixin3/ Michele Simionato From musiccomposition at gmail.com Tue Aug 26 18:42:21 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 26 Aug 2008 11:42:21 -0500 Subject: [Python-Dev] Mercurial mirrors In-Reply-To: References: Message-ID: <1afaf6160808260942t27091332ncc8b786e5781ffbb@mail.gmail.com> On Tue, Aug 26, 2008 at 10:53 AM, Antoine Pitrou wrote: > > Hello, > > Thanks to Neil Schemenauer, we now have some Mercurial mirrors hosted at > http://code.python.org/hg/ Cool! It's nice to have these become "official". My hg branches are all pointing to your site. Can I easily relocate the parent branch? -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From barry at python.org Tue Aug 26 19:18:01 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 26 Aug 2008 13:18:01 -0400 Subject: [Python-Dev] Mercurial mirrors In-Reply-To: <1afaf6160808260942t27091332ncc8b786e5781ffbb@mail.gmail.com> References: <1afaf6160808260942t27091332ncc8b786e5781ffbb@mail.gmail.com> Message-ID: <2B7774E9-1B08-4D7B-8CBA-EA2FDE44F44F@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 26, 2008, at 12:42 PM, Benjamin Peterson wrote: > On Tue, Aug 26, 2008 at 10:53 AM, Antoine Pitrou > wrote: >> >> Hello, >> >> Thanks to Neil Schemenauer, we now have some Mercurial mirrors >> hosted at >> http://code.python.org/hg/ > > Cool! It's nice to have these become "official". My hg branches are > all pointing to your site. Can I easily relocate the parent branch? Actually, right now anything that's not Subversion is still experimental. But agreed that it's nice Mercurial and Bazaar are available. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSLQ6yXEjvBPtnXfVAQK/fgP/aRTdNeqdWddQetANArV8zpYLbS7TUUbl b/h0p6fWJdSf4ZPkuN4yjXZz8F+4QXc+B1qF8qxEdtc9BjvSBbgOED/GEjF++9OJ fQ2JpT01+Cj2Z8rl034leTrGq1CQ9uDC7YTWJblJjyFiFiCEzSiiJ6aETe3LDpwe ZJT6OxyQVwU= =7YSu -----END PGP SIGNATURE----- From rowen at u.washington.edu Tue Aug 26 19:59:09 2008 From: rowen at u.washington.edu (Russell E. Owen) Date: Tue, 26 Aug 2008 10:59:09 -0700 Subject: [Python-Dev] Fix python darwin build to look for non-framework Tk if not a framework build? Message-ID: Python's own setup.py file seems to only look for a Framework Tcl/Tk on darwin. This is a headache if one is trying to build a non-framework python that uses a non-framework tcl/tk. Any chance of getting this fixed? I'm willing to work on a patch if it has any chance of being accepted. Any hints on how to determine if it is a framework build from within setup.py would be most welcome! -- Russell From brett at python.org Tue Aug 26 20:47:32 2008 From: brett at python.org (Brett Cannon) Date: Tue, 26 Aug 2008 11:47:32 -0700 Subject: [Python-Dev] Mercurial mirrors In-Reply-To: References: Message-ID: On Tue, Aug 26, 2008 at 8:53 AM, Antoine Pitrou wrote: > > Hello, > > Thanks to Neil Schemenauer, we now have some Mercurial mirrors hosted at > http://code.python.org/hg/ > > Here are the URLs for each of the available repositories: > - http://code.python.org/hg/trunk/ > - http://code.python.org/hg/branches/py3k/ > - http://code.python.org/hg/branches/release2.5-maint/ > > For Mercurial beginners, each of the URLs above references both a human-readable > Web interface to browse the repositories (view changesets, subscribe to an RSS > feed, annotate source files...), and a machine-queryable address to "clone" the > repositories for local, offline use. > > That is, if you run "hg clone http://code.python.org/hg/trunk/", you'll get a > "trunk" subdirectory containing the whole trunk history in a local repository, > as well as a working copy. You can then do, e.g. "hg annotate setup.py" at light > speed. > > The Mercurial mirrors are sync'ed with the SVN repo every 5 minutes. They are > read-only, that is you cannot push to them. > But can we push branches up to our personal directories on code.python.org like we can with bzr? -Brett From python at rcn.com Tue Aug 26 20:56:48 2008 From: python at rcn.com (Raymond Hettinger) Date: Tue, 26 Aug 2008 15:56:48 -0300 Subject: [Python-Dev] Things to Know About Super References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> Message-ID: [Raymond] >> Cooperative multiple inheritance is *not* about mixing two unrelated >> parents that just happen to use the same method name but have >> different semantics and were not designed to cooperate with each other. >> >> The A-B-C-D diagrams and foo/bar methods in the examples are >> deceptive because they silently drop the precondition of cooperation >> while attempting to demonstrate a supposed flaw in the tool. [Michele] > They just show that the tool is delicate and not easy to use. To me, they miss the point. Simply, if you don't have diamonds, then super() is easy to use and if you do have have diamonds, then super() is the *only* way to do it. Diamonds impose a set of design constraints (making the classes cooperative). The A-B-C-D diagrams ignore this point and make super() seem like an accident waiting to happen. In contrast, with a concrete example, accidental non-cooperativeness would be more self-evident. The problem isn't that super() is fragile. The problem is that a cooperative design pattern requires you to actually make the classes cooperate. This is no different than a visitor design pattern requiring that you attach a conformant visit() method for on each of the visited classes. [Michele] > In my experience, one can go a long way using composition instead of > inheritance. While that is often true, I don't think it applies to the case of cooperative multiple inheritence. To achieve substantially the same functionality using composition, you would likely have to re-invent much of what super() does for us automatically, and you would still be imposing constraits on the composed classes that are substantially the same as what you have with inheritance. [Michele] > I also think that Python would not lose much without cooperative > multiple inheritance. I would state this differently: "The use cases for cooperative multiple inheritence don't arise often in practice; so, if we dropped support for those cases, you probably wouldn't notice until you encountered one of the rare occasions where it was the right answer to your problem." There was some quote floating around that expressed the situation well -- it went something like: "Python makes most problems easy and hard problems possible". The use cases for cooperative multiple inheritance fall in the latter category. BTW, I really like your paper explaining the MRO. Excellent work. Raymond From armin.ronacher at active-4.com Tue Aug 26 21:05:19 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Tue, 26 Aug 2008 19:05:19 +0000 (UTC) Subject: [Python-Dev] Confusing listreverseiterator Behavior Message-ID: Hi, I stumbled upon a confusing listreverseiterator behavior: >>> l = [1, 2, 3, 4] >>> i = iter(l) >>> ri = reversed(l) >>> len(i) Traceback (most recent call last): File " ", line 1, in TypeError: object of type 'listiterator' has no len() >>> len(ri) 4 >>> ri.next() 4 >>> len(ri) 3 This is the only reverse iterator with that sort of behavior. Is that intentional if yes, why? I stumbled upon that when writing a templating engine that tried to lazily get the length of the sequence / iterator but failed doing so after the first iteration because the length of the reverse iterator changes every iteration. Regards, Armin From hall.jeff at gmail.com Tue Aug 26 21:21:15 2008 From: hall.jeff at gmail.com (Jeff Hall) Date: Tue, 26 Aug 2008 15:21:15 -0400 Subject: [Python-Dev] Confusing listreverseiterator Behavior In-Reply-To: References: Message-ID: <1bc395c10808261221n5e6e7c88le0f30f7199e0cee2@mail.gmail.com> Unless I'm misconstruing something the problem is that reversed returns two different object types depending on if it's a list or a tuple >>> l = [1,2,3,4] >>> i = iter(l) >>> ri = reversed(l) >>> l [1, 2, 3, 4] >>> ri >>> i >>> t = (1,2,3,4) >>> it = iter(t) >>> rit = reversed(t) >>> it >>> rit >>> reversing a tuple (or a string) returns a "reversed object" reversing a list returns a "listreverseiterator" definitely an inconsistency -------------- next part -------------- An HTML attachment was scrubbed... URL: From hall.jeff at gmail.com Tue Aug 26 21:47:13 2008 From: hall.jeff at gmail.com (Jeff Hall) Date: Tue, 26 Aug 2008 15:47:13 -0400 Subject: [Python-Dev] Confusing listreverseiterator Behavior In-Reply-To: <1bc395c10808261221n5e6e7c88le0f30f7199e0cee2@mail.gmail.com> References: <1bc395c10808261221n5e6e7c88le0f30f7199e0cee2@mail.gmail.com> Message-ID: <1bc395c10808261247n5b658197g8872539a757e7c33@mail.gmail.com> I realized after I fired off my response that this was still bugging me... it appears that the documentation is incorrect from 2.1 Built-in Functions (v2.5 in case it matters... a quick search of bugs doesn't seem to show anything though) *reversed*( seq) Return a reverse iterator. seq must be an object which supports the sequence protocol (the __len__() method and the __getitem__()method with integer arguments starting at 0). New in version 2.4. the above appears to only be true for lists. For tuples and strings it creates a reverse OBJECT which behaves slightly differently (notably by not including a len() method as you noticed) I can't find how to actually create a "tuplereverseiterator" or "stringreverseiterator" objects... nor does there appear to be a way to create a "reversed" object from a list... Just tested this s = 'bob' t = (1,2,3,4) l = [1,2,3,4) rs = reversed(s) rt = reversed(t) rl = reversed(l) type(rs) type(rt) type(rl) type(rs) == type(rt) True type(rs) == type(rl) False Surely this isn't intentional? -------- Haikus are easy Most make very little sense Refrigerator -------------- next part -------------- An HTML attachment was scrubbed... URL: From armin.ronacher at active-4.com Tue Aug 26 22:04:03 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Tue, 26 Aug 2008 20:04:03 +0000 (UTC) Subject: [Python-Dev] Confusing listreverseiterator Behavior References: <1bc395c10808261221n5e6e7c88le0f30f7199e0cee2@mail.gmail.com> <1bc395c10808261247n5b658197g8872539a757e7c33@mail.gmail.com> Message-ID: Jeff Hall gmail.com> writes: > reversed( > seq) > Return a reverse iterator. seq must be an object which > supports the sequence protocol (the __len__() method and the __getitem__() method with integer arguments starting at > 0). New in version 2.4. the above appears to only be true for lists. For > tuples and strings it creates a reverse OBJECT which behaves slightly > differently (notably by not including a len() method as you noticed) > I can't find how to actually create a "tuplereverseiterator" or > "stringreverseiterator" objects... nor does there appear to be a way to > create a "reversed" object from a list... That's an implementation detail what exactly the reverse iterator is. The same applies to iter() calls. iter("foo") for example returns a iterator type, iter([]) returns a list iterator. The thing you quoted above is the requirement for the object that you pass to reverse(), not the object returned which is some kind of iterator that happens to iterate over the sequence in reverse. The problem I personally have with it is that the listreverseiterator is the only iterator in the standard library that changes the length during iteration and that it's the only reverse iterator that has a length. Even more stunning as the normal iterator doesn't have one. Regards, Armin From python at rcn.com Tue Aug 26 22:05:46 2008 From: python at rcn.com (Raymond Hettinger) Date: Tue, 26 Aug 2008 17:05:46 -0300 Subject: [Python-Dev] Confusing listreverseiterator Behavior References: Message-ID: From: "Armin Ronacher" >>>> len(ri) > 4 >>>> ri.next() > 4 >>>> len(ri) > 3 > > This is the only reverse iterator with that sort of behavior. Use the bug tracker please and assign to me. At one time, some iterators had the ability to know their own length and that would change as the iterator got consumed. Later, it was decided that iterators should not report length and should instead report a length hint. It looks like listreversediterator got missed when this was changed. Raymond From g.brandl at gmx.net Tue Aug 26 22:30:56 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 26 Aug 2008 22:30:56 +0200 Subject: [Python-Dev] Confusing listreverseiterator Behavior In-Reply-To: <1bc395c10808261247n5b658197g8872539a757e7c33@mail.gmail.com> References: <1bc395c10808261221n5e6e7c88le0f30f7199e0cee2@mail.gmail.com> <1bc395c10808261247n5b658197g8872539a757e7c33@mail.gmail.com> Message-ID: Jeff Hall schrieb: > I realized after I fired off my response that this was still bugging > me... it appears that the documentation is incorrect > > from 2.1 Built-in Functions (v2.5 in case it matters... a quick search > of bugs doesn't seem to show anything though) > > *reversed*( seq) > > Return a reverse iterator. seq must be an object which supports the > sequence protocol (the __len__() method and the __getitem__() method > with integer arguments starting at |0|). New in version 2.4. > > the above appears to only be true for lists. Not at all. (I think you misread; the __len__ method must be present on the argument, not the returned object.) > For tuples and strings it > creates a reverse OBJECT which behaves slightly differently (notably by > not including a len() method as you noticed) > > I can't find how to actually create a "tuplereverseiterator" or > "stringreverseiterator" objects... nor does there appear to be a way to > create a "reversed" object from a list... You don't need to. An object returned by reversed() only needs to follow the iterator protocol. Whether it is a listreverseiterator or a general reversed object doesn't matter. In fact, reversed() calls __reversed__ on its argument if it exists, so that custom types may provide their own optimized reverse iterator. Georg From musiccomposition at gmail.com Tue Aug 26 22:35:52 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 26 Aug 2008 15:35:52 -0500 Subject: [Python-Dev] Fix python darwin build to look for non-framework Tk if not a framework build? In-Reply-To: References: Message-ID: <1afaf6160808261335u2dfb92m6a538b2dbdf1b886@mail.gmail.com> On Tue, Aug 26, 2008 at 12:59 PM, Russell E. Owen wrote: > Python's own setup.py file seems to only look for a Framework Tcl/Tk on > darwin. This is a headache if one is trying to build a non-framework > python that uses a non-framework tcl/tk. > > Any chance of getting this fixed? I'm willing to work on a patch if it > has any chance of being accepted. Any hints on how to determine if it is > a framework build from within setup.py would be most welcome! If you do, please file a ticket on the tracker with your patch. Thanks! > > -- Russell > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From steve at holdenweb.com Tue Aug 26 23:10:10 2008 From: steve at holdenweb.com (Steve Holden) Date: Tue, 26 Aug 2008 17:10:10 -0400 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> Message-ID: Michele Simionato wrote: > On Tue, Aug 26, 2008 at 5:32 PM, Raymond Hettinger wrote: [...] >> I think it is a non-sequiter to reason from "diamond diagrams are >> complicated" to "mixins should be disallowed". Instead, I think it better >> to simply recommend that a key to happiness is to keep various mixin classes >> completely orthogonal to one another (no overlapping method names). > > I not completely against multiple inheritance. I am against multiple inheritance > as it is now. A restricted form of multiple inheritance in which mixins classes > are guaranteed to be orthogonal would be fine with me (provided it is > not abused). > This concept exists already in other languages, the orthogonal mixins > are called "traits". > If you aren't aware of it you should take a look at Enthought's traits package. It's part of the Enthought Tool Suite (ETS). https://svn.enthought.com/enthought/wiki While I too appreciate your comments about super I believe you have perhaps overdone it. I do look forward to seeing the edited edition as a part of the documentation. [Hint: the *docs* aren't in feature freeze ;-)] regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From solipsis at pitrou.net Wed Aug 27 00:26:27 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 26 Aug 2008 22:26:27 +0000 (UTC) Subject: [Python-Dev] Mercurial mirrors References: <1afaf6160808260942t27091332ncc8b786e5781ffbb@mail.gmail.com> Message-ID: Benjamin Peterson gmail.com> writes: > > Cool! It's nice to have these become "official". My hg branches are > all pointing to your site. Can I easily relocate the parent branch? Just edit .hg/hgrc in your branches and modify the "default" value in the "[paths]" section. Then "hg in" to be sure everything went ok. Regards Antoine. From solipsis at pitrou.net Wed Aug 27 00:28:24 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 26 Aug 2008 22:28:24 +0000 (UTC) Subject: [Python-Dev] Mercurial mirrors References: Message-ID: Brett Cannon python.org> writes: > > But can we push branches up to our personal directories on > code.python.org like we can with bzr? If you have an ssh access to code.python.org, it should be easy. However, giving other people anonymous read-only access would require some manual configuration I think (not lots of, though). From solipsis at pitrou.net Wed Aug 27 01:13:13 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 26 Aug 2008 23:13:13 +0000 (UTC) Subject: [Python-Dev] class attributes slower in py3k References: <1219604656.5635.2.camel@fsol> Message-ID: Antoine Pitrou pitrou.net> writes: > > > NormalClassAttribute: 339ms 340ms 0.28us 1.111ms > > > > Over twice as slow? > > Yes, should be investigated. > [...] > > > > SpecialClassAttribute: 534ms 535ms 0.45us 1.121ms > > > > ~4x slower! > > Should be investigated as well. It turns out that these two slowdowns are due to classes always being new-style in py3k. Indeed, if I add "__metaclass__ = type" at the beginning of Tools/pybench/Lookups.py, 2.6 becomes as slow as 3.0. Regards Antoine. From barry at python.org Wed Aug 27 01:33:38 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 26 Aug 2008 19:33:38 -0400 Subject: [Python-Dev] Mercurial mirrors In-Reply-To: References: Message-ID: <01F759F8-D271-411B-BC52-7D621C274D61@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 26, 2008, at 6:28 PM, Antoine Pitrou wrote: > Brett Cannon python.org> writes: >> >> But can we push branches up to our personal directories on >> code.python.org like we can with bzr? > > If you have an ssh access to code.python.org, it should be easy. > However, giving > other people anonymous read-only access would require some manual > configuration > I think (not lots of, though). It should not be too hard to adapt the generalizations we made when we brought up the Bazaar repository. Unfortunately, I don't have time to do it, but Thomas and Martin (who are both also probably pretty swamped) probably know how we did it. If not before, we should spend time after the releases in making this available. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSLSS03EjvBPtnXfVAQKAVQP+KCbGOHU0xjfR1mghhdPhlzhHslC1Jwc+ WJAIoDfO/qWGwzBBImIoEqsAqdEBWVbaEjQN57EtL0LRUoR0S3bJdg9P/EeOBDlv 2RHxYZomMFE4rlUJer27oHVBXRO/MgIfWiePQEHowEQQ1pe4INnqiF0OeMlou8sb fYsrpEIun5k= =gtAb -----END PGP SIGNATURE----- From michele.simionato at gmail.com Wed Aug 27 03:16:04 2008 From: michele.simionato at gmail.com (Michele Simionato) Date: Wed, 27 Aug 2008 03:16:04 +0200 Subject: [Python-Dev] Things to Know About Super In-Reply-To: References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> Message-ID: <4edc17eb0808261816r7dd2eff9hb42772e907e3e9bf@mail.gmail.com> On Tue, Aug 26, 2008 at 8:56 PM, Raymond Hettinger wrote: > I would state this differently: "The use cases for cooperative multiple > inheritence don't arise often in practice; so, if we dropped support > for those cases, you probably wouldn't notice until you encountered > one of the rare occasions where it was the right answer to your problem." > > There was some quote floating around that expressed the situation > well -- it went something like: "Python makes most problems easy > and hard problems possible". The use cases for cooperative multiple > inheritance fall in the latter category. It is just a matter of how rare the use cases really are. Cooperative methods has been introduced 6+ years ago. In all this time surely they must have been used. How many compelling uses of cooperation we can find in real life code? For instance in the standard library or in some well known framework? This is a serious question I have been wanting to ask for years. I am sure people here can find some example, so just give me a pointer and we will see. > BTW, I really like your paper explaining the MRO. Excellent work. The issue with that paper is that I wrote it when my Python experience was reduced to six month and my experience with real life large object oriented frameworks was zero. Nowadays I value simplicity more. From aleaxit at gmail.com Wed Aug 27 03:30:17 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Tue, 26 Aug 2008 18:30:17 -0700 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <4edc17eb0808261816r7dd2eff9hb42772e907e3e9bf@mail.gmail.com> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> <4edc17eb0808261816r7dd2eff9hb42772e907e3e9bf@mail.gmail.com> Message-ID: On Tue, Aug 26, 2008 at 6:16 PM, Michele Simionato wrote: ... > It is just a matter of how rare the use cases really are. Cooperative > methods has been introduced 6+ years ago. In all this time surely > they must have been used. How many compelling uses of cooperation > we can find in real life code? For instance in the standard library or > in some well known framework? This is a serious question I have been > wanting to ask for years. I am sure people here can find some example, > so just give me a pointer and we will see. http://www.koders.com/default.aspx?s=super&btn=&la=Python&li=* finds over 5,000 hits, but it would take substantial work to sift through them (in particular because not all refer to the built-in super, as you'll see even in the first page!) -- and a random hit I found by going to p.7 is really bad...: """Mixin to enable reification.""" def __init__(self): super(ReificationStore, self).__init__() [there's *nothing else* in this __init__!]. Alex From mhammond at skippinet.com.au Wed Aug 27 05:08:49 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Wed, 27 Aug 2008 13:08:49 +1000 Subject: [Python-Dev] subprocess insufficiently platform-independent? In-Reply-To: References: Message-ID: <056601c907f2$3d5f36e0$b81da4a0$@com.au> Guido quotes a colleague: > """ > Given a straightforward command list like: > > cmd = ['svn', 'ls', 'http://rietveld.googlecode.com/svn/trunk'] > > You apparently cannot pass this list to any subprocess function > (subprocess.call() or otherwise) with a set of arguments that allow it > to "just work" on both Windows and non-Windows systems. > > If you call: > > subprocess.call(cmd, shell=False) > > Then it works on Linux, but fails on Windows because it does not > perform the Windows %PATHEXT% search that allows it to find that > "svn.exe" is the actual executable to be invoked. I can't reproduce this as described. >>> subprocess.call(['svn', 'ls'], shell=False) svn: '.' is not a working copy 1 The reason this works is that Windows itself (CreateProcess) has support both for implied '.exe' extensions and searching $PATH. Thus, PATHEXT handling isn't required for executables. I *can* reproduce with another extension - eg, 'svn.py' - for this test I had a 'foo.py' on my PATH (but not in the cwd), and .py in PATHEXT. >>> import subprocess >>> subprocess.call(['foo'], shell=False) ... fails >>> subprocess.call(['foo'], shell=True) Hello 0 So I can't see this problem for 'svn.exe' and at face value the behaviour on Windows looks quite correct - you *do* need the shell for this functionality on Windows (in the same way you would to execute a .bat file, for example) > If you call: > > subprocess.call(cmd, shell=True) > > Then it works on Windows (it finds the "svn.exe" executable), but it > fails on Linux because it *only* executes the first argument in the > list ("svn") and does not pass the remaining arguments in the list to > the "svn" invocation. It sounds like in this particular example at least, this behaviour on Linux is the problem? Mark From pje at telecommunity.com Wed Aug 27 05:15:36 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 26 Aug 2008 23:15:36 -0400 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <4edc17eb0808261816r7dd2eff9hb42772e907e3e9bf@mail.gmail.co m> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> <4edc17eb0808261816r7dd2eff9hb42772e907e3e9bf@mail.gmail.com> Message-ID: <20080827031445.C9B8B3A4079@sparrow.telecommunity.com> At 03:16 AM 8/27/2008 +0200, Michele Simionato wrote: >It is just a matter of how rare the use cases really are. Cooperative >methods has been introduced 6+ years ago. In all this time surely >they must have been used. How many compelling uses of cooperation >we can find in real life code? For instance in the standard library or >in some well known framework? This is a serious question I have been >wanting to ask for years. I am sure people here can find some example, >so just give me a pointer and we will see. ISTR pointing out on more than one occasion that a major use case for co-operative super() is in the implementation of metaclasses. The __init__ and __new__ signatures are fixed, multiple inheritance is possible, and co-operativeness is a must (as the base class methods *must* be called). I'm hard-pressed to think of a metaclass constructor or initializer that I've written in the last half-decade or more where I didn't use super() to make it co-operative. That, IMO, is a compelling use case even if there were not a single other example of the need for super. However, I'm pretty sure I've had other cases where it was necessary to co-operate in cases where multiple inheritance occurred later; ie. where it was possible for a subclass to add a new class between parents. Remember that subclasses of a new-style class do not always have the same MRO tail as the original class; i.e., a subclass of "class A(B, C):" is only constrained to have [A...B...C] in its MRO; semi-arbitrary classes may be inserted between e.g. A and B. So, a new-style class cannot, as a general rule, statically determine what base class implementation of a method should be invoked. I personally consider the rare case where I have to force such static knowledge to be an unfortunate wart in the design (of that code, not Python). From michele.simionato at gmail.com Wed Aug 27 05:18:01 2008 From: michele.simionato at gmail.com (Michele Simionato) Date: Wed, 27 Aug 2008 05:18:01 +0200 Subject: [Python-Dev] Things to Know About Super In-Reply-To: References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> <4edc17eb0808261816r7dd2eff9hb42772e907e3e9bf@mail.gmail.com> Message-ID: <4edc17eb0808262018yc904c4dq5320e165ac7dd4c5@mail.gmail.com> On Wed, Aug 27, 2008 at 3:30 AM, Alex Martelli wrote: > On Tue, Aug 26, 2008 at 6:16 PM, Michele Simionato > wrote: > ... >> It is just a matter of how rare the use cases really are. Cooperative >> methods has been introduced 6+ years ago. In all this time surely >> they must have been used. How many compelling uses of cooperation >> we can find in real life code? For instance in the standard library or >> in some well known framework? This is a serious question I have been >> wanting to ask for years. I am sure people here can find some example, >> so just give me a pointer and we will see. > > http://www.koders.com/default.aspx?s=super&btn=&la=Python&li=* finds > over 5,000 hits, but it would take substantial work to sift through > them (in particular because not all refer to the built-in super, as > you'll see even in the first page!) Yep. Notice (I am sure you understood the point correctly, but just to clarify) that I am not interested in random occurrences of super, but in code/frameworks expressly designed to leverage on cooperation and doing it in a compelling way. IOW, I want to see cases where using cooperation is really better than relying on other techniques. Guido gives an example in http://www.python.org/download/releases/2.2.3/descrintro/#cooperation with a .save method, so in theory there are good use cases, but I wonder in practice how common they are and if they are frequent enough to justify the added complication. M.S. From michele.simionato at gmail.com Wed Aug 27 05:48:06 2008 From: michele.simionato at gmail.com (Michele Simionato) Date: Wed, 27 Aug 2008 05:48:06 +0200 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> Message-ID: <4edc17eb0808262048j200f4640v10dff6bc11c434ad@mail.gmail.com> On Tue, Aug 26, 2008 at 6:13 PM, Michele Simionato wrote: > I not completely against multiple inheritance. I am against multiple inheritance > as it is now. A restricted form of multiple inheritance in which mixins classes > are guaranteed to be orthogonal would be fine with me (provided it is > not abused). > This concept exists already in other languages, the orthogonal mixins > are called "traits". I must correct myself here. Even if for practical purposes traits look like a restricted multiple inheritance, in principle it is better to think of them as of an enhanced single inheritance. With traits there is always a single superclass: traits are just single inheritance with a nice syntax to include methods (like in Ruby) and a guarantee that methods will not be overridden silently (this one is missing in Ruby). M.S. From michele.simionato at gmail.com Wed Aug 27 07:24:39 2008 From: michele.simionato at gmail.com (Michele Simionato) Date: Wed, 27 Aug 2008 07:24:39 +0200 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <20080827031445.C9B8B3A4079@sparrow.telecommunity.com> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> <4edc17eb0808261816r7dd2eff9hb42772e907e3e9bf@mail.gmail.com> <20080827031445.C9B8B3A4079@sparrow.telecommunity.com> Message-ID: <4edc17eb0808262224g59c959c9s8a15f778695b8e7a@mail.gmail.com> On Wed, Aug 27, 2008 at 5:15 AM, Phillip J. Eby wrote: > ISTR pointing out on more than one occasion that a major use case for > co-operative super() is in the implementation of metaclasses. The __init__ > and __new__ signatures are fixed, multiple inheritance is possible, and > co-operativeness is a must (as the base class methods *must* be called). > I'm hard-pressed to think of a metaclass constructor or initializer that > I've written in the last half-decade or more where I didn't use super() to > make it co-operative. > > That, IMO, is a compelling use case even if there were not a single other > example of the need for super. I have been giving a lot of thought to this use case, at least from the time of the metaclass conflict recipe. I have always wondered why the recipe had to be so complicated. At the end, I have come to the conclusion that the problem was not with the recipe but with multiple inheritance itself. Let me explain the argument. A possible use case for multiple inheritance of metaclasses is the following: suppose I have a DebugMeta metaclass which adds some debugging support to classes; now I want to apply it to a third party framework which uses a FrameworkMeta metaclass internally. Let us suppose the framework author wrote its metaclass correctly, by supporting cooperation: .. code-block:: python class FrameworkMeta(type): def __new__(mcl, name, bases, dic): print "Adding framework features to %s" % name return super(FrameworkMeta, mcl).__new__(mcl, name, bases, dic) >>> class FrameworkClass(object): ... __metaclass__ = FrameworkMeta Adding framework features to FrameworkClass Moreover, suppose I wrote my DebugMeta to support cooperation correctly: .. code-block:: python class DebugMeta(type): def __new__(mcl, name, bases, dic): print "Adding debugging features to %s" % name return super(DebugMeta, mcl).__new__(mcl, name, bases, dic) Now I can add the debugging features to a class in this way: .. code-block:: python class DebugFrameworkMeta(DebugMeta, FrameworkMeta): pass >>> class DebugFrameworkClass(FrameworkClass): ... __metaclass__ = DebugFrameworkMeta Adding debugging features to DebugFrameworkClass Adding framework features to DebugFrameworkClass As you see everything works fine. Now, lets travel in the fictional world of a fictional language called T-Python which is just like Python, except it lacks multiple inheritance but has some support for traits. By this I mean that there is an ``include_mixin`` function working more or less like this (it could be enhanced but I am keeping it dead simple here for the sake of the argument): .. code-block:: python def include_mixin(mixin, cls): # could be extended to use more mixins # traits as in Squeak take the precedence over the base class dic = vars(mixin).copy() # could be extended to walk the ancestors return type(mixin.__name__ + cls.__name__, (cls,), dic) I will argue that T-Python is not worse than Python for this use case (composition of metaclasses). In the fictional world there is not need for super since all hierarchies are linear and you can just call the base class; FrameworkMeta could have been written as .. code-block:: python class FrameworkMeta2(type): def __new__(mcl, name, bases, dic): print "Adding framework features to %s" % name return type.__new__(mcl, name, bases, dic) and DebugMetas as .. code-block:: python class DebugMeta2(type): def __new__(mcl, name, bases, dic): print "Adding debugging features to %s" % name return mcl.__base__.__new__(mcl, name, bases, dic) Notice that DebugMeta2 is performing a sort of cooperative call here (``mcl.__base__.__new__``) but dead simple since there is just one base class. The analogous of FrameworkClass can be defined as >>> class FrameworkClass2(object): ... __metaclass__ = FrameworkMeta2 Adding framework features to FrameworkClass2 and the analogous of DebugFrameworkClass as >>> class DebugFrameworkClass2(FrameworkClass2): ... __metaclass__ = DebugFrameworkMeta2 Adding debugging features to DebugFrameworkClass2 Adding framework features to DebugFrameworkClass2 So, as you see, it works. Checks of the kind ``isinstance(DebugFrameworkClass2, DebugMeta2)`` would fail, but this is not a big issue (isinstance should not be used, or you could register DebugMeta2 as a base class even if it is not by using Python 2.6 ABC's). Now, I am not claiming that I have thought of all possible usages of multiple inheritance and metaclasses: however I have not found yet a use case that I could not rewrite by using single-inheritance + traits as the one I have just shown. Possibly there are cases which are difficult to rewrite: but how common are they? Notice that I am not advocating rewriting Python. The argument here is purely hyphotetic and concerning a fictional language: I just want to understand if full multiple inheritance is really that useful or not, as a matter of principle. M.S. From michele.simionato at gmail.com Wed Aug 27 08:02:47 2008 From: michele.simionato at gmail.com (Michele Simionato) Date: Wed, 27 Aug 2008 08:02:47 +0200 Subject: [Python-Dev] Things to Know About Super In-Reply-To: References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> Message-ID: <4edc17eb0808262302gc770ec6s5d72f5db011f9626@mail.gmail.com> On Tue, Aug 26, 2008 at 11:10 PM, Steve Holden wrote: > If you aren't aware of it you should take a look at Enthought's traits > package. It's part of the Enthought Tool Suite (ETS). I know of the existence of that framework, however it is quite large and I don't see the relation with the concept of traits I have in mind, which is more or less the one described here: http://www.iam.unibe.ch/%7Escg/Archive/Papers/Scha03aTraits.pdf Basically, these are the properties of traits: 1. the methods/attributes in a trait go logically together; 2. if a trait enhances a class, then all subclasses are enhanced too; 3. if a trait has methods in common with the class, then the methods defined in the class have the precedence; 4. the ordering of traits is not important, i.e. enhancing a class first with trait T1 and then with trait T2 or viceversa is the same; 5. if traits T1 and T2 have names in common, enhancing a class both with T1 and T2 raises an error unless there is an explicitoverriding; 6. if a trait has methods in common with the base class, then the trait methods have the precedence; Properties from 4 to 6 are the distinguishing properties of traits with respect to multiple inheritance and mixins. From kilian.klimek at googlemail.com Wed Aug 27 08:46:15 2008 From: kilian.klimek at googlemail.com (Kilian Klimek) Date: Wed, 27 Aug 2008 08:46:15 +0200 Subject: [Python-Dev] the explicit self Message-ID: <647a40920808262346y5c5593fal285afe85a39a27b4@mail.gmail.com> Hello, i know this has been discusses very much, i'm sorry, but i can't help it. In a nutshell, the proposal is as follows: 1. Self remains explicit (like it is now). 2. if a class is a subclass of a special class, e.g. named 'selfless', the self parameter is not required and a special variable, named 'this' is provided. For example: class Foo (selfless): def __init__ (x, y): this.x = x ... A patch for 3.0b3 implementing this can be found at http://www-lehre.inf.uos.de/~kklimek/misc/python_slp_8.diff regards, Kilian Klimek -------------- next part -------------- An HTML attachment was scrubbed... URL: From cesare.dimauro at a-tono.com Wed Aug 27 09:14:05 2008 From: cesare.dimauro at a-tono.com (Cesare Di Mauro) Date: Wed, 27 Aug 2008 09:14:05 +0200 Subject: [Python-Dev] the explicit self In-Reply-To: <647a40920808262346y5c5593fal285afe85a39a27b4@mail.gmail.com> References: <647a40920808262346y5c5593fal285afe85a39a27b4@mail.gmail.com> Message-ID: On 27 agu 2008 at 08:46:15, Kilian Klimek wrote: > Hello, > > i know this has been discusses very much, i'm sorry, > but i can't help it. In a nutshell, the proposal is as > follows: > > 1. Self remains explicit (like it is now). > 2. if a class is a subclass of a special class, e.g. > named 'selfless', the self parameter is not required > and a special variable, named 'this' is provided. > > > For example: > > class Foo (selfless): > def __init__ (x, y): > this.x = x > ... > > A patch for 3.0b3 implementing this can be found at > http://www-lehre.inf.uos.de/~kklimek/misc/python_slp_8.diff > > regards, > Kilian Klimek > I disagree. From "The Zen of Python": Explicit is better than implicit. Readability counts. Special cases aren't special enough to break the rules. There should be one-- and preferably only one --obvious way to do it. Cheers, Cesare From gherron at islandtraining.com Wed Aug 27 09:12:50 2008 From: gherron at islandtraining.com (Gary Herron) Date: Wed, 27 Aug 2008 00:12:50 -0700 Subject: [Python-Dev] the explicit self In-Reply-To: <647a40920808262346y5c5593fal285afe85a39a27b4@mail.gmail.com> References: <647a40920808262346y5c5593fal285afe85a39a27b4@mail.gmail.com> Message-ID: <48B4FE72.6050200@islandtraining.com> Kilian Klimek wrote: > Hello, > > i know this has been discusses very much, i'm sorry, > but i can't help it. In a nutshell, the proposal is as > follows: > > 1. Self remains explicit (like it is now). > 2. if a class is a subclass of a special class, e.g. > named 'selfless', the self parameter is not required > and a special variable, named 'this' is provided. > > > For example: > > class Foo (selfless): > def __init__ (x, y): > this.x = x > ... > > A patch for 3.0b3 implementing this can be found at > http://www-lehre.inf.uos.de/~kklimek/misc/python_slp_8.diff > Why not just do this? class Foo: def __init__ (this, x, y): this.x = x It's fewer characters, it gets rid of the "self" you seem to dread, and it requires no patches or changes of any kind to Python. And most importantly, has no need to introduce any "magic" into the language. Gary Herron > > regards, > Kilian Klimek > ------------------------------------------------------------------------ > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/gherron%40islandtraining.com > From tjreedy at udel.edu Wed Aug 27 09:40:01 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 27 Aug 2008 03:40:01 -0400 Subject: [Python-Dev] the explicit self In-Reply-To: <647a40920808262346y5c5593fal285afe85a39a27b4@mail.gmail.com> References: <647a40920808262346y5c5593fal285afe85a39a27b4@mail.gmail.com> Message-ID: Kilian Klimek wrote: > i know this has been discusses very much, There is a related discussion on python-ideas right now and was a long discussion on python-list/c.l.p within the last month. And the month before..... Either would have been the place to post this. > i'm sorry, but i can't help it. ??? From g.brandl at gmx.net Wed Aug 27 09:47:10 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 27 Aug 2008 09:47:10 +0200 Subject: [Python-Dev] the explicit self In-Reply-To: <647a40920808262346y5c5593fal285afe85a39a27b4@mail.gmail.com> References: <647a40920808262346y5c5593fal285afe85a39a27b4@mail.gmail.com> Message-ID: Kilian Klimek schrieb: > Hello, > > i know this has been discusses very much, i'm sorry, > but i can't help it. In a nutshell, the proposal is as > follows: > > 1. Self remains explicit (like it is now). > 2. if a class is a subclass of a special class, e.g. > named 'selfless', the self parameter is not required > and a special variable, named 'this' is provided. > > > For example: > > class Foo (selfless): > def __init__ (x, y): > this.x = x > ... > > A patch for 3.0b3 implementing this can be found at > http://www-lehre.inf.uos.de/~kklimek/misc/python_slp_8.diff Just a note about the patch: it introduces the implicit "this" by injecting it into the function's globals. Due to how Python looks up globals and locals, this will be much slower than having a local called "self". Georg From greg.ewing at canterbury.ac.nz Wed Aug 27 09:54:47 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 27 Aug 2008 19:54:47 +1200 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <20080827031445.C9B8B3A4079@sparrow.telecommunity.com> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> <4edc17eb0808261816r7dd2eff9hb42772e907e3e9bf@mail.gmail.com> <20080827031445.C9B8B3A4079@sparrow.telecommunity.com> Message-ID: <48B50847.6060703@canterbury.ac.nz> Phillip J. Eby wrote: > ISTR pointing out on more than one occasion that a major use case for > co-operative super() is in the implementation of metaclasses. The > __init__ and __new__ signatures are fixed, multiple inheritance is > possible, and co-operativeness is a must Do you have a real-life example of this where multiple inheritance is actually used? A non-contrived example or two would be a good thing to have in tutorials etc. where super() is discussed. It would help to convey the kinds of situations in which use of super() is and is not appropriate. -- Greg From fijall at gmail.com Wed Aug 27 10:07:28 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 27 Aug 2008 10:07:28 +0200 Subject: [Python-Dev] the explicit self In-Reply-To: <647a40920808262346y5c5593fal285afe85a39a27b4@mail.gmail.com> References: <647a40920808262346y5c5593fal285afe85a39a27b4@mail.gmail.com> Message-ID: <693bc9ab0808270107o74dd7417la66dd6c552b6a2ce@mail.gmail.com> You can provide selfless class as a class with special metaclass that overloads __new__ and changes signature of each method. Not sure how good is this, but requires no changes to the language and will work as you want. Cheers, fijal On Wed, Aug 27, 2008 at 8:46 AM, Kilian Klimek wrote: > Hello, > > i know this has been discusses very much, i'm sorry, > but i can't help it. In a nutshell, the proposal is as > follows: > > 1. Self remains explicit (like it is now). > 2. if a class is a subclass of a special class, e.g. > named 'selfless', the self parameter is not required > and a special variable, named 'this' is provided. > > > For example: > > class Foo (selfless): > def __init__ (x, y): > this.x = x > ... > > A patch for 3.0b3 implementing this can be found at > http://www-lehre.inf.uos.de/~kklimek/misc/python_slp_8.diff > > regards, > Kilian Klimek > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com > > From fuzzyman at voidspace.org.uk Wed Aug 27 10:21:08 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 27 Aug 2008 09:21:08 +0100 Subject: [Python-Dev] the explicit self In-Reply-To: <693bc9ab0808270107o74dd7417la66dd6c552b6a2ce@mail.gmail.com> References: <647a40920808262346y5c5593fal285afe85a39a27b4@mail.gmail.com> <693bc9ab0808270107o74dd7417la66dd6c552b6a2ce@mail.gmail.com> Message-ID: <48B50E74.4090805@voidspace.org.uk> Maciej Fijalkowski wrote: > You can provide selfless class as a class with special metaclass that > overloads __new__ and changes signature of each method. Not sure how > good is this, but requires no changes to the language and will work as > you want. > > Are you advocating this Maciej? ;-) There's an example that does this here: http://www.voidspace.org.uk/python/articles/metaclasses.shtml#the-selfless-metaclass Michael > Cheers, > fijal > > On Wed, Aug 27, 2008 at 8:46 AM, Kilian Klimek > wrote: > >> Hello, >> >> i know this has been discusses very much, i'm sorry, >> but i can't help it. In a nutshell, the proposal is as >> follows: >> >> 1. Self remains explicit (like it is now). >> 2. if a class is a subclass of a special class, e.g. >> named 'selfless', the self parameter is not required >> and a special variable, named 'this' is provided. >> >> >> For example: >> >> class Foo (selfless): >> def __init__ (x, y): >> this.x = x >> ... >> >> A patch for 3.0b3 implementing this can be found at >> http://www-lehre.inf.uos.de/~kklimek/misc/python_slp_8.diff >> >> regards, >> Kilian Klimek >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com >> >> >> > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.resolverhacks.net/ http://wwww.theotherdelia.co.uk/ From mal at egenix.com Wed Aug 27 12:53:16 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 27 Aug 2008 12:53:16 +0200 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <48B50847.6060703@canterbury.ac.nz> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> <4edc17eb0808261816r7dd2eff9hb42772e907e3e9bf@mail.gmail.com> <20080827031445.C9B8B3A4079@sparrow.telecommunity.com> <48B50847.6060703@canterbury.ac.nz> Message-ID: <48B5321C.2080303@egenix.com> On 2008-08-27 09:54, Greg Ewing wrote: > Do you have a real-life example of this where multiple > inheritance is actually used? > > A non-contrived example or two would be a good thing to > have in tutorials etc. where super() is discussed. It > would help to convey the kinds of situations in which > use of super() is and is not appropriate. The typical use is in mixin classes that can be used to add functionality to base classes, something you often find in application frameworks, e.g. class NewComponent(Feature1Mixin, Feature2Mixin, BaseComponent): ... If the mixin classes have to override one of the methods defined in BaseComponent, then they must pay attention to all other mixin classes used to define the NewComponent. Without super() (or some other mechanism of accessing the base method, like e.g. mxTools' basemethod() for classic classes), the mixins could potentially override methods defined by other mixin classes which would then not get called. As example, think of a typical application server method def process_request(self, request): ... To work properly, each implementation of the method in the mixin classes and base class will have to be called - in the order they were defined in the class definition. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 27 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From kilian.klimek at googlemail.com Wed Aug 27 15:47:18 2008 From: kilian.klimek at googlemail.com (Kilian Klimek) Date: Wed, 27 Aug 2008 15:47:18 +0200 Subject: [Python-Dev] the explicit self In-Reply-To: References: <647a40920808262346y5c5593fal285afe85a39a27b4@mail.gmail.com> Message-ID: <647a40920808270647t5b67005cyb7f7c574b8d1075f@mail.gmail.com> On Wed, Aug 27, 2008 at 9:14 AM, Cesare Di Mauro wrote: > On 27 agu 2008 at 08:46:15, Kilian Klimek > wrote: > > > Hello, > > > > i know this has been discusses very much, i'm sorry, > > but i can't help it. In a nutshell, the proposal is as > > follows: > > > > 1. Self remains explicit (like it is now). > > 2. if a class is a subclass of a special class, e.g. > > named 'selfless', the self parameter is not required > > and a special variable, named 'this' is provided. > > > > > > For example: > > > > class Foo (selfless): > > def __init__ (x, y): > > this.x = x > > ... > > > > A patch for 3.0b3 implementing this can be found at > > http://www-lehre.inf.uos.de/~kklimek/misc/python_slp_8.diff > > > > regards, > > Kilian Klimek > > > > I disagree. From "The Zen of Python": > > Explicit is better than implicit. no point in discussing this, but ... someone else emailed me this one too and i don't see the point: writing down 'self' as the first paramenter of your method does not make it any more explicit what it means. (If you want to consider readability: even worse, you can name it whatever you want). Saying "your method must accept an extra parameter (which most people call 'self') that carries all object attributes" is hardly any more explicit then saying "there is a special variable (which is always named 'this') that carries all object attributes". > > Readability counts. > Special cases aren't special enough to break the rules. > There should be one-- and preferably only one --obvious way to do it. > > Cheers, > Cesare > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aleaxit at gmail.com Wed Aug 27 16:30:29 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Wed, 27 Aug 2008 07:30:29 -0700 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <4edc17eb0808262224g59c959c9s8a15f778695b8e7a@mail.gmail.com> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> <4edc17eb0808261816r7dd2eff9hb42772e907e3e9bf@mail.gmail.com> <20080827031445.C9B8B3A4079@sparrow.telecommunity.com> <4edc17eb0808262224g59c959c9s8a15f778695b8e7a@mail.gmail.com> Message-ID: On Tue, Aug 26, 2008 at 10:24 PM, Michele Simionato wrote: ... > .. code-block:: python > > def include_mixin(mixin, cls): # could be extended to use more mixins > # traits as in Squeak take the precedence over the base class > dic = vars(mixin).copy() # could be extended to walk the ancestors > return type(mixin.__name__ + cls.__name__, (cls,), dic) I don't see any use of this in your following example so I assume you're introducing it just to be able to say that: > In the fictional world there is not need for super since > all hierarchies are linear and you can just call the base class; Nevertheless I must be missing something in the following example: > FrameworkMeta could have been written as > > .. code-block:: python > > class FrameworkMeta2(type): > def __new__(mcl, name, bases, dic): > print "Adding framework features to %s" % name > return type.__new__(mcl, name, bases, dic) > > > and DebugMetas as > > .. code-block:: python > > class DebugMeta2(type): > def __new__(mcl, name, bases, dic): > print "Adding debugging features to %s" % name > return mcl.__base__.__new__(mcl, name, bases, dic) > > > Notice that DebugMeta2 is performing a sort of cooperative call here > (``mcl.__base__.__new__``) but dead simple since there is just one base class. > > The analogous of FrameworkClass can be defined as > >>>> class FrameworkClass2(object): > ... __metaclass__ = FrameworkMeta2 > Adding framework features to FrameworkClass2 > > and the analogous of DebugFrameworkClass as > >>>> class DebugFrameworkClass2(FrameworkClass2): > ... __metaclass__ = DebugFrameworkMeta2 What's DebugFrameworkMeta2? I assume it's some kind of mix but I don't see it defined anywhere so I'm having to guess. > Adding debugging features to DebugFrameworkClass2 > Adding framework features to DebugFrameworkClass2 But where does DebugMeta2 get injected or otherwise get into the picture in this example, so that the "Adding debugging features" print executes? I don't see any code in your example that performs this injection. Maybe you're missing a key bit where you build DebugFrameworkMeta2 by using that example include_mixin you have? I'd like to understand what, in this example, removes the apparent "fragility" (or, lack of flexibility) where DebugMeta2 specifically uses mcl.__base__ -- IOW, if I have another "mixin metaclass" just like DebugMeta2 but called (say) RemotableMeta which does a print "Adding remoting features" and then also calls mcl.__base__.__new__(mcl ... just like DebugMeta2, what gets both of their __new__ methods called in the right order? Maybe you could help me understand this by giving a fully executable Python snippet using __bases__[0] instead of the hypothetical __base__? A;ex From michele.simionato at gmail.com Wed Aug 27 16:36:15 2008 From: michele.simionato at gmail.com (Michele Simionato) Date: Wed, 27 Aug 2008 16:36:15 +0200 Subject: [Python-Dev] Things to Know About Super In-Reply-To: References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> <4edc17eb0808261816r7dd2eff9hb42772e907e3e9bf@mail.gmail.com> <20080827031445.C9B8B3A4079@sparrow.telecommunity.com> <4edc17eb0808262224g59c959c9s8a15f778695b8e7a@mail.gmail.com> Message-ID: <4edc17eb0808270736s2a5b3211x15c20aa8f74f48eb@mail.gmail.com> On Wed, Aug 27, 2008 at 4:30 PM, Alex Martelli wrote: > Maybe you could help me understand this by giving a fully executable > Python snippet using __bases__[0] instead of the hypothetical > __base__? Sorry Alex, I have the fully functional snippet but evidently I have sent some other blurb instead (it was early in the morning) It is on my machine at home and now I am at work, so have patience ;) Michele From guido at python.org Wed Aug 27 18:43:25 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 27 Aug 2008 09:43:25 -0700 Subject: [Python-Dev] subprocess insufficiently platform-independent? In-Reply-To: <056601c907f2$3d5f36e0$b81da4a0$@com.au> References: <056601c907f2$3d5f36e0$b81da4a0$@com.au> Message-ID: On Tue, Aug 26, 2008 at 8:08 PM, Mark Hammond wrote: > Guido quotes a colleague: > >> """ >> Given a straightforward command list like: >> >> cmd = ['svn', 'ls', 'http://rietveld.googlecode.com/svn/trunk'] >> >> You apparently cannot pass this list to any subprocess function >> (subprocess.call() or otherwise) with a set of arguments that allow it >> to "just work" on both Windows and non-Windows systems. >> >> If you call: >> >> subprocess.call(cmd, shell=False) >> >> Then it works on Linux, but fails on Windows because it does not >> perform the Windows %PATHEXT% search that allows it to find that >> "svn.exe" is the actual executable to be invoked. > > I can't reproduce this as described. Which Windows version? This sounds like one of those things that could well vary by Windows version; if it works for you in Vista it may well be broken in XP. It could also vary by other setup parameters besides PATHEXT. >>>> subprocess.call(['svn', 'ls'], shell=False) > svn: '.' is not a working copy > 1 > > The reason this works is that Windows itself (CreateProcess) has support > both for implied '.exe' extensions and searching $PATH. Thus, PATHEXT > handling isn't required for executables. > > I *can* reproduce with another extension - eg, 'svn.py' - for this test I > had a 'foo.py' on my PATH (but not in the cwd), and .py in PATHEXT. > >>>> import subprocess >>>> subprocess.call(['foo'], shell=False) > ... fails >>>> subprocess.call(['foo'], shell=True) > Hello > 0 > > So I can't see this problem for 'svn.exe' and at face value the behaviour on > Windows looks quite correct - you *do* need the shell for this functionality > on Windows (in the same way you would to execute a .bat file, for example) > >> If you call: >> >> subprocess.call(cmd, shell=True) >> >> Then it works on Windows (it finds the "svn.exe" executable), but it >> fails on Linux because it *only* executes the first argument in the >> list ("svn") and does not pass the remaining arguments in the list to >> the "svn" invocation. > > It sounds like in this particular example at least, this behaviour on Linux > is the problem? I think so yes. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From curt at hagenlocher.org Wed Aug 27 19:03:20 2008 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Wed, 27 Aug 2008 10:03:20 -0700 Subject: [Python-Dev] subprocess insufficiently platform-independent? In-Reply-To: References: <056601c907f2$3d5f36e0$b81da4a0$@com.au> Message-ID: On Wed, Aug 27, 2008 at 9:43 AM, Guido van Rossum wrote: > On Tue, Aug 26, 2008 at 8:08 PM, Mark Hammond wrote: >>> >>> Then it works on Linux, but fails on Windows because it does not >>> perform the Windows %PATHEXT% search that allows it to find that >>> "svn.exe" is the actual executable to be invoked. >> >> I can't reproduce this as described. > > Which Windows version? This sounds like one of those things that could > well vary by Windows version; if it works for you in Vista it may well > be broken in XP. It could also vary by other setup parameters besides > PATHEXT. When passing the executable name to CreateProcess via the lpCommandLine parameter, PATH is considered but PATHEXT is ignored. The only extension that's automatically appended is ".exe", and only if no other extension is present. This has been true for as long as I can remember. I've found the documentation for CreateProcess (http://msdn.microsoft.com/en-us/library/ms682425.aspx) to be pretty reliable. And the mention of a ".com" in the docs suggests that the description has been around for a while... -- Curt Hagenlocher curt at hagenlocher.org From fredrik at pythonware.com Wed Aug 27 19:26:09 2008 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 27 Aug 2008 19:26:09 +0200 Subject: [Python-Dev] the explicit self In-Reply-To: <647a40920808270647t5b67005cyb7f7c574b8d1075f@mail.gmail.com> References: <647a40920808262346y5c5593fal285afe85a39a27b4@mail.gmail.com> <647a40920808270647t5b67005cyb7f7c574b8d1075f@mail.gmail.com> Message-ID: Kilian Klimek wrote: > Saying "your method must accept an extra parameter (which most people > call 'self') that carries all object attributes" is hardly any more > explicit then saying "there is a special variable (which is always named > 'this') that carries all object attributes". in this context, implicit and explicit refers to the code, not the documentation. From fredrik at pythonware.com Wed Aug 27 19:34:14 2008 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 27 Aug 2008 19:34:14 +0200 Subject: [Python-Dev] subprocess insufficiently platform-independent? In-Reply-To: References: <056601c907f2$3d5f36e0$b81da4a0$@com.au> Message-ID: Guido van Rossum wrote: >> I can't reproduce this as described. > > Which Windows version? This sounds like one of those things that could > well vary by Windows version; if it works for you in Vista it may well > be broken in XP. It could also vary by other setup parameters besides > PATHEXT. It works the same way on XP, at least: >>> import subprocess >>> cmd = ['svn', 'ls', '.'] >>> subprocess.call(cmd, shell=False) svn: '.' is not a working copy 1 According to the MS docs, CreateProcess works the same way on at least 2K, XP and Vista. The documentation is a bit vague (as usual), but it contains an example that implies that CreateProcess always adds ".exe" if not already there, and that you need to use the command interpreter (that is, shell=True) if you want to run something that's not a Windows executable module (e.g. a batch file). From fredrik at pythonware.com Wed Aug 27 19:36:12 2008 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 27 Aug 2008 19:36:12 +0200 Subject: [Python-Dev] subprocess insufficiently platform-independent? In-Reply-To: References: <056601c907f2$3d5f36e0$b81da4a0$@com.au> Message-ID: Curt Hagenlocher wrote: > I've found the documentation for CreateProcess > (http://msdn.microsoft.com/en-us/library/ms682425.aspx) to be pretty > reliable. And the mention of a ".com" in the docs suggests that the > description has been around for a while... And I just described it as pretty vague ;-) Reading it again, I realize that I completely missed the part that says: "If the file name does not contain an extension, .exe is appended. Therefore, if the file name extension is .com, this parameter must include the .com extension. If the file name ends in a period (.) with no extension, or if the file name contains a path, .exe is not appended." From fredrik at pythonware.com Wed Aug 27 20:21:21 2008 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 27 Aug 2008 20:21:21 +0200 Subject: [Python-Dev] confusing exec error message in 3.0 Message-ID: (using 3.0a4) >>> exec(open("file.py")) Traceback (most recent call last): File " ", line 1, in TypeError: exec() arg 1 must be a string, file, or code object, not TextIOWrapper so what's "file" referring to here? (the above works under 2.5, of course) From fredrik at pythonware.com Wed Aug 27 20:35:29 2008 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 27 Aug 2008 20:35:29 +0200 Subject: [Python-Dev] confusing exec error message in 3.0 In-Reply-To: References: Message-ID: Fredrik Lundh wrote: > (using 3.0a4) ahem. I could have sworn that I installed a beta, but I guess the Windows builds weren't fully synchronized when I did that. I still get the same error after updating to 3.0b2, though. (the download page still says "This is an alpha release", btw.) From g.brandl at gmx.net Thu Aug 28 00:39:01 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 28 Aug 2008 00:39:01 +0200 Subject: [Python-Dev] confusing exec error message in 3.0 In-Reply-To: References: Message-ID: Fredrik Lundh schrieb: > (using 3.0a4) > > >>> exec(open("file.py")) > Traceback (most recent call last): > File " ", line 1, in > TypeError: exec() arg 1 must be a string, file, or code object, not > TextIOWrapper > > so what's "file" referring to here? > > (the above works under 2.5, of course) See http://bugs.python.org/issue1762972 -- it has been decided to drop that possibility. I've a patch that fixes the wrong error message in http://bugs.python.org/issue3706. Georg From Ben.Cottrell at nominum.com Thu Aug 28 00:28:50 2008 From: Ben.Cottrell at nominum.com (Ben.Cottrell at nominum.com) Date: Wed, 27 Aug 2008 15:28:50 -0700 Subject: [Python-Dev] Reference leak in thread._local Message-ID: <20080827222850.AAC451A8206@shell-ng.nominum.com> I noticed that thread._local can leak references if objects are being stored inside the thread._local object whose destructors might release the GIL. The way this happens is that in Modules/threadmodule.c, in the _ldict() function, it does things like this: Py_CLEAR(self->dict); Py_INCREF(ldict); self->dict = ldict; If the Py_CLEAR ends up taking away the last reference to an object contained in the dict, and a thread context switch occurs during that object's deallocation, then self->dict might not be NULL on return from Py_CLEAR; another thread might have run, accessed something in the same thread._local object, and caused self->dict to be set to something else (and Py_INCREF'ed). So when we blindly do the assignment into self->dict, we may be overwriting a valid reference, and not properly Py_DECREFing it. The recent change (revision 64601 to threadmodule.c) did not address context switches during the Py_CLEAR call; only context switches during tp_init. The attached patch (against trunk) is my first attempt at fixing this. It detects if self->dict has been set to something else after the Py_CLEAR, and retries the Py_CLEAR (because _ldict really only cares about installing the proper value of self->dict for the currently running thread). However, I am still uncomfortable about the fact that local_getattro and local_setattro discard the value returned from _ldict, and instead hand off control to the PyObject_Generic layer and trust that by the time self->dict is actually used, it still has the correct value for the current thread. Would it be better to, say, inline a copy of the PyObject_Generic* functions inside local_getattro/local_setattro, and force the operations to be done on the actual dict returned by _ldict()? Thanks, ~Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/x-patch Size: 672 bytes Desc: not available URL: From fuzzyman at voidspace.org.uk Thu Aug 28 00:55:56 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 27 Aug 2008 23:55:56 +0100 Subject: [Python-Dev] confusing exec error message in 3.0 In-Reply-To: References: Message-ID: <48B5DB7C.8020103@voidspace.org.uk> Georg Brandl wrote: > Fredrik Lundh schrieb: > >> (using 3.0a4) >> >> >>> exec(open("file.py")) >> Traceback (most recent call last): >> File " ", line 1, in >> TypeError: exec() arg 1 must be a string, file, or code object, not >> TextIOWrapper >> >> so what's "file" referring to here? >> >> (the above works under 2.5, of course) >> > > See http://bugs.python.org/issue1762972 -- it has been decided to drop > that possibility. > > I've a patch that fixes the wrong error message in http://bugs.python.org/issue3706. > In order to obtain a string from a Python source code file, honouring encoding cookies, the tokenize module has a 'detect_encoding' function that could be useful. Michael > Georg > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From aahz at pythoncraft.com Thu Aug 28 01:41:59 2008 From: aahz at pythoncraft.com (Aahz) Date: Wed, 27 Aug 2008 16:41:59 -0700 Subject: [Python-Dev] Reference leak in thread._local In-Reply-To: <20080827222850.AAC451A8206@shell-ng.nominum.com> References: <20080827222850.AAC451A8206@shell-ng.nominum.com> Message-ID: <20080827234159.GA25481@panix.com> On Wed, Aug 27, 2008, Ben.Cottrell at nominum.com wrote: > > I noticed that thread._local can leak references if objects are > being stored inside the thread._local object whose destructors > might release the GIL. Please post this bug report & patch to bugs.python.org -- otherwise, it will almost certainly get lost. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Adopt A Process -- stop killing all your children! From greg.ewing at canterbury.ac.nz Thu Aug 28 02:14:15 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 28 Aug 2008 12:14:15 +1200 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <48B5321C.2080303@egenix.com> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> <4edc17eb0808261816r7dd2eff9hb42772e907e3e9bf@mail.gmail.com> <20080827031445.C9B8B3A4079@sparrow.telecommunity.com> <48B50847.6060703@canterbury.ac.nz> <48B5321C.2080303@egenix.com> Message-ID: <48B5EDD7.5050802@canterbury.ac.nz> M.-A. Lemburg wrote: > The typical use is in mixin classes that can be used to > add functionality to base classes... But this is just another waffly made-up example. I'm talking about real-life use cases from actual code that's in use. -- Greg From guido at python.org Thu Aug 28 03:10:00 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 27 Aug 2008 18:10:00 -0700 Subject: [Python-Dev] confusing exec error message in 3.0 In-Reply-To: References: Message-ID: 2008/8/27 Fredrik Lundh : > Fredrik Lundh wrote: > >> (using 3.0a4) > > ahem. I could have sworn that I installed a beta, but I guess the Windows builds weren't fully synchronized when I did that. I still get the same error after updating to 3.0b2, though. It's still there. It's a hold-over from 2.x where opened files were indeed usable for exec(). It was a rarely-used feature that turned out to be hard to implement given the continued reliance of the parser on while the I/O system no longer uses that, so it was dropped with very little protests. Please do fix it! -- --Guido van Rossum (home page: http://www.python.org/~guido/) -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Thu Aug 28 03:21:15 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 28 Aug 2008 11:21:15 +1000 Subject: [Python-Dev] confusing exec error message in 3.0 In-Reply-To: References: Message-ID: <200808281121.15908.steve@pearwood.info> On Thu, 28 Aug 2008 08:39:01 am Georg Brandl wrote: > Fredrik Lundh schrieb: > > (using 3.0a4) > > > > >>> exec(open("file.py")) > > > > Traceback (most recent call last): > > File " ", line 1, in > > TypeError: exec() arg 1 must be a string, file, or code object, not > > TextIOWrapper > > > > so what's "file" referring to here? > > > > (the above works under 2.5, of course) > > See http://bugs.python.org/issue1762972 -- it has been decided to > drop that possibility. Hmmm... I have a concern with one of the patches in that issue; I refer to patch that changes the semantics of module's __file__ attribute. Guido noted that exec(open(M.__file__).read(), M.__dict__) almost worked as a replacement for reload(), except that there were issues with file extensions (.py, .pyc, .pyo and even things like .pyc24). So it was decided that M.__file__ should always point to the source file. But now that reload() is now moved into the imp module, I don't see that the justification for changing the semantics of M.__file__ still exists. imp.reload(M) is much better for interactive use than exec(open(M.__file__).read(), M.__dict__). Is there still a justification for having M.__file__ point to the source even if the module was actually loaded from the .pyc version? If so, what is that? Some years ago, as a newbie, I was having trouble with reload() repeatedly not picking up changes I was making to a module. The problem turned out to be user-error, but how I discovered that was by looking at M.__file__ and noticing that Python was loading the .pyc file instead of the .py file I was expecting. Had M.__file__ given misleading information, I would have been mislead for much longer. Here's a small contrived example under Python 2.5: >>> import parrot >>> parrot.__file__ 'parrot.py' >>> # pretend that I made changes to the source ... # in my editor, but forgot to save them ... reload(parrot) >>> parrot.__file__ 'parrot.pyc' I don't think M.__file__ should lie and say it was loaded from a file that it wasn't loaded from. It's useful to be able to look at a module and see what file it was actually loaded from. -- Steven From guido at python.org Thu Aug 28 05:38:14 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 27 Aug 2008 20:38:14 -0700 Subject: [Python-Dev] confusing exec error message in 3.0 In-Reply-To: <200808281121.15908.steve@pearwood.info> References: <200808281121.15908.steve@pearwood.info> Message-ID: On Wed, Aug 27, 2008 at 6:21 PM, Steven D'Aprano wrote: > On Thu, 28 Aug 2008 08:39:01 am Georg Brandl wrote: >> Fredrik Lundh schrieb: >> > (using 3.0a4) >> > >> > >>> exec(open("file.py")) >> > >> > Traceback (most recent call last): >> > File " ", line 1, in >> > TypeError: exec() arg 1 must be a string, file, or code object, not >> > TextIOWrapper >> > >> > so what's "file" referring to here? >> > >> > (the above works under 2.5, of course) >> >> See http://bugs.python.org/issue1762972 -- it has been decided to >> drop that possibility. > > Hmmm... I have a concern with one of the patches in that issue; I refer > to patch that changes the semantics of module's __file__ attribute. > > Guido noted that exec(open(M.__file__).read(), M.__dict__) almost worked > as a replacement for reload(), except that there were issues with file > extensions (.py, .pyc, .pyo and even things like .pyc24). So it was > decided that M.__file__ should always point to the source file. > > But now that reload() is now moved into the imp module, I don't see that > the justification for changing the semantics of M.__file__ still > exists. imp.reload(M) is much better for interactive use than > exec(open(M.__file__).read(), M.__dict__). > > Is there still a justification for having M.__file__ point to the source > even if the module was actually loaded from the .pyc version? If so, > what is that? > > Some years ago, as a newbie, I was having trouble with reload() > repeatedly not picking up changes I was making to a module. The problem > turned out to be user-error, but how I discovered that was by looking > at M.__file__ and noticing that Python was loading the .pyc file > instead of the .py file I was expecting. Had M.__file__ given > misleading information, I would have been mislead for much longer. > Here's a small contrived example under Python 2.5: > >>>> import parrot >>>> parrot.__file__ > 'parrot.py' >>>> # pretend that I made changes to the source > ... # in my editor, but forgot to save them > ... reload(parrot) > >>>> parrot.__file__ > 'parrot.pyc' > > > I don't think M.__file__ should lie and say it was loaded from a file > that it wasn't loaded from. It's useful to be able to look at a module > and see what file it was actually loaded from. While appreciate the use case, there are way more use cases where there's code that must painstakingly strip the trailing 'c' or 'o' from __file__ in order to read the source code. Perhaps we should have a separate API for finding out whether a module was loaded from source or from a .pyc file; but I think it would be better to have such an API somewhere in the imp module. It's also possible to follow what goes on by watching the verbose -v output. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From michele.simionato at gmail.com Thu Aug 28 06:35:08 2008 From: michele.simionato at gmail.com (Michele Simionato) Date: Thu, 28 Aug 2008 06:35:08 +0200 Subject: [Python-Dev] Things to Know About Super In-Reply-To: References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> <4edc17eb0808261816r7dd2eff9hb42772e907e3e9bf@mail.gmail.com> <20080827031445.C9B8B3A4079@sparrow.telecommunity.com> <4edc17eb0808262224g59c959c9s8a15f778695b8e7a@mail.gmail.com> Message-ID: <4edc17eb0808272135q44985feaj1ba550022919eaac@mail.gmail.com> Alex Martelli wrote: > What's DebugFrameworkMeta2? I assume it's some kind of mix but I > don't see it defined anywhere so I'm having to guess. Sorry Alex, here is definition which got lost in cut&paste: DebugFrameworkMeta2 = include_mixin(DebugMeta2, FrameworkMeta2) > I'd like to understand what, in this example, removes the apparent > "fragility" (or, lack of flexibility) where DebugMeta2 specifically > uses mcl.__base__ -- IOW, if I have another "mixin metaclass" just > like DebugMeta2 but called (say) RemotableMeta which does a print > "Adding remoting features" and then also calls > mcl.__base__.__new__(mcl ... just like DebugMeta2, what gets both of > their __new__ methods called in the right order? If you want to reimplement full cooperation between mixins classes you must rework a bit the example, but it does not take a big effort (see later). However my main point is: do we really want cooperative methods? Multiple inheritance of metaclasses is perhaps the strongest use case for multiple inheritance, but is it strong enough? I mean, in real code how many times did I need that? I would not mind make life harder for gurus and simpler for application programmers. I do not think removing cooperation would be so bad in practice. In many practical cases, one could just write the metaclass by hand, in this example something like class RemotableDebugFrameworkMeta(FrameworkMeta): def __new__(mcl, name, bases, dic): print "Adding remoting features to %s" % name print "Adding debugging features to %s" % name return FrameworkMeta.__new__(mcl, name, bases, dic) Maybe you would need to duplicate a couple of lines and/or to introduce an helper function, but you would have the benefit of having a more explicit metaclass, with a simpler hierarchy (see the appendix for an alternative solution). > Maybe you could help me understand this by giving a fully executable > Python snippet using __bases__[0] instead of the hypothetical > __base__? To the best of my knowledge __base__ is a valid class attribute, it denotes the "right" class to use as base. Usually it is the same as bases[0], but there is at least one case when it is different, when composing old style and new style classes: >>> class OldStyle: pass >>> class NewStyle(object): pass >>> class New(OldStyle, NewStyle): pass >>> New.__bases__[0] >>> New.__base__ Appendix: how to reimplement cooperation in a single-inheritance world ---------------------------------------------------------------------------- Quoting Raymond: "To achieve substantially the same functionality, you would likely have to re-invent much of what super() does for us automatically, and you would still be imposing constraits on the composed classes that are substantially the same as what you have with inheritance." Raymond of course is right, but I am arguing that one does not really need to re-invent cooperation because the use case for cooperation are exceedingly rare. Still, if one really wants to reimplement cooperation, here is my take at the challenge: $ cat cooperative_mixins.py "Implements cooperative mixins using multiple-inheritance only" ## everything in three lines def include_mixin(mixin, cls): # could be extended to use more mixins # traits as in Squeak take the precedence over the base class dic = vars(mixin).copy() # could be extended to walk the ancestors dic['_%s__super' % mixin.__name__] = cls return type(mixin.__name__ + cls.__name__, (cls,), dic) ## examples: class FrameworkMeta(type): # example metaclass def __new__(mcl, name, bases, dic): print "Adding framework features to %s" % name return type.__new__(mcl, name, bases, dic) class DebugMeta(type): # mixin metaclass def __new__(mcl, name, bases, dic): print "Adding debugging features to %s" % name return mcl.__super.__new__(mcl, name, bases, dic) class RemotableMeta(type): # another mixin metaclass def __new__(mcl, name, bases, dic): print "Adding remoting features to %s" % name return mcl.__super.__new__(mcl, name, bases, dic) class FrameworkClass(object): __metaclass__ = FrameworkMeta DebugFrameworkMeta = include_mixin(DebugMeta, FrameworkMeta) print '**************** creating DebugFrameworkClass' class DebugFrameworkClass(FrameworkClass): __metaclass__ = DebugFrameworkMeta RemotableDebugFrameworkMeta = include_mixin( RemotableMeta, DebugFrameworkMeta) print '**************** creating RemotableDebugFrameworkClass' class RemotableDebugFrameworkClass(FrameworkClass): __metaclass__ = RemotableDebugFrameworkMeta From steve at pearwood.info Thu Aug 28 07:14:40 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 28 Aug 2008 15:14:40 +1000 Subject: [Python-Dev] confusing exec error message in 3.0 In-Reply-To: References: <200808281121.15908.steve@pearwood.info> Message-ID: <200808281514.41216.steve@pearwood.info> On Thu, 28 Aug 2008 01:38:14 pm Guido van Rossum wrote: > On Wed, Aug 27, 2008 at 6:21 PM, Steven D'Aprano wrote: ... > > I don't think M.__file__ should lie and say it was loaded from a > > file that it wasn't loaded from. It's useful to be able to look at > > a module and see what file it was actually loaded from. > > While appreciate the use case, there are way more use cases where > there's code that must painstakingly strip the trailing 'c' or 'o' > from __file__ in order to read the source code. Perhaps we should > have a separate API for finding out whether a module was loaded from > source or from a .pyc file; but I think it would be better to have > such an API somewhere in the imp module. It's also possible to follow > what goes on by watching the verbose -v output. Thinking more about it, I'm quite enthusiastic about Brett Cannon's suggestion that modules should grow a __source__ attribute as well as __file__. One disadvantage of a separate API is that it's relatively hard to discover, compared to dir(module) or help(module). I don't think I would have thought to look at a module "imp" (implementation?) when I needed that information. But perhaps that's a separate issue. How would such an API work? Knowing nothing concrete about the implementation, I can think of two ways: (1) Look at the module object and extract information from it that recorded what file it came from at the time, even if the Python path (or its contents) changed. This is the 2.x semantics of __file__. (2) Look at the Python path as it exists now, and try to predict (postdict?) which file might have been used to import the module. Of the two, I feel that postdiction is the worse solution. I imagine that this behaviour would be undesirable: # not real code >>> sys.path.append('somewhere') >>> import parrot # imports from 'somewhere/parrot.pyc' >>> parrot.__file__ 'somewhere/parrot.py' >>> del sys.path[-1] >>> sys.path.append('somewhere_else/different') >>> imp.where_from(parrot) 'somewhere_else/different/parrot.py' But maybe that can't happen and I'm worrying for nothing. So what happens now? Does this need a PEP for further discussion? -- Steven From steve at holdenweb.com Thu Aug 28 09:34:27 2008 From: steve at holdenweb.com (Steve Holden) Date: Thu, 28 Aug 2008 03:34:27 -0400 Subject: [Python-Dev] confusing exec error message in 3.0 In-Reply-To: References: <200808281121.15908.steve@pearwood.info> Message-ID: Guido van Rossum wrote: > On Wed, Aug 27, 2008 at 6:21 PM, Steven D'Aprano wrote: >> On Thu, 28 Aug 2008 08:39:01 am Georg Brandl wrote: >>> Fredrik Lundh schrieb: >>>> (using 3.0a4) >>>> >>>> >>> exec(open("file.py")) >>>> >>>> Traceback (most recent call last): >>>> File " ", line 1, in >>>> TypeError: exec() arg 1 must be a string, file, or code object, not >>>> TextIOWrapper >>>> >>>> so what's "file" referring to here? >>>> >>>> (the above works under 2.5, of course) >>> See http://bugs.python.org/issue1762972 -- it has been decided to >>> drop that possibility. >> Hmmm... I have a concern with one of the patches in that issue; I refer >> to patch that changes the semantics of module's __file__ attribute. >> >> Guido noted that exec(open(M.__file__).read(), M.__dict__) almost worked >> as a replacement for reload(), except that there were issues with file >> extensions (.py, .pyc, .pyo and even things like .pyc24). So it was >> decided that M.__file__ should always point to the source file. >> >> But now that reload() is now moved into the imp module, I don't see that >> the justification for changing the semantics of M.__file__ still >> exists. imp.reload(M) is much better for interactive use than >> exec(open(M.__file__).read(), M.__dict__). >> >> Is there still a justification for having M.__file__ point to the source >> even if the module was actually loaded from the .pyc version? If so, >> what is that? >> >> Some years ago, as a newbie, I was having trouble with reload() >> repeatedly not picking up changes I was making to a module. The problem >> turned out to be user-error, but how I discovered that was by looking >> at M.__file__ and noticing that Python was loading the .pyc file >> instead of the .py file I was expecting. Had M.__file__ given >> misleading information, I would have been mislead for much longer. >> Here's a small contrived example under Python 2.5: >> >>>>> import parrot >>>>> parrot.__file__ >> 'parrot.py' >>>>> # pretend that I made changes to the source >> ... # in my editor, but forgot to save them >> ... reload(parrot) >> >>>>> parrot.__file__ >> 'parrot.pyc' >> >> >> I don't think M.__file__ should lie and say it was loaded from a file >> that it wasn't loaded from. It's useful to be able to look at a module >> and see what file it was actually loaded from. > > While appreciate the use case, there are way more use cases where > there's code that must painstakingly strip the trailing 'c' or 'o' > from __file__ in order to read the source code. Perhaps we should have > a separate API for finding out whether a module was loaded from source > or from a .pyc file; but I think it would be better to have such an > API somewhere in the imp module. It's also possible to follow what > goes on by watching the verbose -v output. > Painstakingly? Surely the pain level isn't that high, and I agree with Michael that the __file__ information would be better as the literal truth. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From solipsis at pitrou.net Thu Aug 28 12:28:53 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 28 Aug 2008 10:28:53 +0000 (UTC) Subject: [Python-Dev] Stable / unstable buildbots Message-ID: Hello everyone, What is the rationale behind the distinction between "stable" and "unstable" buildbots? I ask that because the OpenBSD buildbot has failed compiling 3.0 for quite some time, but since that buildbot was in the "unstable" bunch, it was not discovered until someone filed a bug report for it (see http://bugs.python.org/issue3696). Regards Antoine. From barry at python.org Thu Aug 28 14:16:41 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 28 Aug 2008 08:16:41 -0400 Subject: [Python-Dev] Stable / unstable buildbots In-Reply-To: References: Message-ID: <04B40C81-D9DA-4F68-B23F-79FC31609E30@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 28, 2008, at 6:28 AM, Antoine Pitrou wrote: > What is the rationale behind the distinction between "stable" and > "unstable" > buildbots? > I ask that because the OpenBSD buildbot has failed compiling 3.0 for > quite some > time, but since that buildbot was in the "unstable" bunch, it was > not discovered > until someone filed a bug report for it (see http://bugs.python.org/issue3696) > . Mostly, it was a determination that Neal and I made at Pycon, about which bots we should "trust" to judge the health of the trees. I don't think the current list needs to be set in stone, and in fact several of the "stable" bots have had simple svn or other non-tree related problems for a while. As for the "unstable" ones, I personally rarely check them. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSLaXKXEjvBPtnXfVAQJfqgQAiyoqqUNQWByKEalKmWpRyqlX56PJIzPB S5gqubUZOvUTjVvHgvUtePPmJwEntyE+WcWXrX2tRlmMiCwantjgYOuNJeqonuwE BVZRrIjU/mayM9gRiN9NhuGHfV/YiLKpte+DKzuYZwGW2TnaCM3ijWe2QuMxmgqZ MMAO1w+M1kE= =qIFo -----END PGP SIGNATURE----- From facundobatista at gmail.com Thu Aug 28 14:27:02 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 28 Aug 2008 09:27:02 -0300 Subject: [Python-Dev] Stable / unstable buildbots In-Reply-To: <04B40C81-D9DA-4F68-B23F-79FC31609E30@python.org> References: <04B40C81-D9DA-4F68-B23F-79FC31609E30@python.org> Message-ID: 2008/8/28 Barry Warsaw : > bots we should "trust" to judge the health of the trees. I don't think the > current list needs to be set in stone, and in fact several of the "stable" > bots have had simple svn or other non-tree related problems for a while. Maybe a good requisite to move a buildbot from unstable to stable is to find a champion for it. I mean, something that can test on that platform and cares enough about it to, or fix the issue himself/herself, or find who broke it and bother the responsible until it gets fixed. Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From solipsis at pitrou.net Thu Aug 28 15:33:21 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 28 Aug 2008 13:33:21 +0000 (UTC) Subject: [Python-Dev] Stable / unstable buildbots References: <04B40C81-D9DA-4F68-B23F-79FC31609E30@python.org> Message-ID: Facundo Batista gmail.com> writes: > > Maybe a good requisite to move a buildbot from unstable to stable is > to find a champion for it. I mean, something that can test on that > platform and cares enough about it to, or fix the issue > himself/herself, or find who broke it and bother the responsible until > it gets fixed. By that metric, I fear that the only remaining buildbots would be the Linux/Windows x86/x64 ones. I'm not sure anyone here, for example, cares really much about Sparc buildbots, apart from the fact that having red "stable" buildbots doesn't make Python look very good, and we try to avoid that. But then Python wouldn't be really cross-platform anymore, or just in a very theoretical way. Also, lacking stable buildbots means some chunks of the language or stdlib aren't even tested anymore. For example, the bigmem/bigaddrspace tests are completely broken in the py3k branch. Regards Antoine. From facundobatista at gmail.com Thu Aug 28 15:38:40 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 28 Aug 2008 10:38:40 -0300 Subject: [Python-Dev] Stable / unstable buildbots In-Reply-To: References: <04B40C81-D9DA-4F68-B23F-79FC31609E30@python.org> Message-ID: 2008/8/28 Antoine Pitrou : > By that metric, I fear that the only remaining buildbots would be the > Linux/Windows x86/x64 ones. I'm not sure anyone here, for example, cares really Note that I meant to "move from unstable to stable, starting from the actual state", not to "decide again which ones will be stables or not". I agree with you in all the other matters. -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From pje at telecommunity.com Thu Aug 28 17:30:39 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 28 Aug 2008 11:30:39 -0400 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <4edc17eb0808272135q44985feaj1ba550022919eaac@mail.gmail.co m> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> <4edc17eb0808261816r7dd2eff9hb42772e907e3e9bf@mail.gmail.com> <20080827031445.C9B8B3A4079@sparrow.telecommunity.com> <4edc17eb0808262224g59c959c9s8a15f778695b8e7a@mail.gmail.com> <4edc17eb0808272135q44985feaj1ba550022919eaac@mail.gmail.com> Message-ID: <20080828152941.CD4E43A410E@sparrow.telecommunity.com> At 06:35 AM 8/28/2008 +0200, Michele Simionato wrote: >Multiple inheritance of metaclasses is perhaps >the strongest use case for multiple inheritance, but is it strong >enough? I mean, in real code how many times did I need that? >I would not mind make life harder for gurus and simpler for >application programmers. Then you need to leave MI and co-operation the hell alone. Right now, an application programmer can mix metaclasses like this: class FooBar(Foo, Bar): class __metaclass__(Foo.__class__, Bar.__class__): pass ... Or, in 3.x: class FooBarClass(Foo.__class__, Bar.__class__): pass class FooBar(Foo, Bar, metaclass=FooBarClass): ... Either way, this is useful in cases where Foo and Bar come from different frameworks. That's the *only* way to get such things to co-operate, in fact. >I do not think removing cooperation >would be so bad in practice. In many practical cases, one could just write >the metaclass by hand, How is that making things easier for application programmers? >Maybe you would need to duplicate a couple of lines and/or to introduce >an helper function, ...which then has to have an agreed-upon protocol that all metaclass authors have to follow... which we already have... but which you're proposing to get rid of... so we can re-invent it lots of times... in mutually incompatible ways. :) From michele.simionato at gmail.com Thu Aug 28 17:50:29 2008 From: michele.simionato at gmail.com (Michele Simionato) Date: Thu, 28 Aug 2008 17:50:29 +0200 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <20080828152941.CD4E43A410E@sparrow.telecommunity.com> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> <4edc17eb0808261816r7dd2eff9hb42772e907e3e9bf@mail.gmail.com> <20080827031445.C9B8B3A4079@sparrow.telecommunity.com> <4edc17eb0808262224g59c959c9s8a15f778695b8e7a@mail.gmail.com> <4edc17eb0808272135q44985feaj1ba550022919eaac@mail.gmail.com> <20080828152941.CD4E43A410E@sparrow.telecommunity.com> Message-ID: <4edc17eb0808280850xa842656l9fea70acc0717bce@mail.gmail.com> On Aug 28, 5:30 pm, "Phillip J. Eby" wrote: > How is that making things easier for application programmers? We have different definitions of "application programmer". For me a typical application programmer is somebody who never fiddles with metaclasses, which are the realm of framework builders. But the borders are fluid, I agree. > >Maybe you would need to duplicate a couple of lines and/or to introduce > >an helper function, > > ...which then has to have an agreed-upon protocol that all metaclass > authors have to follow... which we already have... but which you're > proposing to get rid of... so we can re-invent it lots of > times... in mutually incompatible ways. :) Notice that I was discussing an hypothetical language. I was arguing that in principle one could write a different language from Python, with single inheritance only, and not lose much expressivity. I am not advocating any change to current Python. My point was in language design: I want to know how much I can remove from a language and still have something useful, in the spirit of the famous Saint-Exupery quote. Michele Simionato From jjb5 at cornell.edu Thu Aug 28 20:36:00 2008 From: jjb5 at cornell.edu (Joel Bender) Date: Thu, 28 Aug 2008 14:36:00 -0400 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <48B50847.6060703@canterbury.ac.nz> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> <4edc17eb0808261816r7dd2eff9hb42772e907e3e9bf@mail.gmail.com> <20080827031445.C9B8B3A4079@sparrow.telecommunity.com> <48B50847.6060703@canterbury.ac.nz> Message-ID: <48B6F010.9000704@cornell.edu> Greg, > Do you have a real-life example of this where multiple > inheritance is actually used? I have built a framework that I have called the "capability pattern" which uses multiple inheritance in a way that might be unique (I'm not familiar enough with other frameworks to know for sure). There are two classes, a Collector and a Capability. The result of calling a function of the collector is a list of results of calling the functions of the bound capabilities. For example, these three are capability classes: class X(Capability): def f(self): return 'X.f' class Y(Capability): def g(self): return 'Y.g' class Z(Capability): def f(self): return 'Z.f' def g(self): return 'Z.g' Now to create a sample collector: class A(Collector, X, Y, Z): pass Calling A().f() returns ['X.f', 'Z.f']. I use this pattern in a web application. The do_GET call is mapped into do_SHOW, and each 'capability' can return something from its do_SHOW (usually a
element) and the results are sent back to the user. In my case I have lots of combinations of capabilities that can be mixed together. I decided to use multiple inheritance over other patterns because I wanted to leverage isinstance(obj,Y) to indicate that some object has some capability, and not having to duplicate the method resolution order code for other kinds of methods is really nice. > A non-contrived example or two would be a good thing to > have in tutorials etc. where super() is discussed. It > would help to convey the kinds of situations in which > use of super() is and is not appropriate. So this is a collection of cooperative classes, and super() isn't used. Joel From pje at telecommunity.com Thu Aug 28 20:54:08 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 28 Aug 2008 14:54:08 -0400 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <4edc17eb0808280850xa842656l9fea70acc0717bce@mail.gmail.com > References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> <4edc17eb0808261816r7dd2eff9hb42772e907e3e9bf@mail.gmail.com> <20080827031445.C9B8B3A4079@sparrow.telecommunity.com> <4edc17eb0808262224g59c959c9s8a15f778695b8e7a@mail.gmail.com> <4edc17eb0808272135q44985feaj1ba550022919eaac@mail.gmail.com> <20080828152941.CD4E43A410E@sparrow.telecommunity.com> <4edc17eb0808280850xa842656l9fea70acc0717bce@mail.gmail.com> Message-ID: <20080828185312.943163A408D@sparrow.telecommunity.com> At 05:50 PM 8/28/2008 +0200, Michele Simionato wrote: >On Aug 28, 5:30 pm, "Phillip J. Eby" wrote: > > How is that making things easier for application programmers? > >We have different definitions of "application programmer". For me a typical >application programmer is somebody who never fiddles with metaclasses, >which are the realm of framework builders. Application programmers use frameworks, and sometimes more than one. If they're subclassing from two different frameworks, each using a different metaclass, they will need to also multiple-inherit the metaclass. This is in fact so annoying that I created a "universal metaclass" in DecoratorTools whose sole function is to delegate metaclass __new__, __init__, and __call__ to class-level methods (e.g. __class_new__, __class_call__, etc.), thereby eliminating the need to have custom metaclasses for most use cases in the first place. Now, wherever possible, I use that single metaclass in my frameworks, so that there's no need to mix them. That, IMO, would be a more useful change than getting rid of super(); it would get rid of the explicit metaclass mixing. (It would still not remove the need for co-operative methods, as the class-delegated methods still need to be co-operative for MI to work.) There are, of course, other ways to create co-operative function calls besides super(), and I've certainly created more a few of them in my time. (E.g. generic function method combination, instancemethod() chains, and next-method-iterators, to name the ones that occur to me right off.) But these are more for cases where super() is wholly inadequate to the purpose, and none are anywhere near as convenient as super(). From fuzzyman at voidspace.org.uk Thu Aug 28 21:31:16 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 28 Aug 2008 20:31:16 +0100 Subject: [Python-Dev] Documentation Error for __hash__ Message-ID: <48B6FD04.6080008@voidspace.org.uk> Hello all, The documentation for __hash__ seems to be outdated. I'm happy to submit a patch, so long as I am not misunderstanding something. http://docs.python.org/dev/reference/datamodel.html#object.__hash__ The documentation states: If a class does not define a __cmp__() or __eq__() method it should not define a __hash__() operation either; if it defines __cmp__() or __eq__() but not __hash__(), its instances will not be usable as dictionary keys. If a class defines mutable objects and implements a __cmp__() or __eq__() method, it should not implement __hash__(), since the dictionary implementation requires that a key?s hash value is immutable (if the object?s hash value changes, it will be in the wrong hash bucket). This may have been true for old style classes, but as new style classes inherit a default __hash__ from object - mutable objects *will* be usable as dictionary keys (hashed on identity) *unless* they implement a __hash__ method that raises a type error. Shouldn't the advice be that classes that implement comparison methods should always implement __hash__ (wasn't this nearly enforced?), and that mutable objects should raise a TypeError in __hash__. Additionally the following documentation states that __reversed__ is new in Python 2.6 and I think it was actually new in Python 2.4 (it certainly works for Python 2.5 and IronPython 1 which targets 2.4...). http://docs.python.org/dev/reference/datamodel.html#object.__reversed__ Michael Foord -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From tjreedy at udel.edu Thu Aug 28 21:59:44 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 28 Aug 2008 15:59:44 -0400 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <4edc17eb0808280850xa842656l9fea70acc0717bce@mail.gmail.com> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808260913v323a700pbeabbec84d0f1406@mail.gmail.com> <4edc17eb0808261816r7dd2eff9hb42772e907e3e9bf@mail.gmail.com> <20080827031445.C9B8B3A4079@sparrow.telecommunity.com> <4edc17eb0808262224g59c959c9s8a15f778695b8e7a@mail.gmail.com> <4edc17eb0808272135q44985feaj1ba550022919eaac@mail.gmail.com> <20080828152941.CD4E43A410E@sparrow.telecommunity.com> <4edc17eb0808280850xa842656l9fea70acc0717bce@mail.gmail.com> Message-ID: Michele Simionato wrote: > Notice that I was discussing an hypothetical language. I was arguing > that in principle > one could write a different language from Python, with single inheritance only, > and not lose much expressivity. I am not advocating any change to > current Python. Since this is a list for discussing changing current Python, you will have to excuse misunderstanding on that point. I do realize that you began with a concrete offer of improving the documentation of the super you hypothetically would like to be rid of ;-) From hall.jeff at gmail.com Thu Aug 28 22:42:18 2008 From: hall.jeff at gmail.com (Jeff Hall) Date: Thu, 28 Aug 2008 16:42:18 -0400 Subject: [Python-Dev] Documentation Error for __hash__ In-Reply-To: <48B6FD04.6080008@voidspace.org.uk> References: <48B6FD04.6080008@voidspace.org.uk> Message-ID: <1bc395c10808281342v5d9a0d48sf748758a7b0fe6dd@mail.gmail.com> I'm not sure about the first but as for the __reversed__ we had a discussion yesterday and it was indeed added in 2.4 (oddly, my 2.5 documentation has this correct... ) -- Haikus are easy Most make very little sense Refrigerator -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Thu Aug 28 23:51:45 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 29 Aug 2008 07:51:45 +1000 Subject: [Python-Dev] Documentation Error for __hash__ In-Reply-To: <48B6FD04.6080008@voidspace.org.uk> References: <48B6FD04.6080008@voidspace.org.uk> Message-ID: <48B71DF1.8020902@gmail.com> Michael Foord wrote: > This may have been true for old style classes, but as new style classes > inherit a default __hash__ from object - mutable objects *will* be > usable as dictionary keys (hashed on identity) *unless* they implement a > __hash__ method that raises a type error. > > Shouldn't the advice be that classes that implement comparison methods > should always implement __hash__ (wasn't this nearly enforced?), and > that mutable objects should raise a TypeError in __hash__. Hmm, I thought I fixed those docs when I did the PyObject_HashNotImplemented patch... (you can set __hash__ = None rather than raising a TypeError explicitly now so that collections.Hashable correctly returns False - the slot machinery in typeobject.c will then take of turning that into PyObject_HashNotImplemented at the C level). However, looking at the relevant checkins, it appears I only updated the C API docs and didn't correct the library or language references. I reopened http://bugs.python.org/issue2235 - there are some other doc fixes I need to do this weekend for RC1, so I can tackle this at the same time. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From tjreedy at udel.edu Thu Aug 28 23:55:14 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 28 Aug 2008 17:55:14 -0400 Subject: [Python-Dev] Documentation Error for __hash__ In-Reply-To: <1bc395c10808281342v5d9a0d48sf748758a7b0fe6dd@mail.gmail.com> References: <48B6FD04.6080008@voidspace.org.uk> <1bc395c10808281342v5d9a0d48sf748758a7b0fe6dd@mail.gmail.com> Message-ID: Jeff Hall wrote: > I'm not sure about the first but as for the __reversed__ we had a > discussion yesterday and it was indeed added in 2.4 (oddly, my 2.5 > documentation has this correct... ) 2.4 doc: reversed( seq) Return a reverse iterator. seq must be an object which supports the sequence protocol (the __len__() method and the __getitem__() method with integer arguments starting at 0). New in version 2.4. [no mention of __reversed__] 3.3.6 Additional methods for emulation of sequence types [ditto] However, I confirmed that __reversed__ is used by reverse() class C: def __reversed__(self): return 'abc' c=C() print(reversed(c)) >>> abc From matt.giuca at gmail.com Fri Aug 29 00:00:06 2008 From: matt.giuca at gmail.com (Matt Giuca) Date: Fri, 29 Aug 2008 08:00:06 +1000 Subject: [Python-Dev] Documentation Error for __hash__ In-Reply-To: <48B6FD04.6080008@voidspace.org.uk> References: <48B6FD04.6080008@voidspace.org.uk> Message-ID: > This may have been true for old style classes, but as new style classes > inherit a default __hash__ from object - mutable objects *will* be usable as > dictionary keys (hashed on identity) *unless* they implement a __hash__ > method that raises a type error. > I always thought this was a bug in new-style classes (due to the fact that, as you say, they inherit __hash__ from object whether it's wanted or not). However, it seems to be fixed in Python 3.0. So this documentation is only "wrong" for Python 2.x branch. See: Python 2.6b3+ (trunk:66055, Aug 29 2008, 07:50:39) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class X(object): ... def __eq__(self, other): ... return True ... >>> x = X() >>> hash(x) -1211564180 versus Python 3.0b3+ (py3k:66055M, Aug 29 2008, 07:52:23) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class X(object): ... def __eq__(self, other): ... return True ... >>> x = X() >>> hash(x) Traceback (most recent call last): File " ", line 1, in TypeError: unhashable type: 'X' Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Fri Aug 29 03:39:01 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 29 Aug 2008 13:39:01 +1200 Subject: [Python-Dev] confusing exec error message in 3.0 In-Reply-To: <200808281121.15908.steve@pearwood.info> References: <200808281121.15908.steve@pearwood.info> Message-ID: <48B75335.6090502@canterbury.ac.nz> Steven D'Aprano wrote: > I don't think M.__file__ should lie and say it was loaded from a file > that it wasn't loaded from. It's useful to be able to look at a module > and see what file it was actually loaded from. On the other hand, it could be useful to be able to find the source file for a module, regardless of whether it was loaded from a .py, .pyc or .pyo. Maybe there should be a __source__ attribute for that? Or is os.path.splitext(M.__file__)[0] + ".py" considered good enough for that? -- Greg From ijmorlan at cs.uwaterloo.ca Fri Aug 29 03:50:14 2008 From: ijmorlan at cs.uwaterloo.ca (Isaac Morland) Date: Thu, 28 Aug 2008 21:50:14 -0400 (EDT) Subject: [Python-Dev] confusing exec error message in 3.0 In-Reply-To: <48B75335.6090502@canterbury.ac.nz> References: <200808281121.15908.steve@pearwood.info> <48B75335.6090502@canterbury.ac.nz> Message-ID: On Fri, 29 Aug 2008, Greg Ewing wrote: > Steven D'Aprano wrote: > >> I don't think M.__file__ should lie and say it was loaded from a file that >> it wasn't loaded from. It's useful to be able to look at a module and see >> what file it was actually loaded from. > > On the other hand, it could be useful to be able to find > the source file for a module, regardless of whether it > was loaded from a .py, .pyc or .pyo. > > Maybe there should be a __source__ attribute for that? > > Or is os.path.splitext(M.__file__)[0] + ".py" considered > good enough for that? I think .__source__ is a much better idea. It's more explicit to start with, and on top of that it can simply be None when there is no source file available, which is much more explicit than "file not found means no source file available". Also, if a feature is ever added to allow telling Python to put the compiled files in a separate tree (like javac -d directory in the Java universe) then it will be even more clear that computing the source location from __file__ is not the way to go. PS, in case it's not clear, I want that feature, in order to keep .pyc files out of my SVN checkouts. I hate having automatically generated files in my workspace. Isaac Morland CSCF Web Guru DC 2554C, x36650 WWW Software Specialist From guido at python.org Fri Aug 29 04:25:47 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 28 Aug 2008 19:25:47 -0700 Subject: [Python-Dev] confusing exec error message in 3.0 In-Reply-To: References: <200808281121.15908.steve@pearwood.info> Message-ID: 2008/8/28 Steve Holden : > Guido van Rossum wrote: >> On Wed, Aug 27, 2008 at 6:21 PM, Steven D'Aprano wrote: >>> On Thu, 28 Aug 2008 08:39:01 am Georg Brandl wrote: >>>> Fredrik Lundh schrieb: >>>>> (using 3.0a4) >>>>> >>>>> >>> exec(open("file.py")) >>>>> >>>>> Traceback (most recent call last): >>>>> File " ", line 1, in >>>>> TypeError: exec() arg 1 must be a string, file, or code object, not >>>>> TextIOWrapper >>>>> >>>>> so what's "file" referring to here? >>>>> >>>>> (the above works under 2.5, of course) >>>> See http://bugs.python.org/issue1762972 -- it has been decided to >>>> drop that possibility. >>> Hmmm... I have a concern with one of the patches in that issue; I refer >>> to patch that changes the semantics of module's __file__ attribute. >>> >>> Guido noted that exec(open(M.__file__).read(), M.__dict__) almost worked >>> as a replacement for reload(), except that there were issues with file >>> extensions (.py, .pyc, .pyo and even things like .pyc24). So it was >>> decided that M.__file__ should always point to the source file. >>> >>> But now that reload() is now moved into the imp module, I don't see that >>> the justification for changing the semantics of M.__file__ still >>> exists. imp.reload(M) is much better for interactive use than >>> exec(open(M.__file__).read(), M.__dict__). >>> >>> Is there still a justification for having M.__file__ point to the source >>> even if the module was actually loaded from the .pyc version? If so, >>> what is that? >>> >>> Some years ago, as a newbie, I was having trouble with reload() >>> repeatedly not picking up changes I was making to a module. The problem >>> turned out to be user-error, but how I discovered that was by looking >>> at M.__file__ and noticing that Python was loading the .pyc file >>> instead of the .py file I was expecting. Had M.__file__ given >>> misleading information, I would have been mislead for much longer. >>> Here's a small contrived example under Python 2.5: >>> >>>>>> import parrot >>>>>> parrot.__file__ >>> 'parrot.py' >>>>>> # pretend that I made changes to the source >>> ... # in my editor, but forgot to save them >>> ... reload(parrot) >>> >>>>>> parrot.__file__ >>> 'parrot.pyc' >>> >>> >>> I don't think M.__file__ should lie and say it was loaded from a file >>> that it wasn't loaded from. It's useful to be able to look at a module >>> and see what file it was actually loaded from. >> >> While appreciate the use case, there are way more use cases where >> there's code that must painstakingly strip the trailing 'c' or 'o' >> from __file__ in order to read the source code. Perhaps we should have >> a separate API for finding out whether a module was loaded from source >> or from a .pyc file; but I think it would be better to have such an >> API somewhere in the imp module. It's also possible to follow what >> goes on by watching the verbose -v output. >> > Painstakingly? Surely the pain level isn't that high, and I agree with > Michael that the __file__ information would be better as the literal truth. Some people have custom mods to Python that change the extension, and then the c/o stripping code breaks. What is the literal truth depends on your use case. It is the literal truth that the source file is the .py file. -- --Guido van Rossum (home page: http://www.python.org/~guido/) -------------- next part -------------- An HTML attachment was scrubbed... URL: From michele.simionato at gmail.com Fri Aug 29 06:07:27 2008 From: michele.simionato at gmail.com (Michele Simionato) Date: Fri, 29 Aug 2008 06:07:27 +0200 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <20080828185312.943163A408D@sparrow.telecommunity.com> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808261816r7dd2eff9hb42772e907e3e9bf@mail.gmail.com> <20080827031445.C9B8B3A4079@sparrow.telecommunity.com> <4edc17eb0808262224g59c959c9s8a15f778695b8e7a@mail.gmail.com> <4edc17eb0808272135q44985feaj1ba550022919eaac@mail.gmail.com> <20080828152941.CD4E43A410E@sparrow.telecommunity.com> <4edc17eb0808280850xa842656l9fea70acc0717bce@mail.gmail.com> <20080828185312.943163A408D@sparrow.telecommunity.com> Message-ID: <4edc17eb0808282107q65969a63qb6b8745461b57b44@mail.gmail.com> On Thu, Aug 28, 2008 at 8:54 PM, Phillip J. Eby wrote: > I created a "universal metaclass" in > DecoratorTools whose sole function is to delegate metaclass __new__, > __init__, and __call__ to class-level methods (e.g. __class_new__, > __class_call__, etc.), thereby eliminating the need to have custom > metaclasses for most use cases in the first place. Now, wherever possible, > I use that single metaclass in my frameworks, so that there's no need to mix > them. easy_installed DecoratorTools and found it: classy_class. >From the point of view of the code, this is a beautiful and elegant snippet. However, suppose that from tomorrow everybody starts using it. Since metaclasses would become so easy to use, possibly a lot of people would take advantage of them. Then we would have potentially complex (multiple) inheritance hierarchies with chains of methods (_class__new__/_class__init__) calling themselves cooperatively in the MRO. Would the resulting code be readable? How easy would be for an average framework user to understand what is happening to his class? I think class decorators would be a much better solution than classy_class for most use cases and we should push that way, not the cooperative inheritance way. Generally speaking I like more solutions bases on functional composition (as in WSGI that you know very well) than on method cooperation. Rather than improve the support for inheritance, I would like (in an ideal world) to reduce it, to make easier the choice for people between inheritance and alternatives (object composition, delegation, functional composition). In the real world, I am content in documenting the pitfalls of super, warn people about the dangers of complex design involving multiple inheritance and cooperation, and suggest alternatives. Michele Simionato From pje at telecommunity.com Fri Aug 29 06:22:39 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 29 Aug 2008 00:22:39 -0400 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <4edc17eb0808282107q65969a63qb6b8745461b57b44@mail.gmail.co m> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808261816r7dd2eff9hb42772e907e3e9bf@mail.gmail.com> <20080827031445.C9B8B3A4079@sparrow.telecommunity.com> <4edc17eb0808262224g59c959c9s8a15f778695b8e7a@mail.gmail.com> <4edc17eb0808272135q44985feaj1ba550022919eaac@mail.gmail.com> <20080828152941.CD4E43A410E@sparrow.telecommunity.com> <4edc17eb0808280850xa842656l9fea70acc0717bce@mail.gmail.com> <20080828185312.943163A408D@sparrow.telecommunity.com> <4edc17eb0808282107q65969a63qb6b8745461b57b44@mail.gmail.com> Message-ID: <20080829042146.6DC123A408D@sparrow.telecommunity.com> At 06:07 AM 8/29/2008 +0200, Michele Simionato wrote: >On Thu, Aug 28, 2008 at 8:54 PM, Phillip J. Eby wrote: > > I created a "universal metaclass" in > > DecoratorTools whose sole function is to delegate metaclass __new__, > > __init__, and __call__ to class-level methods (e.g. __class_new__, > > __class_call__, etc.), thereby eliminating the need to have custom > > metaclasses for most use cases in the first place. Now, wherever possible, > > I use that single metaclass in my frameworks, so that there's no > need to mix > > them. > >easy_installed DecoratorTools and found it: classy_class. > >From the point of view of the code, this is a beautiful and elegant >snippet. However, suppose that from tomorrow everybody starts >using it. Since metaclasses would become so easy to use, possibly a >lot of people would take advantage of them. That was sort of the idea. ;-) > Then we would have >potentially complex (multiple) inheritance hierarchies with >chains of methods (_class__new__/_class__init__) calling >themselves cooperatively in the MRO. Would the resulting >code be readable? Readability's orthogonal. Some of them might be readable, some not. Depends on who's writing them. :) >How easy would be for an average framework user >to understand what is happening to his class? You're right, let's abolish inheritance, too, because then you might have to read more than one class to see what's happening. >I think class decorators would be a much better solution than >classy_class for most use cases Obviously, I disagree. :) You'll notice that DecoratorTools supports class decorators for Python 2.3 and up (actually, I think that particular bit worked in 2.2 as well). So, it's not the absence of class decorators that motivated the 'classy' mixin. >Generally speaking I like >more solutions bases on functional composition (as in WSGI >that you know very well) than on method cooperation. Rather than >improve the support for inheritance, I would like (in an ideal >world) to reduce it, to make easier the choice for people between >inheritance and alternatives (object composition, delegation, functional >composition). In the real world, I am content in documenting the >pitfalls of super, warn people about the dangers of complex >design involving multiple inheritance and cooperation, and suggest >alternatives. Naturally, if you can design a system to use delegates instead of class hierarchy to represent a chain of responsibility, it might well be an improvement. But there are tradeoffs, and no matter what you are going to end up coding chains of responsibility. Co-operative inheritance is a nice solution for chains of responsibility that can be expressed in a class hierarchy, and are no more "dangerous" than any other sort of chain of responsibility. In fact, they are in some ways less so since the patterns are likely to be better documented than anything you come up with on your own. From michele.simionato at gmail.com Fri Aug 29 06:44:24 2008 From: michele.simionato at gmail.com (Michele Simionato) Date: Fri, 29 Aug 2008 06:44:24 +0200 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <20080829042146.6DC123A408D@sparrow.telecommunity.com> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <20080827031445.C9B8B3A4079@sparrow.telecommunity.com> <4edc17eb0808262224g59c959c9s8a15f778695b8e7a@mail.gmail.com> <4edc17eb0808272135q44985feaj1ba550022919eaac@mail.gmail.com> <20080828152941.CD4E43A410E@sparrow.telecommunity.com> <4edc17eb0808280850xa842656l9fea70acc0717bce@mail.gmail.com> <20080828185312.943163A408D@sparrow.telecommunity.com> <4edc17eb0808282107q65969a63qb6b8745461b57b44@mail.gmail.com> <20080829042146.6DC123A408D@sparrow.telecommunity.com> Message-ID: <4edc17eb0808282144l6d81979bic95ae8c11d69cd37@mail.gmail.com> On Fri, Aug 29, 2008 at 6:22 AM, Phillip J. Eby wrote: > You're right, let's abolish inheritance, too, because then you might have to > read more than one class to see what's happening. You are joking, but I actually took this idea quite seriously. Once (four years ago or so) I did implement an object system from scratch in Scheme, completely without inheritance, to see how far it would go. It didn't go far, of course (nor I did expect it to go very far) but at least I learned exactly what (single) inheritance was good for. OTOH, for what concerns multiple inheritance, I am still not convinced it is really worth it. I mean, the MRO is beautiful, elegant and all that on paper, but on real-life code things as different, especially from the side of the users of frameworks heavily based on inheritance. > Naturally, if you can design a system to use delegates instead of class > hierarchy to represent a chain of responsibility, it might well be an > improvement. But there are tradeoffs, and no matter what you are going to > end up coding chains of responsibility. Agreed, it is all about tradeoffs. We have a different opinion on what a good tradeoff is in this case, but that's fine. I guess it depends on personal experience and the kind of code one has to work with. For instance I never had to integrated different frameworks using different metaclasses in my daily work, so I don't see a very strong case for classy_class over class decorators, but I could change my mind in the future, who knows? Anyway, It would be nice to have a good simple *real life* use case of cooperative inheritance not involving metaclasses, suitable for a beginners' tutorial about super, but I haven't found one yet :-( M.S. From mal at egenix.com Fri Aug 29 11:53:28 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 29 Aug 2008 11:53:28 +0200 Subject: [Python-Dev] Documentation Error for __hash__ In-Reply-To: <48B6FD04.6080008@voidspace.org.uk> References: <48B6FD04.6080008@voidspace.org.uk> Message-ID: <48B7C718.7020802@egenix.com> On 2008-08-28 21:31, Michael Foord wrote: > Hello all, > > The documentation for __hash__ seems to be outdated. I'm happy to submit > a patch, so long as I am not misunderstanding something. > > http://docs.python.org/dev/reference/datamodel.html#object.__hash__ > > The documentation states: > > If a class does not define a __cmp__() or __eq__() method it should not > define a __hash__() operation either; if it defines __cmp__() or > __eq__() but not __hash__(), its instances will not be usable as > dictionary keys. If a class defines mutable objects and implements a > __cmp__() or __eq__() method, it should not implement __hash__(), since > the dictionary implementation requires that a key's hash value is > immutable (if the object's hash value changes, it will be in the wrong > hash bucket). > > > This may have been true for old style classes, but as new style classes > inherit a default __hash__ from object - mutable objects *will* be > usable as dictionary keys (hashed on identity) *unless* they implement a > __hash__ method that raises a type error. Being hashable is a different from being usable as dictionary key. Dictionaries perform the lookup based on the hash value, but will then have to check for hash collisions based on an equal comparison. If an object does not define an equal comparison, then it is not usable as dictionary key. > Shouldn't the advice be that classes that implement comparison methods > should always implement __hash__ (wasn't this nearly enforced?), It's probably a good idea to implement __hash__ for objects that implement comparisons, but it won't always work and it is certainly not needed, unless you intend to use them as dictionary keys. > and that mutable objects should raise a TypeError in __hash__. That's a good idea, even though it's not needed either ;-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 29 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From matt.giuca at gmail.com Fri Aug 29 13:03:22 2008 From: matt.giuca at gmail.com (Matt Giuca) Date: Fri, 29 Aug 2008 21:03:22 +1000 Subject: [Python-Dev] Documentation Error for __hash__ In-Reply-To: <48B7C718.7020802@egenix.com> References: <48B6FD04.6080008@voidspace.org.uk> <48B7C718.7020802@egenix.com> Message-ID: > Being hashable is a different from being usable as dictionary key. > > Dictionaries perform the lookup based on the hash value, but will > then have to check for hash collisions based on an equal comparison. > > If an object does not define an equal comparison, then it is not > usable as dictionary key. > But if an object defines *neither* __eq__ or __hash__, then by default it is usable as a dictionary key (using the id() of the object for both default equality and hashing, which is fine, and works for all user-defined types by default). An object defining __hash__ but not __eq__ is not problematic, since it still uses id() default for equality. (It just has potentially bad dictionary performance, if lots of things hash the same even though they aren't equal). This it not a problem by definition because *it is officially "okay" for two objects to hash the same, even if they aren't equal, though undesirable*. So all hashable objects are usable as dictionary keys, are they not? (As far as I know it isn't possible to have an object that does not have an equality comparison, unless you explicitly override __eq__ and have it raise a TypeError for some reason). It's probably a good idea to implement __hash__ for objects that > implement comparisons, but it won't always work and it is certainly > not needed, unless you intend to use them as dictionary keys. > But from what I know, it is a *bad* idea to implement __hash__ for any mutable object with non-reference equality (where equality depends on the mutable state), as an unbreakable rule. This is because if they are inserted into a dictionary, then mutated, they may suddenly be in the wrong bucket. This is why all the mutable types in Python with non-reference equality (eg. list, set, dict) are explicitly not hashable, while the immutable types (eg. tuple, frozenset, str) are hashable, and so are the mutable types with reference equality (eg. functions, user-defined classes by default). > > > and that mutable objects should raise a TypeError in __hash__. > > That's a good idea, even though it's not needed either ;-) > So I think my above "axioms" are a better (less restrictive, and still correct) rule than this one. It's OK for a mutable object to define __hash__, as long as its __eq__ doesn't depend upon its mutable state. For example, you can insert a function object into a dictionary, and mutate its closure, and it won't matter, because neither the hash nor the equality of the object is changing. It's only types like list and dict, with deep equality, where you run into this hash table problem. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at voidspace.org.uk Fri Aug 29 13:22:00 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 29 Aug 2008 12:22:00 +0100 Subject: [Python-Dev] Documentation Error for __hash__ In-Reply-To: <48B7C718.7020802@egenix.com> References: <48B6FD04.6080008@voidspace.org.uk> <48B7C718.7020802@egenix.com> Message-ID: <48B7DBD8.8070205@voidspace.org.uk> M.-A. Lemburg wrote: > On 2008-08-28 21:31, Michael Foord wrote: > >> Hello all, >> >> The documentation for __hash__ seems to be outdated. I'm happy to submit >> a patch, so long as I am not misunderstanding something. >> >> http://docs.python.org/dev/reference/datamodel.html#object.__hash__ >> >> The documentation states: >> >> If a class does not define a __cmp__() or __eq__() method it should not >> define a __hash__() operation either; if it defines __cmp__() or >> __eq__() but not __hash__(), its instances will not be usable as >> dictionary keys. If a class defines mutable objects and implements a >> __cmp__() or __eq__() method, it should not implement __hash__(), since >> the dictionary implementation requires that a key's hash value is >> immutable (if the object's hash value changes, it will be in the wrong >> hash bucket). >> >> >> This may have been true for old style classes, but as new style classes >> inherit a default __hash__ from object - mutable objects *will* be >> usable as dictionary keys (hashed on identity) *unless* they implement a >> __hash__ method that raises a type error. >> > > Being hashable is a different from being usable as dictionary key. > > Dictionaries perform the lookup based on the hash value, but will > then have to check for hash collisions based on an equal comparison. > > If an object does not define an equal comparison, then it is not > usable as dictionary key. > > >> Shouldn't the advice be that classes that implement comparison methods >> should always implement __hash__ (wasn't this nearly enforced?), >> > > It's probably a good idea to implement __hash__ for objects that > implement comparisons, but it won't always work and it is certainly > not needed, unless you intend to use them as dictionary keys. > > So you're suggesting that we document something like. Classes that represent mutable values and define equality methods are free to define __hash__ so long as you don't mind them being used incorrectly if treated as dictionary keys... Technically true, but not very helpful in my opinion... :-) Michael >> and that mutable objects should raise a TypeError in __hash__. >> > > That's a good idea, even though it's not needed either ;-) > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.resolverhacks.net/ http://www.theotherdelia.co.uk/ From mal at egenix.com Fri Aug 29 14:43:11 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 29 Aug 2008 14:43:11 +0200 Subject: [Python-Dev] Documentation Error for __hash__ In-Reply-To: References: <48B6FD04.6080008@voidspace.org.uk> <48B7C718.7020802@egenix.com> Message-ID: <48B7EEDF.70305@egenix.com> On 2008-08-29 13:03, Matt Giuca wrote: >> Being hashable is a different from being usable as dictionary key. >> >> Dictionaries perform the lookup based on the hash value, but will >> then have to check for hash collisions based on an equal comparison. >> >> If an object does not define an equal comparison, then it is not >> usable as dictionary key. >> > > But if an object defines *neither* __eq__ or __hash__, then by default it is > usable as a dictionary key (using the id() of the object for both default > equality and hashing, which is fine, and works for all user-defined types by > default). > > An object defining __hash__ but not __eq__ is not problematic, since it > still uses id() default for equality. (It just has potentially bad > dictionary performance, if lots of things hash the same even though they > aren't equal). This it not a problem by definition because *it is officially > "okay" for two objects to hash the same, even if they aren't equal, though > undesirable*. Note that only instances have the default hash value id(obj). This is not true in general. Most types don't implement the tp_hash slot and thus are not hashable. Indeed, mutable types should not implement that slot unless they know what they're doing :-) > So all hashable objects are usable as dictionary keys, are they not? (As far > as I know it isn't possible to have an object that does not have an equality > comparison, unless you explicitly override __eq__ and have it raise a > TypeError for some reason). Sorry, I wasn't clear enough: with "not defining an equal comparison" I meant that an equal comparison does not succeed, ie. raises an exception or returns Py_NotImplemented (at the C level). Due to the default of using the id(obj) as fallback for the equal comparison, this has to be explicitly coded for. If this is not the case (and that's probably the most common situation), then you're right: hashable implies usable as dictionary key. >> It's probably a good idea to implement __hash__ for objects that >> implement comparisons, but it won't always work and it is certainly >> not needed, unless you intend to use them as dictionary keys. >> > > But from what I know, it is a *bad* idea to implement __hash__ for any > mutable object with non-reference equality (where equality depends on the > mutable state), as an unbreakable rule. This is because if they are inserted > into a dictionary, then mutated, they may suddenly be in the wrong bucket. > This is why all the mutable types in Python with non-reference equality (eg. > list, set, dict) are explicitly not hashable, while the immutable types (eg. > tuple, frozenset, str) are hashable, and so are the mutable types with > reference equality (eg. functions, user-defined classes by default). Right. By implementing __hash__ in classes you have the explicit choice of either raising an exception or returning a useful hash value. Again, the situation is better at the C level, since types don't have a default tp_hash implementation, so have to explicitly code such a slot in order for hash(obj) to work. >>> and that mutable objects should raise a TypeError in __hash__. >> That's a good idea, even though it's not needed either ;-) >> > > So I think my above "axioms" are a better (less restrictive, and still > correct) rule than this one. It's OK for a mutable object to define > __hash__, as long as its __eq__ doesn't depend upon its mutable state. For > example, you can insert a function object into a dictionary, and mutate its > closure, and it won't matter, because neither the hash nor the equality of > the object is changing. It's only types like list and dict, with deep > equality, where you run into this hash table problem. I think the documentation needs to be changed to make the defaults explicit. The documentation should probably say: "If you implement __cmp__ or __eq__ on a class, also implement a __hash__ method (and either have it raise an exception or return a valid non-changing hash value for the object)." "If you implement __hash__ on classes, you should consider implementing __eq__ and/or __cmp__ as well, in order to control how dictionaries use your objects." In general, it's probably best to always implement both methods on classes, even if the application will just use one of them. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 29 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From matt.giuca at gmail.com Fri Aug 29 14:47:48 2008 From: matt.giuca at gmail.com (Matt Giuca) Date: Fri, 29 Aug 2008 22:47:48 +1000 Subject: [Python-Dev] Documentation Error for __hash__ In-Reply-To: <48B7DBD8.8070205@voidspace.org.uk> References: <48B6FD04.6080008@voidspace.org.uk> <48B7C718.7020802@egenix.com> <48B7DBD8.8070205@voidspace.org.uk> Message-ID: > >> It's probably a good idea to implement __hash__ for objects that >> implement comparisons, but it won't always work and it is certainly >> not needed, unless you intend to use them as dictionary keys. >> >> >> > > > So you're suggesting that we document something like. > > Classes that represent mutable values and define equality methods are free > to define __hash__ so long as you don't mind them being used incorrectly if > treated as dictionary keys... > > Technically true, but not very helpful in my opinion... :-) No, I think he was suggesting we document that if a class overrides __eq__, it's a good idea to also implement __hash__, so it can be used as a dictionary key. However I have issues with this. First, he said: "It's probably a good idea to implement __hash__ for objects that implement comparisons, but it won't always work and it is certainly not needed, unless you intend to use them as dictionary keys." You can't say "certainly not needed unless you intend to use them as dictionary keys", since if you are defining an object, you never know when someone else will want to use them as a dict key (or in a set, mind!) So *if possible*, it is a good idea to implement __hash__ if you are implementing __eq__. But also, it needs to be very clear that if you *should not* implement __hash__ on a mutable object -- and it already is. So basically the docs should suggest that it is a good idea to implement __hash__ if you are implementing __eq__ on an immutable object. HOWEVER, There are two contradictory pieces of information in the docs. a) "if it defines __cmp__() or __eq__() but not __hash__() , its instances will not be usable as dictionary keys." versus b) "User-defined classes have __cmp__() and __hash__() methods by default; with them, all objects compare unequal and x.__hash__() returns id(x)." Note that these statements are somewhat contradictory: if a class has a __hash__ method by default (as b suggests), then it isn't possible to "not have a __hash__" (as suggested by a). In Python 2, statement (a) is true for old-style classes only, while statement (b) is true for new style classes only. This distinction needs to be made. (For old-style classes, it isn't the case that it has a __hash__ method by default - rather that the hash() function knows how to deal with objects without a __hash__ method, by calling id()). In Python 3, statement (a) is true always, while statement (b) is not (in fact just the same as old-style classes are in Python 2). So the Python 3 docs can get away with being simpler (without having to handle that weird case). I just saw Marc-Andre's new email come in; I'll look at that now. Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From matt.giuca at gmail.com Fri Aug 29 15:07:25 2008 From: matt.giuca at gmail.com (Matt Giuca) Date: Fri, 29 Aug 2008 23:07:25 +1000 Subject: [Python-Dev] Documentation Error for __hash__ In-Reply-To: <48B7EEDF.70305@egenix.com> References: <48B6FD04.6080008@voidspace.org.uk> <48B7C718.7020802@egenix.com> <48B7EEDF.70305@egenix.com> Message-ID: > Note that only instances have the default hash value id(obj). This > is not true in general. Most types don't implement the tp_hash > slot and thus are not hashable. Indeed, mutable types should not > implement that slot unless they know what they're doing :-) By "instances" you mean "instances of user-defined classes"? (I carefully avoid the term "instance" on its own, since I believe that was phased out when they merged types and classes; it probably still exists in the C API but shouldn't be mentioned in Python-facing documentation). But anyway, yes, we should make that distinction. Sorry, I wasn't clear enough: with "not defining an equal comparison" > I meant that an equal comparison does not succeed, ie. raises an > exception or returns Py_NotImplemented (at the C level). Oh OK. I didn't even realise it was "valid" or "usual" to explicitly block __eq__ like that. > Again, the situation is better at the C level, since types > don't have a default tp_hash implementation, so have to explicitly > code such a slot in order for hash(obj) to work. Yes but I gather that this "data model" document we are talking about is not designed for C authors, but Python authors, so it should be written for the point of view of people coding only in Python. (Only the "Extending and Embedding" and the "C API" documents are for C authors). The documentation should probably say: > > "If you implement __cmp__ or > __eq__ on a class, also implement a __hash__ method (and either > have it raise an exception or return a valid non-changing hash > value for the object)." > I agree, except maybe not for the Python 3 docs. As long as the behaviour I am observing is well-defined and not just a side-effect which could go away -- that is, if you define __eq__/__cmp__ but not __hash__, in a user-defined class, it raises a TypeError -- then I think it isn't necessary to recommend implementing a __hash__ method and raising a TypeError. Better just to leave as-is ("if it defines __cmp__() or __eq__() but not __hash__() , its instances will not be usable as dictionary keys") and clarify the later statement. > > "If you implement __hash__ on classes, you should consider implementing > __eq__ and/or __cmp__ as well, in order to control how dictionaries use > your objects." I don't think I agree with that. I'm not sure why you'd implement __hash__ without __eq__ and/or __cmp__, but it doesn't cause issues so we may as well not address it. > In general, it's probably best to always implement both methods > on classes, even if the application will just use one of them. > Well it certainly is for new-style classes in the 2.x branch. I don't think you should implement __hash__ in Python 3 if you just want a non-hashable object (since this is the default behaviour anyway). A lot of my opinion here, though, which doesn't count very much -- so I'm just making suggestions. Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From python-3000-list at trentnelson.com Fri Aug 29 13:30:08 2008 From: python-3000-list at trentnelson.com (Trent Nelson) Date: Fri, 29 Aug 2008 11:30:08 +0000 Subject: [Python-Dev] [Python-3000] what version of bsddb to use/accept for 2.6/3.0 In-Reply-To: References: Message-ID: <20080829113008.GA29787@wind.teleri.net> On Sun, Aug 24, 2008 at 06:15:06PM -0700, Neal Norwitz wrote: > It looks like the WIndows buildbots use 4.4.20. Unfortunately, the > Windows bots aren't in great shape either. It won't be that hard to bump 3.0 on Windows to use 4.7 (assuming that the 3.0 bsddb codebase has Jesus's new code). I'll be able to take a look at this early next week. Trent. From ncoghlan at gmail.com Fri Aug 29 18:04:52 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 30 Aug 2008 02:04:52 +1000 Subject: [Python-Dev] Documentation Error for __hash__ In-Reply-To: References: <48B6FD04.6080008@voidspace.org.uk> <48B7C718.7020802@egenix.com> <48B7DBD8.8070205@voidspace.org.uk> Message-ID: <48B81E24.2070906@gmail.com> Matt Giuca wrote: > > > It's probably a good idea to implement __hash__ for objects that > implement comparisons, but it won't always work and it is certainly > not needed, unless you intend to use them as dictionary keys. > > > > > > So you're suggesting that we document something like. > > Classes that represent mutable values and define equality methods > are free to define __hash__ so long as you don't mind them being > used incorrectly if treated as dictionary keys... > > Technically true, but not very helpful in my opinion... :-) > > > No, I think he was suggesting we document that if a class overrides > __eq__, it's a good idea to also implement __hash__, so it can be used > as a dictionary key. I strongly advise people interested in this topic to take a closer look at the functionality that was added to address issue 2235. The "don't inherit __hash__" behaviour has been backported from 3.0 and broke a bunch of code, but the naive fix of reverting to the 2.5 behaviour proceeded to make the 2.6 version of collections.Hashable completely useless (since pretty much *everything* then claimed to be hashable, including all of the container types in the standard library). After thrashing out those problems, the 2.6 behaviour ended up being: - __hash__ is still inherited by default - you can block inheritance explicitly by setting "__hash__ = None" in the class definition or on the class object - for a class defined in C, you can block __hash__ inheritance by setting the tp_hash slot to PyObject_HashNotImplemented - using one of the above approaches will cause hash(obj) to raise a TypeError, as will calling the tp_hash slot directly - unlike defining your own __hash__ method, using one of the above techniques will also mean that isinstance(obj, collections.Hashable) will also return False - defining __eq__ and/or __cmp__ without defining __hash__ will lead to a Py3k Warning being raised on the class definition when run with the -3 switch The Python 3.0 behaviour is similar to the above, except that the unconditional inheritance of __hash__ is gone. If you define __eq__ or __cmp__ without defining __hash__, __hash__ will NOT be inherited from the parent class - instead, it will be set to None. The documentation of all of this could definitely use a bit more work - that's why I reopened 2235 in response to Michael's post. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From status at bugs.python.org Fri Aug 29 18:06:34 2008 From: status at bugs.python.org (Python tracker) Date: Fri, 29 Aug 2008 18:06:34 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20080829160634.CBBD978357@psf.upfronthosting.co.za> ACTIVITY SUMMARY (08/22/08 - 08/29/08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2029 open (+66) / 13524 closed (+16) / 15553 total (+82) Open issues with patches: 655 Average duration of open issues: 704 days. Median duration of open issues: 1760 days. Open Issues Breakdown open 2015 (+65) pending 14 ( +1) Issues Created Or Reopened (84) _______________________________ __eq__ / __hash__ check doesn't take inheritance into account 08/28/08 http://bugs.python.org/issue2235 reopened ncoghlan patch Objects/obmalloc.c:529: warning: comparison is always false due 08/24/08 http://bugs.python.org/issue3642 reopened christian.heimes 64bit IA5 Encoding should be in the default encodings 08/22/08 CLOSED http://bugs.python.org/issue3649 created pascal.bach Memory leak in bytes.split() 08/22/08 CLOSED http://bugs.python.org/issue3650 created amaury.forgeotdarc patch, needs review eval() leaks 1 reference every time 08/22/08 CLOSED http://bugs.python.org/issue3651 created amaury.forgeotdarc patch, needs review Remove DeprecationWarning in _warnings about 'line' 08/23/08 http://bugs.python.org/issue3652 created brett.cannon segfault calling sys.excepthook with non-Exception argument 08/23/08 CLOSED http://bugs.python.org/issue3653 created ajaksu2 patch, needs review Duplicated test name in regex test script 08/23/08 CLOSED http://bugs.python.org/issue3654 created mrabarnett latexwriter: \footnotemark can only take numbers as arguments 08/23/08 http://bugs.python.org/issue3655 created pv unicode encoding has lots of leaks of bytes 08/24/08 CLOSED http://bugs.python.org/issue3656 created nnorwitz pickle can pickle the wrong function 08/24/08 http://bugs.python.org/issue3657 created nnorwitz fix for pychecker property complaints 08/24/08 http://bugs.python.org/issue3658 created skip.montanaro patch, easy, needs review sqlite: enumeration value 'TYPE_STRING' not handled in switch 08/24/08 http://bugs.python.org/issue3659 created christian.heimes reference leaks in 3.0 08/24/08 http://bugs.python.org/issue3660 created nnorwitz patch sys.call_tracing segfaults 08/24/08 http://bugs.python.org/issue3661 created ajaksu2 _fileio._FileIO segfaults 08/24/08 CLOSED http://bugs.python.org/issue3662 created ajaksu2 needs review Extra DECREF on syntax errors 08/24/08 CLOSED http://bugs.python.org/issue3663 created amaury.forgeotdarc patch, needs review Pickler.dump from a badly initialized Pickler segfaults 08/24/08 http://bugs.python.org/issue3664 created ajaksu2 Support \u and \U escapes in regexes 08/24/08 http://bugs.python.org/issue3665 created georg.brandl patch atexit.register with bad input segfaults on exit 08/24/08 http://bugs.python.org/issue3666 created ajaksu2 Reloading an extension module always leaks 08/24/08 http://bugs.python.org/issue3667 created amaury.forgeotdarc patch, needs review "s*" argument parser marker leaks memory 08/24/08 http://bugs.python.org/issue3668 created amaury.forgeotdarc patch, needs review sqlite3.Connection.iterdump docs pythonicity 08/24/08 http://bugs.python.org/issue3669 created lehmannro patch Reporting bugs - no such sections 08/24/08 CLOSED http://bugs.python.org/issue3670 created kjohnson What's New in 2.6 - corrections 08/24/08 http://bugs.python.org/issue3671 created kjohnson Ill-formed surrogates not treated as errors during encoding/deco 08/24/08 http://bugs.python.org/issue3672 created Rhamphoryncus bsddb module leaks memory 08/25/08 http://bugs.python.org/issue3673 created nnorwitz patch, patch test_dbm_ndbm skip is unexpected on win32? 08/25/08 http://bugs.python.org/issue3674 created ocean-city patch Python 2.6 can't read sets pickled with Python 3.0 08/25/08 http://bugs.python.org/issue3675 created hagen patch Obsolete references to PEP 291 in py3k lib 08/25/08 http://bugs.python.org/issue3676 created orent importing from UNC roots doesn't work 08/25/08 http://bugs.python.org/issue3677 created krisvale Ignored LDFLAGS during linking libpython$(VERSION).so 08/25/08 http://bugs.python.org/issue3678 created Arfrever patch, easy, needs review pressing HOME key in IDLE editor ends IDLE 08/25/08 http://bugs.python.org/issue3679 created tjd Cycles with some iterator are leaking. 08/25/08 http://bugs.python.org/issue3680 created schuppenies patch Cannot read saved csv file in a single run 08/25/08 http://bugs.python.org/issue3681 created paul test_math: math.log(-ninf) fails to raise exception on OpenBSD 08/26/08 http://bugs.python.org/issue3682 created djmdjm compilation --without-threads fails 08/26/08 http://bugs.python.org/issue3683 created djmdjm needs review traceback.format_exception_only() misplaces the caret for certai 08/26/08 http://bugs.python.org/issue3684 created brodierao patch Crash while compiling Python 3000 in OpenBSD 4.4 08/26/08 http://bugs.python.org/issue3685 created henry.precheur patch PKG-INFO file should differentiate between authors and maintaine 08/26/08 http://bugs.python.org/issue3686 created lemburg Popen() object stdout attribute reassignment behaviour 08/26/08 http://bugs.python.org/issue3687 created vincent.legoll open() rejects bytes as filename 08/26/08 CLOSED http://bugs.python.org/issue3688 created dlitz listreverseiterator has a decreasing len() 08/26/08 http://bugs.python.org/issue3689 created laxrulz777 sys.getsizeof wrong for Py3k bool objects 08/26/08 http://bugs.python.org/issue3690 created schuppenies patch Incorrect variable reference 08/26/08 CLOSED http://bugs.python.org/issue3691 created timh improper scope in list comprehension, when used in class declara 08/26/08 CLOSED http://bugs.python.org/issue3692 created kaizhu Obscure array.array error message 08/27/08 http://bugs.python.org/issue3693 created tjreedy Undetected error in _struct.pack_into 08/27/08 http://bugs.python.org/issue3694 created ajaksu2 patch Source tarball for Sphinx 0.4.3 missing 08/27/08 http://bugs.python.org/issue3695 created hircus Error parsing arguments on OpenBSD <= 4.4 08/27/08 http://bugs.python.org/issue3696 created henry.precheur patch, needs review "Fatal Python error: Cannot recover from stack overflow" on Wind 08/27/08 http://bugs.python.org/issue3697 created pitrou patch, needs review incompatible arguments in warning formatting for idle 08/27/08 http://bugs.python.org/issue3698 created gcoffin test_bigaddrspace broken 08/27/08 http://bugs.python.org/issue3699 created pitrou test_bigmem broken 08/27/08 http://bugs.python.org/issue3700 created pitrou test_ntpath.test_relpath fails when launched from a different Wi 08/27/08 http://bugs.python.org/issue3701 created pitrou test_urllib2.test_trivial fails when run from another Windows dr 08/27/08 http://bugs.python.org/issue3702 created pitrou open() on directory raises IOError with unhelpful message 08/27/08 http://bugs.python.org/issue3703 created hagen patch, needs review cookielib doesn't handle URLs with / in parameters 08/27/08 http://bugs.python.org/issue3704 created andyk patch, needs review py3k aborts if "-c" or "-m" is given a non-ascii value 08/27/08 http://bugs.python.org/issue3705 created pitrou patch, needs review Fix error message for wrong exec() argument type 08/27/08 CLOSED http://bugs.python.org/issue3706 created georg.brandl patch, patch, needs review help('finally') behaves bizarrely 08/27/08 http://bugs.python.org/issue3707 created ms patch, easy, needs review os.urandom(1.1): infinite loop 08/27/08 http://bugs.python.org/issue3708 created ajaksu2 patch, easy, needs review BaseHTTPRequestHandler innefficient when sending HTTP header 08/27/08 http://bugs.python.org/issue3709 created yangman Reference leak in thread._local 08/28/08 http://bugs.python.org/issue3710 created tamino patch docs still say to use .dll for compiled extensions 08/28/08 CLOSED http://bugs.python.org/issue3711 created gagenellina patch memoryview leaks references 08/28/08 http://bugs.python.org/issue3712 created amaury.forgeotdarc patch, needs review Compile warning for Objects/stringlib 08/28/08 CLOSED http://bugs.python.org/issue3713 created kirkshorts patch nntplib module broken by str to unicode conversion 08/28/08 http://bugs.python.org/issue3714 created hdima patch hashlib's docstring throws exception in pydoc 08/28/08 http://bugs.python.org/issue3715 created erickt mistake in 3.4.2 Customizing attribute access 08/28/08 http://bugs.python.org/issue3716 created Alex7564 Py_InitModule* is still referenced in docs 08/28/08 http://bugs.python.org/issue3717 created erickt environment variable MACHDEP and python build system 08/28/08 http://bugs.python.org/issue3718 created rpetrov patch platform.py: _syscmd_file() can't handle target path with space 08/29/08 http://bugs.python.org/issue3719 created jfdp segfault in for loop with evil iterator 08/29/08 http://bugs.python.org/issue3720 created gideon patch, needs review invalid literal for int() with base 16: '' 08/29/08 http://bugs.python.org/issue3721 created xhomol11 print followed by exception eats print with doctest 08/29/08 http://bugs.python.org/issue3722 created cjw296 Py_NewInterpreter does not work 08/29/08 http://bugs.python.org/issue3723 created amaury.forgeotdarc math.log(x, 10) gives different result than math.log10(x) 08/29/08 http://bugs.python.org/issue3724 created segfaulthunter patch telnetlib module broken by str to unicode conversion 08/29/08 http://bugs.python.org/issue3725 created hdima Allow ', ' delimiters in logging.config.fileConfig() 08/29/08 http://bugs.python.org/issue3726 created wcmaier patch poplib module broken by str to unicode conversion 08/29/08 http://bugs.python.org/issue3727 created hdima imaplib module broken by str to unicode conversion 08/29/08 CLOSED http://bugs.python.org/issue3728 created hdima SystemError on calling len() if __len__() doesn't return an int 08/29/08 http://bugs.python.org/issue3729 created hagen patch Update BaseHTTPServer docs 08/29/08 http://bugs.python.org/issue3730 created catlee Issues Now Closed (35) ______________________ LookupError: unknown encoding: X-MAC-JAPANESE 317 days http://bugs.python.org/issue1276 lemburg patch Error using >>> from OpenGL.GLUT import * 301 days http://bugs.python.org/issue1346 igorcamp Using __(get|set|del)slice__ needs a Py3K warning 160 days http://bugs.python.org/issue2351 benjamin.peterson patch, needs review Fixer to handle new places where parentheses are needed 158 days http://bugs.python.org/issue2367 benjamin.peterson patch, 26backport bytes() should respect __bytes__ 161 days http://bugs.python.org/issue2415 benjamin.peterson patch, needs review urllib2 can't handle http://www.wikispaces.com 157 days http://bugs.python.org/issue2464 orsenthil patch Restore isinstance and issubclass speed in 2.6 147 days http://bugs.python.org/issue2534 pitrou patch, patch, needs review Locking should be removed from the new buffer protocol 80 days http://bugs.python.org/issue3046 teoliphant patch bytearrays are not thread safe 66 days http://bugs.python.org/issue3139 pitrou patch multiprocessing build fails on Solaris 10 68 days http://bugs.python.org/issue3149 pitrou 2to3 Fix_imports optimization 59 days http://bugs.python.org/issue3218 benjamin.peterson patch invalid call to PyMem_Free() in fileio_init() 49 days http://bugs.python.org/issue3304 nnorwitz patch, easy making partial functions comparable 7 days http://bugs.python.org/issue3564 tjreedy SPAM 5 days http://bugs.python.org/issue3579 tjreedy invalid exception context 8 days http://bugs.python.org/issue3611 amaury.forgeotdarc patch, needs review shutil.rmtree() fails on invalid filename 8 days http://bugs.python.org/issue3616 pitrou test issues on 64bit windows 2 days http://bugs.python.org/issue3625 mhammond patch, patch, 64bit apple security patches need to be forward ported to py3k 3 days http://bugs.python.org/issue3627 nnorwitz Add more checks to testcapi 2 days http://bugs.python.org/issue3643 benjamin.peterson patch IA5 Encoding should be in the default encodings 3 days http://bugs.python.org/issue3649 pascal.bach Memory leak in bytes.split() 0 days http://bugs.python.org/issue3650 amaury.forgeotdarc patch, needs review eval() leaks 1 reference every time 4 days http://bugs.python.org/issue3651 benjamin.peterson patch, needs review segfault calling sys.excepthook with non-Exception argument 1 days http://bugs.python.org/issue3653 benjamin.peterson patch, needs review Duplicated test name in regex test script 1 days http://bugs.python.org/issue3654 georg.brandl unicode encoding has lots of leaks of bytes 0 days http://bugs.python.org/issue3656 amaury.forgeotdarc _fileio._FileIO segfaults 0 days http://bugs.python.org/issue3662 nnorwitz needs review Extra DECREF on syntax errors 2 days http://bugs.python.org/issue3663 pitrou patch, needs review Reporting bugs - no such sections 0 days http://bugs.python.org/issue3670 kjohnson open() rejects bytes as filename 0 days http://bugs.python.org/issue3688 amaury.forgeotdarc Incorrect variable reference 0 days http://bugs.python.org/issue3691 benjamin.peterson improper scope in list comprehension, when used in class declara 0 days http://bugs.python.org/issue3692 georg.brandl Fix error message for wrong exec() argument type 0 days http://bugs.python.org/issue3706 georg.brandl patch, patch, needs review docs still say to use .dll for compiled extensions 0 days http://bugs.python.org/issue3711 georg.brandl patch Compile warning for Objects/stringlib 0 days http://bugs.python.org/issue3713 christian.heimes patch imaplib module broken by str to unicode conversion 0 days http://bugs.python.org/issue3728 hdima Top Issues Most Discussed (10) ______________________________ 23 eval() leaks 1 reference every time 4 days closed http://bugs.python.org/issue3651 10 "s*" argument parser marker leaks memory 5 days open http://bugs.python.org/issue3668 9 import of site.py fails on startup 57 days open http://bugs.python.org/issue3279 8 Restore isinstance and issubclass speed in 2.6 147 days closed http://bugs.python.org/issue2534 7 Error parsing arguments on OpenBSD <= 4.4 2 days open http://bugs.python.org/issue3696 7 IA5 Encoding should be in the default encodings 3 days closed http://bugs.python.org/issue3649 6 os.urandom(1.1): infinite loop 2 days open http://bugs.python.org/issue3708 6 Objects/obmalloc.c:529: warning: comparison is always false due 5 days open http://bugs.python.org/issue3642 6 test_cpickle crash on AMD64 Windows build 8 days open http://bugs.python.org/issue3640 6 python3.0 interpreter on Cygwin ignores all arguments 8 days open http://bugs.python.org/issue3626 From ncoghlan at gmail.com Fri Aug 29 18:15:20 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 30 Aug 2008 02:15:20 +1000 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <4edc17eb0808282144l6d81979bic95ae8c11d69cd37@mail.gmail.com> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <20080827031445.C9B8B3A4079@sparrow.telecommunity.com> <4edc17eb0808262224g59c959c9s8a15f778695b8e7a@mail.gmail.com> <4edc17eb0808272135q44985feaj1ba550022919eaac@mail.gmail.com> <20080828152941.CD4E43A410E@sparrow.telecommunity.com> <4edc17eb0808280850xa842656l9fea70acc0717bce@mail.gmail.com> <20080828185312.943163A408D@sparrow.telecommunity.com> <4edc17eb0808282107q65969a63qb6b8745461b57b44@mail.gmail.com> <20080829042146.6DC123A408D@sparrow.telecommunity.com> <4edc17eb0808282144l6d81979bic95ae8c11d69cd37@mail.gmail.com> Message-ID: <48B82098.8090106@gmail.com> Michele Simionato wrote: > OTOH, for what concerns multiple inheritance, I am still not > convinced it is really worth it. I mean, the MRO is beautiful, > elegant and all that on paper, but on real-life code things as different, > especially from the side of the users of frameworks heavily > based on inheritance. The mixin methods in the ABC machinery would be a lot less useful without multiple inheritance (and the collections ABCs would be a whole lot harder to define and to write). So if you're looking for use cases for multiple inheritance, I'd suggest starting with the Python 2.6 collections module and seeing how you would go about rewriting it using only single inheritance. I believe the new io module is also fairly dependent on multiple inheritance. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From michele.simionato at gmail.com Fri Aug 29 19:46:36 2008 From: michele.simionato at gmail.com (Michele Simionato) Date: Fri, 29 Aug 2008 19:46:36 +0200 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <48B82098.8090106@gmail.com> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808272135q44985feaj1ba550022919eaac@mail.gmail.com> <20080828152941.CD4E43A410E@sparrow.telecommunity.com> <4edc17eb0808280850xa842656l9fea70acc0717bce@mail.gmail.com> <20080828185312.943163A408D@sparrow.telecommunity.com> <4edc17eb0808282107q65969a63qb6b8745461b57b44@mail.gmail.com> <20080829042146.6DC123A408D@sparrow.telecommunity.com> <4edc17eb0808282144l6d81979bic95ae8c11d69cd37@mail.gmail.com> <48B82098.8090106@gmail.com> Message-ID: <4edc17eb0808291046j62d6ff9dh612e7d0698bbfba6@mail.gmail.com> On Fri, Aug 29, 2008 at 6:15 PM, Nick Coghlan wrote: > The mixin methods in the ABC machinery would be a lot less useful > without multiple inheritance (and the collections ABCs would be a whole > lot harder to define and to write). > > So if you're looking for use cases for multiple inheritance, I'd suggest > starting with the Python 2.6 collections module and seeing how you would > go about rewriting it using only single inheritance. I believe the new > io module is also fairly dependent on multiple inheritance. I am very well aware of the collection module and the ABC mechanism. However, you are missing that mixins can be implemented in a single-inheritance world without the complications of the MRO. See my answer to Alex Martelli in this same thread. From casey at pandora.com Fri Aug 29 20:33:35 2008 From: casey at pandora.com (Casey Duncan) Date: Fri, 29 Aug 2008 12:33:35 -0600 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <4edc17eb0808291046j62d6ff9dh612e7d0698bbfba6@mail.gmail.com> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808272135q44985feaj1ba550022919eaac@mail.gmail.com> <20080828152941.CD4E43A410E@sparrow.telecommunity.com> <4edc17eb0808280850xa842656l9fea70acc0717bce@mail.gmail.com> <20080828185312.943163A408D@sparrow.telecommunity.com> <4edc17eb0808282107q65969a63qb6b8745461b57b44@mail.gmail.com> <20080829042146.6DC123A408D@sparrow.telecommunity.com> <4edc17eb0808282144l6d81979bic95ae8c11d69cd37@mail.gmail.com> <48B82098.8090106@gmail.com> <4edc17eb0808291046j62d6ff9dh612e7d0698bbfba6@mail.gmail.com> Message-ID: <5A0A8180-273D-4739-869C-8F5A35184FC5@pandora.com> On Aug 29, 2008, at 11:46 AM, Michele Simionato wrote: > On Fri, Aug 29, 2008 at 6:15 PM, Nick Coghlan > wrote: >> The mixin methods in the ABC machinery would be a lot less useful >> without multiple inheritance (and the collections ABCs would be a >> whole >> lot harder to define and to write). >> >> So if you're looking for use cases for multiple inheritance, I'd >> suggest >> starting with the Python 2.6 collections module and seeing how you >> would >> go about rewriting it using only single inheritance. I believe the >> new >> io module is also fairly dependent on multiple inheritance. > > I am very well aware of the collection module and the ABC mechanism. > However, you are missing that mixins can be implemented in a single- > inheritance > world without the complications of the MRO. See my answer to Alex > Martelli in this same thread. As interesting as this conversation is at a meta-level, I'm not sure how much more can be accomplished here by debating the merits of multiple vs. single inheritance. Unfortunately I think this is a case where there is not just one good way to do it in all cases, especially given the subjective nature of "good" in this context. This is what I take away from this: - super() is tricky to use at best, and its documentation is inaccurate and incomplete. I think it should also be made more clear that super() is really mostly useful for framework developers, including users extending frameworks. Unfortunately many frameworks require you to extend them in order to write useful applications in my experience, so it trickles down to the app developer at times. In short, more correct documentation == good. - The difficulties of super() are really symptomatic of the difficulties of multiple inheritance. I think it's clear that multiple inheritance is here to stay in Python, and it solves certain classes of problems quite well. But, it requires careful consideration, and it's easy to get carried away and create a huge mess (ala Zope2, which I am all too familiar), and it can hinder code clarity as much as it facilitates reuse. - There are good alternatives to multiple inheritance for many cases, but there are cases where multiple inheritance is arguably best. Traits are a possible alternative that deserve further study. I think that study would be greatly aided by a 3rd-party library implementing traits for Python. If traits are to gain any traction or ever be considered for inclusion into the language such a library would need to exist first and demonstrate its utility. I know I'm probably just stating the obvious here, but I found it therapeutic ;^) -Casey From glyph at divmod.com Fri Aug 29 22:31:07 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Fri, 29 Aug 2008 20:31:07 -0000 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <5A0A8180-273D-4739-869C-8F5A35184FC5@pandora.com> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808272135q44985feaj1ba550022919eaac@mail.gmail.com> <20080828152941.CD4E43A410E@sparrow.telecommunity.com> <4edc17eb0808280850xa842656l9fea70acc0717bce@mail.gmail.com> <20080828185312.943163A408D@sparrow.telecommunity.com> <4edc17eb0808282107q65969a63qb6b8745461b57b44@mail.gmail.com> <20080829042146.6DC123A408D@sparrow.telecommunity.com> <4edc17eb0808282144l6d81979bic95ae8c11d69cd37@mail.gmail.com> <48B82098.8090106@gmail.com> <4edc17eb0808291046j62d6ff9dh612e7d0698bbfba6@mail.gmail.com> <5A0A8180-273D-4739-869C-8F5A35184FC5@pandora.com> Message-ID: <20080829203107.25821.320776057.divmod.xquotient.15101@joule.divmod.com> On 06:33 pm, casey at pandora.com wrote: >On Aug 29, 2008, at 11:46 AM, Michele Simionato wrote: >>On Fri, Aug 29, 2008 at 6:15 PM, Nick Coghlan >>wrote: >>I am very well aware of the collection module and the ABC mechanism. >>However, you are missing that mixins can be implemented in a single- >>inheritance >>world without the complications of the MRO. See my answer to Alex >>Martelli in this same thread. >- super() is tricky to use at best, and its documentation is >inaccurate and incomplete. I think it should also be made more clear >that super() is really mostly useful for framework developers, >including users extending frameworks. Unfortunately many frameworks >require you to extend them in order to write useful applications in my >experience, so it trickles down to the app developer at times. In >short, more correct documentation == good. >I know I'm probably just stating the obvious here, but I found it >therapeutic ;^) I think this is a problem with this topic. Everyone writing about super() seems to be not just clearing up the documentation issues that surround it, but venting from personal frustrations with using it as well. I confess that I have done the same - if not in widely-publicized articles, at least on IRC and mailing list posts. I think it would benefit everyone if this discussion would end up with some patches to the library documentation that documented the semantics of super() more completely in the reference documentation and the "multiple inheritance" area of the tutorial, so that when people *do* run in to difficulties there is a very clear, central explanation of what it's supposed to do. Personally I think the thing that really should be pointed out is that it may behave in a confusing manner if the signature of the method being invoked is not the same on all classes in the same inheritance hierarchy. Theoretical problems aside, 99% of the trouble I've had with super() has to do with __init__ methods. I'll try my hand at such a patch over the weekend, but I'd be grateful for some pointers on a "quick start" for that. I am a complete newb at modifying the official documentation, and I seem to recall from a prior (failed) attempt that the tools are a bit difficult to work with. From python at rcn.com Fri Aug 29 22:39:39 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 29 Aug 2008 13:39:39 -0700 Subject: [Python-Dev] Things to Know About Super References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808272135q44985feaj1ba550022919eaac@mail.gmail.com><20080828152941.CD4E43A410E@sparrow.telecommunity.com><4edc17eb0808280850xa842656l9fea70acc0717bce@mail.gmail.com><20080828185312.943163A408D@sparrow.telecommunity.com><4edc17eb0808282107q65969a63qb6b8745461b57b44@mail.gmail.com><20080829042146.6DC123A408D@sparrow.telecommunity.com><4edc17eb0808282144l6d81979bic95ae8c11d69cd37@mail.gmail.com><48B82098.8090106@gmail.com><4edc17eb0808291046j62d6ff9dh612e7d0698bbfba6@mail.gmail.com><5A0A8180-273D-4739-869C-8F5A35184FC5@pandora.com> <20080829203107.25821.320776057.divmod.xquotient.15101@joule.divmod.com> Message-ID: <23438612C5C6482F89B23B262ECE4259@RaymondLaptop1> From: > I think it would benefit everyone if this discussion would end up with > some patches to the library documentation that documented the semantics > of super() more completely in the reference documentation and the > "multiple inheritance" area of the tutorial, so that when people *do* > run in to difficulties there is a very clear, central explanation of > what it's supposed to do. Am working on a doc patch, will post this weekend. Raymond From michele.simionato at gmail.com Sat Aug 30 06:16:15 2008 From: michele.simionato at gmail.com (Michele Simionato) Date: Sat, 30 Aug 2008 06:16:15 +0200 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <5A0A8180-273D-4739-869C-8F5A35184FC5@pandora.com> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <20080828152941.CD4E43A410E@sparrow.telecommunity.com> <4edc17eb0808280850xa842656l9fea70acc0717bce@mail.gmail.com> <20080828185312.943163A408D@sparrow.telecommunity.com> <4edc17eb0808282107q65969a63qb6b8745461b57b44@mail.gmail.com> <20080829042146.6DC123A408D@sparrow.telecommunity.com> <4edc17eb0808282144l6d81979bic95ae8c11d69cd37@mail.gmail.com> <48B82098.8090106@gmail.com> <4edc17eb0808291046j62d6ff9dh612e7d0698bbfba6@mail.gmail.com> <5A0A8180-273D-4739-869C-8F5A35184FC5@pandora.com> Message-ID: <4edc17eb0808292116q39763348g162b8f90787e8e6c@mail.gmail.com> On Fri, Aug 29, 2008 at 8:33 PM, Casey Duncan wrote: > - There are good alternatives to multiple inheritance for many cases, but > there are cases where multiple inheritance is arguably best. Maybe, but I am still biased in the opposite direction ;) >Traits are a > possible alternative that deserve further study. I think that study would be > greatly aided by a 3rd-party library implementing traits for Python. If > traits are to gain any traction or ever be considered for inclusion into the > language such a library would need to exist first and demonstrate its > utility. I wrote a trait library which I plan to release soon or later. However it is intended as a proof of concept, not as a candidate for inclusion in the standard library. As of now, I don't think we should have a different way of doing mixins in the standard library. There should be only one obvious way and the obvious way in current Python is multiple inheritance as it is now. The proof of concept is important for educational purposes, to open the mind to alternatives, to give inspiration to the creators of new languages: it is not intended to add complication (whenever small) to current Python. Having said that, maybe once I release the library people will start using it in production and will ask for inclusion for the standard library, but this is not my goal now. This happened for my decorator module years ago: when I wrote it I did not expect people to use it, I saw it as a temporary hack until we got a better support for fiddling with function signatures in the standard library. Nevertheless now a lot of people are using it and I am not even sure it is a good thing (I have seen many decorator abuses out there). This the strange thing that happens when you release a library: you will never now what people will end up using it for ;) Michele Simionato From michal at yogi.htu.tuwien.ac.at Sat Aug 30 12:04:41 2008 From: michal at yogi.htu.tuwien.ac.at (Michal Revucky) Date: Sat, 30 Aug 2008 12:04:41 +0200 Subject: [Python-Dev] script containing all opcs Message-ID: <20080830100441.GC29943@yogi> hello everyone, i would like to get a python script which executes all interpreter's opcodes, or how am i supposed to create such script... i just need to make sure that all opcodes (as defined in Include/opcode.h) are executed by this scrip i need this script for testing purposes while rewritting the python's ceval.c code... bye michal From g.brandl at gmx.net Sat Aug 30 15:15:09 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 30 Aug 2008 15:15:09 +0200 Subject: [Python-Dev] script containing all opcs In-Reply-To: <20080830100441.GC29943@yogi> References: <20080830100441.GC29943@yogi> Message-ID: Michal Revucky schrieb: > hello everyone, > > i would like to get a python script which executes all interpreter's opcodes, or > how am i supposed to create such script... i just need to make sure that all > opcodes (as defined in Include/opcode.h) are executed by this scrip > i need this script for testing purposes while rewritting the python's ceval.c > code... This question might better suit comp.lang.python, but nevertheless: The opcodes are quite well documented, in the documentation of the "dis" module. Many of them serve a specific purpose (like implementing `x[y]` or `raise x`) and should therefore be easy to test. For the others, like ROT_TWO, you'll need to look into Python/compile.c to find out in which cases they are generated. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ondrej at certik.cz Sat Aug 30 20:41:30 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Sat, 30 Aug 2008 20:41:30 +0200 Subject: [Python-Dev] Documentation Error for __hash__ In-Reply-To: <48B81E24.2070906@gmail.com> References: <48B6FD04.6080008@voidspace.org.uk> <48B7C718.7020802@egenix.com> <48B7DBD8.8070205@voidspace.org.uk> <48B81E24.2070906@gmail.com> Message-ID: <85b5c3130808301141t4f2b24d8ybd544de0f26d7f4c@mail.gmail.com> > I strongly advise people interested in this topic to take a closer look > at the functionality that was added to address issue 2235. The "don't > inherit __hash__" behaviour has been backported from 3.0 and broke a > bunch of code, but the naive fix of reverting to the 2.5 behaviour > proceeded to make the 2.6 version of collections.Hashable completely > useless (since pretty much *everything* then claimed to be hashable, > including all of the container types in the standard library). > > After thrashing out those problems, the 2.6 behaviour ended up being: > - __hash__ is still inherited by default > - you can block inheritance explicitly by setting "__hash__ = None" in > the class definition or on the class object > - for a class defined in C, you can block __hash__ inheritance by > setting the tp_hash slot to PyObject_HashNotImplemented > - using one of the above approaches will cause hash(obj) to raise a > TypeError, as will calling the tp_hash slot directly > - unlike defining your own __hash__ method, using one of the above > techniques will also mean that isinstance(obj, collections.Hashable) > will also return False > - defining __eq__ and/or __cmp__ without defining __hash__ will lead to > a Py3k Warning being raised on the class definition when run with the -3 > switch > > The Python 3.0 behaviour is similar to the above, except that the > unconditional inheritance of __hash__ is gone. If you define __eq__ or > __cmp__ without defining __hash__, __hash__ will NOT be inherited from > the parent class - instead, it will be set to None. > > The documentation of all of this could definitely use a bit more work - > that's why I reopened 2235 in response to Michael's post. Couple months ago I was trying to figure out how __eq__, __cmp__, __hash__ and similar methods are called in Python and in which order and wrote a docs here: http://docs.sympy.org/python-comparisons.html Feel free to reuse any of it. I wrote it to myself, as I didn't find the official docs very clear. Ondrej From jnoller at gmail.com Sat Aug 30 21:13:32 2008 From: jnoller at gmail.com (Jesse Noller) Date: Sat, 30 Aug 2008 15:13:32 -0400 Subject: [Python-Dev] Documentation Error for __hash__ In-Reply-To: <85b5c3130808301141t4f2b24d8ybd544de0f26d7f4c@mail.gmail.com> References: <48B6FD04.6080008@voidspace.org.uk> <48B7C718.7020802@egenix.com> <48B7DBD8.8070205@voidspace.org.uk> <48B81E24.2070906@gmail.com> <85b5c3130808301141t4f2b24d8ybd544de0f26d7f4c@mail.gmail.com> Message-ID: <9AE6735E-45F6-4A66-B749-1DB7A3D39EA3@gmail.com> On Aug 30, 2008, at 2:41 PM, "Ondrej Certik" wrote: >> I strongly advise people interested in this topic to take a closer >> look >> at the functionality that was added to address issue 2235. The "don't >> inherit __hash__" behaviour has been backported from 3.0 and broke a >> bunch of code, but the naive fix of reverting to the 2.5 behaviour >> proceeded to make the 2.6 version of collections.Hashable completely >> useless (since pretty much *everything* then claimed to be hashable, >> including all of the container types in the standard library). >> >> After thrashing out those problems, the 2.6 behaviour ended up being: >> - __hash__ is still inherited by default >> - you can block inheritance explicitly by setting "__hash__ = None" >> in >> the class definition or on the class object >> - for a class defined in C, you can block __hash__ inheritance by >> setting the tp_hash slot to PyObject_HashNotImplemented >> - using one of the above approaches will cause hash(obj) to raise a >> TypeError, as will calling the tp_hash slot directly >> - unlike defining your own __hash__ method, using one of the above >> techniques will also mean that isinstance(obj, collections.Hashable) >> will also return False >> - defining __eq__ and/or __cmp__ without defining __hash__ will >> lead to >> a Py3k Warning being raised on the class definition when run with >> the -3 >> switch >> >> The Python 3.0 behaviour is similar to the above, except that the >> unconditional inheritance of __hash__ is gone. If you define __eq__ >> or >> __cmp__ without defining __hash__, __hash__ will NOT be inherited >> from >> the parent class - instead, it will be set to None. >> >> The documentation of all of this could definitely use a bit more >> work - >> that's why I reopened 2235 in response to Michael's post. > > Couple months ago I was trying to figure out how __eq__, __cmp__, > __hash__ and similar methods are called in Python and in which order > and wrote a docs here: > > http://docs.sympy.org/python-comparisons.html > > Feel free to reuse any of it. I wrote it to myself, as I didn't find > the official docs very clear. > > Ondrej Ondrej, a patch that improves the official docs would be welcome and still potentially make 2.6/3.0 From ondrej at certik.cz Sat Aug 30 21:35:21 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Sat, 30 Aug 2008 21:35:21 +0200 Subject: [Python-Dev] Documentation Error for __hash__ In-Reply-To: <9AE6735E-45F6-4A66-B749-1DB7A3D39EA3@gmail.com> References: <48B6FD04.6080008@voidspace.org.uk> <48B7C718.7020802@egenix.com> <48B7DBD8.8070205@voidspace.org.uk> <48B81E24.2070906@gmail.com> <85b5c3130808301141t4f2b24d8ybd544de0f26d7f4c@mail.gmail.com> <9AE6735E-45F6-4A66-B749-1DB7A3D39EA3@gmail.com> Message-ID: <85b5c3130808301235t7ce79662t767c2831e736d808@mail.gmail.com> >> Ondrej > > Ondrej, a patch that improves the official docs would be welcome and still > potentially make 2.6/3.0 That'd be awesome. I need to finish my thesis in the next couple days, so I'd welcome if anyone could just take it and put usefult things in. I could get to do it the next week the earliest. Ondrej From solipsis at pitrou.net Sat Aug 30 22:06:00 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 30 Aug 2008 20:06:00 +0000 (UTC) Subject: [Python-Dev] The recursion checking problem Message-ID: Hi, I was working on a recursion overflow checking bug (http://bugs.python.org/issue2548) and, while I've managed to produce a working patch, I've also become uncomfortable with the very idea of trying to plug all those holes just for the sake of plugging them. I'll try to explain why, by describing the conflicting factors I've identified: - more and more, we are adding calls to Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() all over the interpreter, to avoid special/obscure cases of undetected infinite recursion; this can probably be considered a good thing, but: - after a recursion error has been raised (technically a RuntimeError), usually some code has to do cleanup after noticing the exception; this cleanup now can very easily bump into the recursion limit again, due to the point mentioned above (the funniest example of this is PyErr_ExceptionMatches, which makes a call to PyObject_IsSubclass which itself increases the recursion count because __subclasscheck__ can be recursively invoked...). - to counter the latter problem, py3k has introduced a somewhat smarter mechanism (which I've tracked down to a commit in the defunct p3yk branch by Martin): when the recursion limit is exceeded, a special flag named "overflowed" is set in the thread state structure which disables the primary recursion check, so that cleanup code has a bit of room to increase the recursion count a bit. A secondary recursion check exists (equal to the primary one /plus/ 50) and, if it is reached, the interpreter aborts with a fatal error. The "overflowed" flag is cleared when the recursion count drops below the primary recursion limit /minus/ 50. Now it looks rather smart but: - unfortunately, some functions inside the interpreter discard every exception by design. The primary example is PyDict_GetItem(), which is certainly used quite a lot :-)... When PyDict_GetItem() returns NULL, the caller can only assume that the key isn't in the dict, it has no way to know that there was a critical problem due to a recursion overflow. I encountered the latter problem when trying to backport the py3k recursion overflow algorithm to trunk. A fatal error suddenly appeared in test_cpickle, and it turned out that the recursion count was exceeded in PyObject_RichCompare(), the error was then cleared in PyDict_GetItem(), but the "overflowed" flag was still set so that a subsequent recursion overflow would trigger the secondary check and lead to the fatal error. I guess that, if it doesn't happen in py3k, it's just by chance: the recursion overflow is probably happening at another point where errors don't get discarded. Indeed, the failure I got on trunk was manifesting itself when running "regrtest.py test_cpickle" but not directly "test_cpickle.py"... which shows how delicate the recursion mechanism has become. My attempt to solve the latter problem while still backporting the py3k scheme involves clearing the "overflowed" flag in PyErr_Clear(). This makes all tests pass ok, but also means the "overflowed" flag loses a lot of its meaning... since PyErr_Clear() is called in a lot of places (and, especially, in PyDict_GetItem()). Also, at this point I fear that the solution to the problem is becoming, because of its complexity, perhaps worse than the problem itself. That's why I'm bringing it here, to have your opinion. (I also suggest that we stop trying to fix recursion checking bugs until the stable release, so as to give us some time to do the Right Thing later - if there is such a thing) Regards Antoine. From brett at python.org Sat Aug 30 22:29:12 2008 From: brett at python.org (Brett Cannon) Date: Sat, 30 Aug 2008 13:29:12 -0700 Subject: [Python-Dev] The recursion checking problem In-Reply-To: References: Message-ID: On Sat, Aug 30, 2008 at 1:06 PM, Antoine Pitrou wrote: > > Hi, > > I was working on a recursion overflow checking bug > (http://bugs.python.org/issue2548) and, while I've managed to produce a working > patch, I've also become uncomfortable with the very idea of trying to plug all > those holes just for the sake of plugging them. I'll try to explain why, by > describing the conflicting factors I've identified: > > - more and more, we are adding calls to Py_EnterRecursiveCall() and > Py_LeaveRecursiveCall() all over the interpreter, to avoid special/obscure > cases of undetected infinite recursion; this can probably be considered a good > thing, but: > > - after a recursion error has been raised (technically a RuntimeError), usually > some code has to do cleanup after noticing the exception; this cleanup now can > very easily bump into the recursion limit again, due to the point mentioned > above (the funniest example of this is PyErr_ExceptionMatches, which makes a > call to PyObject_IsSubclass which itself increases the recursion count because > __subclasscheck__ can be recursively invoked...). > > - to counter the latter problem, py3k has introduced a somewhat smarter > mechanism (which I've tracked down to a commit in the defunct p3yk branch by > Martin): when the recursion limit is exceeded, a special flag named > "overflowed" is set in the thread state structure which disables the primary > recursion check, so that cleanup code has a bit of room to increase the > recursion count a bit. A secondary recursion check exists (equal to the primary > one /plus/ 50) and, if it is reached, the interpreter aborts with a fatal error. > The "overflowed" flag is cleared when the recursion count drops below the > primary recursion limit /minus/ 50. Now it looks rather smart but: > > - unfortunately, some functions inside the interpreter discard every exception > by design. The primary example is PyDict_GetItem(), which is certainly used > quite a lot :-)... When PyDict_GetItem() returns NULL, the caller can only > assume that the key isn't in the dict, it has no way to know that there was a > critical problem due to a recursion overflow. > As the comment says for PyDict_GetItem(), it's a relic from the days when there was no way to call Python code when making the call. That is no longer true (and is probably true for a lot of places where a similar assumption is made). > I encountered the latter problem when trying to backport the py3k recursion > overflow algorithm to trunk. A fatal error suddenly appeared in test_cpickle, > and it turned out that the recursion count was exceeded in > PyObject_RichCompare(), the error was then cleared in PyDict_GetItem(), but the > "overflowed" flag was still set so that a subsequent recursion overflow would > trigger the secondary check and lead to the fatal error. > > I guess that, if it doesn't happen in py3k, it's just by chance: the recursion > overflow is probably happening at another point where errors don't get > discarded. Indeed, the failure I got on trunk was manifesting itself when > running "regrtest.py test_cpickle" but not directly "test_cpickle.py"... which > shows how delicate the recursion mechanism has become. > > My attempt to solve the latter problem while still backporting the py3k scheme > involves clearing the "overflowed" flag in PyErr_Clear(). This makes all tests > pass ok, but also means the "overflowed" flag loses a lot of its meaning... > since PyErr_Clear() is called in a lot of places (and, especially, in > PyDict_GetItem()). > > > Also, at this point I fear that the solution to the problem is becoming, > because of its complexity, perhaps worse than the problem itself. That's why > I'm bringing it here, to have your opinion. > Well, For Py3K at least we might need to consider going through the C API and fixing it so that these incorrect assumptions that functions like PyDict_GetItem() make are no longer made by introducing some new functions that behave in a "better" way. And for the recursion issue, I think it stems from corners that are cut in the C API by us. We inline functions all over the place, assume that Python's implementation underneath the hood is going to make calls that stay in C, etc. But as time has gone on and we have added flexibility to Python, more and more places have a chance to call Python code and trigger issues. > (I also suggest that we stop trying to fix recursion checking bugs until the > stable release, so as to give us some time to do the Right Thing later - if > there is such a thing) > I have no problem punting for now; there is no way I would be willing to wager any amount of money that the recursion check covered all cases. I have always viewed the check as a bonus sanity check, but not something to heavily rely upon. -Brett From stephane at harobed.org Sat Aug 30 22:30:54 2008 From: stephane at harobed.org (=?iso-8859-1?q?KLEIN_St=E9phane?=) Date: Sat, 30 Aug 2008 20:30:54 +0000 (UTC) Subject: [Python-Dev] [optparse] I wonder if optparse is dead ? because... Message-ID: Hello, I wonder if optparse is dead ? because : * svn access is down (svn co svn://starship.python.net/optik/trunk optik) ( http://sourceforge.net/mailarchive/forum.php? thread_name=466F541C.6010804%40users.sourceforge.net&forum_name=optik- users ) * last release is out 2 years ago ( http://sourceforge.net/mailarchive/forum.php? thread_name=20060723161651.GA2076%40cthulhu.gerg.ca&forum_name=optik- users ) * Greg Ward mail ( gward at python.net ) and is home page (http:// www.gerg.ca/) are away * I'm interested by this feature http://sourceforge.net/tracker/index.php? func=detail&aid=1819510&group_id=38019&atid=421097 but it hasn't comment since one year ago. Otherwise optparse is a great module ! Thanks for your information, Stephane From g.brandl at gmx.net Sat Aug 30 23:09:05 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 30 Aug 2008 23:09:05 +0200 Subject: [Python-Dev] Documentation Error for __hash__ In-Reply-To: <85b5c3130808301235t7ce79662t767c2831e736d808@mail.gmail.com> References: <48B6FD04.6080008@voidspace.org.uk> <48B7C718.7020802@egenix.com> <48B7DBD8.8070205@voidspace.org.uk> <48B81E24.2070906@gmail.com> <85b5c3130808301141t4f2b24d8ybd544de0f26d7f4c@mail.gmail.com> <9AE6735E-45F6-4A66-B749-1DB7A3D39EA3@gmail.com> <85b5c3130808301235t7ce79662t767c2831e736d808@mail.gmail.com> Message-ID: Ondrej Certik schrieb: >>> Ondrej >> >> Ondrej, a patch that improves the official docs would be welcome and still >> potentially make 2.6/3.0 > > That'd be awesome. I need to finish my thesis in the next couple days, > so I'd welcome if anyone could just take it and put usefult things in. > I could get to do it the next week the earliest. Doc patches are always accepted, and I intend to update the online 2.6 and 3.0 docs continually even when the finals are released, so there is no particular deadline you have to make. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Sat Aug 30 23:11:21 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 30 Aug 2008 23:11:21 +0200 Subject: [Python-Dev] [optparse] I wonder if optparse is dead ? because... In-Reply-To: References: Message-ID: KLEIN St?phane schrieb: > Hello, > > I wonder if optparse is dead ? because : > > * svn access is down (svn co svn://starship.python.net/optik/trunk optik) > ( http://sourceforge.net/mailarchive/forum.php? > thread_name=466F541C.6010804%40users.sourceforge.net&forum_name=optik- > users ) > > * last release is out 2 years ago > ( http://sourceforge.net/mailarchive/forum.php? > thread_name=20060723161651.GA2076%40cthulhu.gerg.ca&forum_name=optik- > users ) > > * Greg Ward mail ( gward at python.net ) and is home page (http:// > www.gerg.ca/) are away > > * I'm interested by this feature > http://sourceforge.net/tracker/index.php? > func=detail&aid=1819510&group_id=38019&atid=421097 > but it hasn't comment since one year ago. > > Otherwise optparse is a great module ! Well, the module itself certainly isn't dead since it's distributed with Python. What might be dead is the standalone version, Optik. However, we have so far delegated feature requests to Greg and expected them to be included in Optik first before being merged to the core. If Optik is no longer maintained, we can do this directly in the core, with Python's release cycle. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ziade.tarek at gmail.com Sat Aug 30 23:15:13 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 30 Aug 2008 23:15:13 +0200 Subject: [Python-Dev] [optparse] I wonder if optparse is dead ? because... In-Reply-To: References: Message-ID: <94bdd2610808301415laaf9e01rc0a8637f90b17f3d@mail.gmail.com> On Sat, Aug 30, 2008 at 10:30 PM, KLEIN St?phane wrote: > Hello, > > I wonder if optparse is dead ? because : > > * svn access is down (svn co svn://starship.python.net/optik/trunk optik) > ( http://sourceforge.net/mailarchive/forum.php? > thread_name=466F541C.6010804%40users.sourceforge.net&forum_name=optik- > users ) > > * last release is out 2 years ago > ( http://sourceforge.net/mailarchive/forum.php? > thread_name=20060723161651.GA2076%40cthulhu.gerg.ca&forum_name=optik- > users ) > Hello, It has been integrated in Python since then, afaik http://svn.python.org/projects/python/trunk/Lib/optparse.py > > * Greg Ward mail ( gward at python.net ) and is home page (http:// > www.gerg.ca/) are away > > * I'm interested by this feature > http://sourceforge.net/tracker/index.php? > func=detail&aid=1819510&group_id=38019&atid=421097 > but it hasn't comment since one year ago. > > Otherwise optparse is a great module ! > > Thanks for your information, > Stephane > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ondrej at certik.cz Sat Aug 30 23:24:45 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Sat, 30 Aug 2008 23:24:45 +0200 Subject: [Python-Dev] Documentation Error for __hash__ In-Reply-To: References: <48B6FD04.6080008@voidspace.org.uk> <48B7C718.7020802@egenix.com> <48B7DBD8.8070205@voidspace.org.uk> <48B81E24.2070906@gmail.com> <85b5c3130808301141t4f2b24d8ybd544de0f26d7f4c@mail.gmail.com> <9AE6735E-45F6-4A66-B749-1DB7A3D39EA3@gmail.com> <85b5c3130808301235t7ce79662t767c2831e736d808@mail.gmail.com> Message-ID: <85b5c3130808301424j5da8308fg212e2a06b84d5981@mail.gmail.com> Hi Georg! On Sat, Aug 30, 2008 at 11:09 PM, Georg Brandl wrote: > Ondrej Certik schrieb: >>>> Ondrej >>> >>> Ondrej, a patch that improves the official docs would be welcome and still >>> potentially make 2.6/3.0 >> >> That'd be awesome. I need to finish my thesis in the next couple days, >> so I'd welcome if anyone could just take it and put usefult things in. >> I could get to do it the next week the earliest. > > Doc patches are always accepted, and I intend to update the online 2.6 and > 3.0 docs continually even when the finals are released, so there is no > particular deadline you have to make. This is awesome, I didn't know you take care of the official docs too. I'll send the doc patches when I get to it, unless someone else does it before me. Ondrej From stephane at harobed.org Sat Aug 30 23:29:19 2008 From: stephane at harobed.org (=?iso-8859-1?q?KLEIN_St=E9phane?=) Date: Sat, 30 Aug 2008 21:29:19 +0000 (UTC) Subject: [Python-Dev] [optparse] I wonder if optparse is dead ? because... References: <94bdd2610808301415laaf9e01rc0a8637f90b17f3d@mail.gmail.com> Message-ID: Le Sat, 30 Aug 2008 23:15:13 +0200, Tarek Ziad? a ?crit?: > On Sat, Aug 30, 2008 at 10:30 PM, KLEIN St?phane > wrote: > >> Hello, >> >> I wonder if optparse is dead ? because : >> >> * svn access is down (svn co svn://starship.python.net/optik/trunk >> optik) ( http://sourceforge.net/mailarchive/forum.php? >> thread_name=466F541C.6010804%40users.sourceforge.net&forum_name=optik- >> users ) >> >> * last release is out 2 years ago >> ( http://sourceforge.net/mailarchive/forum.php? >> thread_name=20060723161651.GA2076%40cthulhu.gerg.ca&forum_name=optik- >> users ) >> >> > Hello, > > It has been integrated in Python since then, afaik > > http://svn.python.org/projects/python/trunk/Lib/optparse.py > Yes, I know but there are this lines in optparse.py source code : """ By Greg Ward Originally distributed as Optik; see http://optik.sourceforge.net/ . If you have problems with this module, please do not file bugs, patches, or feature requests with Python; instead, use Optik's SourceForge project page: http://sourceforge.net/projects/optik For support, use the optik-users at lists.sourceforge.net mailing list (http://lists.sourceforge.net/lists/listinfo/optik-users). """ then if I read this, I need to send my request feature to Optik web site but it appear asleep. Regards, Stephane From michele.simionato at gmail.com Sun Aug 31 09:12:27 2008 From: michele.simionato at gmail.com (Michele Simionato) Date: Sun, 31 Aug 2008 09:12:27 +0200 Subject: [Python-Dev] Things to Know About Super In-Reply-To: <4edc17eb0808292116q39763348g162b8f90787e8e6c@mail.gmail.com> References: <54eae8eb-6cb1-4088-a277-7fad9b4225fd@m45g2000hsb.googlegroups.com> <4edc17eb0808280850xa842656l9fea70acc0717bce@mail.gmail.com> <20080828185312.943163A408D@sparrow.telecommunity.com> <4edc17eb0808282107q65969a63qb6b8745461b57b44@mail.gmail.com> <20080829042146.6DC123A408D@sparrow.telecommunity.com> <4edc17eb0808282144l6d81979bic95ae8c11d69cd37@mail.gmail.com> <48B82098.8090106@gmail.com> <4edc17eb0808291046j62d6ff9dh612e7d0698bbfba6@mail.gmail.com> <5A0A8180-273D-4739-869C-8F5A35184FC5@pandora.com> <4edc17eb0808292116q39763348g162b8f90787e8e6c@mail.gmail.com> Message-ID: <4edc17eb0808310012w4cd0f340u70f3664aa938be6d@mail.gmail.com> On Sat, Aug 30, 2008 at 6:16 AM, Michele Simionato > I wrote a trait library which I plan to release soon or later. Ok, just for the people here that cannot wait I have prepared a pre-alpha snapshot and uploaded it to my site: http://www.phyast.pitt.edu/~micheles/python/strait.html At some moment I want to release it officially, but as of now I do not feel I have nailed out all the details and there may be difficulties I have not foreseen. If it is so, I am sure Phillip will find out all the loopholes ;) Nevertheless, I feel that I have covered out a lot of use cases that I cared to cover, and that there is something good in there, so have fun with this foolish attempt of putting multiple inheritance straight! ;-) Michele Simionato From jcea at jcea.es Sun Aug 31 16:43:34 2008 From: jcea at jcea.es (Jesus Cea) Date: Sun, 31 Aug 2008 16:43:34 +0200 Subject: [Python-Dev] Email update Message-ID: <48BAAE16.9010200@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm getting some private email about python issues in "jcea at argo.es". This email address works for now, but it is "deprecated". My current email is "jcea at jcea.es". Please, update your addressbooks, and any list/contact info out there. I already updated my mailman subscriptions months ago. My old email can vanish without warning. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSLquCZlgi5GaxT1NAQKQRwP/fI7xR3O87aPRSunGIibVqLEAwiSczc8N diWZ2QxSmr2GnL7tDWXU9B/ROxuFL3Rhti21T/BfLDDPn5pi9YmwVkKji4Jt+v1L UROgaJ5nnv1CBiLvKPDSkIoHoFKa4Gx1hIWihEy6Oed8mA8qKP+rm0tX0NfWJaRo ofEja3FooDs= =wYIM -----END PGP SIGNATURE-----
RetroSearch is an open source project built by @garambo
| Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4