Showing content from https://mail.python.org/pipermail/python-dev/2009-January.txt below:
References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> Message-ID: <930F189C8A437347B80DF2C156F7EC7F04D7C90D51@exchis.ccp.ad.local> Are you all certain that this mapping from a generator expression to a foor loop isn't just a happy coincidence? After all, the generator statement is just a generalization of the list comprehension and that doesn't map quite so directly. I have always taken both expressions at face value, and not tried to map them into something else. Why should you, since they are designed to match the grammar of the english language and make perfect sense if you read them as you would a herring recipe. The suggested "while" keyword expands on this easy to understand theme, and the fact that it doesn't fit a mapping that was probably never intentional shouldn't detract from that. K -----Original Message----- From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Terry Reedy Sent: 19. jan?ar 2009 17:52 To: python-dev at python.org Subject: Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions The other, posted by Steven Bethard, is that it fundamentally breaks the current semantics of abbreviating (except for iteration variable scoping) an 'equivalent' for loop. This should have been listed in the PEP under Objections (or whatever the section. I did not bother to second his objection there but will now. -1 From skip at pobox.com Mon Jan 19 22:52:04 2009 From: skip at pobox.com (skip at pobox.com) Date: Mon, 19 Jan 2009 15:52:04 -0600 Subject: [Python-Dev] Hmmm... __PyObject_NextNotImplemented when tracing lines In-Reply-To: References: <20090119165137.0991CD33C78@montanaro.dyndns.org> Message-ID: <18804.62980.134343.770744@montanaro.dyndns.org> Alexander> I cannot reproduce this on my Mac. It looks like you may Alexander> have an out of date python.exe in your sandbox. Please check Alexander> that Alexander> $ nm python.exe | grep PyObject_NextNotImplemented Alexander> 00052940 T __PyObject_NextNotImplemented Alexander> has a "T" in the second column. Alexander> Note that this symbol seem to have been added recently: ... I see what the problem is now. I configured with --enable-shared. Even though my sandbox was up-to-date, my python.exe was current and my libpython2.7.dylib held the symbol in question the *installed* version of libpython2.7.dylib didn't have the symbol: % nm python.exe | grep PyObject_NextNotImplemented % nm libpython2.7.dylib | egrep PyObject_NextNotImplemented 00054200 T __PyObject_NextNotImplemented % nm /Users/skip/local/lib/libpython2.7.dylib | egrep PyObject_NextNotImplemented % I've been bitten by this before. When building with --enable-shared the executable appears to have the installed shared library location bound into it: % otool -L python.exepython.exe: /Users/skip/local/lib/libpython2.7.dylib (compatibility version 2.7.0, current version 2.7.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.3) Is this something that we can fix/work around? It seems to me like a wart, probably platform-dependent. Alexander> I am puzzled, however, that you don't see problems in a Alexander> non-coverage run. Yeah, that mystifies me as well, especially given the above output from nm and otool. Skip From LambertDW at Corning.com Mon Jan 19 20:00:02 2009 From: LambertDW at Corning.com (Lambert, David W (S&T)) Date: Mon, 19 Jan 2009 14:00:02 -0500 Subject: [Python-Dev] Add a "while" clause to generator expressions References: Message-ID: <84B204FFB016BA4984227335D8257FBA2736E7@CVCV0XI05.na.corning.com> The proposal is similar to the c do statement do statement while (expression); which for whatever reason (infrequency?) like the switch statement have rightly not been adopted into python. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/ms-tnef Size: 2686 bytes Desc: not available URL: From tjreedy at udel.edu Tue Jan 20 00:52:07 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 19 Jan 2009 18:52:07 -0500 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F04D7C90D51@exchis.ccp.ad.local> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <930F189C8A437347B80DF2C156F7EC7F04D7C90D51@exchis.ccp.ad.local> Message-ID: Kristj?n Valur J?nsson wrote: > Are you all certain that this mapping from a generator expression to > a foor loop isn't just a happy coincidence? Yes, The manual *defines* the meaning of a comprehension in terms of the corresponding nested statements. "The comprehension consists of a single expression followed by at least one for clause and zero or more for or if clauses. In this case, the elements of the new container are those that would be produced by considering each of the for or if clauses a block, nesting from left to right, and evaluating the expression to produce an element each time the innermost block is reached." (3.0 doc) It was originally defined as exactly equivalent (with empty list initialization added). The only intended change of the slightly softer newer version is that the target name bindings do not escape the scope of the comprehension. The proposed change would BREAK the definition and intent of what a comprehension means. > After all, the generator statement is just a generalization > of the list comprehension I would call it a variation. Syntactically, a generator statement is a comprehension with parentheses instead of square brackets or curly braces. tjr From python at rcn.com Tue Jan 20 00:56:22 2009 From: python at rcn.com (Raymond Hettinger) Date: Mon, 19 Jan 2009 15:56:22 -0800 Subject: [Python-Dev] Copyright notices in modules Message-ID: <94F1BCD4C7B448C8AE5C8C1FA34DD661@RaymondLaptop1> Why does numbers.py say: # Copyright 2007 Google, Inc. All Rights Reserved. # Licensed to PSF under a Contributor Agreement. Weren't there multiple contributors including non-google people? Does Google want to be associated with code that was submitted with no tests? Do we want this sort of stuff in the code? If someone signs a contributor agreement, can we forgo the external copyright comments? Do we want to make a practice of every contributor commenting in the name of the company they were working for at the time (if so, I would have to add the comment to a lot of modules)? Does the copyright concept even apply to an abstract base class (I thought APIs were not subject to copyright, just like database layouts and language definitions)? Raymond From ironfroggy at gmail.com Tue Jan 20 01:42:12 2009 From: ironfroggy at gmail.com (Calvin Spealman) Date: Mon, 19 Jan 2009 19:42:12 -0500 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <76fd5acf0901190847y1c28b659o3e3fb7e4549cabcc@mail.gmail.com> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <76fd5acf0901190829r3c0133e6i49154bae5163483f@mail.gmail.com> <5d1a32000901190841k18450400nbd61fdf878522e59@mail.gmail.com> <76fd5acf0901190847y1c28b659o3e3fb7e4549cabcc@mail.gmail.com> Message-ID: <76fd5acf0901191642q16d6b37cy25a1247aa73e7389@mail.gmail.com> OK, I still don't like the general idea, but I have a suggestion for what I think is a more favorable syntax, anyway. Basically, adding the allowance of an 'else break' when there is an if clause in a generator expression or list comprehension. I still dont think it should be done, but it is a more consistent syntax. It has the obvious problem of looking like it might allow an alternative expression, like the if-expression. prime = (p for p in sieve() if p < 1000 else break) On Mon, Jan 19, 2009 at 11:47 AM, Calvin Spealman wrote: > On Mon, Jan 19, 2009 at 11:41 AM, Gerald Britton > wrote: >> Thanks Calvin, >> >> Could you please expand on your thoughts about possible confusion? >> That is, how do you see a programmer becoming confused if this option >> were added to the syntax. > > I think that the difference between these two lines is too easy to > miss among other code: > > prime = (p for p in sieve() while p < 1000) > > and: > > prime = (p for p in sieve() if p < 1000) > > The very distinctly different behavior is two important to look so > similar at a glance. And, again, I just don't see takewhile being used > all that often. Not nearly often enough to warrant replacing it with a > special syntax! Only 178 results from Google CodeSearch, while chain, > groupby, and repeat get 4000, 3000, and 1000 respectively. Should > those be given their own syntax? > >> On Mon, Jan 19, 2009 at 11:29 AM, Calvin Spealman wrote: >>> I am really unconvinced of the utility of this proposal and quite >>> convinced of the confusing factor it may well add to the current >>> syntax. I would like to see more applicable examples. It would replace >>> uses of takewhile, but that isn't a really often used function. So, is >>> there any evidence to support that making this a new syntax would find >>> so many more uses of the construct to be worth it? I believe not. >>> >>> On Mon, Jan 19, 2009 at 10:10 AM, Gerald Britton >>> wrote: >>>> Please find below PEP 3142: Add a "while" clause to generator >>>> expressions. I'm looking for feedback and discussion. >>>> >>>> >>>> PEP: 3142 >>>> Title: Add a "while" clause to generator expressions >>>> Version: $Revision: 68715 $ >>>> Last-Modified: $Date: 2009-01-18 11:28:20 +0100 (So, 18. Jan 2009) $ >>>> Author: Gerald Britton >>>> Status: Draft >>>> Type: Standards Track >>>> Content-Type: text/plain >>>> Created: 12-Jan-2009 >>>> Python-Version: 3.0 >>>> Post-History: >>>> >>>> >>>> Abstract >>>> >>>> This PEP proposes an enhancement to generator expressions, adding a >>>> "while" clause to complement the existing "if" clause. >>>> >>>> >>>> Rationale >>>> >>>> A generator expression (PEP 289 [1]) is a concise method to serve >>>> dynamically-generated objects to list comprehensions (PEP 202 [2]). >>>> Current generator expressions allow for an "if" clause to filter >>>> the objects that are returned to those meeting some set of >>>> criteria. However, since the "if" clause is evaluated for every >>>> object that may be returned, in some cases it is possible that all >>>> objects would be rejected after a certain point. For example: >>>> >>>> g = (n for n in range(100) if n*n < 50) >>>> >>>> which is equivalent to the using a generator function >>>> (PEP 255 [3]): >>>> >>>> def __gen(exp): >>>> for n in exp: >>>> if n*n < 50: >>>> yield n >>>> g = __gen(iter(range(10))) >>>> >>>> would yield 0, 1, 2, 3, 4, 5, 6 and 7, but would also consider >>>> the numbers from 8 to 99 and reject them all since n*n >= 50 for >>>> numbers in that range. Allowing for a "while" clause would allow >>>> the redundant tests to be short-circuited: >>>> >>>> g = (n for n in range(100) while n*n < 50) >>>> >>>> would also yield 0, 1, 2, 3, 4, 5, 6 and 7, but would stop at 8 >>>> since the condition (n*n < 50) is no longer true. This would be >>>> equivalent to the generator function: >>>> >>>> def __gen(exp): >>>> for n in exp: >>>> if n*n < 50: >>>> yield n >>>> else: >>>> break >>>> g = __gen(iter(range(100))) >>>> >>>> Currently, in order to achieve the same result, one would need to >>>> either write a generator function such as the one above or use the >>>> takewhile function from itertools: >>>> >>>> from itertools import takewhile >>>> g = takewhile(lambda n: n*n < 50, range(100)) >>>> >>>> The takewhile code achieves the same result as the proposed syntax, >>>> albeit in a longer (some would say "less-elegant") fashion. Also, >>>> the takewhile version requires an extra function call (the lambda >>>> in the example above) with the associated performance penalty. >>>> A simple test shows that: >>>> >>>> for n in (n for n in range(100) if 1): pass >>>> >>>> performs about 10% better than: >>>> >>>> for n in takewhile(lambda n: 1, range(100)): pass >>>> >>>> though they achieve similar results. (The first example uses a >>>> generator; takewhile is an iterator). If similarly implemented, >>>> a "while" clause should perform about the same as the "if" clause >>>> does today. >>>> >>>> The reader may ask if the "if" and "while" clauses should be >>>> mutually exclusive. There are good examples that show that there >>>> are times when both may be used to good advantage. For example: >>>> >>>> p = (p for p in primes() if p > 100 while p < 1000) >>>> >>>> should return prime numbers found between 100 and 1000, assuming >>>> I have a primes() generator that yields prime numbers. Of course, this >>>> could also be achieved like this: >>>> >>>> p = (p for p in (p for p in primes() if p > 100) while p < 1000) >>>> >>>> which is syntactically simpler. Some may also ask if it is possible >>>> to cover dropwhile() functionality in a similar way. I initially thought >>>> of: >>>> >>>> p = (p for p in primes() not while p < 100) >>>> >>>> but I am not sure that I like it since it uses "not" in a non-pythonic >>>> fashion, I think. >>>> >>>> Adding a "while" clause to generator expressions maintains the >>>> compact form while adding a useful facility for short-circuiting >>>> the expression. >>>> >>>> Implementation: >>>> >>>> I am willing to assist in the implementation of this feature, although I have >>>> not contributed to Python thus far and would definitely need mentoring. (At >>>> this point I am not quite sure where to begin.) Presently though, I would >>>> find it challenging to fit this work into my existing workload. >>>> >>>> >>>> Acknowledgements >>>> >>>> Raymond Hettinger first proposed the concept of generator >>>> expressions in January 2002. >>>> >>>> >>>> References >>>> >>>> [1] PEP 289: Generator Expressions >>>> http://www.python.org/dev/peps/pep-0289/ >>>> >>>> [2] PEP 202: List Comprehensions >>>> http://www.python.org/dev/peps/pep-0202/ >>>> >>>> [3] PEP 255: Simple Generators >>>> http://www.python.org/dev/peps/pep-0255/ >>>> >>>> >>>> Copyright >>>> >>>> This document has been placed in the public domain. >>>> >>>> >>>> Local Variables: >>>> mode: indented-text >>>> indent-tabs-mode: nil >>>> sentence-end-double-space: t >>>> fill-column: 70 >>>> coding: utf-8 >>>> End: >>>> >>>> _______________________________________________ >>>> Python-Dev mailing list >>>> Python-Dev at python.org >>>> http://mail.python.org/mailman/listinfo/python-dev >>>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com >>>> >>>> >>> >>> >>> >>> -- >>> Read my blog! I depend on your acceptance of my opinion! I am interesting! >>> http://techblog.ironfroggy.com/ >>> Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy >>> >> > > > > -- > Read my blog! I depend on your acceptance of my opinion! I am interesting! > http://techblog.ironfroggy.com/ > Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy > -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy From brett at python.org Tue Jan 20 03:24:42 2009 From: brett at python.org (Brett Cannon) Date: Mon, 19 Jan 2009 18:24:42 -0800 Subject: [Python-Dev] Questions/comments on documentation formatting Message-ID: I have been writing up the initial docs for importlib and four things struck me: 1. Why is three space indents the preferred indentation level? 2. Should we start using function annotations? 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [, c=None]])``) really necessary when default argument values are present? And do we really need to nest the brackets when it is obvious that having on optional argument means the rest are optional as well? 4. The var directive is not working even though the docs list it as a valid directive; so is it still valid and something is broken, or the docs need to be updated? -Brett From stephen at xemacs.org Tue Jan 20 03:51:41 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 20 Jan 2009 11:51:41 +0900 Subject: [Python-Dev] Copyright notices in modules In-Reply-To: <94F1BCD4C7B448C8AE5C8C1FA34DD661@RaymondLaptop1> References: <94F1BCD4C7B448C8AE5C8C1FA34DD661@RaymondLaptop1> Message-ID: <877i4qzo3m.fsf@xemacs.org> Raymond Hettinger writes: > Does the copyright concept even apply to an abstract base class (I > thought APIs were not subject to copyright, just like database > layouts and language definitions)? Yes, it does, although a public API per se is not subject to copyright, because there's only one way to do it. Any comments, internal implementation (eg, names of persistent state variables, members, and constants, and the very existence of those identifiers), and tests are subject to copyright because they are expressive. I believe that a private API also can be subject to copyright, though I'm not as sure of that. The point being that there are good APIs and bad APIs that expose the same functionality, so that API design is expressive. However, if you expose the API and license people to use it, that license makes it impossible to restrict them from using it thereafter. Caveat: IANAL, and this is under U.S. law. From cs at zip.com.au Tue Jan 20 03:54:49 2009 From: cs at zip.com.au (Cameron Simpson) Date: Tue, 20 Jan 2009 13:54:49 +1100 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <76fd5acf0901191642q16d6b37cy25a1247aa73e7389@mail.gmail.com> Message-ID: <20090120025449.GA3484@cskk.homeip.net> On 19Jan2009 19:42, Calvin Spealman wrote: | OK, I still don't like the general idea, but I have a suggestion for | what I think is a more favorable syntax, anyway. Basically, adding the | allowance of an 'else break' when there is an if clause in a generator | expression or list comprehension. I still dont think it should be | done, but it is a more consistent syntax. It has the obvious problem | of looking like it might allow an alternative expression, like the | if-expression. | | prime = (p for p in sieve() if p < 1000 else break) If I'm reading your suggestion correctly it changes feel of the "if" part quite fandamentally. A bare "if" acts as a filter, yielding only the items matching the condition. With an "else break" it yields only the items matching the condition up to the first which fails. For a range like "p < 1000" this isn't so different, but if the condition doesn't apply to just the leading items then this form becomes almost useless. Consider "p % 3 == 0". Normally that would yield every third item. With "else break" it would yield just one item. Any non-continuous filter has the same issue. In my opinion this makes the construct far less applicable than it would appear at first glance. So -1 from me. I saw someone else mention takewhile(), and that's my preferred way of doing the original suggestion ("[ ... while some-constraint ]"), and thus I'm -1 on the original suggestion too (also for the if/while confusion mentioned in the thread). Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Drive Agressively Rash Magnificently - Nankai Leathers From benjamin at python.org Tue Jan 20 04:01:08 2009 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 19 Jan 2009 21:01:08 -0600 Subject: [Python-Dev] Questions/comments on documentation formatting In-Reply-To: References: Message-ID: <1afaf6160901191901g1d634947h8d53862282c18b0e@mail.gmail.com> On Mon, Jan 19, 2009 at 8:24 PM, Brett Cannon wrote: > I have been writing up the initial docs for importlib and four things struck me: > > 1. Why is three space indents the preferred indentation level? Because it matches nicely up with the length of directives: .. somedirective:: blah ^^^ > > 2. Should we start using function annotations? No, I think that information is better stored in the function description. > > 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [, > c=None]])``) really necessary when default argument values are > present? And do we really need to nest the brackets when it is obvious > that having on optional argument means the rest are optional as well? Actually, the defaults are usually documented in the description not the signature. > > 4. The var directive is not working even though the docs list it as a > valid directive; so is it still valid and something is broken, or the > docs need to be updated? The docs should be updated. "data" is the one to use now. -- Regards, Benjamin From scott+python-dev at scottdial.com Tue Jan 20 04:02:04 2009 From: scott+python-dev at scottdial.com (Scott Dial) Date: Mon, 19 Jan 2009 22:02:04 -0500 Subject: [Python-Dev] Questions/comments on documentation formatting In-Reply-To: References: Message-ID: <49753EAC.6030901@scottdial.com> Brett Cannon wrote: > 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [, > c=None]])``) really necessary when default argument values are > present? And do we really need to nest the brackets when it is obvious > that having on optional argument means the rest are optional as well? I can't think of an example off the top of my head, but I'm certain the point of nesting the brackets is to delimit the optional arguments into groups. Documenting your fxn() examples as "fxn(a [, b=None, c=None])" would imply that if you provide 'b' then you must provide 'c', or if we abandon nested brackets, it's ambiguous as to the requirements. Imagine seeing "foo(a [, b=None, c=None [, d=None]])" and I think the rationale for such notation becomes clear. -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From brett at python.org Tue Jan 20 04:11:31 2009 From: brett at python.org (Brett Cannon) Date: Mon, 19 Jan 2009 19:11:31 -0800 Subject: [Python-Dev] Questions/comments on documentation formatting In-Reply-To: <1afaf6160901191901g1d634947h8d53862282c18b0e@mail.gmail.com> References: <1afaf6160901191901g1d634947h8d53862282c18b0e@mail.gmail.com> Message-ID: On Mon, Jan 19, 2009 at 19:01, Benjamin Peterson wrote: > On Mon, Jan 19, 2009 at 8:24 PM, Brett Cannon wrote: >> I have been writing up the initial docs for importlib and four things struck me: >> >> 1. Why is three space indents the preferred indentation level? > > Because it matches nicely up with the length of directives: > > .. somedirective:: blah > ^^^ > >> >> 2. Should we start using function annotations? > > No, I think that information is better stored in the function description. > Why? Putting it in the signature makes it very succinct and a simple glance at the doc to see what type/ABC is expected. >> >> 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [, >> c=None]])``) really necessary when default argument values are >> present? And do we really need to nest the brackets when it is obvious >> that having on optional argument means the rest are optional as well? > > Actually, the defaults are usually documented in the description not > the signature. > OK, but that doesn't make it optimal. And that still doesn't answer my question of whether all of those nested brackets are truly necessary. >> >> 4. The var directive is not working even though the docs list it as a >> valid directive; so is it still valid and something is broken, or the >> docs need to be updated? > > The docs should be updated. "data" is the one to use now. So the 'data' directive turns into any variable, not just a module variables? -Brett From benjamin at python.org Tue Jan 20 04:19:17 2009 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 19 Jan 2009 21:19:17 -0600 Subject: [Python-Dev] Questions/comments on documentation formatting In-Reply-To: References: <1afaf6160901191901g1d634947h8d53862282c18b0e@mail.gmail.com> Message-ID: <1afaf6160901191919t5d80dbeete4ee13b2c82772f2@mail.gmail.com> On Mon, Jan 19, 2009 at 9:11 PM, Brett Cannon wrote: > On Mon, Jan 19, 2009 at 19:01, Benjamin Peterson wrote: >> On Mon, Jan 19, 2009 at 8:24 PM, Brett Cannon wrote: >>> >>> 2. Should we start using function annotations? >> >> No, I think that information is better stored in the function description. >> > > Why? Putting it in the signature makes it very succinct and a simple > glance at the doc to see what type/ABC is expected. Well, I guess it's just not been explored. Feel free to try it out if you wish, though. > >>> >>> 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [, >>> c=None]])``) really necessary when default argument values are >>> present? And do we really need to nest the brackets when it is obvious >>> that having on optional argument means the rest are optional as well? >> >> Actually, the defaults are usually documented in the description not >> the signature. >> > > OK, but that doesn't make it optimal. And that still doesn't answer my > question of whether all of those nested brackets are truly necessary. All I can say is that it is the style/convention. > >>> >>> 4. The var directive is not working even though the docs list it as a >>> valid directive; so is it still valid and something is broken, or the >>> docs need to be updated? >> >> The docs should be updated. "data" is the one to use now. > > So the 'data' directive turns into any variable, not just a module variables? "data" is for module level objects. If you're documenting properties or attributes in classes, use "attribute". -- Regards, Benjamin From python at rcn.com Tue Jan 20 04:23:22 2009 From: python at rcn.com (Raymond Hettinger) Date: Mon, 19 Jan 2009 19:23:22 -0800 Subject: [Python-Dev] Questions/comments on documentation formatting References: Message-ID: <096C7BCCCDD040149931A722E080E305@RaymondLaptop1> From: "Brett Cannon" > 1. Why is three space indents the preferred indentation level? I've also wondered about this. It is somewhat incovenient when bringing in code samples from files with four space indents. Raymond From benjamin at python.org Tue Jan 20 04:30:02 2009 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 19 Jan 2009 21:30:02 -0600 Subject: [Python-Dev] Questions/comments on documentation formatting In-Reply-To: <096C7BCCCDD040149931A722E080E305@RaymondLaptop1> References: <096C7BCCCDD040149931A722E080E305@RaymondLaptop1> Message-ID: <1afaf6160901191930w41f9e88i4ba6be9c98278618@mail.gmail.com> On Mon, Jan 19, 2009 at 9:23 PM, Raymond Hettinger wrote: > From: "Brett Cannon" >> >> 1. Why is three space indents the preferred indentation level? > > I've also wondered about this. It is somewhat incovenient > when bringing in code samples from files with four space indents. It's just reST indentation that is 3 spaces. Code examples in the reST can be 4 spaces. -- Regards, Benjamin From ben+python at benfinney.id.au Tue Jan 20 04:49:54 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 20 Jan 2009 14:49:54 +1100 Subject: [Python-Dev] Questions/comments on documentation formatting References: <096C7BCCCDD040149931A722E080E305@RaymondLaptop1> <1afaf6160901191930w41f9e88i4ba6be9c98278618@mail.gmail.com> Message-ID: <87ocy24owt.fsf@benfinney.id.au> "Benjamin Peterson" writes: > On Mon, Jan 19, 2009 at 9:23 PM, Raymond Hettinger wrote: > > From: "Brett Cannon" > >> > >> 1. Why is three space indents the preferred indentation level? > > > > I've also wondered about this. It is somewhat incovenient > > when bringing in code samples from files with four space indents. > > It's just reST indentation that is 3 spaces. It doesn't have to be. When writing reST, I always make directives so they will line up nicely at 4-space indents: ===== Normal paragraph .. Comment .. foo:: bar ===== -- \ ?I was the kid next door's imaginary friend.? ?Emo Philips | `\ | _o__) | Ben Finney From python at rcn.com Tue Jan 20 04:50:25 2009 From: python at rcn.com (Raymond Hettinger) Date: Mon, 19 Jan 2009 19:50:25 -0800 Subject: [Python-Dev] Questions/comments on documentation formatting References: <096C7BCCCDD040149931A722E080E305@RaymondLaptop1> <1afaf6160901191930w41f9e88i4ba6be9c98278618@mail.gmail.com> Message-ID: <25D181D6496549F59FBADF8445F08C4E@RaymondLaptop1> I have another question about doc formatting. What controls whether section headers get urls with a custom named jump target instead of a default name like "id1"? In particular, look at the urls for: http://docs.python.org/dev/library/collections.html#id1 versus http://docs.python.org/dev/library/collections.html#abcs-abstract-base-classes I would like all of the targets to have meaningful names. Raymond From ben+python at benfinney.id.au Tue Jan 20 05:06:51 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 20 Jan 2009 15:06:51 +1100 Subject: [Python-Dev] Questions/comments on documentation formatting References: <096C7BCCCDD040149931A722E080E305@RaymondLaptop1> <1afaf6160901191930w41f9e88i4ba6be9c98278618@mail.gmail.com> <25D181D6496549F59FBADF8445F08C4E@RaymondLaptop1> Message-ID: <87k58q4o4k.fsf@benfinney.id.au> "Raymond Hettinger" writes: > What controls whether section headers get urls with a custom named > jump target instead of a default name like "id1"? > > In particular, look at the urls for: > http://docs.python.org/dev/library/collections.html#id1 versus Hmm. Immediately preceding the element, there is also an empty element with a meaningful ID, "counter-objects", that can be used to get to the same position in the document: http://docs.python.org/dev/library/collections.html#counter-objects However, the problem is that URL isn't used for the ?Permalink to this headline? link. Perhaps a bug report to the Docutils folks is in order. -- \ ?Nature is trying very hard to make us succeed, but nature does | `\ not depend on us. We are not the only experiment.? ?Richard | _o__) Buckminster Fuller, 1978-04-30 | Ben Finney From brett at python.org Tue Jan 20 06:56:28 2009 From: brett at python.org (Brett Cannon) Date: Mon, 19 Jan 2009 21:56:28 -0800 Subject: [Python-Dev] Questions/comments on documentation formatting In-Reply-To: <1afaf6160901191919t5d80dbeete4ee13b2c82772f2@mail.gmail.com> References: <1afaf6160901191901g1d634947h8d53862282c18b0e@mail.gmail.com> <1afaf6160901191919t5d80dbeete4ee13b2c82772f2@mail.gmail.com> Message-ID: On Mon, Jan 19, 2009 at 19:19, Benjamin Peterson wrote: > On Mon, Jan 19, 2009 at 9:11 PM, Brett Cannon wrote: >> On Mon, Jan 19, 2009 at 19:01, Benjamin Peterson wrote: >>> On Mon, Jan 19, 2009 at 8:24 PM, Brett Cannon wrote: >>>> >>>> 2. Should we start using function annotations? >>> >>> No, I think that information is better stored in the function description. >>> >> >> Why? Putting it in the signature makes it very succinct and a simple >> glance at the doc to see what type/ABC is expected. > > Well, I guess it's just not been explored. Feel free to try it out if > you wish, though. > I just might. >> >>>> >>>> 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [, >>>> c=None]])``) really necessary when default argument values are >>>> present? And do we really need to nest the brackets when it is obvious >>>> that having on optional argument means the rest are optional as well? >>> >>> Actually, the defaults are usually documented in the description not >>> the signature. >>> >> >> OK, but that doesn't make it optimal. And that still doesn't answer my >> question of whether all of those nested brackets are truly necessary. > > All I can say is that it is the style/convention. > Right, which is why I am questioning it. =) >> >>>> >>>> 4. The var directive is not working even though the docs list it as a >>>> valid directive; so is it still valid and something is broken, or the >>>> docs need to be updated? >>> >>> The docs should be updated. "data" is the one to use now. >> >> So the 'data' directive turns into any variable, not just a module variables? > > "data" is for module level objects. If you're documenting properties > or attributes in classes, use "attribute". Then what are we supposed to use for arguments? Just ``literal``? -Brett From brett at python.org Tue Jan 20 06:58:25 2009 From: brett at python.org (Brett Cannon) Date: Mon, 19 Jan 2009 21:58:25 -0800 Subject: [Python-Dev] Questions/comments on documentation formatting In-Reply-To: <25D181D6496549F59FBADF8445F08C4E@RaymondLaptop1> References: <096C7BCCCDD040149931A722E080E305@RaymondLaptop1> <1afaf6160901191930w41f9e88i4ba6be9c98278618@mail.gmail.com> <25D181D6496549F59FBADF8445F08C4E@RaymondLaptop1> Message-ID: On Mon, Jan 19, 2009 at 19:50, Raymond Hettinger wrote: > I have another question about doc formatting. > > What controls whether section headers get urls with a custom named jump > target instead of a default name like "id1"? > > In particular, look at the urls for: > http://docs.python.org/dev/library/collections.html#id1 versus > > http://docs.python.org/dev/library/collections.html#abcs-abstract-base-classes > I would like all of the targets to have meaningful names. Not sure from a sphinx perspective, but Docutils does this automatically. You can also always specify the anchor point manually:: .. _abcs-abstract-base-classes Abstract Base Classes ------------------------------ to get what you want. -Brett From brett at python.org Tue Jan 20 07:03:15 2009 From: brett at python.org (Brett Cannon) Date: Mon, 19 Jan 2009 22:03:15 -0800 Subject: [Python-Dev] Questions/comments on documentation formatting In-Reply-To: <49753EAC.6030901@scottdial.com> References: <49753EAC.6030901@scottdial.com> Message-ID: On Mon, Jan 19, 2009 at 19:02, Scott Dial wrote: > Brett Cannon wrote: >> 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [, >> c=None]])``) really necessary when default argument values are >> present? And do we really need to nest the brackets when it is obvious >> that having on optional argument means the rest are optional as well? > > I can't think of an example off the top of my head, but I'm certain the > point of nesting the brackets is to delimit the optional arguments into > groups. Documenting your fxn() examples as "fxn(a [, b=None, c=None])" > would imply that if you provide 'b' then you must provide 'c', or if we > abandon nested brackets, it's ambiguous as to the requirements. Imagine > seeing "foo(a [, b=None, c=None [, d=None]])" and I think the rationale > for such notation becomes clear. Well, that is such a rare case that I don't know if it warrants the line noise in the argument declaration. That argument also doesn't make sense in the face of ``fxn(a [, a=None [, b=None]])`` where 'b' almost always has no connection to 'a', but is still supposed to be listed that way because of positional arguments being optional. I understand using them for C functions where there is no such thing as a default argument, but it just doesn't make a ton of sense for Python code. I don't know of anyone who was confused by what help() spit out and not having fancy bracketing. -Brett From python at rcn.com Tue Jan 20 08:50:35 2009 From: python at rcn.com (Raymond Hettinger) Date: Mon, 19 Jan 2009 23:50:35 -0800 Subject: [Python-Dev] Questions/comments on documentation formatting References: <096C7BCCCDD040149931A722E080E305@RaymondLaptop1> <1afaf6160901191930w41f9e88i4ba6be9c98278618@mail.gmail.com> <25D181D6496549F59FBADF8445F08C4E@RaymondLaptop1> Message-ID: <04AD60E3A4594B78B93222A88E86B21A@RaymondLaptop1> >> In particular, look at the urls for: >> http://docs.python.org/dev/library/collections.html#id1 versus >> >> http://docs.python.org/dev/library/collections.html#abcs-abstract-base-classes >> I would like all of the targets to have meaningful names. [Brett] > Not sure from a sphinx perspective, but Docutils does this > automatically. You can also always specify the anchor point manually:: > > .. _abcs-abstract-base-classes > > Abstract Base Classes > ------------------------------ > > to get what you want. Thanks for the note. It pointed me to the real problem which was that manual anchor points can interfere with the automatically generated names if their names are the same. The solution was to *remove* the manually generated anchor points. Raymond From mal at egenix.com Tue Jan 20 10:17:37 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 20 Jan 2009 10:17:37 +0100 Subject: [Python-Dev] Copyright notices in modules In-Reply-To: <94F1BCD4C7B448C8AE5C8C1FA34DD661@RaymondLaptop1> References: <94F1BCD4C7B448C8AE5C8C1FA34DD661@RaymondLaptop1> Message-ID: <497596B1.4060600@egenix.com> On 2009-01-20 00:56, Raymond Hettinger wrote: > Why does numbers.py say: > > # Copyright 2007 Google, Inc. All Rights Reserved. > # Licensed to PSF under a Contributor Agreement. Because that's where that file originated, I guess. This is part of what you have to do for things that are licensed to the PSF under a contributor agreement: http://www.python.org/psf/contrib/contrib-form/ """ Contributor shall identify each Contribution by placing the following notice in its source code adjacent to Contributor's valid copyright notice: "Licensed to PSF under a Contributor Agreement." """ > Weren't there multiple contributors including non-google people? The initial contribution was done by Google (Jeffrey Yasskin AFAIK) and that's where the above lines originated from. > Does Google want to be associated with code that > was submitted with no tests? Only Google can comment on this. > Do we want this sort of stuff in the code? Yes, it is required by the contrib forms. > If someone signs a contributor agreement, can we > forgo the external copyright comments? No. See above. Only the copyright owner can remove such notices. > Do we want to make a practice of every contributor > commenting in the name of the company they were > working for at the time (if so, I would have to add > the comment to a lot of modules)? That depends on the contract a contributor has with the company that funded the work. It's quite common for such contracts to include a clause stating that all IP generated during work time is owned by the employer. > Does the copyright concept even apply to an > abstract base class (I thought APIs were not > subject to copyright, just like database layouts > and language definitions)? It applies to the written program text. You are probably thinking about other IP rights such as patents or designs. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 20 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From fuzzyman at voidspace.org.uk Tue Jan 20 11:02:14 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 20 Jan 2009 10:02:14 +0000 Subject: [Python-Dev] Copyright notices in modules In-Reply-To: <497596B1.4060600@egenix.com> References: <94F1BCD4C7B448C8AE5C8C1FA34DD661@RaymondLaptop1> <497596B1.4060600@egenix.com> Message-ID: <4975A126.6070309@voidspace.org.uk> M.-A. Lemburg wrote: > [snip...] > >> Does the copyright concept even apply to an >> abstract base class (I thought APIs were not >> subject to copyright, just like database layouts >> and language definitions)? >> > > It applies to the written program text. You are probably > thinking about other IP rights such as patents or designs. > > You need to read Van Lindberg's excellent book on intellectual property rights and open source (which is about American law and European law will be different). Mere collections of facts are not copyrightable as they are not creative (the basis of copyright) and this is presumed to apply to parts of software like header files and interface descriptions - which could easily apply to ABCs in Python. I recommend his book by the way - I'm about half way through so far and it is highly readable Michael Foord -- http://www.ironpythoninaction.com/ From fuzzyman at voidspace.org.uk Tue Jan 20 11:06:38 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 20 Jan 2009 10:06:38 +0000 Subject: [Python-Dev] Questions/comments on documentation formatting In-Reply-To: References: <49753EAC.6030901@scottdial.com> Message-ID: <4975A22E.3030402@voidspace.org.uk> Brett Cannon wrote: > On Mon, Jan 19, 2009 at 19:02, Scott Dial > wrote: > >> Brett Cannon wrote: >> >>> 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [, >>> c=None]])``) really necessary when default argument values are >>> present? And do we really need to nest the brackets when it is obvious >>> that having on optional argument means the rest are optional as well? >>> >> I can't think of an example off the top of my head, but I'm certain the >> point of nesting the brackets is to delimit the optional arguments into >> groups. Documenting your fxn() examples as "fxn(a [, b=None, c=None])" >> would imply that if you provide 'b' then you must provide 'c', or if we >> abandon nested brackets, it's ambiguous as to the requirements. Imagine >> seeing "foo(a [, b=None, c=None [, d=None]])" and I think the rationale >> for such notation becomes clear. >> > > Well, that is such a rare case that I don't know if it warrants the > line noise in the argument declaration. That argument also doesn't > make sense in the face of ``fxn(a [, a=None [, b=None]])`` where 'b' > almost always has no connection to 'a', but is still supposed to be > listed that way because of positional arguments being optional. I > understand using them for C functions where there is no such thing as > a default argument, but it just doesn't make a ton of sense for Python > code. I don't know of anyone who was confused by what help() spit out > and not having fancy bracketing. > > I think the square bracketing is ugly and does nothing for clarity or readability. The sooner it can be phased out the better. Function annotations should probably only be used in API descriptions where those annotations actually exist - otherwise when there are real annotations you have a conflict on how to indicate that in the documentation. Michael > -Brett > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ From ncoghlan at gmail.com Tue Jan 20 11:15:49 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 20 Jan 2009 20:15:49 +1000 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F04D7C90D51@exchis.ccp.ad.local> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <930F189C8A437347B80DF2C156F7EC7F04D7C90D51@exchis.ccp.ad.local> Message-ID: <4975A455.6080208@gmail.com> Kristj?n Valur J?nsson wrote: > Are you all certain that this mapping from a generator expression to > a foor loop isn't just a happy coincidence? After all, the generator > statement is just a generalization of the list comprehension and that > doesn't map quite so directly. The mapping of the for and if clauses is identical for both generator expressions and the various flavours of comprehension. It's only the outer wrappings (creating a generator/dict/set/list) and the innermost loop body (yield statement/item assignment/set add/list append) that differ between the constructs. As Terry noted, it's even defined that way in the language reference - the expressions are pure syntactic sugar for the corresponding statements. While this doesn't often matter in practice (since people tend to switch to using the statement based versions rather than writing convoluted multiple clause comprehensions), it's still an important symmetry that matters greatly to Python VM implementers so any proposed changes need to take it into account. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From mal at egenix.com Tue Jan 20 13:28:54 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 20 Jan 2009 13:28:54 +0100 Subject: [Python-Dev] Copyright notices in modules In-Reply-To: <4975A126.6070309@voidspace.org.uk> References: <94F1BCD4C7B448C8AE5C8C1FA34DD661@RaymondLaptop1> <497596B1.4060600@egenix.com> <4975A126.6070309@voidspace.org.uk> Message-ID: <4975C386.10901@egenix.com> On 2009-01-20 11:02, Michael Foord wrote: > M.-A. Lemburg wrote: >> [snip...] >> >>> Does the copyright concept even apply to an >>> abstract base class (I thought APIs were not >>> subject to copyright, just like database layouts >>> and language definitions)? >>> >> >> It applies to the written program text. You are probably >> thinking about other IP rights such as patents or designs. >> >> > > You need to read Van Lindberg's excellent book on intellectual property > rights and open source (which is about American law and European law > will be different). Mere collections of facts are not copyrightable as > they are not creative (the basis of copyright) and this is presumed to > apply to parts of software like header files and interface descriptions > - which could easily apply to ABCs in Python. I doubt that you can make such assumptions in general. It's a case-by-case decision and also one that depends on the copyright law or convention you assume. See e.g. the WIPO copyright treaty: http://www.wipo.int/treaties/en/ip/wct/trtdocs_wo033.html#P56_5626 and the Berne Convention: http://www.wipo.int/treaties/en/ip/berne/trtdocs_wo001.html#P85_10661 and TRIPS: http://www.wto.org/english/docs_e/legal_e/27-trips_04_e.htm#1 That said, for numbers.py there's certainly enough creativity in that file to enjoy copyright protection. > I recommend his book by the way - I'm about half way through so far and > it is highly readable Thanks for the pointer. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 20 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From luke.leighton at googlemail.com Tue Jan 20 14:02:25 2009 From: luke.leighton at googlemail.com (Luke Kenneth Casson Leighton) Date: Tue, 20 Jan 2009 13:02:25 +0000 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now Message-ID: folks, hi, this is a fairly important issue for python development interoperability - martin mentioned that releases of mingw-compiled python, if done with a non-interoperable version of msvcrt, would cause much mayhem. well, compiling python on mingw with msvcr80 _can_ be done; using it can also be a simple matter of creating a python.exe.manifest file, but i can't actually do any testing because it doesn't work under wine. so, pending any further advice and guidance from anyone which allows me to successfully proceed, i'm not going to continue to compile - or release - python2.5 *or* python2.6 builds (when i get round to that) using msvcr80 or msvcr9X. one issue in favour of this decision is that the DLL that's produced by the autoconf build process is "libpython2.5.dll.a" - not "python2.5.dll". it has a different name. it should be abundantly clear to users and developers that "if name equals libpython2.5.dll.a then duh build equals different". additionally, the setup.py distutils all goes swimmingly well and lovely - using libpython2.5.dll.a. the only issue which _is_ going to throw a spanner in the works is that people who download win32-built precompiled c-based modules are going to find that hey, "it don't work!" and the answer will have to be "go get a version of that module, compiled with mingw, not MSVC". of course - if python for win32 ENTIRELY DROPPED msvc as a development platform, and went for an entirely free software development toolchain, then this problem goes away. thoughts, anyone? l. From tlesher at gmail.com Tue Jan 20 14:11:25 2009 From: tlesher at gmail.com (Tim Lesher) Date: Tue, 20 Jan 2009 08:11:25 -0500 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: Message-ID: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> On Tue, Jan 20, 2009 at 08:02, Luke Kenneth Casson Leighton wrote: > of course - if python for win32 ENTIRELY DROPPED msvc as a development > platform, and went for an entirely free software development > toolchain, then this problem goes away. That's a non-starter for anyone who incorporates Python in an existing MSVC-based development environment. When in Rome... -- Tim Lesher From fuzzyman at voidspace.org.uk Tue Jan 20 14:13:16 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 20 Jan 2009 13:13:16 +0000 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> Message-ID: <4975CDEC.8010408@voidspace.org.uk> Tim Lesher wrote: > On Tue, Jan 20, 2009 at 08:02, Luke Kenneth Casson Leighton > wrote: > >> of course - if python for win32 ENTIRELY DROPPED msvc as a development >> platform, and went for an entirely free software development >> toolchain, then this problem goes away. >> > > That's a non-starter for anyone who incorporates Python in an existing > MSVC-based development environment. > > When in Rome... > > There would also be a significant performance cost. The PGO (Profile Guided Optimisation) compilation of Visual Studio is impressive. Michael -- http://www.ironpythoninaction.com/ From lists at cheimes.de Tue Jan 20 14:44:03 2009 From: lists at cheimes.de (Christian Heimes) Date: Tue, 20 Jan 2009 14:44:03 +0100 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: Message-ID: Luke Kenneth Casson Leighton schrieb: > of course - if python for win32 ENTIRELY DROPPED msvc as a development > platform, and went for an entirely free software development > toolchain, then this problem goes away. > > thoughts, anyone? That's not going to happen anytime soon. As long as Microsoft Visual Studio support is feasible, we will stick to VS. WINE support is not a top priority for us. Christian From lkcl at lkcl.net Tue Jan 20 15:18:42 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Tue, 20 Jan 2009 14:18:42 +0000 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> Message-ID: On Tue, Jan 20, 2009 at 1:11 PM, Tim Lesher wrote: > On Tue, Jan 20, 2009 at 08:02, Luke Kenneth Casson Leighton > wrote: >> of course - if python for win32 ENTIRELY DROPPED msvc as a development >> platform, and went for an entirely free software development >> toolchain, then this problem goes away. > > That's a non-starter for anyone who incorporates Python in an existing > MSVC-based development environment. surely incorporating libpython2.5.dll.a or libpython2.6.dll.a, along with the .def and the importlib that's generated with dlldump, unless i'm missing something, would be a simple matter, yes? > When in Rome... yeah they said the same thing about "gas ovens", too. not the nazi gas ovens, the phrase my mother used to say "if someone stuck their head in a gas oven, would you do the same?". > There would also be a significant performance cost. > The PGO (Profile Guided Optimisation) compilation of > Visual Studio is impressive. i'd say "great" - but given a choice of "impressive profile guided optimisation plus a proprietary compiler, proprietary operating system _and_ being forced to purchase a system _capable_ of running said proprietary compiler, said proprietary operating system, _and_ giving up free software principles _and_ having to go through patch-pain, install-pain _and_ being forced to use a GUI-based IDE for compilation" or "free software tools and downloads the use of which means i am beholden to NOONE", it's a simple choice for me to make - maybe not for other people. l. From luke.leighton at googlemail.com Tue Jan 20 15:20:07 2009 From: luke.leighton at googlemail.com (Luke Kenneth Casson Leighton) Date: Tue, 20 Jan 2009 14:20:07 +0000 Subject: [Python-Dev] one last go at msvcr80 / msvcr90 assemblies - mingw build of python Message-ID: could someone kindly send me the assembly files that are created by a proprietary win32 build of python2.5, 2.6 and trunk, the ones used to create the dll _and_ the python.exe many thanks. From gerald.britton at gmail.com Tue Jan 20 15:24:32 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Tue, 20 Jan 2009 09:24:32 -0500 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <20090120141824.GN11140@ruber.office.udmvt.ru> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <20090120141824.GN11140@ruber.office.udmvt.ru> Message-ID: <5d1a32000901200624t243fc943m404a887aef6d0853@mail.gmail.com> hmmm...doesn't: if n*n < 50 or raise StopIteration() really mean, "Return an integer in the range 0-99 if n-squared is less than fifty or the statement 'raise StopIteration()' returns True" ? I'm not sure that that will work. On Tue, Jan 20, 2009 at 9:18 AM, wrote: > On Mon, Jan 19, 2009 at 10:10:00AM -0500, Gerald Britton wrote: >> Please find below PEP 3142: Add a "while" clause to generator >> expressions. I'm looking for feedback and discussion. >> > ... >> g = (n for n in range(100) while n*n < 50) > > May I suggest you this variant? > > def raiseStopIteration(): > raise StopIteration > > g = (n for n in range(100) if n*n < 50 or raiseStopIteration()) > > Well, there are more characters... > > But it is not using any syntax changes and does not require any approval > to be functional. Yet it is as fast as the proposed variant, does not require > modules and, I hope, will not confuse you or anyone else. > > > -- > Alexey G. Shpagin > From python-3000 at udmvt.ru Tue Jan 20 15:18:24 2009 From: python-3000 at udmvt.ru (python-3000 at udmvt.ru) Date: Tue, 20 Jan 2009 18:18:24 +0400 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> Message-ID: <20090120141824.GN11140@ruber.office.udmvt.ru> On Mon, Jan 19, 2009 at 10:10:00AM -0500, Gerald Britton wrote: > Please find below PEP 3142: Add a "while" clause to generator > expressions. I'm looking for feedback and discussion. > ... > g = (n for n in range(100) while n*n < 50) May I suggest you this variant? def raiseStopIteration(): raise StopIteration g = (n for n in range(100) if n*n < 50 or raiseStopIteration()) Well, there are more characters... But it is not using any syntax changes and does not require any approval to be functional. Yet it is as fast as the proposed variant, does not require modules and, I hope, will not confuse you or anyone else. -- Alexey G. Shpagin From benjamin at python.org Tue Jan 20 15:26:38 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 20 Jan 2009 08:26:38 -0600 Subject: [Python-Dev] Questions/comments on documentation formatting In-Reply-To: References: <1afaf6160901191901g1d634947h8d53862282c18b0e@mail.gmail.com> <1afaf6160901191919t5d80dbeete4ee13b2c82772f2@mail.gmail.com> Message-ID: <1afaf6160901200626p66f731c8nfbbfce4d6d20eca1@mail.gmail.com> On Mon, Jan 19, 2009 at 11:56 PM, Brett Cannon wrote: > On Mon, Jan 19, 2009 at 19:19, Benjamin Peterson wrote: >> On Mon, Jan 19, 2009 at 9:11 PM, Brett Cannon wrote: >>> On Mon, Jan 19, 2009 at 19:01, Benjamin Peterson wrote: >>>> On Mon, Jan 19, 2009 at 8:24 PM, Brett Cannon wrote: >>>>> >>>>> 2. Should we start using function annotations? >>>> >>>> No, I think that information is better stored in the function description. >>>> >>> >>> Why? Putting it in the signature makes it very succinct and a simple >>> glance at the doc to see what type/ABC is expected. >> >> Well, I guess it's just not been explored. Feel free to try it out if >> you wish, though. >> > > I just might. We might be opening a can of worms, though. Do we document everything that takes a dictionary argument with collections.Mapping or everything that takes a integer numbers.Rationale? What if multiple types are possible? >>>>> 4. The var directive is not working even though the docs list it as a >>>>> valid directive; so is it still valid and something is broken, or the >>>>> docs need to be updated? >>>> >>>> The docs should be updated. "data" is the one to use now. >>> >>> So the 'data' directive turns into any variable, not just a module variables? >> >> "data" is for module level objects. If you're documenting properties >> or attributes in classes, use "attribute". > > Then what are we supposed to use for arguments? Just ``literal``? No, use *some_argument*. -- Regards, Benjamin From p.f.moore at gmail.com Tue Jan 20 15:39:37 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 20 Jan 2009 14:39:37 +0000 Subject: [Python-Dev] Questions/comments on documentation formatting In-Reply-To: <1afaf6160901200626p66f731c8nfbbfce4d6d20eca1@mail.gmail.com> References: <1afaf6160901191901g1d634947h8d53862282c18b0e@mail.gmail.com> <1afaf6160901191919t5d80dbeete4ee13b2c82772f2@mail.gmail.com> <1afaf6160901200626p66f731c8nfbbfce4d6d20eca1@mail.gmail.com> Message-ID: <79990c6b0901200639y16d7d856o4917b38b92d5be52@mail.gmail.com> 2009/1/20 Benjamin Peterson : > We might be opening a can of worms, though. Do we document everything > that takes a dictionary argument with collections.Mapping or > everything that takes a integer numbers.Rationale? What if multiple > types are possible? No. Only document things as taking an ABC argument if they actually *do* only take that ABC. def f(dct): return dct['a'] does not require a collections.Mapping argument, just something that implements indexing-by-strings. Even with ABCs available, I thought that duck typing was still expected to be the norm. If a function does a type-test for an ABC, it makes sense to document it as requiring that ABC (to flag to users that they may need to register their own types with the ABC), Otherwise, it does not. Paul. From ronaldoussoren at mac.com Tue Jan 20 16:01:43 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Tue, 20 Jan 2009 16:01:43 +0100 Subject: [Python-Dev] bundlebuilder broken in 2.6 In-Reply-To: References: <7043CB7C-18F4-4E16-AE0C-CDA6BA311044@barrys-emacs.org> Message-ID: On 18 Jan, 2009, at 18:10, Barry Scott wrote: >> >> >> While the build should be fixed for 2.6+ (I'll send a patch), note >> that >> bundlebuilder is gone in 3.0. > > What is the replacement for bundlebuilder for 3.0? Lack of > bundlebuilder becomes a serious porting problem for me. > I deliver pysvn WOrkbench as a bundle to simplify installation > by my users. py2app, which hasn't been ported to python 3.0 yet (AFAIK). Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available URL: From arigo at tunes.org Tue Jan 20 15:56:40 2009 From: arigo at tunes.org (Armin Rigo) Date: Tue, 20 Jan 2009 15:56:40 +0100 Subject: [Python-Dev] Adapt test suite for other Python impls Message-ID: <20090120145640.GA4958@code0.codespeak.net> Hi all, There is a pending patch issue at http://bugs.python.org/issue4242 which proposes to tag, in the CPython test suite, which tests are general language tests (the vast majority) and which ones are specific to CPython. The patch would add a couple of helpful functions to test_support.py (http://bugs.python.org/file12718/test-impl-details-2.diff). A bientot, Armin. From curt at hagenlocher.org Tue Jan 20 16:29:54 2009 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Tue, 20 Jan 2009 07:29:54 -0800 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> Message-ID: On Tue, Jan 20, 2009 at 6:18 AM, Luke Kenneth Casson Leighton wrote: > > yeah they said the same thing about "gas ovens", too. not the nazi > gas ovens, the phrase my mother used to say "if someone stuck their > head in a gas oven, would you do the same?". I don't know who is forcing you to use a platform that you hate so much, but I respectfully suggest that this person is not on any of these mailing lists. -- Curt Hagenlocher curt at hagenlocher.org From python-3000 at udmvt.ru Tue Jan 20 16:38:40 2009 From: python-3000 at udmvt.ru (Alexey G. Shpagin) Date: Tue, 20 Jan 2009 19:38:40 +0400 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <5d1a32000901200624t243fc943m404a887aef6d0853@mail.gmail.com> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <20090120141824.GN11140@ruber.office.udmvt.ru> <5d1a32000901200624t243fc943m404a887aef6d0853@mail.gmail.com> Message-ID: <20090120153840.GO11140@ruber.office.udmvt.ru> On Tue, Jan 20, 2009 at 09:24:32AM -0500, Gerald Britton wrote: > hmmm...doesn't: > > if n*n < 50 or raise StopIteration() > > really mean, "Return an integer in the range 0-99 if n-squared is less > than fifty or the statement 'raise StopIteration()' returns True" ? > > I'm not sure that that will work. Well, your variant will trigger syntax error (and will not work surely). To make it work we need a function, that raises StopIteration. exactly as I have suggested. > > On Tue, Jan 20, 2009 at 9:18 AM, wrote: > > On Mon, Jan 19, 2009 at 10:10:00AM -0500, Gerald Britton wrote: > >> Please find below PEP 3142: Add a "while" clause to generator > >> expressions. I'm looking for feedback and discussion. > >> > > ... > >> g = (n for n in range(100) while n*n < 50) > > > > May I suggest you this variant? > > > > def raiseStopIteration(): > > raise StopIteration > > > > g = (n for n in range(100) if n*n < 50 or raiseStopIteration()) -- Alexey G. Shpagin From benjamin at python.org Tue Jan 20 16:43:33 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 20 Jan 2009 09:43:33 -0600 Subject: [Python-Dev] Questions/comments on documentation formatting In-Reply-To: <79990c6b0901200639y16d7d856o4917b38b92d5be52@mail.gmail.com> References: <1afaf6160901191901g1d634947h8d53862282c18b0e@mail.gmail.com> <1afaf6160901191919t5d80dbeete4ee13b2c82772f2@mail.gmail.com> <1afaf6160901200626p66f731c8nfbbfce4d6d20eca1@mail.gmail.com> <79990c6b0901200639y16d7d856o4917b38b92d5be52@mail.gmail.com> Message-ID: <1afaf6160901200743s4d7e43b0j640d55939e6e2595@mail.gmail.com> On Tue, Jan 20, 2009 at 8:39 AM, Paul Moore wrote: > 2009/1/20 Benjamin Peterson : >> We might be opening a can of worms, though. Do we document everything >> that takes a dictionary argument with collections.Mapping or >> everything that takes a integer numbers.Rationale? What if multiple >> types are possible? > > No. Only document things as taking an ABC argument if they actually > *do* only take that ABC. > > def f(dct): > return dct['a'] > > does not require a collections.Mapping argument, just something that > implements indexing-by-strings. Even with ABCs available, I thought > that duck typing was still expected to be the norm. That's exactly why I don't think ABCs would do much good. There are almost no functions which absolutely require a certain interface. So use of annotations would be rare. > > If a function does a type-test for an ABC, it makes sense to document > it as requiring that ABC (to flag to users that they may need to > register their own types with the ABC), Otherwise, it does not. > > Paul. > -- Regards, Benjamin From gerald.britton at gmail.com Tue Jan 20 16:45:27 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Tue, 20 Jan 2009 10:45:27 -0500 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <20090120153840.GO11140@ruber.office.udmvt.ru> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <20090120141824.GN11140@ruber.office.udmvt.ru> <5d1a32000901200624t243fc943m404a887aef6d0853@mail.gmail.com> <20090120153840.GO11140@ruber.office.udmvt.ru> Message-ID: <5d1a32000901200745i4d925e9alaa9daa7472d59198@mail.gmail.com> OK, so your suggestion: g = (n for n in range(100) if n*n < 50 or raiseStopIteration()) really means "return in in the range 0-99 if n-squared is less than 50 or the function raiseStopIteration() returns True". How would this get the generator to stop once n*n >=50? It looks instead like the first time around, StopIteration will be raised and (presumably) the generator will terminate. On Tue, Jan 20, 2009 at 10:38 AM, Alexey G. Shpagin wrote: > On Tue, Jan 20, 2009 at 09:24:32AM -0500, Gerald Britton wrote: >> hmmm...doesn't: >> >> if n*n < 50 or raise StopIteration() >> >> really mean, "Return an integer in the range 0-99 if n-squared is less >> than fifty or the statement 'raise StopIteration()' returns True" ? >> >> I'm not sure that that will work. > Well, your variant will trigger syntax error (and will not work surely). > > To make it work we need a function, that raises StopIteration. > exactly as I have suggested. > >> >> On Tue, Jan 20, 2009 at 9:18 AM, wrote: >> > On Mon, Jan 19, 2009 at 10:10:00AM -0500, Gerald Britton wrote: >> >> Please find below PEP 3142: Add a "while" clause to generator >> >> expressions. I'm looking for feedback and discussion. >> >> >> > ... >> >> g = (n for n in range(100) while n*n < 50) >> > >> > May I suggest you this variant? >> > > >> > def raiseStopIteration(): >> > raise StopIteration >> > >> > g = (n for n in range(100) if n*n < 50 or raiseStopIteration()) > > -- > Alexey G. Shpagin > From stephen at xemacs.org Tue Jan 20 16:54:57 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 21 Jan 2009 00:54:57 +0900 Subject: [Python-Dev] Copyright notices in modules In-Reply-To: <4975C386.10901@egenix.com> References: <94F1BCD4C7B448C8AE5C8C1FA34DD661@RaymondLaptop1> <497596B1.4060600@egenix.com> <4975A126.6070309@voidspace.org.uk> <4975C386.10901@egenix.com> Message-ID: <87fxjegege.fsf@xemacs.org> M.-A. Lemburg writes: > On 2009-01-20 11:02, Michael Foord wrote: > > Mere collections of facts are not copyrightable as they are not > > creative (the basis of copyright) That's incorrect in the U.S.; what is copyrightable is an *original work of expression fixed in some medium*. "Original" is closely related to "creative", but it's not the same. The emphasis is on novelty, not on the intellectual power involved. So, for example, you can copyright a set of paint splashes on paper, as long as they're *new* paint splashes. The real issue here, however, is "expression". What's important is whether there are different ways to say it. So you can indeed copyright the phone book or a dictionary, which *does* protect such things as unusual use of typefaces or color to aid understanding. What you can't do is prevent someone from publishing another phone book or dictionary based on the same facts, and since "put it in alphabetical order" hasn't been an original form of expression since Aristotle or so, they can alphabetize their phone book or dictionary, and it is going to look a lot like yours. On the other hand, ABCs are not a "mere collection of facts". They are subject to various forms of organization (top down, bottom up, alphabetical order, etc), and that organization will in general be copyrightable. Also, unless your ABCs are all independent of each other, you will be making choices about when to derive and when to define from scratch. That aspect of organization is expressive, and once written down ("fixed in a medium") it is copyrightable. > > I recommend his book by the way - I'm about half way through so far and > > it is highly readable Larry Rosen's book is also good. From python-3000 at udmvt.ru Tue Jan 20 16:57:55 2009 From: python-3000 at udmvt.ru (Alexey G. Shpagin) Date: Tue, 20 Jan 2009 19:57:55 +0400 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <5d1a32000901200745i4d925e9alaa9daa7472d59198@mail.gmail.com> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <20090120141824.GN11140@ruber.office.udmvt.ru> <5d1a32000901200624t243fc943m404a887aef6d0853@mail.gmail.com> <20090120153840.GO11140@ruber.office.udmvt.ru> <5d1a32000901200745i4d925e9alaa9daa7472d59198@mail.gmail.com> Message-ID: <20090120155755.GP11140@ruber.office.udmvt.ru> On Tue, Jan 20, 2009 at 10:45:27AM -0500, Gerald Britton wrote: > OK, so your suggestion: > > g = (n for n in range(100) if n*n < 50 or raiseStopIteration()) > > really means "return in in the range 0-99 if n-squared is less than 50 > or the function raiseStopIteration() returns True". > > How would this get the generator to stop once n*n >=50? It looks > instead like the first time around, StopIteration will be raised and > (presumably) the generator will terminate. Just test it. After the generator is terminated, no one will call range(100).next() method, if I really understand you. Maybe (as suggested before with 'if ... else break`) we should rename function raiseStopIteration() to else_break(), since it looks to me being a 'if ... else break's implementation with functions. Example will look like g = (n for n in range(100) if n*n < 50 or else_break()) That's to the matter of taste, I think. -- Alexey G. Shpagin From gerald.britton at gmail.com Tue Jan 20 17:07:21 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Tue, 20 Jan 2009 11:07:21 -0500 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <20090120155755.GP11140@ruber.office.udmvt.ru> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <20090120141824.GN11140@ruber.office.udmvt.ru> <5d1a32000901200624t243fc943m404a887aef6d0853@mail.gmail.com> <20090120153840.GO11140@ruber.office.udmvt.ru> <5d1a32000901200745i4d925e9alaa9daa7472d59198@mail.gmail.com> <20090120155755.GP11140@ruber.office.udmvt.ru> Message-ID: <5d1a32000901200807x21b94817o614e7e4fbadd396b@mail.gmail.com> Yup, I tried your idea and it does work as I intended. It looks a little better than using takewhile, but not (to me anyway) as nice as my original suggestion. Still, if my idea is ultimately rejected (looks that way at the moment), this is a good alternative. On Tue, Jan 20, 2009 at 10:57 AM, Alexey G. Shpagin wrote: > On Tue, Jan 20, 2009 at 10:45:27AM -0500, Gerald Britton wrote: >> OK, so your suggestion: >> >> g = (n for n in range(100) if n*n < 50 or raiseStopIteration()) >> >> really means "return in in the range 0-99 if n-squared is less than 50 >> or the function raiseStopIteration() returns True". >> >> How would this get the generator to stop once n*n >=50? It looks >> instead like the first time around, StopIteration will be raised and >> (presumably) the generator will terminate. > > Just test it. After the generator is terminated, no one will call > range(100).next() > method, if I really understand you. > > Maybe (as suggested before with 'if ... else break`) we should rename > function raiseStopIteration() to else_break(), > since it looks to me being a 'if ... else break's implementation with functions. > > Example will look like > g = (n for n in range(100) if n*n < 50 or else_break()) > > That's to the matter of taste, I think. > > -- > Alexey G. Shpagin > From sturla at molden.no Tue Jan 20 17:08:48 2009 From: sturla at molden.no (Sturla Molden) Date: Tue, 20 Jan 2009 17:08:48 +0100 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <5d1a32000901200745i4d925e9alaa9daa7472d59198@mail.gmail.com> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <20090120141824.GN11140@ruber.office.udmvt.ru> <5d1a32000901200624t243fc943m404a887aef6d0853@mail.gmail.com> <20090120153840.GO11140@ruber.office.udmvt.ru> <5d1a32000901200745i4d925e9alaa9daa7472d59198@mail.gmail.com> Message-ID: <4975F710.9030404@molden.no> On 1/20/2009 4:45 PM, Gerald Britton wrote: > OK, so your suggestion: > > g = (n for n in range(100) if n*n < 50 or raiseStopIteration()) > > really means "return in in the range 0-99 if n-squared is less than 50 > or the function raiseStopIteration() returns True". > > How would this get the generator to stop once n*n >=50? It looks > instead like the first time around, StopIteration will be raised and > (presumably) the generator will terminate. I still find it odd to invent new syntax for simple things like def quit(): raise StopIteration gen = itertools.imap( lambda x: x if x <= 50 else quit(), (i for i in range(100)) ) for i in gen: print i Sturla Molden From algorias at yahoo.com Tue Jan 20 17:32:24 2009 From: algorias at yahoo.com (Vitor Bosshard) Date: Tue, 20 Jan 2009 08:32:24 -0800 (PST) Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <20090120141824.GN11140@ruber.office.udmvt.ru> Message-ID: <616836.15316.qm@web54408.mail.yahoo.com> ----- Mensaje original ---- > De: "python-3000 at udmvt.ru" > Para: Gerald Britton > CC: python-dev at python.org > Enviado: martes, 20 de enero, 2009 11:18:24 > Asunto: Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions > > May I suggest you this variant? > > ??? def raiseStopIteration(): > ??? ??? raise StopIteration > > ??? g = (n for n in range(100) if n*n < 50 or raiseStopIteration()) > > Well, there are more characters... > > But it is not using any syntax changes and does not require any approval > to be functional. Yet it is as fast as the proposed variant, does not require > modules and, I hope, will not confuse you or anyone else. > This works as a generator, but not as a list comprehension. The exception is propagated instead of just cutting short the loop: >>> def r(): raise StopIteration >>> print [i for i in range(10) if i**2 < 50 or r()] Traceback (most recent call last): ? File " ", line 1, in ??? print [i for i in range(10) if i**2 < 50 or r()] ? File " ", line 1, in r ??? def r(): raise StopIteration StopIteration >>> Vitor ?Todo sobre la Liga Mexicana de f?tbol! Estadisticas, resultados, calendario, fotos y m?s:< http://espanol.sports.yahoo.com/ From gerald.britton at gmail.com Tue Jan 20 17:40:07 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Tue, 20 Jan 2009 11:40:07 -0500 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <616836.15316.qm@web54408.mail.yahoo.com> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <20090120141824.GN11140@ruber.office.udmvt.ru> <616836.15316.qm@web54408.mail.yahoo.com> Message-ID: <5d1a32000901200840o70b4173ao34ef28feb7802edb@mail.gmail.com> Right, but the PEP is only about generator expressions. On Tue, Jan 20, 2009 at 11:32 AM, Vitor Bosshard wrote: > ----- Mensaje original ---- >> De: "python-3000 at udmvt.ru" >> Para: Gerald Britton >> CC: python-dev at python.org >> Enviado: martes, 20 de enero, 2009 11:18:24 >> Asunto: Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions >> >> May I suggest you this variant? >> >> def raiseStopIteration(): >> raise StopIteration >> >> g = (n for n in range(100) if n*n < 50 or raiseStopIteration()) >> >> Well, there are more characters... >> >> But it is not using any syntax changes and does not require any approval >> to be functional. Yet it is as fast as the proposed variant, does not require >> modules and, I hope, will not confuse you or anyone else. >> > > This works as a generator, but not as a list comprehension. The exception is propagated instead of just cutting short the loop: > >>>> def r(): raise StopIteration >>>> print [i for i in range(10) if i**2 < 50 or r()] > Traceback (most recent call last): > File " ", line 1, in > print [i for i in range(10) if i**2 < 50 or r()] > File " ", line 1, in r > def r(): raise StopIteration > StopIteration >>>> > > > Vitor > > > ?Todo sobre la Liga Mexicana de f?tbol! Estadisticas, resultados, calendario, fotos y m?s:< > http://espanol.sports.yahoo.com/ > From fuzzyman at voidspace.org.uk Tue Jan 20 17:54:50 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 20 Jan 2009 16:54:50 +0000 Subject: [Python-Dev] Copyright notices in modules In-Reply-To: <87fxjegege.fsf@xemacs.org> References: <94F1BCD4C7B448C8AE5C8C1FA34DD661@RaymondLaptop1> <497596B1.4060600@egenix.com> <4975A126.6070309@voidspace.org.uk> <4975C386.10901@egenix.com> <87fxjegege.fsf@xemacs.org> Message-ID: <497601DA.5090204@voidspace.org.uk> Stephen J. Turnbull wrote: > M.-A. Lemburg writes: > > On 2009-01-20 11:02, Michael Foord wrote: > > > > Mere collections of facts are not copyrightable as they are not > > > creative (the basis of copyright) > > That's incorrect in the U.S.; what is copyrightable is an *original > work of expression fixed in some medium*. "Original" is closely > related to "creative", but it's not the same. The emphasis is on > novelty, not on the intellectual power involved. So, for example, you > can copyright a set of paint splashes on paper, as long as they're > *new* paint splashes. > No but expression is more strongly related to creative. > The real issue here, however, is "expression". What's important is > whether there are different ways to say it. So you can indeed > copyright the phone book or a dictionary, which *does* protect such > things as unusual use of typefaces or color to aid understanding. > What you can't do is prevent someone from publishing another phone > book or dictionary based on the same facts, and since "put it in > alphabetical order" hasn't been an original form of expression since > Aristotle or so, they can alphabetize their phone book or dictionary, > and it is going to look a lot like yours. > > On the other hand, ABCs are not a "mere collection of facts". They are > subject to various forms of organization (top down, bottom up, > alphabetical order, etc), and that organization will in general be > copyrightable. Also, unless your ABCs are all independent of each > other, you will be making choices about when to derive and when to > define from scratch. That aspect of organization is expressive, and > once written down ("fixed in a medium") it is copyrightable. > As you say - mere ordering does not render something copyrightable. Phone books and maps deliberately insert fictitious data in order to be eligible for copyright under these terms. On the other hand I'm inclined to believe that there is enough original expression in the ABCs to be copyrightable. It's a basically irrelevant point though. :-) Michael > > > I recommend his book by the way - I'm about half way through so far and > > > it is highly readable > > Larry Rosen's book is also good. > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From solipsis at pitrou.net Tue Jan 20 17:56:06 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 20 Jan 2009 16:56:06 +0000 (UTC) Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <20090120141824.GN11140@ruber.office.udmvt.ru> <5d1a32000901200624t243fc943m404a887aef6d0853@mail.gmail.com> <20090120153840.GO11140@ruber.office.udmvt.ru> <5d1a32000901200745i4d925e9alaa9daa7472d59198@mail.gmail.com> <20090120155755.GP11140@ruber.office.udmvt.ru> Message-ID: Alexey G. Shpagin udmvt.ru> writes: > > Example will look like > g = (n for n in range(100) if n*n < 50 or else_break()) Please don't suggest any hack involving raising StopIteration as part of a conditional statement in a generator expression. It might work today, but it might as well break tomorrow as it's only a side-effect of the implementation, not an official property of the language. Regards Antoine. From algorias at yahoo.com Tue Jan 20 18:00:14 2009 From: algorias at yahoo.com (Vitor Bosshard) Date: Tue, 20 Jan 2009 09:00:14 -0800 (PST) Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <20090120141824.GN11140@ruber.office.udmvt.ru> <616836.15316.qm@web54408.mail.yahoo.com> <5d1a32000901200840o70b4173ao34ef28feb7802edb@mail.gmail.com> Message-ID: <592455.44583.qm@web54409.mail.yahoo.com> ----- Mensaje original ---- > De: Gerald Britton > Para: Vitor Bosshard > CC: python-3000 at udmvt.ru; python-dev at python.org > Enviado: martes, 20 de enero, 2009 13:40:07 > Asunto: Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions > > Right, but the PEP is only about generator expressions. > Yes, but consistency with list comprehensions would be a nice thing to have, which is absent from both?the "or raise()"?idiom and the takewhile one (which is, by definition, a generator). The new syntax wouldn't have this issue. I'm not in favor of the change, just pointing this out. Vitor ?Todo sobre la Liga Mexicana de f?tbol! Estadisticas, resultados, calendario, fotos y m?s:< http://espanol.sports.yahoo.com/ From ironfroggy at gmail.com Tue Jan 20 18:34:43 2009 From: ironfroggy at gmail.com (Calvin Spealman) Date: Tue, 20 Jan 2009 12:34:43 -0500 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <592455.44583.qm@web54409.mail.yahoo.com> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <20090120141824.GN11140@ruber.office.udmvt.ru> <616836.15316.qm@web54408.mail.yahoo.com> <5d1a32000901200840o70b4173ao34ef28feb7802edb@mail.gmail.com> <592455.44583.qm@web54409.mail.yahoo.com> Message-ID: <76fd5acf0901200934g4d188c92l48f0c0ace18b1af4@mail.gmail.com> On Tue, Jan 20, 2009 at 12:00 PM, Vitor Bosshard wrote: > > > ----- Mensaje original ---- >> De: Gerald Britton >> Para: Vitor Bosshard >> CC: python-3000 at udmvt.ru; python-dev at python.org >> Enviado: martes, 20 de enero, 2009 13:40:07 >> Asunto: Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions >> >> Right, but the PEP is only about generator expressions. >> > > Yes, but consistency with list comprehensions would be a nice thing to have, which is absent from both the "or raise()" idiom and the takewhile one (which is, by definition, a generator). The new syntax wouldn't have this issue. > > I'm not in favor of the change, just pointing this out. I saw this to, and do want to throw in my two cents that it should be consistent between them. We should not add something to one and not the other. If the PEP, even if its rejected, doesn't change to reflect that its suggestion is for both generator expressions and list comprehensions, I think it should be considered invalid from the start. We should never add syntax that makes list(<...>) != [<...>] (where <...> is my stupid expression placeholder). > Vitor > > > ?Todo sobre la Liga Mexicana de f?tbol! Estadisticas, resultados, calendario, fotos y m?s:< > http://espanol.sports.yahoo.com/ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com > -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy From gerald.britton at gmail.com Tue Jan 20 18:46:55 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Tue, 20 Jan 2009 12:46:55 -0500 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <616836.15316.qm@web54408.mail.yahoo.com> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <20090120141824.GN11140@ruber.office.udmvt.ru> <616836.15316.qm@web54408.mail.yahoo.com> Message-ID: <5d1a32000901200946q306ed9e2i385443d585171b77@mail.gmail.com> I wonder if this is a bug? On Tue, Jan 20, 2009 at 11:32 AM, Vitor Bosshard wrote: > ----- Mensaje original ---- >> De: "python-3000 at udmvt.ru" >> Para: Gerald Britton >> CC: python-dev at python.org >> Enviado: martes, 20 de enero, 2009 11:18:24 >> Asunto: Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions >> >> May I suggest you this variant? >> >> def raiseStopIteration(): >> raise StopIteration >> >> g = (n for n in range(100) if n*n < 50 or raiseStopIteration()) >> >> Well, there are more characters... >> >> But it is not using any syntax changes and does not require any approval >> to be functional. Yet it is as fast as the proposed variant, does not require >> modules and, I hope, will not confuse you or anyone else. >> > > This works as a generator, but not as a list comprehension. The exception is propagated instead of just cutting short the loop: > >>>> def r(): raise StopIteration >>>> print [i for i in range(10) if i**2 < 50 or r()] > Traceback (most recent call last): > File " ", line 1, in > print [i for i in range(10) if i**2 < 50 or r()] > File " ", line 1, in r > def r(): raise StopIteration > StopIteration >>>> > > > Vitor > > > ?Todo sobre la Liga Mexicana de f?tbol! Estadisticas, resultados, calendario, fotos y m?s:< > http://espanol.sports.yahoo.com/ > From ironfroggy at gmail.com Tue Jan 20 18:54:03 2009 From: ironfroggy at gmail.com (Calvin Spealman) Date: Tue, 20 Jan 2009 12:54:03 -0500 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <5d1a32000901200946q306ed9e2i385443d585171b77@mail.gmail.com> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <20090120141824.GN11140@ruber.office.udmvt.ru> <616836.15316.qm@web54408.mail.yahoo.com> <5d1a32000901200946q306ed9e2i385443d585171b77@mail.gmail.com> Message-ID: <76fd5acf0901200954p4d2b812ex1a9fa753c762164@mail.gmail.com> On Tue, Jan 20, 2009 at 12:46 PM, Gerald Britton wrote: > I wonder if this is a bug? I don't think so, but its interesting nonetheless. passing a generator expression to list() involves two loops: the list construction and the generator expression. So, a StopIteration from whatever the GE is iterating over is caught by the GE mechanics, and anything else in the clauses can be caught by the list constructor. If the same thing is done in a LC, such an exception from the clause has nothing to catch it. It is not raised as part of iterating over something. I don't think we'd want to just start swallowing errors here, as it would change defined behavior. > On Tue, Jan 20, 2009 at 11:32 AM, Vitor Bosshard wrote: >> ----- Mensaje original ---- >>> De: "python-3000 at udmvt.ru" >>> Para: Gerald Britton >>> CC: python-dev at python.org >>> Enviado: martes, 20 de enero, 2009 11:18:24 >>> Asunto: Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions >>> >>> May I suggest you this variant? >>> >>> def raiseStopIteration(): >>> raise StopIteration >>> >>> g = (n for n in range(100) if n*n < 50 or raiseStopIteration()) >>> >>> Well, there are more characters... >>> >>> But it is not using any syntax changes and does not require any approval >>> to be functional. Yet it is as fast as the proposed variant, does not require >>> modules and, I hope, will not confuse you or anyone else. >>> >> >> This works as a generator, but not as a list comprehension. The exception is propagated instead of just cutting short the loop: >> >>>>> def r(): raise StopIteration >>>>> print [i for i in range(10) if i**2 < 50 or r()] >> Traceback (most recent call last): >> File " ", line 1, in >> print [i for i in range(10) if i**2 < 50 or r()] >> File " ", line 1, in r >> def r(): raise StopIteration >> StopIteration >>>>> >> >> >> Vitor >> >> >> ?Todo sobre la Liga Mexicana de f?tbol! Estadisticas, resultados, calendario, fotos y m?s:< >> http://espanol.sports.yahoo.com/ >> > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com > -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy From rdmurray at bitdance.com Tue Jan 20 19:08:00 2009 From: rdmurray at bitdance.com (rdmurray at bitdance.com) Date: Tue, 20 Jan 2009 13:08:00 -0500 (EST) Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <20090120141824.GN11140@ruber.office.udmvt.ru> <5d1a32000901200624t243fc943m404a887aef6d0853@mail.gmail.com> <20090120153840.GO11140@ruber.office.udmvt.ru> <5d1a32000901200745i4d925e9alaa9daa7472d59198@mail.gmail.com> <20090120155755.GP11140@ruber.office.udmvt.ru> Message-ID: On Tue, 20 Jan 2009 at 16:56, Antoine Pitrou wrote: > Alexey G. Shpagin udmvt.ru> writes: >> >> Example will look like >> g = (n for n in range(100) if n*n < 50 or else_break()) > > Please don't suggest any hack involving raising StopIteration as part of a > conditional statement in a generator expression. It might work today, but it > might as well break tomorrow as it's only a side-effect of the implementation, > not an official property of the language. Doing the above is, by definition, no different from raising StopIteration in a for loop inside a generator function. The language reference does document the raising of a StopIteration as signaling the exhaustion of the generator. In addition, the 3.0 docs (but, oddly, not the 2.6 docs) say in the 'for' loop documentation: "When the items are exhausted (which is immediately when the list is empty or an iterator raises a StopIteration exception)"). The difference in behavior between raising StopIteration in a list comprehension versus a generator expression are consistent with the above, by the way. If you raise StopIteration in a function whose definition is the same as the list comprehension but you are building the list as you go and only returning it when it is complete, then the StopIteration would propagate upward with no values returned (ie: in a for loop it would look like an empty list). I don't know about other people, but I have certainly assumed that raising StopIteration was a legitimate way to terminate an iterator, and have written code accordingly. If this is not true, it should probably be explicitly documented in the language reference somewhere. --RDM From guido at python.org Tue Jan 20 20:24:39 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 20 Jan 2009 11:24:39 -0800 Subject: [Python-Dev] __del__ and tp_dealloc in the IO lib In-Reply-To: References: <52dc1c820901181738h53e7ecf9if254a91de2025722@mail.gmail.com> Message-ID: On Sun, Jan 18, 2009 at 11:49 PM, Adam Olsen wrote: > On Sun, Jan 18, 2009 at 9:32 PM, Guido van Rossum wrote: >> On Sun, Jan 18, 2009 at 5:38 PM, Gregory P. Smith wrote: >>> +1 on getting rid of the IOBase __del__ in the C rewrite in favor of >>> tp_dealloc. >>> >>> On Sun, Jan 18, 2009 at 11:53 PM, Christian Heimes wrote: >>>> >>>> Brett Cannon schrieb: >>>> > Fine by me. People should be using the context manager for guaranteed >>>> > file closure anyway IMO. >>> >>> Yes they should. (how I really really wish i didn't have to use 2.4 anymore >>> ;) >> >> Come on, the open-try-use-finally-close idiom isn't *that* bad... >> >>> But lets at least be clear that is never acceptable for a python >>> implementation to leak file descriptors/handles (or other system resources), >>> they should be closed and released whenever the particular GC implementation >>> gets around to it. >> >> I would like to make a stronger promise. I think that for files open >> for *writing*, all data written to the file should be flushed to disk >> before the fd is closed. This is the real reason for having the >> __del__: closing the fd is done by the C implementation of FileIO, but >> since (until the rewrite in C) the buffer management is all in Python >> (both the binary I/O buffer and the additional text I/O buffer), I >> felt the downside of having a __del__ method was preferable over the >> possibility of output files not being flushed (which is always >> nightmarish to debug). >> >> Of course, once both layers of buffering are implemented in C, the >> need for __del__ to do this goes away, and I would be fine with doing >> it all in tp_alloc. >> >>>> You make a very good point! Perhaps we should stop promising that files >>>> get closed as soon as possible and encourage people in using the with >>>> statement. >>>> >>>> Christian >>> >>> eegads, do we actually -promise- that somewhere? If so I'll happily go >>> update those docs with a caveat. >> >> I don't think we've promised that ever since the days when JPython >> (with a P!) was young... >> >>> I regularly point out in code reviews that the very convenient and common >>> idiom of open(name, 'w').write(data) doesn't guarantee when the file will be >>> closed; its up to the GC implementation details. Good code should never >>> depend on the GC for a timely release of scarce external resources (file >>> descriptors/handles). >> >> And buffer flushing. While I don't want to guarantee that the buffer >> is flushed ASAP, I do want to continue promising that it is flushed >> before the object is GC'ed and before the fd is closed. > > Could we add a warning if the file has not been explicitly flushed? > Consider removing the implicit flush later, if there's a sufficient > implementation benefit to it. No, I really want to keep the implicit flush, even if it's hard for the implementation. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Tue Jan 20 21:38:05 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 21 Jan 2009 06:38:05 +1000 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <20090120141824.GN11140@ruber.office.udmvt.ru> <5d1a32000901200624t243fc943m404a887aef6d0853@mail.gmail.com> <20090120153840.GO11140@ruber.office.udmvt.ru> <5d1a32000901200745i4d925e9alaa9daa7472d59198@mail.gmail.com> <20090120155755.GP11140@ruber.office.udmvt.ru> Message-ID: <4976362D.5040803@gmail.com> Antoine Pitrou wrote: > Alexey G. Shpagin udmvt.ru> writes: >> Example will look like >> g = (n for n in range(100) if n*n < 50 or else_break()) > > Please don't suggest any hack involving raising StopIteration as part of a > conditional statement in a generator expression. It might work today, but it > might as well break tomorrow as it's only a side-effect of the implementation, > not an official property of the language. As RDM noted, it actually is documented behaviour due to the equivalence between generator expressions and the corresponding generator functions. Writing a separate generator function is typically going to be cleaner and more readable though. Cheers, Nick. P.S. Here's another cute hack for terminating an iterator early: >>> list(iter((n for n in range(10)).next, 5)) [0, 1, 2, 3, 4] (it's nowhere near as flexible as itertools.takewhile, of course) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Tue Jan 20 21:42:36 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 21 Jan 2009 06:42:36 +1000 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <5d1a32000901200946q306ed9e2i385443d585171b77@mail.gmail.com> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <20090120141824.GN11140@ruber.office.udmvt.ru> <616836.15316.qm@web54408.mail.yahoo.com> <5d1a32000901200946q306ed9e2i385443d585171b77@mail.gmail.com> Message-ID: <4976373C.9040808@gmail.com> Gerald Britton wrote: > I wonder if this is a bug? Nope, it's part of the defined behaviour. Avoiding the overhead of the GE machinery is actually the main advantage in using a comprehension over the equivalent generator expression. Deliberately raising StopIteration is about the only way to notice the small semantics difference (in Py3k anyway - in 2.x there are scoping differences as well). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Tue Jan 20 22:01:27 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 21 Jan 2009 07:01:27 +1000 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> Message-ID: <49763BA7.6070802@gmail.com> Luke Kenneth Casson Leighton wrote: > i'd say "great" - but given a choice of "impressive profile guided > optimisation plus a proprietary compiler, proprietary operating system > _and_ being forced to purchase a system _capable_ of running said > proprietary compiler, said proprietary operating system, _and_ giving > up free software principles _and_ having to go through patch-pain, > install-pain _and_ being forced to use a GUI-based IDE for > compilation" or "free software tools and downloads the use of which > means i am beholden to NOONE", it's a simple choice for me to make - > maybe not for other people. It only becomes a problem when someone wants to both support Windows users of their extension modules with pre-built binaries, but *also* doesn't want to set up the appropriate environment for building such binaries (currently a minimum bar of Visual Studio Express on a Windows VM instance). The most common reaction I've seen to this problem from package developers is "I don't run Windows, so if users want pre-built binaries, someone with a Windows environment is going to have to volunteer to provide them". And that seems like a perfectly reasonable way to handle the situation to me. On POSIX systems, GCC does a great job, on Windows, MSVC is better (from a performance point of view). The closed source vs open source, free vs non-free philosophical arguments don't really play a significant part in the decision. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From martin at v.loewis.de Tue Jan 20 22:19:02 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 20 Jan 2009 22:19:02 +0100 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> Message-ID: <49763FC6.9090303@v.loewis.de> >> That's a non-starter for anyone who incorporates Python in an existing >> MSVC-based development environment. > > surely incorporating libpython2.5.dll.a or libpython2.6.dll.a, along > with the .def and the importlib that's generated with dlldump, unless > i'm missing something, would be a simple matter, yes? Not exactly sure what this is, but I believe Python *already* includes such a thing. Regards, Martin From dickinsm at gmail.com Tue Jan 20 23:05:29 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 20 Jan 2009 22:05:29 +0000 Subject: [Python-Dev] Strategies for debugging buildbot failures? In-Reply-To: <1afaf6160901181626j4d92eb31v1ad5400e1a37460b@mail.gmail.com> References: <5c6f2a5d0901181003j96d32a5x35db64bc710d3405@mail.gmail.com> <18803.51331.633341.223823@montanaro.dyndns.org> <1afaf6160901181626j4d92eb31v1ad5400e1a37460b@mail.gmail.com> Message-ID: <5c6f2a5d0901201405v5b893e66qa13ca1dabfdf84d6@mail.gmail.com> Thanks for all the feedback. [Michael Foord] > At Resolver Systems we regularly extend the test framework purely > to provide more diagnostic information in the event of test failures. > We do a lot of functional testing through the UI, which is particularly > prone to intermittent and hard to diagnose failures. Seems like a sound approach in general. It seems awkward to apply the method to this particular failure, though. I guess one would need extra code in regrtest.py to catch the invalid signal, for a start. [Martin v. L?wis] > Buildbot also supports submission of patches directly to the > slaves. This is currently not activated, and clearly requires > some authentication/authorization; if you want to use that, > I'd be happy to experiment with setting it up, though. > [...] > In the past, for the really difficult problems, we arranged to > have the developers get access to the buildbot slaves. Thanks, Martin. I think I've pretty much run out of time to pursue this particular problem for the moment; I may return to it later. It's good to know that these options are available, though. Mark From steve at pearwood.info Tue Jan 20 23:55:59 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 21 Jan 2009 09:55:59 +1100 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <20090120155755.GP11140@ruber.office.udmvt.ru> Message-ID: <200901210955.59699.steve@pearwood.info> On Wed, 21 Jan 2009 03:56:06 am Antoine Pitrou wrote: > Alexey G. Shpagin udmvt.ru> writes: > > Example will look like > > g = (n for n in range(100) if n*n < 50 or else_break()) > > Please don't suggest any hack involving raising StopIteration as part > of a conditional statement in a generator expression. It might work > today, but it might as well break tomorrow as it's only a side-effect > of the implementation, not an official property of the language. If that's the case, then that is a point in favour of the PEP. Personally, I find the proposed syntax change very readable and intuitive. Some have argued that it makes Python harder to learn because it adds one more thing to learn, but that's a trivial objection. It's an extension to the existing syntax, but an obvious one. The difficulty in becoming proficient in a language is not learning the syntax, but in becoming experienced with the libraries, and on that regard the PEP is a win because it simplifies the itertools module by removing takewhile (which unfortunately I find neither readable or intuitive). Another argument against the PEP was that it breaks the correspondence between the generator expression and the equivalent for-loop. I had never even noticed such correspondence before, because to my eyes the most important term is the yielded expression, not the scaffolding around it. In a generator expression, we have: yielded-expr for-clause if-clause while the corresponding nested statements are: for-clause if-clause yielded-expr The three clauses are neither in the same order, nor are they in reverse order. I don't know how important that correspondence is to language implementers, but as a Python programmer, I'd gladly give up that correspondence (which I don't find that great) in order to simplify exiting a generator expression early. So I like the proposed change. I find it elegant and very Pythonic. +1 for me. -- Steven D'Aprano From tjreedy at udel.edu Wed Jan 21 00:09:23 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 20 Jan 2009 18:09:23 -0500 Subject: [Python-Dev] Copyright notices in modules In-Reply-To: <497596B1.4060600@egenix.com> References: <94F1BCD4C7B448C8AE5C8C1FA34DD661@RaymondLaptop1> <497596B1.4060600@egenix.com> Message-ID: M.-A. Lemburg wrote: > On 2009-01-20 00:56, Raymond Hettinger wrote: >> Why does numbers.py say: >> >> # Copyright 2007 Google, Inc. All Rights Reserved. >> # Licensed to PSF under a Contributor Agreement. > > Because that's where that file originated, I guess. This is part > of what you have to do for things that are licensed to the PSF > under a contributor agreement: > > http://www.python.org/psf/contrib/contrib-form/ > > """ > Contributor shall identify each Contribution by placing the following notice in > its source code adjacent to Contributor's valid copyright notice: "Licensed to > PSF under a Contributor Agreement." > """ > >> Weren't there multiple contributors including non-google people? > > The initial contribution was done by Google (Jeffrey Yasskin > AFAIK) and that's where the above lines originated from. Thank you for the explanation, here and below, as far as it goes. But what about the copyrightable and therefore copywrited contributions of others? Does Google (in this case) get an automatic transfer of copyright to Google? A single copyright notice seems to imply that. In the case of minor edits of the original work, perhaps yes. When, for instance, I send an author notice of a typo or a suggested rephasing of a sentence, I consider that a donation to the author. In the case of new work, added to the file by PSF so that the file become a compilation or anthology of the work of several people, I should think not. If there is any copyright notice, then perhaps there should be several -- one for each 'major' (new section) contributor and one for the PSF for the compilation. I have occasional seen such things in printed works. >> Does Google want to be associated with code that >> was submitted with no tests? > > Only Google can comment on this. > >> Do we want this sort of stuff in the code? > > Yes, it is required by the contrib forms. Then it seems to me that there should/could be a notice for each major contributor of independent and separately copyrightable sections. >> If someone signs a contributor agreement, can we >> forgo the external copyright comments? > > No. See above. Only the copyright owner can remove such > notices. > >> Do we want to make a practice of every contributor >> commenting in the name of the company they were >> working for at the time (if so, I would have to add >> the comment to a lot of modules)? > > That depends on the contract a contributor has with the > company that funded the work. It's quite common for such > contracts to include a clause stating that all IP generated > during work time is owned by the employer. > >> Does the copyright concept even apply to an >> abstract base class (I thought APIs were not >> subject to copyright, just like database layouts >> and language definitions)? > > It applies to the written program text. You are probably > thinking about other IP rights such as patents or designs. Bottom line to me. The current notion of copyright does not work too well with evolving, loosely collective works (which eventually become 'folklore'). Terry Jan Reedy From mal at egenix.com Wed Jan 21 00:15:27 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 21 Jan 2009 00:15:27 +0100 Subject: [Python-Dev] Copyright notices in modules In-Reply-To: <87fxjegege.fsf@xemacs.org> References: <94F1BCD4C7B448C8AE5C8C1FA34DD661@RaymondLaptop1> <497596B1.4060600@egenix.com> <4975A126.6070309@voidspace.org.uk> <4975C386.10901@egenix.com> <87fxjegege.fsf@xemacs.org> Message-ID: <49765B0F.7080300@egenix.com> On 2009-01-20 16:54, Stephen J. Turnbull wrote: > M.-A. Lemburg writes: > > On 2009-01-20 11:02, Michael Foord wrote: > > > > Mere collections of facts are not copyrightable as they are not > > > creative (the basis of copyright) > > That's incorrect in the U.S.; what is copyrightable is an *original > work of expression fixed in some medium*. "Original" is closely > related to "creative", but it's not the same. The emphasis is on > novelty, not on the intellectual power involved. So, for example, you > can copyright a set of paint splashes on paper, as long as they're > *new* paint splashes. > > The real issue here, however, is "expression". What's important is > whether there are different ways to say it. So you can indeed > copyright the phone book or a dictionary, which *does* protect such > things as unusual use of typefaces or color to aid understanding. > What you can't do is prevent someone from publishing another phone > book or dictionary based on the same facts, and since "put it in > alphabetical order" hasn't been an original form of expression since > Aristotle or so, they can alphabetize their phone book or dictionary, > and it is going to look a lot like yours. The above argument is what makes copyright so complicated. Computer software has been given the same status as a piece of literary work, so all conventions for such works apply. However, this doesn't necessarily mean that all computer software is copyrightable per-se. The key problem is defining the threshold of originality needed for a work to become copyrightable at all and that's where different jurisdictions use different definitions or guidelines based on case law. http://en.wikipedia.org/wiki/Threshold_of_originality E.g. in Germany it is common not to grant copyright on logos that are used as trademarks. OTOH, use of a logo in the trademark sense automatically makes it a trademark (even without registration). > On the other hand, ABCs are not a "mere collection of facts". They are > subject to various forms of organization (top down, bottom up, > alphabetical order, etc), and that organization will in general be > copyrightable. Also, unless your ABCs are all independent of each > other, you will be making choices about when to derive and when to > define from scratch. That aspect of organization is expressive, and > once written down ("fixed in a medium") it is copyrightable. > > > > I recommend his book by the way - I'm about half way through so far and > > > it is highly readable > > Larry Rosen's book is also good. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 20 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From tjreedy at udel.edu Wed Jan 21 00:27:58 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 20 Jan 2009 18:27:58 -0500 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <5d1a32000901200946q306ed9e2i385443d585171b77@mail.gmail.com> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <20090120141824.GN11140@ruber.office.udmvt.ru> <616836.15316.qm@web54408.mail.yahoo.com> <5d1a32000901200946q306ed9e2i385443d585171b77@mail.gmail.com> Message-ID: Gerald Britton wrote: > I wonder if this is a bug? It is a known glitch reported last summer. Devs decided not to fix because doing so would, in the patches tried, slow list comps significantly. Also, the documented intent and expected usage of StopIteration is this "exception StopIteration Raised by builtin next() and an iterator?s __next__() method to signal that there are no further values." The second clause includes usage in the body of a generator function since that body becomes the __next__ method of the generator-iterator produced by calling the generator function. The meaning of any other usage, such as in the body of a standard function other than next(),(as in the example producing the glitch), is undefined and leads to undefined behavior, which could be different in other implementations and change in future implementations. Terry Jan Reedy From python at rcn.com Wed Jan 21 00:36:44 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 20 Jan 2009 15:36:44 -0800 Subject: [Python-Dev] Copyright notices in modules References: <94F1BCD4C7B448C8AE5C8C1FA34DD661@RaymondLaptop1><497596B1.4060600@egenix.com> Message-ID: <9E704C0F455642AE974D4DAC51534F3F@RaymondLaptop1> [Terry Reedy] > Bottom line to me. The current notion of copyright does not work too > well with evolving, loosely collective works (which eventually become > 'folklore'). I'm at a loss of why the notice needs to be there at all. AFAICT, we've had tons of contributions from googlers and only one has put a Google copyright notice in the source. Raymond From benjamin at python.org Wed Jan 21 01:00:10 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 20 Jan 2009 18:00:10 -0600 Subject: [Python-Dev] Copyright notices in modules In-Reply-To: <9E704C0F455642AE974D4DAC51534F3F@RaymondLaptop1> References: <94F1BCD4C7B448C8AE5C8C1FA34DD661@RaymondLaptop1> <497596B1.4060600@egenix.com> <9E704C0F455642AE974D4DAC51534F3F@RaymondLaptop1> Message-ID: <1afaf6160901201600r56c0e3k5c86ca0e34cf0a72@mail.gmail.com> On Tue, Jan 20, 2009 at 5:36 PM, Raymond Hettinger wrote: > [Terry Reedy] >> >> Bottom line to me. The current notion of copyright does not work too well >> with evolving, loosely collective works (which eventually become >> 'folklore'). > > I'm at a loss of why the notice needs to be there at all. AFAICT, we've > had tons of contributions from googlers and only one has put a Google > copyright notice in the source. Oh? Grepping through the source shows no less than 30 copyright notices from Google. -- Regards, Benjamin From tjreedy at udel.edu Wed Jan 21 01:04:06 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 20 Jan 2009 19:04:06 -0500 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: Message-ID: Luke Kenneth Casson Leighton wrote: > this is a fairly important issue for python development > interoperability - martin mentioned that releases of mingw-compiled > python, if done with a non-interoperable version of msvcrt, would > cause much mayhem. > well, compiling python on mingw with msvcr80 _can_ be done; using it > can also be a simple matter of creating a python.exe.manifest file, > but i can't actually do any testing because it doesn't work under > wine. > so, pending any further advice and guidance from anyone which allows > me to successfully proceed, i'm not going to continue to compile - or > release - python2.5 *or* python2.6 builds (when i get round to that) > using msvcr80 or msvcr9X. > one issue in favour of this decision is that the DLL that's produced > by the autoconf build process is "libpython2.5.dll.a" - not > "python2.5.dll". it has a different name. it should be abundantly > clear to users and developers that "if name equals libpython2.5.dll.a > then duh build equals different". additionally, the setup.py > distutils all goes swimmingly well and lovely - using > libpython2.5.dll.a. > the only issue which _is_ going to throw a spanner in the works is > that people who download win32-built precompiled c-based modules are > going to find that hey, "it don't work!" and the answer will have to > be "go get a version of that module, compiled with mingw, not MSVC". > > of course - if python for win32 ENTIRELY DROPPED msvc as a development > platform, and went for an entirely free software development > toolchain, then this problem goes away. > > thoughts, anyone? As I understand the above, you listed or implied 3 paths other than you completely giving up, which you are not ready to do yet. 1. You release non-interoperable binary, with a modified name to alleviate, but not prevent confusion. 2. You get some sort of help from someone to release an interoperable binary. 3. The devs drop msvc (wink missing ;-). Not surprisingly to me, people on pydev followed herring #3 to explain why not. If you want responses to path 2, a post leaving out 3 and giving more detail might be more successful. All I could do is unzip stuff into a temp directory and run the test suite on my XP mechine. Terry Jan Reedy From guido at python.org Wed Jan 21 01:45:36 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 20 Jan 2009 16:45:36 -0800 Subject: [Python-Dev] Copyright notices in modules In-Reply-To: <9E704C0F455642AE974D4DAC51534F3F@RaymondLaptop1> References: <94F1BCD4C7B448C8AE5C8C1FA34DD661@RaymondLaptop1> <497596B1.4060600@egenix.com> <9E704C0F455642AE974D4DAC51534F3F@RaymondLaptop1> Message-ID: 2009/1/20 Raymond Hettinger : > I'm at a loss of why the notice needs to be there at all. There's a difference between contributing a whole file and contributing a patch. Patches do not require copyright notices. Whole files do. This is not affected by later edits to the file. > AFAICT, we've > had tons of contributions from googlers and only one has put a Google copyright notice in the source. I count 28 .py files with a Google copyright and 127 with other copyrights (not counting the 47 PSF copyrights :-). Why are you picking on Google? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Wed Jan 21 02:20:44 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 20 Jan 2009 17:20:44 -0800 Subject: [Python-Dev] Copyright notices in modules References: <94F1BCD4C7B448C8AE5C8C1FA34DD661@RaymondLaptop1> <497596B1.4060600@egenix.com> <9E704C0F455642AE974D4DAC51534F3F@RaymondLaptop1> Message-ID: [Raymond Hettinger] >> I'm at a loss of why the notice needs to be there at all. [GvR] > There's a difference between contributing a whole file and > contributing a patch. Patches do not require copyright notices. Whole > files do. This is not affected by later edits to the file. That makes sense. In general though, I think if a contributor isn't required by their company to add a copyright, then this sort of thing should be left out of the source code. Most of the contributors here don't seem to copyright-up everything they do (with the exception of big packages or externally maintained resources). If everyone making a significant contribution has a contributor agreement on file, perhaps we can build a list of those in a single file rather than scattering notices throughout the code. I don't see that those benefit anyone (maintainers, the original contributor, or the contributor's company). At least these notices are somewhat innocuous. The ones that were the most irritating are the ones requiring a literal copy of the notice to be placed in the docs. A while back, I spent a day getting us in compliance with those. FWIW, I'm not picking on anyone. I would just like to see a practice emerge where these stop getting added and perhaps start getting removed unless they are actually necessary for some reason (i.e. a company requires it). AFAICT, little notices like the one atop numbers.py don't confer property rights to anyone. The original purpose of a copyright notice has been lost. It has become useless boilerplate, a toothless warning sign about a unclaimable property claim on donated code. Raymond P.S. It seems silly that the copyright on PEP3141 says, "this document has been placed in the public domain" but the code itself has a company copyright. The former seems like something someone would care more about as a creative expression or original research. From guido at python.org Wed Jan 21 03:03:18 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 20 Jan 2009 18:03:18 -0800 Subject: [Python-Dev] Copyright notices in modules In-Reply-To: References: <94F1BCD4C7B448C8AE5C8C1FA34DD661@RaymondLaptop1> <497596B1.4060600@egenix.com> <9E704C0F455642AE974D4DAC51534F3F@RaymondLaptop1> Message-ID: I would be all for cleaning up, if the lawyers agree, but I've spent enough time talking to lawyers for the rest of my life. You know where to reach Van Lindberg. On Tue, Jan 20, 2009 at 5:20 PM, Raymond Hettinger wrote: > > [Raymond Hettinger] >>> >>> I'm at a loss of why the notice needs to be there at all. > > [GvR] >> >> There's a difference between contributing a whole file and >> contributing a patch. Patches do not require copyright notices. Whole >> files do. This is not affected by later edits to the file. > > That makes sense. In general though, I think if a contributor isn't > required by their company to add a copyright, then this sort of thing > should be left out of the source code. Most of the contributors here > don't seem to copyright-up everything they do (with the exception > of big packages or externally maintained resources). > > If everyone making a significant contribution has a contributor agreement > on file, perhaps we can build a list of those in a single file rather than > scattering notices throughout the code. I don't see that those benefit > anyone (maintainers, the original contributor, or the contributor's > company). > > At least these notices are somewhat innocuous. The ones that were > the most irritating are the ones requiring a literal copy of the notice > to be placed in the docs. A while back, I spent a day getting us in > compliance with those. > > FWIW, I'm not picking on anyone. I would just like to see a practice > emerge where these stop getting added and perhaps start getting removed > unless they are actually necessary for some reason (i.e. a company requires > it). > > AFAICT, little notices like the one atop numbers.py don't confer property > rights to anyone. The original purpose of a copyright notice has been lost. > It has become useless boilerplate, a toothless warning sign about a > unclaimable > property claim on donated code. > > > Raymond > > > P.S. It seems silly that the copyright on PEP3141 says, "this document has > been placed in the public domain" but the code itself has a company > copyright. > The former seems like something someone would care more about as a > creative expression or original research. > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Wed Jan 21 05:10:24 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 20 Jan 2009 23:10:24 -0500 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <200901210955.59699.steve@pearwood.info> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <20090120155755.GP11140@ruber.office.udmvt.ru> <200901210955.59699.steve@pearwood.info> Message-ID: Steven D'Aprano wrote: > Another argument against the PEP was that it breaks the correspondence > between the generator expression and the equivalent for-loop. I had > never even noticed such correspondence before, because to my eyes the > most important term is the yielded expression, not the scaffolding > around it. This was a major reason to add comprehensions. Your not noticing a primary design principle is hardly a reason to abandon it. Indeed, I claim that your ignorance shows its validity ;-). > In a generator expression, we have: > > yielded-expr for-clause if-clause > > while the corresponding nested statements are: > > for-clause if-clause yielded-expr > > The three clauses are neither in the same order, nor are they in reverse > order. They are in the same order but rotated, with the last brought around to the front to emphasize it. Did you really not notice that either? >I don't know how important that correspondence is to language > implementers, but as a Python programmer, I'd gladly give up that > correspondence (which I don't find that great) in order to simplify > exiting a generator expression early. > > So I like the proposed change. I find it elegant and very Pythonic. +1 > for me. Ironically, in a thread cross-posted on c.l.p and elsewhere, someone just labeled Python's comprehension syntax as "ad hoc syntax soup". That currently is completely wrong. It is a carefully designed 1 to 1 transformation between multiple nested statements and a single expression. But this proposal ignores and breaks that. Using 'while x' to mean 'if x: break' *is*, to me, 'ad hoc'. So I detest the proposed change. I find it ugly and anti-Pythonic. Terry Jan Reedy From tjreedy at udel.edu Wed Jan 21 08:01:40 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 21 Jan 2009 02:01:40 -0500 Subject: [Python-Dev] Copyright notices in modules In-Reply-To: References: <94F1BCD4C7B448C8AE5C8C1FA34DD661@RaymondLaptop1> <497596B1.4060600@egenix.com> <9E704C0F455642AE974D4DAC51534F3F@RaymondLaptop1> Message-ID: Guido van Rossum wrote: > 2009/1/20 Raymond Hettinger : >> I'm at a loss of why the notice needs to be there at all. > > There's a difference between contributing a whole file and > contributing a patch. Patches do not require copyright notices. Whole > files do. This is not affected by later edits to the file. In my comment, I postulated the situation where the patch consisted of merging in another, independently copyrighted, 'whole file'. Perhaps this has mostly been a non-existent situation and therefor moot. One real situation I was thinking of, unconnected to Google as far as I am aware, is the case of two third-party IP6 modules and the suggestion that they be merged into one stdlib module. If that were accomplished by committing one and merging the other in a patch, it would be unfair (and untrue) to have just one copyright notice. Of course, in this case, I hope the two authors work everything out between themselves first before any submission. I completely understand about strongly preferring programming to lawyer consultation ;-). tjr From kristjan at ccpgames.com Wed Jan 21 11:30:10 2009 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Wed, 21 Jan 2009 10:30:10 +0000 Subject: [Python-Dev] Issue 4448 Message-ID: <930F189C8A437347B80DF2C156F7EC7F04DACA726B@exchis.ccp.ad.local> Hello there. I recently reactivated http://bugs.python.org/issue4448 because of the need to port http://bugs.python.org/issue4879 to 3.1 This isn't a straightforward port because of the changes in the IO library. I'd appreciate if someone could shed some light on the comment in line 268 in Lib/http/client.py. See my last comment in the issue for details. Thanks, Kristj?n -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Wed Jan 21 11:46:46 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 21 Jan 2009 21:46:46 +1100 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <200901210955.59699.steve@pearwood.info> Message-ID: <200901212146.46821.steve@pearwood.info> On Wed, 21 Jan 2009 03:10:24 pm Terry Reedy wrote: > Steven D'Aprano wrote: ... > > In a generator expression, we have: > > > > yielded-expr for-clause if-clause > > > > while the corresponding nested statements are: > > > > for-clause if-clause yielded-expr > > > > The three clauses are neither in the same order, nor are they in > > reverse order. > > They are in the same order but rotated, with the last brought around > to the front to emphasize it. Did you really not notice that either? There are only three items, of course I noticed that there is *some* rearrangement of the first that leads to the second. Out of the six possible permutations of three items, they can all be described in terms of some sort of reflection, rotation or swap. > > I don't know how important that correspondence is to language > > implementers, but as a Python programmer, I'd gladly give up that > > correspondence (which I don't find that great) in order to simplify > > exiting a generator expression early. > > > > So I like the proposed change. I find it elegant and very Pythonic. > > +1 for me. > > Ironically, in a thread cross-posted on c.l.p and elsewhere, someone > just labeled Python's comprehension syntax as "ad hoc syntax soup". Is that Xah Lee? It sounds like the sort of thing he'd say. > That currently is completely wrong. It certainly is wrong. List comps and generator expressions are very elegant, at least to English speakers with a maths background (I personally "got" list comps once I recognised the correspondence to mathematical set notation. I assumed that was deliberate). > It is a carefully designed 1 to > 1 transformation between multiple nested statements and a single > expression. I'm sure that correspondence is obvious to some, but it wasn't obvious to me, and I don't suppose I'm the only one. That's not a criticism of the current syntax. Far from it -- the current syntax is excellent, regardless of whether or not you notice that it corresponds to a if-loop nested inside a for-loop with the contents rotated outside. > But this proposal ignores and breaks that. Using 'while > x' to mean 'if x: break' *is*, to me, 'ad hoc'. But it doesn't mean that. The proposed "while x" has very similar semantics to the "while x" in a while-loop: break when *not* x. > So I detest the proposed change. I find it ugly and anti-Pythonic. To each their own. I find it an elegant extension to the existing syntax. -- Steven D'Aprano From lkcl at lkcl.net Wed Jan 21 12:08:23 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Wed, 21 Jan 2009 11:08:23 +0000 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: <49763BA7.6070802@gmail.com> References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> <49763BA7.6070802@gmail.com> Message-ID: > It only becomes a problem when someone wants to both support Windows > users of their extension modules with pre-built binaries, but *also* > doesn't want to set up the appropriate environment for building such > binaries (currently a minimum bar of Visual Studio Express on a Windows > VM instance). ok - fortunately, thanks to dan kegel for pointing me in the right direction of "winetricks vcrun2005p1" i was able to get a successful build using Microsoft.VC8.CRT assemblies. i say "successful" because Parser/pgen.exe was built and ran, and libpython2.5.dll.a was also successfully built, as was python.exe successfully built. the problem _now_ to overcome is that the bloody libmsvcrt80.a has the wrong definitions, for a 32-bit build! it has functions like _fstat instead of _fstat32 and so on. if this was a 64-bit version of wine i was using mingw32 under, i would not have encountered this issue. amazingly, however, someone _else_ who kindly tried out compiling python2.5 with mingw and msvcr80, native on win32, reported that it was a complete success! as in, "successful build, successful install, successful run of tests, only 4 failed regression tests". i am utterly mystified as to how that happened. next task: beat the crap out of libmsvcr80.a and /mingw/include/*.h, repeat until success. l. From lkcl at lkcl.net Wed Jan 21 12:10:38 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Wed, 21 Jan 2009 11:10:38 +0000 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: <49763FC6.9090303@v.loewis.de> References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> <49763FC6.9090303@v.loewis.de> Message-ID: On Tue, Jan 20, 2009 at 9:19 PM, "Martin v. L?wis" wrote: >>> That's a non-starter for anyone who incorporates Python in an existing >>> MSVC-based development environment. >> >> surely incorporating libpython2.5.dll.a or libpython2.6.dll.a, along >> with the .def and the importlib that's generated with dlldump, unless >> i'm missing something, would be a simple matter, yes? > > Not exactly sure what this is, but I believe Python *already* includes > such a thing. sorry, martin - i thought the win32 builds generated python25.lib, python25.dll and python25.def so as to fit into the 8.3 filename convention. From lkcl at lkcl.net Wed Jan 21 12:42:49 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Wed, 21 Jan 2009 11:42:49 +0000 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> <49763BA7.6070802@gmail.com> Message-ID: > next task: beat the crap out of libmsvcr80.a and /mingw/include/*.h, > repeat until success. https://sourceforge.net/tracker/index.php?func=detail&aid=2134161&group_id=2435&atid=352435 roumen, it looks like you've been and done that, already - thank you! From rdmurray at bitdance.com Wed Jan 21 14:39:26 2009 From: rdmurray at bitdance.com (rdmurray at bitdance.com) Date: Wed, 21 Jan 2009 08:39:26 -0500 (EST) Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <200901212146.46821.steve@pearwood.info> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <200901210955.59699.steve@pearwood.info> <200901212146.46821.steve@pearwood.info> Message-ID: On Wed, 21 Jan 2009 at 21:46, Steven D'Aprano wrote: > On Wed, 21 Jan 2009 03:10:24 pm Terry Reedy wrote: >> It is a carefully designed 1 to >> 1 transformation between multiple nested statements and a single >> expression. > > I'm sure that correspondence is obvious to some, but it wasn't obvious > to me, and I don't suppose I'm the only one. That's not a criticism of > the current syntax. Far from it -- the current syntax is excellent, > regardless of whether or not you notice that it corresponds to a > if-loop nested inside a for-loop with the contents rotated outside. It wasn't obvious to me until I read this thread, but now that I know about it I feel a huge sense of relief. I was never comfortable with extending (or reading an extension of) a list comprehension beyond the obvious yield/for/if pattern before. Now I have a reliable tool to understand any complex list comprehension. I would not want to lose that! >> But this proposal ignores and breaks that. Using 'while >> x' to mean 'if x: break' *is*, to me, 'ad hoc'. > > But it doesn't mean that. The proposed "while x" has very similar > semantics to the "while x" in a while-loop: break when *not* x. Half right. 'while x' in the proposed syntax is equivalent to 'if not x: break', But IMO it goes too far to say it has similar semantics to 'while x' in a while loop. Neither while x*x<4: for x in range(10): yield x*x nor for x in range(10): while x*x<4: yield x*x are the same as for x in range(10): if not x*x<4: break yield x*x I understand that you are saying that 'while x' is used in the same logical sense ("take a different action when x is no longer true"), but that I don't feel that that is enough to say that it has similar semantics. Or, perhaps more accurately, it is just similar enough to be very confusing because it is also different enough to be very surprising. The semantics of 'while' in python includes the bit about creating a loop, and does _not_ include executing a 'break' in the surrounding loop. To give 'while' this new meaning would be, IMO, un-pythonic. (If python had a 'for/while' construct, it would be a different story...and then it would probably already be part of the list comprehension syntax.) >> So I detest the proposed change. I find it ugly and anti-Pythonic. I'd say +1 except that I don't find it ugly, just un-Pythonic :) --RDM From gerald.britton at gmail.com Wed Jan 21 15:38:01 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Wed, 21 Jan 2009 09:38:01 -0500 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <200901210955.59699.steve@pearwood.info> <200901212146.46821.steve@pearwood.info> Message-ID: <5d1a32000901210638y1fc361d0s893a58988b7e9a7a@mail.gmail.com> FWIW, there are a few historic languages that implement a compound for-loop: Algol 68, PL/I, SAS et al allow constructs that, if translated to an equivalent (currently invalid) Python-style syntax would look like this" for in while : Some also allow for an "until" keyword. I'm not suggesting that we need to do this in Python; it's just interesting to note that there is some precedent for this approach. On Wed, Jan 21, 2009 at 8:39 AM, wrote: > On Wed, 21 Jan 2009 at 21:46, Steven D'Aprano wrote: >> >> On Wed, 21 Jan 2009 03:10:24 pm Terry Reedy wrote: >>> >>> It is a carefully designed 1 to >>> 1 transformation between multiple nested statements and a single >>> expression. >> >> I'm sure that correspondence is obvious to some, but it wasn't obvious >> to me, and I don't suppose I'm the only one. That's not a criticism of >> the current syntax. Far from it -- the current syntax is excellent, >> regardless of whether or not you notice that it corresponds to a >> if-loop nested inside a for-loop with the contents rotated outside. > > It wasn't obvious to me until I read this thread, but now that I know > about it I feel a huge sense of relief. I was never comfortable with > extending (or reading an extension of) a list comprehension beyond the > obvious yield/for/if pattern before. Now I have a reliable tool to > understand any complex list comprehension. I would not want to lose that! > >>> But this proposal ignores and breaks that. Using 'while >>> x' to mean 'if x: break' *is*, to me, 'ad hoc'. >> >> But it doesn't mean that. The proposed "while x" has very similar >> semantics to the "while x" in a while-loop: break when *not* x. > > Half right. 'while x' in the proposed syntax is equivalent to 'if not x: > break', But IMO it goes too far to say it has similar semantics to 'while > x' in a while loop. Neither > > while x*x<4: > for x in range(10): > yield x*x > > nor > > for x in range(10): > while x*x<4: > yield x*x > > are the same as > > for x in range(10): > if not x*x<4: break > yield x*x > > I understand that you are saying that 'while x' is used in the same > logical sense ("take a different action when x is no longer true"), > but that I don't feel that that is enough to say that it has similar > semantics. Or, perhaps more accurately, it is just similar enough to be > very confusing because it is also different enough to be very surprising. > The semantics of 'while' in python includes the bit about creating a > loop, and does _not_ include executing a 'break' in the surrounding loop. > To give 'while' this new meaning would be, IMO, un-pythonic. (If python > had a 'for/while' construct, it would be a different story...and then > it would probably already be part of the list comprehension syntax.) > >>> So I detest the proposed change. I find it ugly and anti-Pythonic. > > I'd say +1 except that I don't find it ugly, just un-Pythonic :) > > --RDM > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/gerald.britton%40gmail.com > From algorias at yahoo.com Wed Jan 21 16:27:38 2009 From: algorias at yahoo.com (Vitor Bosshard) Date: Wed, 21 Jan 2009 07:27:38 -0800 (PST) Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <200901210955.59699.steve@pearwood.info> <200901212146.46821.steve@pearwood.info> <5d1a32000901210638y1fc361d0s893a58988b7e9a7a@mail.gmail.com> Message-ID: <549056.3449.qm@web54409.mail.yahoo.com> ----- Mensaje original ---- > De: Gerald Britton > Para: rdmurray at bitdance.com > CC: python-dev at python.org > Enviado: mi?rcoles, 21 de enero, 2009 11:38:01 > Asunto: Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions > > FWIW, there are a few historic languages that implement a compound > for-loop:? Algol 68, PL/I, SAS et al allow constructs that, if > translated to an equivalent (currently invalid) Python-style syntax > would look like this" > > for in while : > ? > ? > > Some also allow for an "until" keyword.? I'm not suggesting that we > need to do this in Python; it's just interesting to note that there is > some precedent for this approach. > Well,?you could propose changing the for loop syntax (and by extension comprehensions and generators). It's?a much more radical proposal, but?it does keep?consistency across the board, which is one of the major flaws of the PEP in its current form. BTW, there is already an "until" keyword in python, it's called "while not" ;) Vitor ?Todo sobre la Liga Mexicana de f?tbol! Estadisticas, resultados, calendario, fotos y m?s:< http://espanol.sports.yahoo.com/ From gerald.britton at gmail.com Wed Jan 21 16:51:39 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Wed, 21 Jan 2009 10:51:39 -0500 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <549056.3449.qm@web54409.mail.yahoo.com> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <200901210955.59699.steve@pearwood.info> <200901212146.46821.steve@pearwood.info> <5d1a32000901210638y1fc361d0s893a58988b7e9a7a@mail.gmail.com> <549056.3449.qm@web54409.mail.yahoo.com> Message-ID: <5d1a32000901210751l2ef9e7fdy378293d937b91138@mail.gmail.com> OK then, what is the feeling out there about extending the "for" syntax in general (and by extension list comprehensions and generator expressions) by adding an optional while clause like this: for in [while [ | not ]: The predicate would be tested after an is taken from and before execution of the . If the predicate evaluates to false, StopIteration would be raised. This construct would be equivalent to: for in : if [not | ]: break Note: this is beyond what I was thinking in the first place, but has arisen from the ensuing discussion. Note 2: this would cover itertools.takewhile but not itertools.dropwhile, AFAICS On Wed, Jan 21, 2009 at 10:27 AM, Vitor Bosshard wrote: > ----- Mensaje original ---- >> De: Gerald Britton >> Para: rdmurray at bitdance.com >> CC: python-dev at python.org >> Enviado: mi?rcoles, 21 de enero, 2009 11:38:01 >> Asunto: Re: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions >> >> FWIW, there are a few historic languages that implement a compound >> for-loop: Algol 68, PL/I, SAS et al allow constructs that, if >> translated to an equivalent (currently invalid) Python-style syntax >> would look like this" >> >> for in while : >> >> >> >> Some also allow for an "until" keyword. I'm not suggesting that we >> need to do this in Python; it's just interesting to note that there is >> some precedent for this approach. >> > > Well, you could propose changing the for loop syntax (and by extension comprehensions and generators). It's a much more radical proposal, but it does keep consistency across the board, which is one of the major flaws of the PEP in its current form. > > BTW, there is already an "until" keyword in python, it's called "while not" ;) > > > Vitor > > > ?Todo sobre la Liga Mexicana de f?tbol! Estadisticas, resultados, calendario, fotos y m?s:< > http://espanol.sports.yahoo.com/ > From ludvig at lericson.se Wed Jan 21 18:32:37 2009 From: ludvig at lericson.se (Ludvig Ericson) Date: Wed, 21 Jan 2009 18:32:37 +0100 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <5d1a32000901210929s66b8233ew32310e3a6a4f392f@mail.gmail.com> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <200901210955.59699.steve@pearwood.info> <200901212146.46821.steve@pearwood.info> <5d1a32000901210638y1fc361d0s893a58988b7e9a7a@mail.gmail.com> <549056.3449.qm@web54409.mail.yahoo.com> <5d1a32000901210751l2ef9e7fdy378293d937b91138@mail.gmail.com> <5d1a32000901210929s66b8233ew32310e3a6a4f392f@mail.gmail.com> Message-ID: <10286A07-55AB-4BA3-9818-DAE5CD5999FD@lericson.se> The following was supposed to go to the list: 18:29 Gerald Britton: > Yes you could have long lines, but you wouldn't have to use it. You > could still code it up as you would today. It might be convenient for > shorter expressions though. > > 12:12 PM Ludvig Ericson: >> On Jan 21, 2009, at 16:51, Gerald Britton wrote: >> >>> for in [while [ | not ]: >>> >> >> (Sorry for just sort of popping in to the list.) >> >> That would make for some very, very long lines. I for one wouldn't >> like >> seeing: >> >>>>> for cart_item in current_user.cart.new_items \ >> ... while cart_item.cant_imagine_more: >> ... >> >> I realize that the other approach--an immediate if-break--wouldn't >> look >> great either, but it wouldn't be cramming that much stuff into one >> line. From aahz at pythoncraft.com Wed Jan 21 18:54:37 2009 From: aahz at pythoncraft.com (Aahz) Date: Wed, 21 Jan 2009 09:54:37 -0800 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <5d1a32000901210751l2ef9e7fdy378293d937b91138@mail.gmail.com> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <200901210955.59699.steve@pearwood.info> <200901212146.46821.steve@pearwood.info> <5d1a32000901210638y1fc361d0s893a58988b7e9a7a@mail.gmail.com> <549056.3449.qm@web54409.mail.yahoo.com> <5d1a32000901210751l2ef9e7fdy378293d937b91138@mail.gmail.com> Message-ID: <20090121175437.GA6175@panix.com> On Wed, Jan 21, 2009, Gerald Britton wrote: > > OK then, what is the feeling out there about extending the "for" > syntax in general (and by extension list comprehensions and generator > expressions) by adding an optional while clause like this: > > for in [while [ | not ]: > What I suggest is that your ideas need more thought before bringing them to python-dev -- I think you should either go back to python-ideas or try comp.lang.python -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From aahz at pythoncraft.com Wed Jan 21 19:00:14 2009 From: aahz at pythoncraft.com (Aahz) Date: Wed, 21 Jan 2009 10:00:14 -0800 Subject: [Python-Dev] PEP 8 and constants Message-ID: <20090121180014.GA16447@panix.com> In comp.lang.python, there has been some discussion of the fact that there are no guidelines in PEP 8 for constants: http://groups.google.com/group/comp.lang.python/browse_thread/thread/ed964fe8ad6da7b7 Is there any sentiment that PEP 8 should be updated to reflect the common usage of ALL_CAPS for constants? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From guido at python.org Wed Jan 21 19:04:19 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Jan 2009 10:04:19 -0800 Subject: [Python-Dev] Copyright notices in modules In-Reply-To: References: <94F1BCD4C7B448C8AE5C8C1FA34DD661@RaymondLaptop1> <497596B1.4060600@egenix.com> <9E704C0F455642AE974D4DAC51534F3F@RaymondLaptop1> Message-ID: On Tue, Jan 20, 2009 at 11:01 PM, Terry Reedy wrote: > Guido van Rossum wrote: >> >> 2009/1/20 Raymond Hettinger : >>> >>> I'm at a loss of why the notice needs to be there at all. >> >> There's a difference between contributing a whole file and >> contributing a patch. Patches do not require copyright notices. Whole >> files do. This is not affected by later edits to the file. > > In my comment, I postulated the situation where the patch consisted of > merging in another, independently copyrighted, 'whole file'. Perhaps this > has mostly been a non-existent situation and therefor moot. > > One real situation I was thinking of, unconnected to Google as far as I am > aware, is the case of two third-party IP6 modules and the suggestion that > they be merged into one stdlib module. If that were accomplished by > committing one and merging the other in a patch, it would be unfair (and > untrue) to have just one copyright notice. Of course, in this case, I hope > the two authors work everything out between themselves first before any > submission. There's nothing top stop you from having multiple copyrights in one file, when that represents the rights of the original authors fairly. > I completely understand about strongly preferring programming to lawyer > consultation ;-). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Jan 21 19:48:59 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Jan 2009 10:48:59 -0800 Subject: [Python-Dev] PEP 8 and constants In-Reply-To: <20090121180014.GA16447@panix.com> References: <20090121180014.GA16447@panix.com> Message-ID: On Wed, Jan 21, 2009 at 10:00 AM, Aahz wrote: > In comp.lang.python, there has been some discussion of the fact that > there are no guidelines in PEP 8 for constants: > > http://groups.google.com/group/comp.lang.python/browse_thread/thread/ed964fe8ad6da7b7 > > Is there any sentiment that PEP 8 should be updated to reflect the common > usage of ALL_CAPS for constants? It makes sense to codify this usage in PEP 8. I think it's by far the most common convention adopted by projects that set their own style guide based on PEP 8 with local additions. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tseaver at palladion.com Wed Jan 21 20:02:55 2009 From: tseaver at palladion.com (Tres Seaver) Date: Wed, 21 Jan 2009 14:02:55 -0500 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <549056.3449.qm@web54409.mail.yahoo.com> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <200901210955.59699.steve@pearwood.info> <200901212146.46821.steve@pearwood.info> <5d1a32000901210638y1fc361d0s893a58988b7e9a7a@mail.gmail.com> <549056.3449.qm@web54409.mail.yahoo.com> Message-ID: <4977715F.8030203@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Vitor Bosshard wrote: >> Some also allow for an "until" keyword. I'm not suggesting that we >> need to do this in Python; it's just interesting to note that there is >> some precedent for this approach. >> > > Well, you could propose changing the for loop syntax (and by > extension comprehensions and generators). It's a much more radical proposal, but > it does keep consistency across the board, which is one of the major > flaws of the PEP in its current form. > > BTW, there is already an "until" keyword in python, it's called "while not" ;) 'until' is used at least in some languages (Pascal, Modula*, maybe Ada?) for a "terminate at bottom" loop (one guaranteed to run at least once): in such cases, the predicate has the negative sense. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJd3Ff+gerLs4ltQ4RAuOQAJ47EA8Cf1KPMdNiZTBiJqweiUNZBgCgsVrc 38fgphB+hjdnTblAQT8Q5tA= =SeEn -----END PGP SIGNATURE----- From tseaver at palladion.com Wed Jan 21 20:02:55 2009 From: tseaver at palladion.com (Tres Seaver) Date: Wed, 21 Jan 2009 14:02:55 -0500 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <549056.3449.qm@web54409.mail.yahoo.com> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <200901210955.59699.steve@pearwood.info> <200901212146.46821.steve@pearwood.info> <5d1a32000901210638y1fc361d0s893a58988b7e9a7a@mail.gmail.com> <549056.3449.qm@web54409.mail.yahoo.com> Message-ID: <4977715F.8030203@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Vitor Bosshard wrote: >> Some also allow for an "until" keyword. I'm not suggesting that we >> need to do this in Python; it's just interesting to note that there is >> some precedent for this approach. >> > > Well, you could propose changing the for loop syntax (and by > extension comprehensions and generators). It's a much more radical proposal, but > it does keep consistency across the board, which is one of the major > flaws of the PEP in its current form. > > BTW, there is already an "until" keyword in python, it's called "while not" ;) 'until' is used at least in some languages (Pascal, Modula*, maybe Ada?) for a "terminate at bottom" loop (one guaranteed to run at least once): in such cases, the predicate has the negative sense. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJd3Ff+gerLs4ltQ4RAuOQAJ47EA8Cf1KPMdNiZTBiJqweiUNZBgCgsVrc 38fgphB+hjdnTblAQT8Q5tA= =SeEn -----END PGP SIGNATURE----- From martin at v.loewis.de Wed Jan 21 20:42:17 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Wed, 21 Jan 2009 20:42:17 +0100 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> <49763FC6.9090303@v.loewis.de> Message-ID: <49777A99.9010607@v.loewis.de> > sorry, martin - i thought the win32 builds generated python25.lib, > python25.dll Correct. > and python25.def No. > so as to fit into the 8.3 filename convention. No. It generates python25.lib because that's the import library for python25.dll. It calls it python25.dll because the lib prefix is atypical for the platform, and also redundant (DLL means "dynamic link library"). The Python binary installer also includes libpython25.a, for use with mingw32. Regards, Martin From rowen at u.washington.edu Wed Jan 21 20:47:52 2009 From: rowen at u.washington.edu (Russell E. Owen) Date: Wed, 21 Jan 2009 11:47:52 -0800 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <200901210955.59699.steve@pearwood.info> <200901212146.46821.steve@pearwood.info> Message-ID: In article , rdmurray at bitdance.com wrote: >... > I understand that you are saying that 'while x' is used in the same > logical sense ("take a different action when x is no longer true"), > but that I don't feel that that is enough to say that it has similar > semantics. Or, perhaps more accurately, it is just similar enough to be > very confusing because it is also different enough to be very surprising. > The semantics of 'while' in python includes the bit about creating a > loop, and does _not_ include executing a 'break' in the surrounding loop. > To give 'while' this new meaning would be, IMO, un-pythonic. (If python > had a 'for/while' construct, it would be a different story...and then > it would probably already be part of the list comprehension syntax.) I agree. I feel that the term "while" is a poor choice for "when this is no longer true then stop". It sounds more like a synonym for "if" to me. I would be much more comfortable using "until" (in the opposite sense to the proposed "while"); it clearly implies "we're done so stop". I don't know if it's a feature that is really useful, but I do think it would be transparent: code that used it would be easily understood. -- Russell From lkcl at lkcl.net Wed Jan 21 20:50:28 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Wed, 21 Jan 2009 19:50:28 +0000 Subject: [Python-Dev] progress: compiling python2.5 under msys (specifically but not exclusively under wine) with msvcr80 Message-ID: this is a progress report on compiling python using entirely free software tools, no proprietary compilers or operating systems involved, yet still linking and successfully running with msvcr80 assemblies. manifests and rc files, which are compiled to internal resources, have been added. various sections which are uniquely identifed by _MSC_VER >= 1400 etc have had to be enabled with corresponding MSVCRT_VERSION >= 0x0800 - in particular, signal handling (PyOS_getsig()). currently, under wine with msvcr80, there looks like there is a bug with a common theme related to threads, but here's a short list: test_array.py is blocking, test_bz2.py is hanging and test_cmd_line.py causes a segfault; test_ctypes is _still_ a bundle of fun. for those people who use native win32 platforms who are compiling up this code, you should have better luck. significantly, the wine developers have been absolutely fantastic, and have fixed several bugs in wine, sometimes within hours, that were found as a result of running the extremely comprehensive python regression tests. the python regression tests are a credit to the collaborative incremental improvement process of free software development. i look forward to seeing the same incremental improvement applied to the development of python, evidence of which would be clearly seen by the acceptance of one of the following patches, one of which is dated 2003: http://bugs.python.org/issue3754 http://bugs.python.org/issue841454 http://bugs.python.org/issue3871 http://bugs.python.org/issue4954 http://bugs.python.org/issue5010 for those people wishing to track and contribute to the development of python for win32 using entirely free software tools, either under wine or native windows, there is a git repository, here, slightly illogically named pythonwine because that's where i started from (cross-compiling python under wine, so i could get at the wine registry from python). obviously, since then, things have... moved on :) http://github.com/lkcl/pythonwine/tree/python_2.5.2_wine l. From lkcl at lkcl.net Wed Jan 21 21:08:05 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Wed, 21 Jan 2009 20:08:05 +0000 Subject: [Python-Dev] progress: compiling python2.5 under msys (specifically but not exclusively under wine) with msvcr80 In-Reply-To: References: Message-ID: > http://bugs.python.org/issue5010 correction: that's http://bugs.python.org/issue5026 apologies for the mix-up. also,for the msvcrt80 build, it is _essential_ that you use a patched version of mingw32-runtime, see: https://sourceforge.net/tracker/index.php?func=detail&aid=2134161&group_id=2435&atid=352435 libmsvcr80.a mistakenly thinks that _fstat exists (it doesn't - only _fstat32 does, and many more). it's quite straightforward to rebuild - just remember to run ./configure --prefix=/mingw and if you want to revert just reinstall mingw runtime .exe l. From tjreedy at udel.edu Wed Jan 21 21:42:05 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 21 Jan 2009 15:42:05 -0500 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <200901212146.46821.steve@pearwood.info> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <200901210955.59699.steve@pearwood.info> <200901212146.46821.steve@pearwood.info> Message-ID: Steven D'Aprano wrote: > On Wed, 21 Jan 2009 03:10:24 pm Terry Reedy wrote: >> Steven D'Aprano wrote: >>> The three clauses are neither in the same order, nor are they in >>> reverse order. >> They are in the same order but rotated, with the last brought around >> to the front to emphasize it. Did you really not notice that either? > > There are only three items, of course I noticed that there is *some* > rearrangement of the first that leads to the second. Out of the six > possible permutations of three items, they can all be described in > terms of some sort of reflection, rotation or swap. Irrelevant. *Every* comprehension, no matter how many clauses, rotates the expression from last to first and keeps the clauses in the same order with the same meaning. Simple rule. >> Ironically, in a thread cross-posted on c.l.p and elsewhere, someone >> just labeled Python's comprehension syntax as "ad hoc syntax soup". > > Is that Xah Lee? It sounds like the sort of thing he'd say. It was the thread he started, but not him. He contributed other idiocies. Terry Jan Reedy From lkcl at lkcl.net Wed Jan 21 22:07:30 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Wed, 21 Jan 2009 21:07:30 +0000 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: <49777A99.9010607@v.loewis.de> References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> <49763FC6.9090303@v.loewis.de> <49777A99.9010607@v.loewis.de> Message-ID: On Wed, Jan 21, 2009 at 7:42 PM, "Martin v. L?wis" wrote: >> sorry, martin - i thought the win32 builds generated python25.lib, >> python25.dll > > Correct. > >> and python25.def > > No. > >> so as to fit into the 8.3 filename convention. > > No. It generates python25.lib because that's the import library > for python25.dll. It calls it python25.dll because the lib prefix > is atypical for the platform, and also redundant (DLL means > "dynamic link library"). > > The Python binary installer also includes libpython25.a, for use > with mingw32. ok, so - different from what's being generated by ./configure under msys under wine or native win32 - what's being generated (libpython 2 . 5 . a and libpython 2 . 5 . dll . a) is more akin to the cygwin environment. therefore, there's absolutely no doubt that the two are completely different. and on that basis, would i be correct in thinking that you _can't_ go linking or building modules or any python win32 code for one and have a hope in hell of using it on the other, and that you would _have_ to rebuild e.g. numpy for use with a mingw32-msys-built version of python? or, is the .pyd loading a bit cleverer (or perhaps a bit less cleverer) than i'm expecting it to be? l. From martin at v.loewis.de Wed Jan 21 22:13:26 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Wed, 21 Jan 2009 22:13:26 +0100 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> <49763FC6.9090303@v.loewis.de> <49777A99.9010607@v.loewis.de> Message-ID: <49778FF6.8070608@v.loewis.de> > ok, so - different from what's being generated by ./configure under > msys under wine or native win32 - what's being generated (libpython 2 > . 5 . a and libpython 2 . 5 . dll . a) is more akin to the cygwin > environment. > > therefore, there's absolutely no doubt that the two are completely different. > > and on that basis, would i be correct in thinking that you _can't_ go > linking or building modules or any python win32 code for one and have > a hope in hell of using it on the other, and that you would _have_ to > rebuild e.g. numpy for use with a mingw32-msys-built version of > python? I can't comment on that, because I don't know what your port does. Does it not produce a .dll containing the majority of Python? And is that not called python25.dll? Regards, Martin From cesare.dimauro at a-tono.com Wed Jan 21 21:48:55 2009 From: cesare.dimauro at a-tono.com (Cesare Di Mauro) Date: Wed, 21 Jan 2009 21:48:55 +0100 (CET) Subject: [Python-Dev] progress: compiling python2.5 under msys (specifically but not exclusively under wine) with msvcr80 In-Reply-To: References: Message-ID: <50019.151.53.150.247.1232570935.squirrel@webmail1.pair.com> Have you made some benchmarks like pystone? Cheers, Cesare On Wed, Jan 21, 2009 08:50PM, Luke Kenneth Casson Leighton wrote: > this is a progress report on compiling python using entirely free > software tools, no proprietary compilers or operating systems > involved, yet still linking and successfully running with msvcr80 > assemblies. manifests and rc files, which are compiled to internal > resources, have been added. > various sections which are uniquely identifed by _MSC_VER >= 1400 etc > have had to be enabled with corresponding MSVCRT_VERSION >= 0x0800 - > in particular, signal handling (PyOS_getsig()). > > currently, under wine with msvcr80, there looks like there is a bug > with a common theme related to threads, but here's a short list: > test_array.py is blocking, test_bz2.py is hanging and test_cmd_line.py > causes a segfault; test_ctypes is _still_ a bundle of fun. for those > people who use native win32 platforms who are compiling up this code, > you should have better luck. > > significantly, the wine developers have been absolutely fantastic, and > have fixed several bugs in wine, sometimes within hours, that were > found as a result of running the extremely comprehensive python > regression tests. > > the python regression tests are a credit to the collaborative > incremental improvement process of free software development. > > i look forward to seeing the same incremental improvement applied to > the development of python, evidence of which would be clearly seen by > the acceptance of one of the following patches, one of which is dated > 2003: > http://bugs.python.org/issue3754 > http://bugs.python.org/issue841454 > http://bugs.python.org/issue3871 > http://bugs.python.org/issue4954 > http://bugs.python.org/issue5010 > > for those people wishing to track and contribute to the development of > python for win32 using entirely free software tools, either under wine > or native windows, there is a git repository, here, slightly > illogically named pythonwine because that's where i started from > (cross-compiling python under wine, so i could get at the wine > registry from python). obviously, since then, things have... moved on > :) > > http://github.com/lkcl/pythonwine/tree/python_2.5.2_wine > > l. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/cesare.dimauro%40a-tono.com > > From g.brandl at gmx.net Wed Jan 21 22:53:42 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 21 Jan 2009 22:53:42 +0100 Subject: [Python-Dev] Questions/comments on documentation formatting In-Reply-To: References: Message-ID: Brett Cannon schrieb: > I have been writing up the initial docs for importlib and four things struck me: > > 1. Why is three space indents the preferred indentation level? As said, it matches directive content with directive headers nicely. Ben's solution is nice as well, but now that we have 3-space I'd rather we stick with 3-space (however, if you don't care, I'll not reformat 4-space indents :) Code in code blocks should use 4-space as usual. > 2. Should we start using function annotations? It's not really supported yet by Sphinx. Also, I don't know if it makes too much sense, given that it will reinforce the thinking of annotations as type declarations. > 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [, > c=None]])``) really necessary when default argument values are > present? And do we really need to nest the brackets when it is obvious > that having on optional argument means the rest are optional as well? We've discussed that once on the doc-SIG, and I agreed that the bracketing is not really pretty, especially if it's heavily nested. Python functions where it makes sense should use the default-value syntax, while C functions without kwargs support need to keep the brackets. Making this consistent throughout the docs is no small task, of course. > 4. The var directive is not working even though the docs list it as a > valid directive; so is it still valid and something is broken, or the > docs need to be updated? (First, you're confusing "directive" and "role" which led to some confusion on Benjamin's part.) Where is a "var" role documented? If it is, it is a bug. Georg From bugtrack at roumenpetrov.info Wed Jan 21 23:51:36 2009 From: bugtrack at roumenpetrov.info (Roumen Petrov) Date: Thu, 22 Jan 2009 00:51:36 +0200 Subject: [Python-Dev] progress: compiling python2.5 under msys (specifically but not exclusively under wine) with msvcr80 In-Reply-To: References: Message-ID: <4977A6F8.4060904@roumenpetrov.info> Terry Reedy wrote: > Luke Kenneth Casson Leighton wrote: > >> i look forward to seeing the same incremental improvement applied to >> the development of python, evidence of which would be clearly seen by >> the acceptance of one of the following patches, one of which is dated >> 2003: > >> http://bugs.python.org/issue841454 > > Against 2.3, rejected due to dependence on SCons. > Also appears to have been incomplete, needing more work. No it was complete but use SCons. Most of changes changes in code you will see again in 3871. >> http://bugs.python.org/issue3754 > > Open by Roumen Petrov, no review, see below. This is again request and the patch is for trunk. It share common idea with 841454:Cross building python for mingw32:Andreas Ames (yxcv):2003-11-13 14:31 1006238:Cross compile patch:Daniel Goertzen (goertzen):2004-08-09 22:05 1597850:Cross compiling patches for MINGW hanwen:2006-11-16 16:57 >> http://bugs.python.org/issue3871 > > Open, from same submitter, only (minor) review by you. > Does this supercede 3754? No. It share common changes to code with 841454, 1006238, 1412448, 1597850. May be 1597850 and 3871 supercede 1412448. The issue3871 raise questions (and include solution/work around) related to: 2942 - mingw/cygwin do not accept asm file as extension source 2445 - Use The CygwinCCompiler Under Cygwin 1706863 - Failed to build Python 2.5.1 with sqlite3 Also issues related to LDFLAGS: 4010 - configure options don't trickle down to distutils 1628484 - Python 2.5 64 bit compile fails on Solaris 10/gcc 4.1.1 [SNIP] From aahz at pythoncraft.com Thu Jan 22 00:39:24 2009 From: aahz at pythoncraft.com (Aahz) Date: Wed, 21 Jan 2009 15:39:24 -0800 Subject: [Python-Dev] Where is Fred Drake? Message-ID: <20090121233924.GA10141@panix.com> Mail to fdrake at acm.org is bouncing; I don't know whether it's a temporary failure. Does anyone have another address for him? ----- Forwarded message from Mail Delivery System ----- > Date: Wed, 21 Jan 2009 22:48:49 +0100 (CET) > From: Mail Delivery System > Subject: Undelivered Mail Returned to Sender > To: webmaster at python.org > Content-Description: Notification > This is the mail system at host bag.python.org. > > I'm sorry to have to inform you that your message could not > be delivered to one or more recipients. It's attached below. > > For further assistance, please send mail to postmaster. > > If you do so, please include this problem report. You can > delete your own text from the attached returned message. > > The mail system > > : host acm.org.s7a1.psmtp.com[64.18.6.14] said: 550 No such > user - psmtp (in reply to RCPT TO command) Content-Description: Delivery report > Reporting-MTA: dns; bag.python.org > X-Postfix-Queue-ID: 24FF41E404E > X-Postfix-Sender: rfc822; webmaster at python.org > Arrival-Date: Wed, 21 Jan 2009 22:48:48 +0100 (CET) > > Final-Recipient: rfc822; fdrake at acm.org > Original-Recipient: rfc822;fdrake at acm.org > Action: failed > Status: 5.0.0 > Remote-MTA: dns; acm.org.s7a1.psmtp.com > Diagnostic-Code: smtp; 550 No such user - psmtp ----- End forwarded message ----- -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From brett at python.org Thu Jan 22 00:56:19 2009 From: brett at python.org (Brett Cannon) Date: Wed, 21 Jan 2009 15:56:19 -0800 Subject: [Python-Dev] Questions/comments on documentation formatting In-Reply-To: References: Message-ID: On Wed, Jan 21, 2009 at 13:53, Georg Brandl wrote: > Brett Cannon schrieb: >> I have been writing up the initial docs for importlib and four things struck me: >> >> 1. Why is three space indents the preferred indentation level? > > As said, it matches directive content with directive headers nicely. > Ben's solution is nice as well, but now that we have 3-space I'd rather > we stick with 3-space (however, if you don't care, I'll not reformat > 4-space indents :) > =) OK. > Code in code blocks should use 4-space as usual. > >> 2. Should we start using function annotations? > > It's not really supported yet by Sphinx. Also, I don't know if it makes > too much sense, given that it will reinforce the thinking of annotations > as type declarations. > Fine by me. >> 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [, >> c=None]])``) really necessary when default argument values are >> present? And do we really need to nest the brackets when it is obvious >> that having on optional argument means the rest are optional as well? > > We've discussed that once on the doc-SIG, and I agreed that the bracketing > is not really pretty, especially if it's heavily nested. Python functions > where it makes sense should use the default-value syntax, while C functions > without kwargs support need to keep the brackets. > That was my thinking. > Making this consistent throughout the docs is no small task, of course. > Nope, but perhaps all new docs should stop their use. >> 4. The var directive is not working even though the docs list it as a >> valid directive; so is it still valid and something is broken, or the >> docs need to be updated? > > (First, you're confusing "directive" and "role" which led to some confusion > on Benjamin's part.) > > Where is a "var" role documented? If it is, it is a bug. http://docs.python.org/dev/3.0/documenting/markup.html#inline-markup. -Brett From benji at benjiyork.com Thu Jan 22 00:56:52 2009 From: benji at benjiyork.com (Benji York) Date: Wed, 21 Jan 2009 18:56:52 -0500 Subject: [Python-Dev] Where is Fred Drake? In-Reply-To: <20090121233924.GA10141@panix.com> References: <20090121233924.GA10141@panix.com> Message-ID: On Wed, Jan 21, 2009 at 6:39 PM, Aahz wrote: > Mail to fdrake at acm.org is bouncing; I don't know whether it's a > temporary failure. Does anyone have another address for him? /me channels Fred: Use freddrake at verizon.net until the acm.org account is back up. -- Benji York From cs at zip.com.au Thu Jan 22 01:40:10 2009 From: cs at zip.com.au (Cameron Simpson) Date: Thu, 22 Jan 2009 11:40:10 +1100 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <4977715F.8030203@palladion.com> Message-ID: <20090122004010.GA14090@cskk.homeip.net> On 21Jan2009 14:02, Tres Seaver wrote: | Vitor Bosshard wrote: | > BTW, there is already an "until" keyword in python, it's called "while not" ;) | | 'until' is used at least in some languages (Pascal, Modula*, maybe Ada?) | for a "terminate at bottom" loop (one guaranteed to run at least once): | in such cases, the predicate has the negative sense. This is a particular flavour of "do ... while" which just happens to read a little better in English. It does sometimes bother me that Python doesn't have do...while when I find my self replicating the loop bottom above the loop. Back at uni we had to implement a small language in our compilers class and the lecturer had specified a proper generic while loop, thus: loop: suite while invariant suite endloop I think the keywords were better than above, but it neatly handled the fact that the while-test must often be preceeded by some setup that would be replicated at the loop bottom in Python and many other languages: setup-invariant-state while test-invariant do stuff setup-invariant-state of which the bare while... and converse do...while loops are particular extremes. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Why doesn't DOS ever say EXCELLENT command or filename? From guido at python.org Thu Jan 22 04:03:53 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Jan 2009 19:03:53 -0800 Subject: [Python-Dev] PEP 8 and constants In-Reply-To: <1afaf6160901211842x57963b26q796bca35975c69e1@mail.gmail.com> References: <20090121180014.GA16447@panix.com> <1afaf6160901211842x57963b26q796bca35975c69e1@mail.gmail.com> Message-ID: Yes, that's what I commonly see. On Wed, Jan 21, 2009 at 6:42 PM, Benjamin Peterson wrote: > On Wed, Jan 21, 2009 at 12:48 PM, Guido van Rossum wrote: >> On Wed, Jan 21, 2009 at 10:00 AM, Aahz wrote: >>> In comp.lang.python, there has been some discussion of the fact that >>> there are no guidelines in PEP 8 for constants: >>> >>> http://groups.google.com/group/comp.lang.python/browse_thread/thread/ed964fe8ad6da7b7 >>> >>> Is there any sentiment that PEP 8 should be updated to reflect the common >>> usage of ALL_CAPS for constants? >> >> It makes sense to codify this usage in PEP 8. I think it's by far the >> most common convention adopted by projects that set their own style >> guide based on PEP 8 with local additions. > > Do you suggest underscores between words in constants as with other names? > > > > -- > Regards, > Benjamin > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Thu Jan 22 04:22:55 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 21 Jan 2009 22:22:55 -0500 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <20090122004010.GA14090@cskk.homeip.net> References: <4977715F.8030203@palladion.com> <20090122004010.GA14090@cskk.homeip.net> Message-ID: Cameron Simpson wrote: > > Back at uni we had to implement a small language in our compilers class > and the lecturer had specified a proper generic while loop, thus: > > loop: > suite > while invariant > suite > endloop In Python, that is spelled while True: suite if not invariant: break suite > I think the keywords were better than above, but it neatly handled the > fact that the while-test must often be preceeded by some setup that > would be replicated at the loop bottom in Python and many other languages: > > setup-invariant-state > while test-invariant > do stuff > setup-invariant-state Good Python programmers do not repeat the setup code like this. See the proper say-it-once way above. This discussion belongs back on Python ideas, where it began and should have stayed. tjr From benjamin at python.org Thu Jan 22 04:34:24 2009 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 21 Jan 2009 21:34:24 -0600 Subject: [Python-Dev] PEP 8 and constants In-Reply-To: References: <20090121180014.GA16447@panix.com> <1afaf6160901211842x57963b26q796bca35975c69e1@mail.gmail.com> Message-ID: <1afaf6160901211934q4538e6f5t2819459bd1bdc4da@mail.gmail.com> >> On Wed, Jan 21, 2009 at 12:48 PM, Guido van Rossum wrote: >>> On Wed, Jan 21, 2009 at 10:00 AM, Aahz wrote: >>>> In comp.lang.python, there has been some discussion of the fact that >>>> there are no guidelines in PEP 8 for constants: >>>> >>>> http://groups.google.com/group/comp.lang.python/browse_thread/thread/ed964fe8ad6da7b7 >>>> >>>> Is there any sentiment that PEP 8 should be updated to reflect the common >>>> usage of ALL_CAPS for constants? >>> >>> It makes sense to codify this usage in PEP 8. I think it's by far the >>> most common convention adopted by projects that set their own style >>> guide based on PEP 8 with local additions. Ok. I added a note about constants in r68849. -- Regards, Benjamin From benjamin at python.org Thu Jan 22 03:42:28 2009 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 21 Jan 2009 20:42:28 -0600 Subject: [Python-Dev] PEP 8 and constants In-Reply-To: References: <20090121180014.GA16447@panix.com> Message-ID: <1afaf6160901211842x57963b26q796bca35975c69e1@mail.gmail.com> On Wed, Jan 21, 2009 at 12:48 PM, Guido van Rossum wrote: > On Wed, Jan 21, 2009 at 10:00 AM, Aahz wrote: >> In comp.lang.python, there has been some discussion of the fact that >> there are no guidelines in PEP 8 for constants: >> >> http://groups.google.com/group/comp.lang.python/browse_thread/thread/ed964fe8ad6da7b7 >> >> Is there any sentiment that PEP 8 should be updated to reflect the common >> usage of ALL_CAPS for constants? > > It makes sense to codify this usage in PEP 8. I think it's by far the > most common convention adopted by projects that set their own style > guide based on PEP 8 with local additions. Do you suggest underscores between words in constants as with other names? -- Regards, Benjamin From aahz at pythoncraft.com Thu Jan 22 07:14:13 2009 From: aahz at pythoncraft.com (Aahz) Date: Wed, 21 Jan 2009 22:14:13 -0800 Subject: [Python-Dev] PEP 8 and constants In-Reply-To: <1afaf6160901211934q4538e6f5t2819459bd1bdc4da@mail.gmail.com> References: <20090121180014.GA16447@panix.com> <1afaf6160901211842x57963b26q796bca35975c69e1@mail.gmail.com> <1afaf6160901211934q4538e6f5t2819459bd1bdc4da@mail.gmail.com> Message-ID: <20090122061413.GA29901@panix.com> On Wed, Jan 21, 2009, Benjamin Peterson wrote: >>> On Wed, Jan 21, 2009 at 12:48 PM, Guido van Rossum wrote: >>>> On Wed, Jan 21, 2009 at 10:00 AM, Aahz wrote: >>>>> >>>>> In comp.lang.python, there has been some discussion of the fact that >>>>> there are no guidelines in PEP 8 for constants: >>>>> >>>>> http://groups.google.com/group/comp.lang.python/browse_thread/thread/ed964fe8ad6da7b7 >>>>> >>>>> Is there any sentiment that PEP 8 should be updated to reflect the common >>>>> usage of ALL_CAPS for constants? >>>> >>>> It makes sense to codify this usage in PEP 8. I think it's by far the >>>> most common convention adopted by projects that set their own style >>>> guide based on PEP 8 with local additions. > > Ok. I added a note about constants in r68849. Thanks! -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From ncoghlan at gmail.com Thu Jan 22 07:33:56 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 22 Jan 2009 16:33:56 +1000 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <5d1a32000901210751l2ef9e7fdy378293d937b91138@mail.gmail.com> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <200901210955.59699.steve@pearwood.info> <200901212146.46821.steve@pearwood.info> <5d1a32000901210638y1fc361d0s893a58988b7e9a7a@mail.gmail.com> <549056.3449.qm@web54409.mail.yahoo.com> <5d1a32000901210751l2ef9e7fdy378293d937b91138@mail.gmail.com> Message-ID: <49781354.2090807@gmail.com> Gerald Britton wrote: > OK then, what is the feeling out there about extending the "for" > syntax in general (and by extension list comprehensions and generator > expressions) by adding an optional while clause like this: > > for in [while [ | not ]: > > > The predicate would be tested after an is taken from > and before execution of the . If the predicate evaluates to > false, StopIteration would be raised. This construct would be > equivalent to: > > for in : > if [not | ]: break > > > Note: this is beyond what I was thinking in the first place, but has > arisen from the ensuing discussion. As Aahz said, this needs to go back to python-ideas or c.l.p to see if it goes anywhere. However, be aware that you're going to need examples from *real code* that show improvements in correctness, readability or speed in order to convince a sufficiently large number of the core devs and/or Guido that such an additional wrinkle to the looping syntax is worth it. A change being clever or cute isn't enough to justify its inclusion - it needs to provide sufficient real world benefit to counter the cost of the feature's development and maintenance, as well as the additional overhead for all users of the language in learning about it. An approach that has been used effectively in the past to argue for new syntax or builtins is to trawl through the standard library and its test suite looking for things that could be simplified by the proposed addition to the language, but appropriate examples could also be drawn from the code bases of other large Python projects (Twisted, Zope, Django, Bazaar, Mercurial... etc). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Thu Jan 22 07:40:34 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 22 Jan 2009 16:40:34 +1000 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <20090122004010.GA14090@cskk.homeip.net> References: <20090122004010.GA14090@cskk.homeip.net> Message-ID: <497814E2.2040400@gmail.com> Cameron Simpson wrote: > On 21Jan2009 14:02, Tres Seaver wrote: > | Vitor Bosshard wrote: > | > BTW, there is already an "until" keyword in python, it's called "while not" ;) > | > | 'until' is used at least in some languages (Pascal, Modula*, maybe Ada?) > | for a "terminate at bottom" loop (one guaranteed to run at least once): > | in such cases, the predicate has the negative sense. > > This is a particular flavour of "do ... while" which just happens > to read a little better in English. It does sometimes bother me that > Python doesn't have do...while when I find my self replicating the loop > bottom above the loop. Adding a do-while construct to Python has already been proposed: http://www.python.org/dev/peps/pep-0315/ It was merely deferred due to only garnering lukewarm support and lack of a reference implementation rather than actually being rejected: http://mail.python.org/pipermail/python-dev/2006-February/060718.html Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Thu Jan 22 07:42:29 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 22 Jan 2009 16:42:29 +1000 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <200901210955.59699.steve@pearwood.info> <200901212146.46821.steve@pearwood.info> Message-ID: <49781555.1020303@gmail.com> Terry Reedy wrote: > Steven D'Aprano wrote: >> Is that Xah Lee? It sounds like the sort of thing he'd say. > > It was the thread he started, but not him. He contributed other idiocies. Xah Lee is still around? I would have expected him to get bored and go away years ago... Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From eric at trueblade.com Thu Jan 22 10:06:24 2009 From: eric at trueblade.com (Eric Smith) Date: Thu, 22 Jan 2009 04:06:24 -0500 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: References: <4977715F.8030203@palladion.com> <20090122004010.GA14090@cskk.homeip.net> Message-ID: <49783710.6070302@trueblade.com> Terry Reedy wrote: > Cameron Simpson wrote: >> >> Back at uni we had to implement a small language in our compilers class >> and the lecturer had specified a proper generic while loop, thus: >> >> loop: >> suite >> while invariant >> suite >> endloop > > In Python, that is spelled > > while True: > suite > if not invariant: break > suite Indeed. This is the well known "loop and a half" problem. Eric. From amauryfa at gmail.com Thu Jan 22 10:18:45 2009 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Thu, 22 Jan 2009 10:18:45 +0100 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> <49763FC6.9090303@v.loewis.de> <49777A99.9010607@v.loewis.de> Message-ID: Hello, On Wed, Jan 21, 2009 at 22:07, Luke Kenneth Casson Leighton wrote: > On Wed, Jan 21, 2009 at 7:42 PM, "Martin v. L?wis" wrote: >>> sorry, martin - i thought the win32 builds generated python25.lib, >>> python25.dll >> >> Correct. >> >>> and python25.def >> >> No. >> >>> so as to fit into the 8.3 filename convention. >> >> No. It generates python25.lib because that's the import library >> for python25.dll. It calls it python25.dll because the lib prefix >> is atypical for the platform, and also redundant (DLL means >> "dynamic link library"). >> >> The Python binary installer also includes libpython25.a, for use >> with mingw32. > > ok, so - different from what's being generated by ./configure under > msys under wine or native win32 - what's being generated (libpython 2 > . 5 . a and libpython 2 . 5 . dll . a) is more akin to the cygwin > environment. > > therefore, there's absolutely no doubt that the two are completely different. > > and on that basis, would i be correct in thinking that you _can't_ go > linking or building modules or any python win32 code for one and have > a hope in hell of using it on the other, and that you would _have_ to > rebuild e.g. numpy for use with a mingw32-msys-built version of > python? > > or, is the .pyd loading a bit cleverer (or perhaps a bit less > cleverer) than i'm expecting it to be? On Windows, you must turn on the --enable_shared option if you want to build extension modules. You could take the cygwin build as an example, see what's done in ./configure.in. -- Amaury Forgeot d'Arc From lkcl at lkcl.net Thu Jan 22 11:44:31 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Thu, 22 Jan 2009 10:44:31 +0000 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: <49778FF6.8070608@v.loewis.de> References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> <49763FC6.9090303@v.loewis.de> <49777A99.9010607@v.loewis.de> <49778FF6.8070608@v.loewis.de> Message-ID: On Wed, Jan 21, 2009 at 9:13 PM, "Martin v. L?wis" wrote: >> ok, so - different from what's being generated by ./configure under >> msys under wine or native win32 - what's being generated (libpython 2 >> . 5 . a and libpython 2 . 5 . dll . a) is more akin to the cygwin >> environment. >> >> therefore, there's absolutely no doubt that the two are completely different. >> >> and on that basis, would i be correct in thinking that you _can't_ go >> linking or building modules or any python win32 code for one and have >> a hope in hell of using it on the other, and that you would _have_ to >> rebuild e.g. numpy for use with a mingw32-msys-built version of >> python? > > I can't comment on that, because I don't know what your port does. > Does it not produce a .dll containing the majority of Python? no, it contains the minimal necessary amount of python modules, exactly like when python is built using cygwin. actualy, there's a few modules that _have_ to be included. roumen discovered that you have to have these: _functools _functoolsmodule.c # Tools for working with functions and callable objects operator operator.c # operator.add() and similar goodies _locale _localemodule.c # -lintl _struct _struct.c _subprocess ../PC/_subprocess.c _winreg ../PC/_winreg.c and i've discovered that when running under wine you have to also have these: _weakref _weakref.c and also when running unde wine with msvcr80, so far, you have to also have these: collections collectionsmodule.c thread threadmodule.c all the rest can be done as .pyd > And is that not called python25.dll? no, it's called libpython2.5.dll.a, just like when python is built using cygwin. the configure scripts, thanks to the cygwin build, already end up copying that to libpython2.5.dll. _not_ python25.dll l. p.s. there's nothing to stop you adding every single module and then renaming the resultant blob to libpython25.dll - i just haven't been given, or found, a good reason to do so yet. From lkcl at lkcl.net Thu Jan 22 11:57:07 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Thu, 22 Jan 2009 10:57:07 +0000 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> <49763FC6.9090303@v.loewis.de> <49777A99.9010607@v.loewis.de> Message-ID: On Thu, Jan 22, 2009 at 9:18 AM, Amaury Forgeot d'Arc wrote: >> or, is the .pyd loading a bit cleverer (or perhaps a bit less >> cleverer) than i'm expecting it to be? > > On Windows, you must turn on the --enable_shared option if you want to > build extension modules. > You could take the cygwin build as an example, see what's done in > ./configure.in. amaury, thank you for mentioning that - yes, as it turns out, all of the mingw ports (dan, roumen etc) do pretty much exactly this. also it turns out that on mingw, if you _don't_ disable shared (i.e. if you try to build a static library) mingw32 gcc runtime utils .16, .17 _and_ .19 all segfault or have runtime assertions when creating the archives!! either ar.exe or ranlib.exe choke themselves to death. which is greaaat. so, i've had to set the variable which specifies the libpython2.5.a static library to "" in order to stop it from being built. it would be helpful if there was a --enable-static=yes/no configure option, but there isn't one. leaving that aside, you understand therefore that dan, roumen and i have all managed to achieve building of .pyd extension modules. so, the question i am asking is: would it be reasonable to expect mingw-compiled .pyd modules to work with a proprietary-compiled msvc python.exe, and vice-versa? l. From tino at wildenhain.de Thu Jan 22 11:48:08 2009 From: tino at wildenhain.de (Tino Wildenhain) Date: Thu, 22 Jan 2009 11:48:08 +0100 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <5d1a32000901190859h6720205o585fee8d19607f2@mail.gmail.com> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <5d1a32000901190837v70242228l381f3801ea1866bb@mail.gmail.com> <5d1a32000901190859h6720205o585fee8d19607f2@mail.gmail.com> Message-ID: <49784EE8.1050704@wildenhain.de> Hi, Gerald Britton wrote: > The sieve is just one example. The basic idea is that for some > infinite generator (even a very simple one) you want to cut it off > after some point. As for the number of characters, I spelled lambda > incorrectly (left out a b) and there should be a space after the colon > to conform to design guides. So, actually the takewhile version is > two characters longer, not counting "import itertools" of course! the only usefull approach I could see is to enable slice syntax on generators which would make it possible to describe the exact or maximum lenght of results you want out of it. something like: >> g=(i for i in xrange(1000))[2:5] >> g.next() # wrapper would now step 2 times w/o yield and 1 with yield 2 >> g.next() 3 >> g.next() 4 >> g.next() Traceback (most recent call last): File " ", line 1, in StopIteration as expected - this could be included into itertools for now. Regards Tino > > On Mon, Jan 19, 2009 at 11:44 AM, Daniel Stutzbach > wrote: >> On Mon, Jan 19, 2009 at 10:37 AM, Gerald Britton >> wrote: >>> prime = (p for p in sieve() while p < 1000) >>> prime = takewhile(lamda p:p<1000, sieve()) >> I'm pretty sure the extra cost of evaluating the lambda at each step is tiny >> compared to the cost of the sieve, so I don't you can make a convincing >> argument on performance. >> >> Also, you know the latter is actually fewer characters, right? :-) >> >> -- >> Daniel Stutzbach, Ph.D. >> President, Stutzbach Enterprises, LLC > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/tino%40wildenhain.de -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3241 bytes Desc: S/MIME Cryptographic Signature URL: From ncoghlan at gmail.com Thu Jan 22 12:42:02 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 22 Jan 2009 21:42:02 +1000 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <49784EE8.1050704@wildenhain.de> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <5d1a32000901190837v70242228l381f3801ea1866bb@mail.gmail.com> <5d1a32000901190859h6720205o585fee8d19607f2@mail.gmail.com> <49784EE8.1050704@wildenhain.de> Message-ID: <49785B8A.7090509@gmail.com> Tino Wildenhain wrote: >>> g=(i for i in xrange(1000))[2:5] >>> g.next() # wrapper would now step 2 times w/o yield and 1 with yield > 2 >>> g.next() > 3 >>> g.next() > 4 >>> g.next() > Traceback (most recent call last): > File " ", line 1, in > StopIteration > > as expected - this could be included into itertools for now. Slicing of arbitrary iterators has been supported by itertools ever since the module was first added to the standard library. >>> from itertools import islice >>> g = islice((i for i in xrange(1000)), 2, 5) >>> list(g) [2, 3, 4] Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From tino at wildenhain.de Thu Jan 22 13:37:22 2009 From: tino at wildenhain.de (Tino Wildenhain) Date: Thu, 22 Jan 2009 13:37:22 +0100 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <49785B8A.7090509@gmail.com> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <5d1a32000901190837v70242228l381f3801ea1866bb@mail.gmail.com> <5d1a32000901190859h6720205o585fee8d19607f2@mail.gmail.com> <49784EE8.1050704@wildenhain.de> <49785B8A.7090509@gmail.com> Message-ID: <49786882.6090802@wildenhain.de> Nick Coghlan wrote: > Tino Wildenhain wrote: >>>> g=(i for i in xrange(1000))[2:5] >>>> g.next() # wrapper would now step 2 times w/o yield and 1 with yield >> 2 >>>> g.next() >> 3 >>>> g.next() >> 4 >>>> g.next() >> Traceback (most recent call last): >> File " ", line 1, in >> StopIteration >> >> as expected - this could be included into itertools for now. > > Slicing of arbitrary iterators has been supported by itertools ever > since the module was first added to the standard library. > >>>> from itertools import islice >>>> g = islice((i for i in xrange(1000)), 2, 5) >>>> list(g) > [2, 3, 4] > Yeah right, I actually believed it is already there but didn't bother to check ;-) Thx Tino -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3241 bytes Desc: S/MIME Cryptographic Signature URL: From catch-all at masklinn.net Thu Jan 22 14:44:41 2009 From: catch-all at masklinn.net (Xavier Morel) Date: Thu, 22 Jan 2009 14:44:41 +0100 Subject: [Python-Dev] PEP 3142: Add a "while" clause to generator expressions In-Reply-To: <49785B8A.7090509@gmail.com> References: <5d1a32000901190710i288bf19ahbbf1c12385f793b0@mail.gmail.com> <5d1a32000901190837v70242228l381f3801ea1866bb@mail.gmail.com> <5d1a32000901190859h6720205o585fee8d19607f2@mail.gmail.com> <49784EE8.1050704@wildenhain.de> <49785B8A.7090509@gmail.com> Message-ID: <154174DF-DA11-4F6C-8A1C-9FDD1C56F260@masklinn.net> On 22 Jan 2009, at 12:42 , Nick Coghlan wrote: > Tino Wildenhain wrote: >>>> g=(i for i in xrange(1000))[2:5] >>>> g.next() # wrapper would now step 2 times w/o yield and 1 with >>>> yield >> 2 >>>> g.next() >> 3 >>>> g.next() >> 4 >>>> g.next() >> Traceback (most recent call last): >> File " ", line 1, in >> StopIteration >> >> as expected - this could be included into itertools for now. > > Slicing of arbitrary iterators has been supported by itertools ever > since the module was first added to the standard library. islice is pretty annoying in some aspects, though. Mainly because it doesn't accept kwargs and defaults to requiring the stop argument. So drop(iterable, n) has to be written islice(iterable, n, None) (and of course the naming isn't ideal), and you can't really use functools.partial since the iterator is the first argument (unless there's a way to partially apply only the tail args without kwargs). From techtonik at gmail.com Thu Jan 22 15:11:31 2009 From: techtonik at gmail.com (anatoly techtonik) Date: Thu, 22 Jan 2009 16:11:31 +0200 Subject: [Python-Dev] Single Sign-On for *.python.org In-Reply-To: <49741F83.9020804@v.loewis.de> References: <76fd5acf0901181147h4773eb43l2746e2a15daf2dd0@mail.gmail.com> <87tz7w85do.fsf@benfinney.id.au> <49741F83.9020804@v.loewis.de> Message-ID: I do have some old patches for roundup that I was unable to test, because of blocking issues with openidenabled python-openid library and my Blogger server. See the top issue with the patch at openidenabled tracker: http://trac.openidenabled.com/trac/query?status=new&status=assigned&status=reopened&project=python-openid&order=priority Judging from complaints in development mailing list during the last three months I may conclude that the library isn't supported anymore. I do not know alternative OpenID implementation for Python, so the only way I see to continue development is to fork the lib. However, it is not really clear to me how to do this in case of Apache license. On Mon, Jan 19, 2009 at 8:36 AM, "Martin v. L?wis" wrote: >> I've also had fruitless discussions about adding OpenID authentication >> to Roundup. > > Did you offer patches to roundup during these discussions? > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/techtonik%40gmail.com > -- --anatoly t. From techtonik at gmail.com Thu Jan 22 16:00:26 2009 From: techtonik at gmail.com (anatoly techtonik) Date: Thu, 22 Jan 2009 17:00:26 +0200 Subject: [Python-Dev] About SCons Re: progress: compiling python2.5 under msys (specifically but not exclusively under wine) with msvcr80 Message-ID: On Thu, Jan 22, 2009 at 12:51 AM, Roumen Petrov wrote: >> >> Against 2.3, rejected due to dependence on SCons. >> Also appears to have been incomplete, needing more work. > > No it was complete but use SCons. Most of changes changes in code you will > see again in 3871. > I would better use SCons for both unix and windows builds. In case of windows for both compilers - mingw and microsoft ones. To port curses extension to windows I need to know what gcc options mean, what are the rules to write Makefiles and how to repeat these rules as well as find options in visual studio interface. Not mentioning various platform-specific defines and warning fixes. -- --anatoly t. From g.brandl at gmx.net Thu Jan 22 19:12:06 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 22 Jan 2009 19:12:06 +0100 Subject: [Python-Dev] Questions/comments on documentation formatting In-Reply-To: References: Message-ID: Brett Cannon schrieb: >>> 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [, >>> c=None]])``) really necessary when default argument values are >>> present? And do we really need to nest the brackets when it is obvious >>> that having on optional argument means the rest are optional as well? >> >> We've discussed that once on the doc-SIG, and I agreed that the bracketing >> is not really pretty, especially if it's heavily nested. Python functions >> where it makes sense should use the default-value syntax, while C functions >> without kwargs support need to keep the brackets. >> > > That was my thinking. > >> Making this consistent throughout the docs is no small task, of course. >> > > Nope, but perhaps all new docs should stop their use. OK. Perhaps we can sprint a bit on automatic replacement at PyCon. >>> 4. The var directive is not working even though the docs list it as a >>> valid directive; so is it still valid and something is broken, or the >>> docs need to be updated? >> >> (First, you're confusing "directive" and "role" which led to some confusion >> on Benjamin's part.) >> >> Where is a "var" role documented? If it is, it is a bug. > > http://docs.python.org/dev/3.0/documenting/markup.html#inline-markup. I assume you're referring to "Variable names are an exception, they should be marked simply with *var*."? Do you have suggestions how to improve clarity? Georg From brett at python.org Thu Jan 22 19:20:38 2009 From: brett at python.org (Brett Cannon) Date: Thu, 22 Jan 2009 10:20:38 -0800 Subject: [Python-Dev] Questions/comments on documentation formatting In-Reply-To: References: Message-ID: On Thu, Jan 22, 2009 at 10:12, Georg Brandl wrote: > Brett Cannon schrieb: > >>>> 3. Are brackets for optional arguments (e.g. ``def fxn(a [, b=None [, >>>> c=None]])``) really necessary when default argument values are >>>> present? And do we really need to nest the brackets when it is obvious >>>> that having on optional argument means the rest are optional as well? >>> >>> We've discussed that once on the doc-SIG, and I agreed that the bracketing >>> is not really pretty, especially if it's heavily nested. Python functions >>> where it makes sense should use the default-value syntax, while C functions >>> without kwargs support need to keep the brackets. >>> >> >> That was my thinking. >> >>> Making this consistent throughout the docs is no small task, of course. >>> >> >> Nope, but perhaps all new docs should stop their use. > > OK. Perhaps we can sprint a bit on automatic replacement at PyCon. > That's a possibility. >>>> 4. The var directive is not working even though the docs list it as a >>>> valid directive; so is it still valid and something is broken, or the >>>> docs need to be updated? >>> >>> (First, you're confusing "directive" and "role" which led to some confusion >>> on Benjamin's part.) >>> >>> Where is a "var" role documented? If it is, it is a bug. >> >> http://docs.python.org/dev/3.0/documenting/markup.html#inline-markup. > > I assume you're referring to "Variable names are an exception, they should > be marked simply with *var*."? Do you have suggestions how to improve > clarity? > "... variables, including function/method arguments, ...". -Brett From g.brandl at gmx.net Thu Jan 22 19:29:48 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 22 Jan 2009 19:29:48 +0100 Subject: [Python-Dev] Questions/comments on documentation formatting In-Reply-To: References: Message-ID: Brett Cannon schrieb: >>>>> 4. The var directive is not working even though the docs list it as a >>>>> valid directive; so is it still valid and something is broken, or the >>>>> docs need to be updated? >>>> >>>> (First, you're confusing "directive" and "role" which led to some confusion >>>> on Benjamin's part.) >>>> >>>> Where is a "var" role documented? If it is, it is a bug. >>> >>> http://docs.python.org/dev/3.0/documenting/markup.html#inline-markup. >> >> I assume you're referring to "Variable names are an exception, they should >> be marked simply with *var*."? Do you have suggestions how to improve >> clarity? >> > > "... variables, including function/method arguments, ...". Thanks, I've changed it a bit along these lines in r68859. Georg From martin at v.loewis.de Thu Jan 22 19:43:57 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 22 Jan 2009 19:43:57 +0100 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> <49763FC6.9090303@v.loewis.de> <49777A99.9010607@v.loewis.de> <49778FF6.8070608@v.loewis.de> Message-ID: <4978BE6D.6060000@v.loewis.de> >> I can't comment on that, because I don't know what your port does. >> Does it not produce a .dll containing the majority of Python? > > no, it contains the minimal necessary amount of python modules, > exactly like when python is built using cygwin. actualy, there's a > few modules that _have_ to be included. That's actually not my question. Do you have a DLL that contains all of Python/*.o and Objects/*.o? >> And is that not called python25.dll? > > no, it's called libpython2.5.dll.a, just like when python is built > using cygwin. the configure scripts, thanks to the cygwin build, > already end up copying that to libpython2.5.dll. > > _not_ python25.dll I'm giving up for now. I don't quite understand why the file gets first called libpython2.5.dll.a, and then renamed to libpython2.5.dll. Could it be perhaps that the .a file is an import library, and actually different from the .dll file? > p.s. there's nothing to stop you adding every single module and then > renaming the resultant blob to libpython25.dll - i just haven't been > given, or found, a good reason to do so yet. That doesn't really matter, I guess. An extension module build by your port will either fail to load into the regular Python (if libpython2.5.dll is not found), or load and then crash (because it uses a different copy of the Python runtime). Likewise vice versa. If this port ever takes off, we get another binary-incompatible Python version. I hope that users will understand that it is disjoint from the python.org version (as they seem to understand fine for the Cygwin build, which already picks up its extension modules also from a disjoint location, which helps to keep the two separate). Regards, Martin From martin at v.loewis.de Thu Jan 22 19:50:32 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 22 Jan 2009 19:50:32 +0100 Subject: [Python-Dev] Single Sign-On for *.python.org In-Reply-To: References: <76fd5acf0901181147h4773eb43l2746e2a15daf2dd0@mail.gmail.com> <87tz7w85do.fsf@benfinney.id.au> <49741F83.9020804@v.loewis.de> Message-ID: <4978BFF8.2020403@v.loewis.de> > I do not know alternative OpenID implementation for Python, so the > only way I see to continue development is to fork the lib. PyPI reports 15 packages when you search for OpenID. Not sure whether any of these are any good. Regards, Martin From jnoller at gmail.com Thu Jan 22 20:06:33 2009 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 22 Jan 2009 14:06:33 -0500 Subject: [Python-Dev] Unable to resolve svn.python.org: OS/X Message-ID: <4222a8490901221106k44dacca2se5454b4885305b7c@mail.gmail.com> The other day, Martin pointed out that my buildslave had gone off the reservation: on restarting it via the "buildbot start ~/buildarea" command - Martin noticed the slave had started throwing the DNS resolution error: closing stdin using PTY: True svn: PROPFIND request failed on '/projects/python/branches/py3k' svn: PROPFIND of '/projects/python/branches/py3k': Could not resolve hostname `svn.python.org': Temporary failure in name resolution (http://svn.python.org) program finished with exit code 1 Apparently, this has bothered a few buildbots. Some quick googling popped up the fix: http://buildbot.net/trac/wiki/UsingLaunchd After dropping the attached plist file in /Library/LaunchDaemons and setting the permissions right (and then chown -R'ing the existing buildarea for the buildbot user to buildbot:daemon) - running "sudo launchctl load org.python.buildbot.slave.plist" brought the buildbot back up in working order. Hopefully this helps out. -jesse -------------- next part -------------- A non-text attachment was scrubbed... Name: org.python.buildbot.slave.plist Type: application/octet-stream Size: 1146 bytes Desc: not available URL: From lkcl at lkcl.net Thu Jan 22 20:17:36 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Thu, 22 Jan 2009 19:17:36 +0000 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: <4978BE6D.6060000@v.loewis.de> References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> <49763FC6.9090303@v.loewis.de> <49777A99.9010607@v.loewis.de> <49778FF6.8070608@v.loewis.de> <4978BE6D.6060000@v.loewis.de> Message-ID: On Thu, Jan 22, 2009 at 6:43 PM, "Martin v. L?wis" wrote: >>> I can't comment on that, because I don't know what your port does. >>> Does it not produce a .dll containing the majority of Python? >> >> no, it contains the minimal necessary amount of python modules, >> exactly like when python is built using cygwin. actualy, there's a >> few modules that _have_ to be included. > > That's actually not my question. ah right - sorry to not quite fully understand. > Do you have a DLL that contains > all of Python/*.o and Objects/*.o? yes. >>> And is that not called python25.dll? >> >> no, it's called libpython2.5.dll.a, just like when python is built >> using cygwin. the configure scripts, thanks to the cygwin build, >> already end up copying that to libpython2.5.dll. >> >> _not_ python25.dll > > I'm giving up for now. I don't quite understand why the file gets > first called libpython2.5.dll.a, and then renamed to libpython2.5.dll. > Could it be perhaps that the .a file is an import library, and actually > different from the .dll file? ... *thinks*... sorry, you're right. it's the way that dlltool is used on cygwin. dlltool on cygwin and gcc on cygwin create files with the following equivalence: python25.lib on msvc <---> libpython2.5.dll.a on cygwin and mingw32 python2.5.dll on msvc <--> libpython2.5.dll on cygwin and mingw32 >> p.s. there's nothing to stop you adding every single module and then >> renaming the resultant blob to libpython25.dll - i just haven't been >> given, or found, a good reason to do so yet. > > That doesn't really matter, I guess. An extension module build by your > port will either fail to load into the regular Python (if > libpython2.5.dll is not found), or load and then crash (because it uses > a different copy of the Python runtime). Likewise vice versa. > > If this port ever takes off, we get another binary-incompatible Python > version. there are at least three [mingw] already. > I hope that users will understand that it is disjoint from > the python.org version (as they seem to understand fine for the > Cygwin build, which already picks up its extension modules also from > a disjoint location, which helps to keep the two separate). there are already no less than _four_ mingw ports of python, of varying degrees. * http://jove.prohosting.com/iwave/ipython/pyMinGW.html * http://sebsauvage.net/python/mingw.html * http://python-mingw.donbennett.org/ * roumen's cross-compile+native port * the port i'm working on - extending roumen's native mingw compile one dates back to... python 2.2 i didn't include that one. another is python2.4. don's work is a cygwin cross-compile (note NOT a "compile of python for cygwin such that you need CYGWIN.DLL to run python"), so, using cygwin under win32 to cross-compile a native python.exe. smart, that. roumen then worked on that further, to make it compile under mingw / msys, not cygwin. and i'm working on taking windows _completely_ out of the loop, by getting python.exe to both compile _and_ run under wine, with the added benefit that if you _did_ happen to want to compile (or run) under either native windows or both, you can. l. From skip at pobox.com Thu Jan 22 20:23:40 2009 From: skip at pobox.com (skip at pobox.com) Date: Thu, 22 Jan 2009 13:23:40 -0600 Subject: [Python-Dev] Unable to resolve svn.python.org: OS/X In-Reply-To: <4222a8490901221106k44dacca2se5454b4885305b7c@mail.gmail.com> References: <4222a8490901221106k44dacca2se5454b4885305b7c@mail.gmail.com> Message-ID: <18808.51132.416898.304436@montanaro.dyndns.org> Jesse> The other day, Martin pointed out that my buildslave had gone off Jesse> the reservation: on restarting it via the "buildbot start Jesse> ~/buildarea" command - Martin noticed the slave had started Jesse> throwing the DNS resolution error: ... Thanks for this. This appears to be exactly what's gone wrong with the OS/X community buildbot I run. I'll try to plop your solution in place when I get home and see how it works after that. Skip From lkcl at lkcl.net Thu Jan 22 20:39:40 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Thu, 22 Jan 2009 19:39:40 +0000 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: <4978BE6D.6060000@v.loewis.de> References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> <49763FC6.9090303@v.loewis.de> <49777A99.9010607@v.loewis.de> <49778FF6.8070608@v.loewis.de> <4978BE6D.6060000@v.loewis.de> Message-ID: > version. I hope that users will understand that it is disjoint from > the python.org version (as they seem to understand fine for the > Cygwin build, which already picks up its extension modules also from > a disjoint location, which helps to keep the two separate). yes i made the default installation location (--prefix=) c:/python2.5 _not_ c:/python25 but obviously it _has_ been necessary to make the installation of modules into the exact same _style_ of location as the msvc build, because it has obviously also been necessary to use PC/getpathp.c not getpath.c so, .pyd modules will get installed in c:/python2.5/lib/site-packages/ and most importantly they'll get _looked_ for in there! for a while, they were being installed in c:/python2.5/lib/python2.5/site-packages which was a bit of a mess - that's the "unix" style of module locations. getpathp.c looks for "Lib/os.py" whilst getpath.c looks for "os.py" there's a whole _stack_ of knock-on effect little things like that. l. From martin at v.loewis.de Thu Jan 22 20:40:19 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 22 Jan 2009 20:40:19 +0100 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> <49763FC6.9090303@v.loewis.de> <49777A99.9010607@v.loewis.de> <49778FF6.8070608@v.loewis.de> <4978BE6D.6060000@v.loewis.de> Message-ID: <4978CBA3.8040905@v.loewis.de> > there are already no less than _four_ mingw ports of python, of varying degrees. > > * http://jove.prohosting.com/iwave/ipython/pyMinGW.html Ok, this one builds pythonXY, so it tries to be compatible with the official distribution (although it seems to link against MSVCRT.dll) > * http://sebsauvage.net/python/mingw.html That's *not* a port of Python to MingW. Instead, it is a set of instructions on how to build Python extension modules, using the official Python binaries, with mingw. I think this is obsolete now, as this now ships with Python itself. > * http://python-mingw.donbennett.org/ This doesn't seem to be distributing binaries. Regards, Martin From lkcl at lkcl.net Thu Jan 22 21:01:16 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Thu, 22 Jan 2009 20:01:16 +0000 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: <4978BE6D.6060000@v.loewis.de> References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> <49763FC6.9090303@v.loewis.de> <49777A99.9010607@v.loewis.de> <49778FF6.8070608@v.loewis.de> <4978BE6D.6060000@v.loewis.de> Message-ID: > That doesn't really matter, I guess. An extension module build by your > port will either fail to load into the regular Python (if > libpython2.5.dll is not found), or load and then crash (because it uses > a different copy of the Python runtime). Likewise vice versa. excellent, excellent that's _really_ good - and here's why: if it is _guaranteed_ to crash, regardless of what i do (because the copy of the python runtime is different), then it _doesn't_ matter what version of msvcrt i link the mingw-built python runtime with, does it? am i right? and if _that's_ the case, i can stop fricking about with msvcr80 :) which would be an absolute godsend because msvcr80 under wine is a frickin nightmare. the python regression tests pretty much hammer the daylights out of wine and it's squeaking in all sorts of weird places. adding in msvcr80, an undocumented transparent "blob" into the mix just makes getting this port fully operational all the more difficult. i'd like to avoid that :) l. From lkcl at lkcl.net Thu Jan 22 21:09:15 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Thu, 22 Jan 2009 20:09:15 +0000 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: <4978CBA3.8040905@v.loewis.de> References: <49763FC6.9090303@v.loewis.de> <49777A99.9010607@v.loewis.de> <49778FF6.8070608@v.loewis.de> <4978BE6D.6060000@v.loewis.de> <4978CBA3.8040905@v.loewis.de> Message-ID: On Thu, Jan 22, 2009 at 7:40 PM, "Martin v. L?wis" wrote: >> there are already no less than _four_ mingw ports of python, of varying degrees. >> >> * http://jove.prohosting.com/iwave/ipython/pyMinGW.html > > Ok, this one builds pythonXY, so it tries to be compatible with the > official distribution (although it seems to link against MSVCRT.dll) > >> * http://sebsauvage.net/python/mingw.html > > That's *not* a port of Python to MingW. Instead, it is a set of > instructions on how to build Python extension modules, using the > official Python binaries, with mingw. oh? ah, sorry, i didn't check . >> * http://python-mingw.donbennett.org/ > > This doesn't seem to be distributing binaries. sourceforge page. i checked the statistics, there don't seem to be very many hits (sorry to hear that don, if you're reading this!) ok. there _is_ a sourceforge page,... yep, downloads here: http://sourceforge.net/project/showfiles.php?group_id=182839 ok , so that makes... 3? From martin at v.loewis.de Thu Jan 22 21:17:48 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 22 Jan 2009 21:17:48 +0100 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> <49763FC6.9090303@v.loewis.de> <49777A99.9010607@v.loewis.de> <49778FF6.8070608@v.loewis.de> <4978BE6D.6060000@v.loewis.de> Message-ID: <4978D46C.3030200@v.loewis.de> > am i right? You should test that. I'm not sure whether it will crash (in particular, it might not on load), but it *might* crash, or fail in strange ways (e.g. when it checks whether something is a string, and decides it is not, because it is looking at the other PyString_Type) > and if _that's_ the case, i can stop fricking about with msvcr80 :) If so, I think there is little point in submitting patches to the Python bug tracker. I'm -1 on supporting two different-but-similar builds on Windows. I could accept a different build *process*, but the outcome must be the same either way. (of course, msvcr80 is irrelevant, because Python had never been using that officially) Regards, Martin From martin at v.loewis.de Thu Jan 22 21:22:33 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 22 Jan 2009 21:22:33 +0100 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: <49763FC6.9090303@v.loewis.de> <49777A99.9010607@v.loewis.de> <49778FF6.8070608@v.loewis.de> <4978BE6D.6060000@v.loewis.de> <4978CBA3.8040905@v.loewis.de> Message-ID: <4978D589.1040003@v.loewis.de> >> This doesn't seem to be distributing binaries. > > sourceforge page. i checked the statistics, there don't seem to be > very many hits (sorry to hear that don, if you're reading this!) ok. > there _is_ a sourceforge page,... yep, downloads here: > > http://sourceforge.net/project/showfiles.php?group_id=182839 Where *exactly* do you get binaries there? All I can find is patches-2.5.1v2.gz Regards, Martin From LambertDW at Corning.com Thu Jan 22 21:29:09 2009 From: LambertDW at Corning.com (Lambert, David W (S&T)) Date: Thu, 22 Jan 2009 15:29:09 -0500 Subject: [Python-Dev] [issue5029] Odd slicing behaviour In-Reply-To: References: Message-ID: <84B204FFB016BA4984227335D8257FBA8490FB@CVCV0XI05.na.corning.com> I cannot find that the documentation states "with negative step swap left with right". This is perhaps non-obvious. It is the words of the tutorial that caused issue 5029 author's confusion. 'a'[0::-1] != [] (is True, author expected False). The tutorial says: "One way to remember how slices work is to think of the indices as pointing between characters, with the left edge of the first character numbered 0. Then the right edge of the last character of a string of n characters has index n, for example:" From lkcl at lkcl.net Thu Jan 22 21:55:33 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Thu, 22 Jan 2009 20:55:33 +0000 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: <4978D46C.3030200@v.loewis.de> References: <49763FC6.9090303@v.loewis.de> <49777A99.9010607@v.loewis.de> <49778FF6.8070608@v.loewis.de> <4978BE6D.6060000@v.loewis.de> <4978D46C.3030200@v.loewis.de> Message-ID: On Thu, Jan 22, 2009 at 8:17 PM, "Martin v. L?wis" wrote: >> am i right? > > You should test that. I'm not sure whether it will crash (in particular, > it might not on load), but it *might* crash, or fail in strange ways > (e.g. when it checks whether something is a string, and decides it is > not, because it is looking at the other PyString_Type) mrmmm... how? apps won't end up loading _both_ libpython2.5.dll _and_ python25.dll (or libpython2.N.dll and python2N.dll) will they? >> and if _that's_ the case, i can stop fricking about with msvcr80 :) > > If so, I think there is little point in submitting patches to the Python > bug tracker. I'm -1 on supporting two different-but-similar builds on > Windows. I could accept a different build *process*, but the outcome > must be the same either way. > > (of course, msvcr80 is irrelevant, because Python had never been using > that officially) oh? i saw the PCbuild8 and thought it was. oh that's even better - if python2.5 only officially support msvcrt whew. ok , i see - python2.6 uses msvcr90. i'll cross that bridge when i come to it. l. > Regards, > Martin > From lkcl at lkcl.net Thu Jan 22 21:56:30 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Thu, 22 Jan 2009 20:56:30 +0000 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: <4978D589.1040003@v.loewis.de> References: <49777A99.9010607@v.loewis.de> <49778FF6.8070608@v.loewis.de> <4978BE6D.6060000@v.loewis.de> <4978CBA3.8040905@v.loewis.de> <4978D589.1040003@v.loewis.de> Message-ID: On Thu, Jan 22, 2009 at 8:22 PM, "Martin v. L?wis" wrote: >>> This doesn't seem to be distributing binaries. >> >> sourceforge page. i checked the statistics, there don't seem to be >> very many hits (sorry to hear that don, if you're reading this!) ok. >> there _is_ a sourceforge page,... yep, downloads here: >> >> http://sourceforge.net/project/showfiles.php?group_id=182839 > > Where *exactly* do you get binaries there? > > All I can find is patches-2.5.1v2.gz doh! From martin at v.loewis.de Thu Jan 22 22:09:51 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 22 Jan 2009 22:09:51 +0100 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: <49763FC6.9090303@v.loewis.de> <49777A99.9010607@v.loewis.de> <49778FF6.8070608@v.loewis.de> <4978BE6D.6060000@v.loewis.de> <4978D46C.3030200@v.loewis.de> Message-ID: <4978E09F.4080507@v.loewis.de> > mrmmm... how? apps won't end up loading _both_ libpython2.5.dll _and_ > python25.dll (or libpython2.N.dll and python2N.dll) will they? Of course they will! python.exe (say, the official one) loads python25.dll. Then, an import is made of a ming-wine extension, say foo.pyd, which is linked with libpython2.5.dll, which then gets loaded. Voila, you have two interpreters in memory, with different type objects, memory heaps, and so on. This was always the problem when an old extension module (say, from 2.4) was loaded into a new interpreter (say, 2.5), then you load both python25.dll and python24.dll, causing crashes. To prevent this issue, Python now checks whether the module is linked with an incorrect pythonxy.dll, but won't catch that libpython2.5.dll is also a VM. >> (of course, msvcr80 is irrelevant, because Python had never been using >> that officially) > > oh? i saw the PCbuild8 and thought it was. oh that's even better - > if python2.5 only officially support msvcrt whew. No, Python 2.5 is linked with msvcr71.dll. PCbuild8 was never officially used. > ok , i see - python2.6 uses msvcr90. Correct. Regards, Martin From techtonik at gmail.com Thu Jan 22 22:36:09 2009 From: techtonik at gmail.com (anatoly techtonik) Date: Thu, 22 Jan 2009 23:36:09 +0200 Subject: [Python-Dev] Single Sign-On for *.python.org In-Reply-To: <4978BFF8.2020403@v.loewis.de> References: <76fd5acf0901181147h4773eb43l2746e2a15daf2dd0@mail.gmail.com> <87tz7w85do.fsf@benfinney.id.au> <49741F83.9020804@v.loewis.de> <4978BFF8.2020403@v.loewis.de> Message-ID: On Thu, Jan 22, 2009 at 8:50 PM, "Martin v. L?wis" wrote: >> I do not know alternative OpenID implementation for Python, so the >> only way I see to continue development is to fork the lib. > > PyPI reports 15 packages when you search for OpenID. Not sure whether > any of these are any good. django-authopenid 0.9.6 7 Openid authentification application for Django - uses openidenabled http://code.google.com/p/django-authopenid/wiki/README plone.app.openid 1.1 7 Plone OpenID authentication support - the same openidenabled requirement http://pypi.python.org/pypi/plone.app.openid/1.1 plone.openid 1.2 7 OpenID authentication support for PAS - GPLd, openidenabled - http://svn.plone.org/svn/plone/plone.openid/trunk/setup.py python-openid 2.2.1 7 OpenID support for servers and consumers. - well, it is the openidenabled lib itself - http://openidenabled.com/python-openid/ silva.pas.openid 1.1 7 OpenID suport for Silva - openidenabled - https://svn.infrae.com/silva.pas.openid/trunk/setup.py TracOpenIDDelegate 1.0 7 Add OpenID delegation links to a Trac site. - merely delegates the auth to other site authopenid_middleware 0.1 6 OpenID authentication middleware for WSGI applications - yep, another openenabled wrapper TGOpenIDLogin 0.1 6 OpenID login controller for TurboGears - guess what? http://nxsy.org/code/tgopenidlogin/ TracAuthOpenId 0.1.9 6 OpenID plugin for Trac - the same middleware wrapper, openidenabled TestOpenID 0.2.2 5 A test consumer and server for Open ID. - demonstration of how to use the python-openid library, enabled gracie 0.2.8 3 Gracie - OpenID provider for local accounts - GPLed and enabled AuthKit 0.4.3 1 An authentication and authorization toolkit for WSGI applications and frameworks - you know the answer - http://authkit.org/trac/browser/AuthKit/trunk/setup.py plone.app.layout 1.1.7 1 Layout mechanisms for Plone - irrelevant Products.SilvaForum 0.3.1 1 Forum for Silva - irrelevant wsgiauth 0.1 - uses openidenabled No option. -- --anatoly t. From skippy.hammond at gmail.com Thu Jan 22 23:16:25 2009 From: skippy.hammond at gmail.com (Mark Hammond) Date: Fri, 23 Jan 2009 09:16:25 +1100 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> <49763FC6.9090303@v.loewis.de> <49777A99.9010607@v.loewis.de> <49778FF6.8070608@v.loewis.de> <4978BE6D.6060000@v.loewis.de> Message-ID: <4978F039.2060008@gmail.com> On 23/01/2009 7:01 AM, Luke Kenneth Casson Leighton wrote: >> That doesn't really matter, I guess. An extension module build by your >> port will either fail to load into the regular Python (if >> libpython2.5.dll is not found), or load and then crash (because it uses >> a different copy of the Python runtime). Likewise vice versa. > > > excellent, excellent that's _really_ good - and here's why: > > if it is _guaranteed_ to crash, regardless of what i do (because the > copy of the python runtime is different), then it _doesn't_ matter > what version of msvcrt i link the mingw-built python runtime with, > does it? I'm very confused about this: It seems you started this work precisely so you can be compatible between MSVC built Python's and mingw builds - ie, this thread starts with you saying: > this is a fairly important issue for python development > interoperability - but now you seem to be saying it is actually a *feature* if they don't work together? > and if _that's_ the case, i can stop fricking about with msvcr80 :) If all you are doing is trying to get a version of Python working under Wine that isn't compatible with MSVC built binaries, I can't work out why you are fricking around with msvcr80 either! Cheers, Mark From bugtrack at roumenpetrov.info Fri Jan 23 00:13:40 2009 From: bugtrack at roumenpetrov.info (Roumen Petrov) Date: Fri, 23 Jan 2009 01:13:40 +0200 Subject: [Python-Dev] progress: compiling python2.5 under msys (specifically but not exclusively under wine) with msvcr80 In-Reply-To: <50019.151.53.150.247.1232570935.squirrel@webmail1.pair.com> References: <50019.151.53.150.247.1232570935.squirrel@webmail1.pair.com> Message-ID: <4978FDA4.8050101@roumenpetrov.info> Cesare Di Mauro wrote: > Have you made some benchmarks like pystone? There is result from pystone test test run an old PC (NT 5.1): - 2.6(official build): 42194,6; 42302,4; 41990,8; 42658,0; 42660,6; 42770,1 average=42429,4 deviation=311,6 - 2.6.1(official build): 35612,1; 35778,8; 35666,7; 35697,9; 35514,9; 35654,0 average=35654,1 deviation=88,1 - trunk(my mingw based build): 35256,7; 35272,5; 35247,2; 35270,7; 35225,6; 35233,5 average=35251,0 deviation=19,2 There is problem with python performance between 2.6 and 2.6.1 ~ 19% :(. Also the test for GCC-mingw is not with same source base. Roumen From bugtrack at roumenpetrov.info Fri Jan 23 00:22:28 2009 From: bugtrack at roumenpetrov.info (Roumen Petrov) Date: Fri, 23 Jan 2009 01:22:28 +0200 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> <49763FC6.9090303@v.loewis.de> <49777A99.9010607@v.loewis.de> <49778FF6.8070608@v.loewis.de> Message-ID: <4978FFB4.1050802@roumenpetrov.info> Luke Kenneth Casson Leighton wrote: > On Wed, Jan 21, 2009 at 9:13 PM, "Martin v. L?wis" wrote: >>> ok, so - different from what's being generated by ./configure under >>> msys under wine or native win32 - what's being generated (libpython 2 >>> . 5 . a and libpython 2 . 5 . dll . a) is more akin to the cygwin >>> environment. >>> >>> therefore, there's absolutely no doubt that the two are completely different. >>> >>> and on that basis, would i be correct in thinking that you _can't_ go >>> linking or building modules or any python win32 code for one and have >>> a hope in hell of using it on the other, and that you would _have_ to >>> rebuild e.g. numpy for use with a mingw32-msys-built version of >>> python? >> I can't comment on that, because I don't know what your port does. >> Does it not produce a .dll containing the majority of Python? > > no, it contains the minimal necessary amount of python modules, > exactly like when python is built using cygwin. actualy, there's a > few modules that _have_ to be included. > > roumen discovered that you have to have these: > > _functools _functoolsmodule.c # Tools for working with functions > and callable objects > operator operator.c # operator.add() and similar goodies > _locale _localemodule.c # -lintl > _struct _struct.c > _subprocess ../PC/_subprocess.c > _winreg ../PC/_winreg.c Yes and this is issue in native build - setup.py fail to load :(. In cross-build where I use python from build system I could produce those as modules. > and i've discovered that when running under wine you have to also have these: > _weakref _weakref.c > > and also when running unde wine with msvcr80, so far, you have to also > have these: > collections collectionsmodule.c > thread threadmodule.c > > all the rest can be done as .pyd [SNIP] Actually I didn't spend time to find why MSVC build include so many modules as build-ins. Roumen From bugtrack at roumenpetrov.info Fri Jan 23 00:46:26 2009 From: bugtrack at roumenpetrov.info (Roumen Petrov) Date: Fri, 23 Jan 2009 01:46:26 +0200 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: <9613db600901200511tebe4f6bya1049ee42acac0fc@mail.gmail.com> <49763FC6.9090303@v.loewis.de> <49777A99.9010607@v.loewis.de> <49778FF6.8070608@v.loewis.de> <4978BE6D.6060000@v.loewis.de> Message-ID: <49790552.8050504@roumenpetrov.info> Luke Kenneth Casson Leighton wrote: >> version. I hope that users will understand that it is disjoint from >> the python.org version (as they seem to understand fine for the >> Cygwin build, which already picks up its extension modules also from >> a disjoint location, which helps to keep the two separate). > > yes i made the default installation location (--prefix=) c:/python2.5 > _not_ c:/python25 but obviously it _has_ been necessary to make the > installation of modules into the exact same _style_ of location as the > msvc build, because it has obviously also been necessary to use > PC/getpathp.c not getpath.c I'm thinking about possibility to avoid compatible paths, i.e. to drop windows specific PC/getpathp.c and to return back to posix getpath.c. The problem is that MSVC based build is not installed in tree compatible to the posix build. Now I think that GCC(mingw) build has to use same tree as other posix builds. Mixing posix build and install (makefile) with paths used by from MSVC build require additional changes either in makefile or in PC/getpathp.c. In the both case change is more the 100 lines and now I prefer mingw to use posix tree. This open another issue. ?he posix build install in fixed location. I think that with a small change in distutils (no more then 10-20 lines) we may overcome this. > so, .pyd modules will get installed in > c:/python2.5/lib/site-packages/ and most importantly they'll get > _looked_ for in there! for a while, they were being installed in > c:/python2.5/lib/python2.5/site-packages which was a bit of a mess - > that's the "unix" style of module locations. getpathp.c looks for > "Lib/os.py" whilst getpath.c looks for "os.py" > > there's a whole _stack_ of knock-on effect little things like that. :) The installation is the last issue. Roumen From rasky at develer.com Fri Jan 23 02:22:15 2009 From: rasky at develer.com (Giovanni Bajo) Date: Fri, 23 Jan 2009 01:22:15 +0000 (UTC) Subject: [Python-Dev] __del__ and tp_dealloc in the IO lib References: <52dc1c820901181738h53e7ecf9if254a91de2025722@mail.gmail.com> Message-ID: On Mon, 19 Jan 2009 01:38:18 +0000, Gregory P. Smith wrote: > I regularly point out in code reviews that the very convenient and > common idiom of open(name, 'w').write(data) doesn't guarantee when the > file will be closed; its up to the GC implementation details. Which, to me, sounds like "please, don't assume that bytes are 8-bits wide; this depends on implementation details of your CPU". CPython will always use reference counting and thus have a simple and clear GC criteria that can be exploited to simplify the code. I personally don't like defensive programming, nor coding for situations that will never arise . When I write CPython applications (thus, for instance, using C extensions), I don't see *any* point in trying to achieve any cross-python-implementation compatibility. I simply don't need it. Probably, library programmers have a different point of view. But I always object when I'm told that I should make my code longer and harder to read only because CPython might stop using reference counting (... when hell freezes over). Back to the topic, please let's keep things as they are now: the file descriptor is automatically closed as soon as the file object is destroyed. If you then feel "safer" always using with or try/finally, nobody is going to complain. And everybody will be happy :) -- Giovanni Bajo Develer S.r.l. http://www.develer.com From curt at hagenlocher.org Fri Jan 23 02:42:29 2009 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Thu, 22 Jan 2009 17:42:29 -0800 Subject: [Python-Dev] __del__ and tp_dealloc in the IO lib In-Reply-To: References: <52dc1c820901181738h53e7ecf9if254a91de2025722@mail.gmail.com> Message-ID: On Thu, Jan 22, 2009 at 5:22 PM, Giovanni Bajo wrote: > On Mon, 19 Jan 2009 01:38:18 +0000, Gregory P. Smith wrote: > >> I regularly point out in code reviews that the very convenient and >> common idiom of open(name, 'w').write(data) doesn't guarantee when the >> file will be closed; its up to the GC implementation details. > > Which, to me, sounds like "please, don't assume that bytes are 8-bits > wide; this depends on implementation details of your CPU". I think it's a lot more like "please, don't assume that there's a Global Interpreter Lock" -- something that the implementation shouldn't change without good reason and sufficient warning, but which isn't actually part of the language specification. And of course, such advice always carries more weight for code that's intended to be reusable than it does for code that has little chance of escaping the application it's in. -- Curt Hagenlocher curt at hagenlocher.org From steve at holdenweb.com Fri Jan 23 02:48:11 2009 From: steve at holdenweb.com (Steve Holden) Date: Thu, 22 Jan 2009 20:48:11 -0500 Subject: [Python-Dev] __del__ and tp_dealloc in the IO lib In-Reply-To: References: <52dc1c820901181738h53e7ecf9if254a91de2025722@mail.gmail.com> Message-ID: Giovanni Bajo wrote: > On Mon, 19 Jan 2009 01:38:18 +0000, Gregory P. Smith wrote: > >> I regularly point out in code reviews that the very convenient and >> common idiom of open(name, 'w').write(data) doesn't guarantee when the >> file will be closed; its up to the GC implementation details. > > Which, to me, sounds like "please, don't assume that bytes are 8-bits > wide; this depends on implementation details of your CPU". > Which it does, assuming you are using (for example) an ancient DECSystem-10. But you really can't assume in your writings about Python that all readers will be using CPython, so it seems like a reasonable point to make. > CPython will always use reference counting and thus have a simple and > clear GC criteria that can be exploited to simplify the code. I > personally don't like defensive programming, nor coding for situations > that will never arise . When I write CPython applications (thus, for > instance, using C extensions), I don't see *any* point in trying to > achieve any cross-python-implementation compatibility. I simply don't > need it. > Who gave you this guarantee of CPython's future behavior? Who knows which implementation of Python will be used to support your code and mine in five years? > Probably, library programmers have a different point of view. As they properly should. > But I > always object when I'm told that I should make my code longer and harder > to read only because CPython might stop using reference counting (... > when hell freezes over). > Ah, religious beliefs ... ;-) > Back to the topic, please let's keep things as they are now: the file > descriptor is automatically closed as soon as the file object is > destroyed. If you then feel "safer" always using with or try/finally, > nobody is going to complain. And everybody will be happy :) And what are the IronPython team, and the Jython team, supposed to do when they get around to implementing Python 3? Clearly (since both teams are already committed to implementing it) the more we can do to accommodate them the better it will be for cross-implementation compatibility. Or did I miss something? You are, of course, free to make whatever assumptions you like about the environment in which your code executes. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From guido at python.org Fri Jan 23 03:42:25 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 22 Jan 2009 18:42:25 -0800 Subject: [Python-Dev] __del__ and tp_dealloc in the IO lib In-Reply-To: References: <52dc1c820901181738h53e7ecf9if254a91de2025722@mail.gmail.com> Message-ID: On Thu, Jan 22, 2009 at 5:22 PM, Giovanni Bajo wrote: > CPython will always use reference counting and thus have a simple and > clear GC criteria that can be exploited to simplify the code. Believe this at your own peril. Once, CPython didn't have GC at all (apart from refcounting). Now it does. There are GC techniques that delay DECREF operations until it's more convenient. If someone finds a way to exploit that technique to save 10% of execution time it would only be right to start using it. You *can* assume that objects that are no longer referenced will *eventually* be GC'ed, and that GC'ing a file means flushing its buffer and closing its file descriptor. You *cannot* assume that objects are *immediately* GC'ed. This is already not always true in CPython for many different reasons, like objects involved in cycles, weak references, or tracebacks saved with exceptions, or perhaps profiling/debugging hooks. If we found a good reason to introduce file objects into some kind of cycle or weak reference dict, I could see file objects getting delayed reclamation even without changes in GC implementation. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From billiejoex at gmail.com Fri Jan 23 03:13:27 2009 From: billiejoex at gmail.com (Giampaolo Rodola') Date: Fri, 23 Jan 2009 03:13:27 +0100 Subject: [Python-Dev] Should ftplib use UTF-8 instead of latin-1 encoding? Message-ID: <729626cc0901221813m72a9896bj30092dbfbd5cb133@mail.gmail.com> Hi, while attempting to port pyftpdlib [1] to Python 3 I have noticed that ftplib differs from the previous 2.x version in that it uses latin-1 to encode everything it's sent over the FTP command channel, but by reading RFC-2640 [2] it seems that UTF-8 should be preferred instead. I'm far from being an expert of encodings, plus the RFC is quite hard to understand, so sorry in advance if I have misunderstood the whole thing. Just wanted to put this up to people more qualified than me. [1] http://code.google.com/p/pyftpdlib [2] http://www.ietf.org/rfc/rfc2640.txt --- Giampaolo http://code.google.com/p/pyftpdlib From brett at python.org Fri Jan 23 06:15:55 2009 From: brett at python.org (Brett Cannon) Date: Thu, 22 Jan 2009 21:15:55 -0800 Subject: [Python-Dev] PEP 374 (DVCS) now in reST Message-ID: I have now converted PEP 374 (http://www.python.org/dev/peps/pep-0374/) from Google Docs to reST and checked it in. I am not going to paste it into an email as it is nearly 1500 lines in reST form. Because there are four authors handling corrections it is a little different than normal on who you contact to suggest changes. For each specific VCS there is a primary author as listed in the PEP in the intro to the Scenarios section. Email the proper author if you find an issue with a specific VCS. Otherwise email me for general PEP issues. Core developers can make changes on their own while taking into account they should let the author in charge of the PEP know if they make a big change. Since I will be the author making the final recommendation I am documenting my thought processes on my decision making for this whole thing as I go along in the PEP so as to be as transparent as possible. I am not even close to being done, so please don't email me about the section. And I would like to thank my co-authors for their time and effort thus far in filling in the PEP on behalf of their favorite DVCS. Everyone has put in a lot of time already with I am sure more time in the future. -Brett From lkcl at lkcl.net Fri Jan 23 07:36:02 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Fri, 23 Jan 2009 06:36:02 +0000 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: <4978E09F.4080507@v.loewis.de> References: <49777A99.9010607@v.loewis.de> <49778FF6.8070608@v.loewis.de> <4978BE6D.6060000@v.loewis.de> <4978D46C.3030200@v.loewis.de> <4978E09F.4080507@v.loewis.de> Message-ID: On Thu, Jan 22, 2009 at 9:09 PM, "Martin v. L?wis" wrote: >> mrmmm... how? apps won't end up loading _both_ libpython2.5.dll _and_ >> python25.dll (or libpython2.N.dll and python2N.dll) will they? > > Of course they will! yeah, silly - i worked that out juust after i pressed "send". > python.exe (say, the official one) loads > python25.dll. Then, an import is made of a ming-wine extension, say > foo.pyd, which is linked with libpython2.5.dll, which then gets loaded. > Voila, you have two interpreters in memory, with different type objects, > memory heaps, and so on. ok, there's a solution for that - the gist of the solution is already implemented in things like Apache Runtime and Apache2 (modules), and is an extremely common standard technique implemented in OS kernels. the "old school" name for it is "vector tables". so you never let the .pyd (or so even) modules "link" to the libpythonN.N.dll, pythonNN.dll (or libpythonN.N.so even), you _pass in_ a pointer to everything it's ever going to need (in its init ) function. > This was always the problem when an old extension module (say, from 2.4) > was loaded into a new interpreter (say, 2.5), then you load both > python25.dll and python24.dll, causing crashes. To prevent this issue, > Python now checks whether the module is linked with an incorrect > pythonxy.dll, but won't catch that libpython2.5.dll is also a VM. ok. i'll look at making libpython2.5.dll equal to python25.dll. >>> (of course, msvcr80 is irrelevant, because Python had never been using >>> that officially) >> >> oh? i saw the PCbuild8 and thought it was. oh that's even better - >> if python2.5 only officially support msvcrt whew. > > No, Python 2.5 is linked with msvcr71.dll. ehn? i don't see that anywhere in any PC/* files - i do see that there's a dependency on .NET SDK 1.1 which uses msvcr71.dll still, 71 is good news - as long as it's not involving assemblies. 2.6 is a different matter, but, thinking about it, i have hopes that the better-tested-codepath of the 2.6 codebase would have better luck with 9.0 [than i had with 2.5 and 8.0] simply because... it's been tested already! [and 2.5 with 8.0 hadn't] l. From lkcl at lkcl.net Fri Jan 23 07:37:25 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Fri, 23 Jan 2009 06:37:25 +0000 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: <49778FF6.8070608@v.loewis.de> <4978BE6D.6060000@v.loewis.de> <4978D46C.3030200@v.loewis.de> <4978E09F.4080507@v.loewis.de> Message-ID: On Fri, Jan 23, 2009 at 6:36 AM, Luke Kenneth Casson Leighton wrote: > On Thu, Jan 22, 2009 at 9:09 PM, "Martin v. L?wis" wrote: >>> mrmmm... how? apps won't end up loading _both_ libpython2.5.dll _and_ >>> python25.dll (or libpython2.N.dll and python2N.dll) will they? >> >> Of course they will! > > yeah, silly - i worked that out juust after i pressed "send". ironically, i started out with the intent of going for python2N.dll interoperability. then i noticed that all the other mingw ports dropped the total-inclusion-of-all-modules .... because you _can_. i should have stuck with my original plan :) From lkcl at lkcl.net Fri Jan 23 08:22:16 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Fri, 23 Jan 2009 07:22:16 +0000 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: <4978F039.2060008@gmail.com> References: <49763FC6.9090303@v.loewis.de> <49777A99.9010607@v.loewis.de> <49778FF6.8070608@v.loewis.de> <4978BE6D.6060000@v.loewis.de> <4978F039.2060008@gmail.com> Message-ID: On Thu, Jan 22, 2009 at 10:16 PM, Mark Hammond wrote: > On 23/01/2009 7:01 AM, Luke Kenneth Casson Leighton wrote: >>> >>> That doesn't really matter, I guess. An extension module build by your >>> port will either fail to load into the regular Python (if >>> libpython2.5.dll is not found), or load and then crash (because it uses >>> a different copy of the Python runtime). Likewise vice versa. >> >> >> excellent, excellent that's _really_ good - and here's why: >> >> if it is _guaranteed_ to crash, regardless of what i do (because the >> copy of the python runtime is different), then it _doesn't_ matter >> what version of msvcrt i link the mingw-built python runtime with, >> does it? > > I'm very confused about this: It seems you started this work precisely so > you can be compatible between MSVC built Python's and mingw builds yeah that's where i _started_ - and after being on this for what nearly eight days straight i was hoping to get away with as little extra work as possible. > - ie, > this thread starts with you saying: > >> this is a fairly important issue for python development >> interoperability > > - but now you seem to be saying it is actually a *feature* if they don't > work together? *sigh* no, that was me getting confused >> and if _that's_ the case, i can stop fricking about with msvcr80 :) > > If all you are doing is trying to get a version of Python working under Wine > that isn't compatible with MSVC built binaries, I can't work out why you are > fricking around with msvcr80 either! ha ha :) existence of PCbuild8 is the main reason :) that and getting the wrong end of the stick. i'll get there eventually. From tjreedy at udel.edu Fri Jan 23 08:45:23 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 23 Jan 2009 02:45:23 -0500 Subject: [Python-Dev] Should ftplib use UTF-8 instead of latin-1 encoding? In-Reply-To: <729626cc0901221813m72a9896bj30092dbfbd5cb133@mail.gmail.com> References: <729626cc0901221813m72a9896bj30092dbfbd5cb133@mail.gmail.com> Message-ID: Giampaolo Rodola' wrote: > Hi, > while attempting to port pyftpdlib [1] to Python 3 I have noticed that > ftplib differs from the previous 2.x version in that it uses latin-1 > to encode everything it's sent over the FTP command channel, but by > reading RFC-2640 [2] it seems that UTF-8 should be preferred instead. > I'm far from being an expert of encodings, plus the RFC is quite hard > to understand, so sorry in advance if I have misunderstood the whole > thing. I read it the same way. The whole point of the RFC is that UTF-8 rather than the very limited latin-1 is needed for true internationalization. > Just wanted to put this up to people more qualified than me. > > > [1] http://code.google.com/p/pyftpdlib > [2] http://www.ietf.org/rfc/rfc2640.txt > > > --- Giampaolo > http://code.google.com/p/pyftpdlib > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org > From martin at v.loewis.de Fri Jan 23 09:06:45 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Fri, 23 Jan 2009 09:06:45 +0100 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: <49777A99.9010607@v.loewis.de> <49778FF6.8070608@v.loewis.de> <4978BE6D.6060000@v.loewis.de> <4978D46C.3030200@v.loewis.de> <4978E09F.4080507@v.loewis.de> Message-ID: <49797A95.6050402@v.loewis.de> > ok, there's a solution for that - the gist of the solution is already > implemented in things like Apache Runtime and Apache2 (modules), and > is an extremely common standard technique implemented in OS kernels. > the "old school" name for it is "vector tables". We might be able to do that in Python 4; it would certainly require a PEP. >> No, Python 2.5 is linked with msvcr71.dll. > > ehn? i don't see that anywhere in any PC/* files - i do see that > there's a dependency on .NET SDK 1.1 which uses msvcr71.dll Take a look at PCbuild/pythoncore.vcproj. It says Version="7.10" This is how you know VS 2003 was used to build Python 2.5, which in turn links in msvcr71.dll. Regards, Martin From martin at v.loewis.de Fri Jan 23 09:08:10 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Fri, 23 Jan 2009 09:08:10 +0100 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: <49778FF6.8070608@v.loewis.de> <4978BE6D.6060000@v.loewis.de> <4978D46C.3030200@v.loewis.de> <4978E09F.4080507@v.loewis.de> Message-ID: <49797AEA.8000307@v.loewis.de> > ironically, i started out with the intent of going for python2N.dll > interoperability. then i noticed that all the other mingw ports > dropped the total-inclusion-of-all-modules .... because you _can_. What modules are built in and what modules are external doesn't affect interoperability wrt. (third-party) extension modules. Regards, Martin From lkcl at lkcl.net Fri Jan 23 09:27:16 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Fri, 23 Jan 2009 08:27:16 +0000 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: <49797AEA.8000307@v.loewis.de> References: <4978BE6D.6060000@v.loewis.de> <4978D46C.3030200@v.loewis.de> <4978E09F.4080507@v.loewis.de> <49797AEA.8000307@v.loewis.de> Message-ID: On Fri, Jan 23, 2009 at 8:08 AM, "Martin v. L?wis" wrote: >> ironically, i started out with the intent of going for python2N.dll >> interoperability. then i noticed that all the other mingw ports >> dropped the total-inclusion-of-all-modules .... because you _can_. > > What modules are built in and what modules are external doesn't affect > interoperability wrt. (third-party) extension modules. ahhh .... sooo.... [said in a japanese kind of way that indicates understanding...] From martin at v.loewis.de Fri Jan 23 09:31:23 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 23 Jan 2009 09:31:23 +0100 Subject: [Python-Dev] Should ftplib use UTF-8 instead of latin-1 encoding? In-Reply-To: <729626cc0901221813m72a9896bj30092dbfbd5cb133@mail.gmail.com> References: <729626cc0901221813m72a9896bj30092dbfbd5cb133@mail.gmail.com> Message-ID: <4979805B.4030107@v.loewis.de> Giampaolo Rodola' wrote: > Hi, > while attempting to port pyftpdlib [1] to Python 3 I have noticed that > ftplib differs from the previous 2.x version in that it uses latin-1 > to encode everything it's sent over the FTP command channel, but by > reading RFC-2640 [2] it seems that UTF-8 should be preferred instead. > I'm far from being an expert of encodings, plus the RFC is quite hard > to understand, so sorry in advance if I have misunderstood the whole > thing. I read it that a conforming client MUST issue a FEAT command, to determine whether the server supports UTF8. One would have to go back to the original FTP RFC, but it seams that, in the absence of server UTF8 support, all path names must be 7-bit clean (which means that ASCII should be the default encoding). In any case, Brett changed the encoding to latin1 in r58378, maybe he can comment. Regards, Martin From martin at v.loewis.de Fri Jan 23 09:39:17 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 23 Jan 2009 09:39:17 +0100 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: References: Message-ID: <49798235.50509@v.loewis.de> > And I would like to thank my co-authors for their time and effort thus > far in filling in the PEP on behalf of their favorite DVCS. Everyone > has put in a lot of time already with I am sure more time in the > future. So what will happen next? ISTM that the PEP is not complete, since it doesn't specify a specific DVCS to migrate to (i.e. it wouldn't be possible to implement the PEP as it stands). Somebody will have to make a decision. Ultimately, Guido will have to approve the PEP, but it might be that he refuses to make a choice of specific DVCS. Traditionally, it is the PEP author who makes all choices (considering suggestions from the community, of course). So what DVCS do the PEP authors recommend? Regards, Martin From eckhardt at satorlaser.com Fri Jan 23 10:19:39 2009 From: eckhardt at satorlaser.com (Ulrich Eckhardt) Date: Fri, 23 Jan 2009 10:19:39 +0100 Subject: [Python-Dev] Update on MS Windows CE port Message-ID: <200901231019.39111.eckhardt@satorlaser.com> Hi! Just a short update on my porting issues. I tried various things in the holidays/vacation, filed several bug reports, got a few patches applied and I think the thing is taking shape now. However, I couldn't work on it for the past two weeks since I'm just too swamped at work here. I haven't given up though! What still needs work? The main component that requires porting is the pythoncore project, but that port isn't finished yet. I'm using the VS8.0 project files as a base and adapted it in the following steps (note that this is preliminary): 0. Check out trunk (i.e. 2.x, not 3.x). 1. Create new project configuration for your CE target and use the win32 one as a base. 2. In the preprocessor settings, add these entries: UNICODE,_UNICODE,UNDER_CE,_WIN32_WCE=$(CEVER) 3. Try compiling. ;) What issues are left? There are two classes of errors you will encounter, those that are related to CE itself (like missing errno and other parts of standard C) and those that are more general, like assuming TCHAR=char. Those that assume that TCHAR=char can also be found with the plain win32 variant by simply adding UNICODE and _UNICODE to the preprocessor defines. What are the future plans? I'm trying to fix errors in pythoncore one by one and provide separate bug reports with patches. Note that there are already several patches in the BTS which would merit reviews if you have time, even if it's only a "patch applies and doesn't cause regressions". When I get pythoncore to compile (and be it by disabling a few builtin modules), I will try to get the commandline interface to work. Note that IIRC there are two commandlines availabel under CE, one is a common cmd.exe, the other is called AYGShell or some such. Both will/might require different approaches. Another problem is the fact that there is not just one target platform but a potentially infinite number of them, since every CE device can have a separate SDK which further might provide different features. This probably requires autogenerated build files or some other autoconf-like procedure that allows enabling or disabling certain features for builds. so much for now Uli -- Sator Laser GmbH Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Sator Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at ************************************************************************************** Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder anderweitig benutzt werden. E-Mails k?nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte ?nderungen enthalten. Sator Laser GmbH ist f?r diese Folgen nicht verantwortlich. ************************************************************************************** From dirkjan at ochtman.nl Fri Jan 23 10:56:54 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Fri, 23 Jan 2009 09:56:54 +0000 (UTC) Subject: [Python-Dev] PEP 374 (DVCS) now in reST References: <49798235.50509@v.loewis.de> Message-ID: Martin v. L?wis v.loewis.de> writes: > Somebody will have to make a decision. Ultimately, Guido will have to > approve the PEP, but it might be that he refuses to make a choice of > specific DVCS. Traditionally, it is the PEP author who makes all > choices (considering suggestions from the community, of course). So > what DVCS do the PEP authors recommend? Brett mentioned in his email that he wasn't ready to make a decision yet, I think? I also think that the PEP could still use some modifications from people who have more experience with the DVCSs. I'll probably send in some suggestions later today. Cheers, Dirkjan From rasky at develer.com Fri Jan 23 11:57:00 2009 From: rasky at develer.com (Giovanni Bajo) Date: Fri, 23 Jan 2009 11:57:00 +0100 Subject: [Python-Dev] __del__ and tp_dealloc in the IO lib In-Reply-To: References: <52dc1c820901181738h53e7ecf9if254a91de2025722@mail.gmail.com> Message-ID: <1232708220.8900.23.camel@ozzu> On gio, 2009-01-22 at 18:42 -0800, Guido van Rossum wrote: > On Thu, Jan 22, 2009 at 5:22 PM, Giovanni Bajo wrote: > > CPython will always use reference counting and thus have a simple and > > clear GC criteria that can be exploited to simplify the code. > > Believe this at your own peril. > > Once, CPython didn't have GC at all (apart from refcounting). Now it > does. There are GC techniques that delay DECREF operations until it's > more convenient. If someone finds a way to exploit that technique to > save 10% of execution time it would only be right to start using it. > > You *can* assume that objects that are no longer referenced will > *eventually* be GC'ed, and that GC'ing a file means flushing its > buffer and closing its file descriptor. You *cannot* assume that > objects are *immediately* GC'ed. This is already not always true in > CPython for many different reasons, like objects involved in cycles, > weak references, I don't understand what you mean with weak references delaying object deallocation. I'm probably missing something here... > or tracebacks saved with exceptions, or perhaps > profiling/debugging hooks. If we found a good reason to introduce file > objects into some kind of cycle or weak reference dict, I could see > file objects getting delayed reclamation even without changes in GC > implementation. That would be break so much code that I doubt that, in practice, you can slip it through within a release. Besides, being able to write simpler code like "for L in open("foo.txt")" is per-se a good reason *not to* put file objects in cycles; so you will probably need more than one good reason to change this. OK, not *you* because of your BDFL powers ;), but anyone else would surely have to face great opposition. The fact that file objects are collected and closed immediately in all reasonable use cases (and even in case of exceptions, that you mention, things get even better with the new semantic of the except clause) is a *good* property of Python. I regularly see people *happy* about it. I miss to understand why many Python developers are so fierce in trying to push the idea of cross-python compatibility (which is something that does simply *not* exist in real world for applications) or to warn about rainy days in the future when this would stop working in CPython. I would strongly prefer that CPython would settle on (= document) using reference counting and immediate destruction so that people can stop making their everyday code more complex with no benefit. You will be losing no more than an open door that nobody has entered in 20 years, and people would only benefit from it. -- Giovanni Bajo Develer S.r.l. http://www.develer.com From solipsis at pitrou.net Fri Jan 23 12:05:48 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 23 Jan 2009 11:05:48 +0000 (UTC) Subject: [Python-Dev] PEP 374 (DVCS) now in reST References: Message-ID: Brett Cannon python.org> writes: > > I have now converted PEP 374 > (http://www.python.org/dev/peps/pep-0374/) from Google Docs to reST > and checked it in. I am not going to paste it into an email as it is > nearly 1500 lines in reST form. It seems the ">>" token is mangled into a French closing quote ("?") inside code snippets. Antoine. From steve at holdenweb.com Fri Jan 23 13:05:59 2009 From: steve at holdenweb.com (Steve Holden) Date: Fri, 23 Jan 2009 07:05:59 -0500 Subject: [Python-Dev] __del__ and tp_dealloc in the IO lib In-Reply-To: <1232708220.8900.23.camel@ozzu> References: <52dc1c820901181738h53e7ecf9if254a91de2025722@mail.gmail.com> <1232708220.8900.23.camel@ozzu> Message-ID: <4979B2A7.705@holdenweb.com> Giovanni Bajo wrote: > On gio, 2009-01-22 at 18:42 -0800, Guido van Rossum wrote: [...] > I miss to understand why many Python developers are so fierce in trying > to push the idea of cross-python compatibility (which is something that > does simply *not* exist in real world for applications) or to warn about > rainy days in the future when this would stop working in CPython. I > would strongly prefer that CPython would settle on (= document) using > reference counting and immediate destruction so that people can stop > making their everyday code more complex with no benefit. You will be > losing no more than an open door that nobody has entered in 20 years, > and people would only benefit from it. Probably because it's good practice to write for compatibility where possible. Cross-OS compatibility isn't possible in the general case either, but it's still a good goal in the cases where it *is* possible. Given that your sample code will generally work even for implementations where garbage collection is used rather than reference counting I fail to understand why you insist so hard that a more restrictive rule should be implemented. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Fri Jan 23 13:05:59 2009 From: steve at holdenweb.com (Steve Holden) Date: Fri, 23 Jan 2009 07:05:59 -0500 Subject: [Python-Dev] __del__ and tp_dealloc in the IO lib In-Reply-To: <1232708220.8900.23.camel@ozzu> References: <52dc1c820901181738h53e7ecf9if254a91de2025722@mail.gmail.com> <1232708220.8900.23.camel@ozzu> Message-ID: <4979B2A7.705@holdenweb.com> Giovanni Bajo wrote: > On gio, 2009-01-22 at 18:42 -0800, Guido van Rossum wrote: [...] > I miss to understand why many Python developers are so fierce in trying > to push the idea of cross-python compatibility (which is something that > does simply *not* exist in real world for applications) or to warn about > rainy days in the future when this would stop working in CPython. I > would strongly prefer that CPython would settle on (= document) using > reference counting and immediate destruction so that people can stop > making their everyday code more complex with no benefit. You will be > losing no more than an open door that nobody has entered in 20 years, > and people would only benefit from it. Probably because it's good practice to write for compatibility where possible. Cross-OS compatibility isn't possible in the general case either, but it's still a good goal in the cases where it *is* possible. Given that your sample code will generally work even for implementations where garbage collection is used rather than reference counting I fail to understand why you insist so hard that a more restrictive rule should be implemented. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From solipsis at pitrou.net Fri Jan 23 13:33:14 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 23 Jan 2009 12:33:14 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?=5F=5Fdel=5F=5F_and_tp=5Fdealloc_in_the_IO?= =?utf-8?q?_lib?= References: <52dc1c820901181738h53e7ecf9if254a91de2025722@mail.gmail.com> <1232708220.8900.23.camel@ozzu> Message-ID: Giovanni Bajo develer.com> writes: > > The fact that file objects are collected and closed immediately in all > reasonable use cases (and even in case of exceptions, that you mention, > things get even better with the new semantic of the except clause) The new except clause removes any external references to the exception, but there's still, AFAIR, the reference cycle through the traceback object, which means the whole thing will still have to wait for a pass of the cyclic garbage collector. Regards Antoine. From fijall at gmail.com Fri Jan 23 14:28:43 2009 From: fijall at gmail.com (Maciej Fijalkowski) Date: Fri, 23 Jan 2009 14:28:43 +0100 Subject: [Python-Dev] __del__ and tp_dealloc in the IO lib In-Reply-To: <1232708220.8900.23.camel@ozzu> References: <52dc1c820901181738h53e7ecf9if254a91de2025722@mail.gmail.com> <1232708220.8900.23.camel@ozzu> Message-ID: <693bc9ab0901230528h1165adag354310f03e065892@mail.gmail.com> > That would be break so much code that I doubt that, in practice, you can > slip it through within a release. Besides, being able to write simpler > code like "for L in open("foo.txt")" is per-se a good reason *not to* > put file objects in cycles; so you will probably need more than one good > reason to change this. OK, not *you* because of your BDFL powers ;), but > anyone else would surely have to face great opposition. note that this is about *writing* files, not reading. You would be surprised how much of the software have already taken care to be cross-platform (ie twisted, django, pylons, ...), not to rely on that and be able to run on any other python implementation. > > The fact that file objects are collected and closed immediately in all > reasonable use cases (and even in case of exceptions, that you mention, > things get even better with the new semantic of the except clause) is a > *good* property of Python. I regularly see people *happy* about it. > > I miss to understand why many Python developers are so fierce in trying > to push the idea of cross-python compatibility (which is something that > does simply *not* exist in real world for applications) or to warn about > rainy days in the future when this would stop working in CPython. I > would strongly prefer that CPython would settle on (= document) using > reference counting and immediate destruction so that people can stop > making their everyday code more complex with no benefit. You will be > losing no more than an open door that nobody has entered in 20 years, > and people would only benefit from it. someone said at some point "noone will ever need more than 650k of RAM". see what has happened. Cheers, fijal From rdmurray at bitdance.com Fri Jan 23 15:57:02 2009 From: rdmurray at bitdance.com (rdmurray at bitdance.com) Date: Fri, 23 Jan 2009 09:57:02 -0500 (EST) Subject: [Python-Dev] __del__ and tp_dealloc in the IO lib In-Reply-To: <1232708220.8900.23.camel@ozzu> References: <52dc1c820901181738h53e7ecf9if254a91de2025722@mail.gmail.com> <1232708220.8900.23.camel@ozzu> Message-ID: On Fri, 23 Jan 2009 at 11:57, Giovanni Bajo wrote: > The fact that file objects are collected and closed immediately in all > reasonable use cases (and even in case of exceptions, that you mention, > things get even better with the new semantic of the except clause) is a > *good* property of Python. I regularly see people *happy* about it. I have never assumed that python closed my files before the end of the program unless I told it to do so, and have always coded accordingly. To do otherwise strikes me as bad coding. I don't believe I ever considered that such an assumption was even thinkable: closing open files when I'm done with them is part of my set of "good programming" habits developed over years of coding, habits that I apply in _any_ language in which I write code. (In fact, it took me a while before I was willing to let python take care of closing the files at program end...and even now I sometimes close files explicitly even in short programs.) Closing file objects is a specific instance of a more general programming rule that goes something like "clean up when you are done". I do in general trust python to clean up python data structures because it knows better than I do when "done" arrives; but when I know when "done" is, I do the cleanup. I love the 'with' statement :) --RDM From guido at python.org Fri Jan 23 16:27:32 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 23 Jan 2009 07:27:32 -0800 Subject: [Python-Dev] __del__ and tp_dealloc in the IO lib In-Reply-To: <1232708220.8900.23.camel@ozzu> References: <52dc1c820901181738h53e7ecf9if254a91de2025722@mail.gmail.com> <1232708220.8900.23.camel@ozzu> Message-ID: On Fri, Jan 23, 2009 at 2:57 AM, Giovanni Bajo wrote: > I miss to understand why many Python developers are so fierce in trying > to push the idea of cross-python compatibility (which is something that > does simply *not* exist in real world for applications) or to warn about > rainy days in the future when this would stop working in CPython. I > would strongly prefer that CPython would settle on (= document) using > reference counting and immediate destruction so that people can stop > making their everyday code more complex with no benefit. You will be > losing no more than an open door that nobody has entered in 20 years, > and people would only benefit from it. You are so very wrong, my son. CPython's implementation strategy *will* evolve. Several groups are hard at work trying to make a faster Python interpreter, and when they succeed, everyone, including you, will want to use their version (or their version may simply *be* the new CPython). Plus, you'd be surprised how many people might want to port existing code (and that may include code that uses C extensions, many of which are also ported) to Jython or IronPython. Your mistake sounds more like "nobody will ever want to run this on Windows, so I won't have to use the os.path module" and other short-sighted ideas. While you may be right in the short run, it may also be the death penalty for a piece of genius code that is found to be unportable. And, by the way, "for line in open(filename): ..." will continue to work. It may just not close the file right away. This is a forgivable sin in a small program that opens a few files only. It only becomes a program when this is itself inside a loop that loops over many filenames -- you could run out of file descriptors. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Fri Jan 23 16:30:59 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 23 Jan 2009 10:30:59 -0500 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <49798235.50509@v.loewis.de> References: <49798235.50509@v.loewis.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Brett, thanks for putting this PEP together! On Jan 23, 2009, at 3:39 AM, Martin v. L?wis wrote: > Somebody will have to make a decision. Ultimately, Guido will have to > approve the PEP, but it might be that he refuses to make a choice of > specific DVCS. Traditionally, it is the PEP author who makes all > choices (considering suggestions from the community, of course). So > what DVCS do the PEP authors recommend? Brett, perhaps you should publish a tentative schedule. Milestones I'd like to see include * Initial impressions section completed * Call for rebuttals * Second draft of impressions * (perhaps multiple) Recommendations to Guido and python-dev * Experimental live branches deployed for testing * Final recommendation * Final decision My understanding is that a final decision will /not/ be made by Pycon. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSXnis3EjvBPtnXfVAQJTqwQAimSA/JDzYN132npgazIag3fwOk36DAJl vvYMXOfWqvfl9DO/8cPF9YFOwF7YdHM8k4wUTmfLYhE8JfefODjrdHkL5pdclwDg Pbb2tjMfl0vBOPeaaPnJ5NKIJMwyRWkVhFMyNU5KmBmVRPJXpAQM23IOORX2dAaI tLONmrvx8K4= =CF18 -----END PGP SIGNATURE----- From rasky at develer.com Fri Jan 23 16:48:21 2009 From: rasky at develer.com (Giovanni Bajo) Date: Fri, 23 Jan 2009 16:48:21 +0100 Subject: [Python-Dev] __del__ and tp_dealloc in the IO lib In-Reply-To: References: <52dc1c820901181738h53e7ecf9if254a91de2025722@mail.gmail.com> <1232708220.8900.23.camel@ozzu> Message-ID: <4979E6C5.3000502@develer.com> On 1/23/2009 4:27 PM, Guido van Rossum wrote: > On Fri, Jan 23, 2009 at 2:57 AM, Giovanni Bajo wrote: >> I miss to understand why many Python developers are so fierce in trying >> to push the idea of cross-python compatibility (which is something that >> does simply *not* exist in real world for applications) or to warn about >> rainy days in the future when this would stop working in CPython. I >> would strongly prefer that CPython would settle on (= document) using >> reference counting and immediate destruction so that people can stop >> making their everyday code more complex with no benefit. You will be >> losing no more than an open door that nobody has entered in 20 years, >> and people would only benefit from it. > > You are so very wrong, my son. CPython's implementation strategy > *will* evolve. Several groups are hard at work trying to make a faster > Python interpreter, and when they succeed, everyone, including you, > will want to use their version (or their version may simply *be* the > new CPython). I'm basing my assumption on 19 years of history of CPython. Please, correct me if I'm wrong, but the only thing that changed is that the cyclic-GC was added so that loops are now collected, but nothing change with respect to cyclic collection. And everybody (including you, IIRC) has always agreed that it would be very very hard to eradicate reference counting from CPython and all the existing extensions; so hard that it is probably more convenient to start a different interpreter implementation. > Plus, you'd be surprised how many people might want to port existing > code (and that may include code that uses C extensions, many of which > are also ported) to Jython or IronPython. I would love to be surprised, in fact! Since I fail to see any business strategy behind such a porting, I don't see this happening very often in the business industry (and even less in the open source community, where there are also political issues between those versions of Python, I would say). I also never met someone that wanted to make a cross-interpreter Python application, nor read about someone that has a reasonable use case for wanting to do that, besides geek fun; which is why I came to this conclusion, though I obviously have access only to a little information compared to other people in here. On the other hand, I see people using IronPython so that they can access to the .NET framework (which can't be ported to other Python versions), or using Java so that they can blend to existing Java programs. And those are perfectly good use cases for the existence of such interpreters, but not for the merits of writing cross-interpreter portable code. I would be pleased if you (or others) could point me to real-world use cases of this cross-interpreter portability. > Your mistake sounds more like "nobody will ever want to run this on > Windows, so I won't have to use the os.path module" and other > short-sighted ideas. While you may be right in the short run, it may > also be the death penalty for a piece of genius code that is found to > be unportable. And in fact, I don't defensively code cross-OS portable code. Python is great in that *most* of what you naturally write is portable; which means that, the day that you need it, it's a breeze to port your code (assuming that you have also picked up the correct extensions, which I always try to do). But that does not mean that I have today to waste time on something that I don't need. > And, by the way, "for line in open(filename): ..." will continue to > work. It may just not close the file right away. This is a forgivable > sin in a small program that opens a few files only. It only becomes a > program when this is itself inside a loop that loops over many > filenames -- you could run out of file descriptors. I do understand this, but I'm sure you realize that there other similars example where the side effects are far worse. Maybe you don't care since you simply decided to declare that code *wrong*. But I'm unsure the community will kindly accept such a deep change in behaviour. Especially within the existing 2.x or 3.x release lines. -- Giovanni Bajo Develer S.r.l. http://www.develer.com From solipsis at pitrou.net Fri Jan 23 16:58:03 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 23 Jan 2009 15:58:03 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?=5F=5Fdel=5F=5F_and_tp=5Fdealloc_in_the_IO?= =?utf-8?q?_lib?= References: <52dc1c820901181738h53e7ecf9if254a91de2025722@mail.gmail.com> <1232708220.8900.23.camel@ozzu> Message-ID: Guido van Rossum python.org> writes: > > And, by the way, "for line in open(filename): ..." will continue to > work. It may just not close the file right away. This is a forgivable > sin in a small program that opens a few files only. It only becomes a > program when this is itself inside a loop that loops over many > filenames -- you could run out of file descriptors. It can also be a problem under Windows where, IIRC, you can't delete a file which is still opened somewhere (even for reading). From steve at holdenweb.com Fri Jan 23 17:36:16 2009 From: steve at holdenweb.com (Steve Holden) Date: Fri, 23 Jan 2009 11:36:16 -0500 Subject: [Python-Dev] __del__ and tp_dealloc in the IO lib In-Reply-To: <4979E6C5.3000502@develer.com> References: <52dc1c820901181738h53e7ecf9if254a91de2025722@mail.gmail.com> <1232708220.8900.23.camel@ozzu> <4979E6C5.3000502@develer.com> Message-ID: Giovanni Bajo wrote: > On 1/23/2009 4:27 PM, Guido van Rossum wrote: >> On Fri, Jan 23, 2009 at 2:57 AM, Giovanni Bajo wrote: >>> I miss to understand why many Python developers are so fierce in trying >>> to push the idea of cross-python compatibility (which is something that >>> does simply *not* exist in real world for applications) or to warn about >>> rainy days in the future when this would stop working in CPython. I >>> would strongly prefer that CPython would settle on (= document) using >>> reference counting and immediate destruction so that people can stop >>> making their everyday code more complex with no benefit. You will be >>> losing no more than an open door that nobody has entered in 20 years, >>> and people would only benefit from it. >> >> You are so very wrong, my son. CPython's implementation strategy >> *will* evolve. Several groups are hard at work trying to make a faster >> Python interpreter, and when they succeed, everyone, including you, >> will want to use their version (or their version may simply *be* the >> new CPython). > > I'm basing my assumption on 19 years of history of CPython. Please, > correct me if I'm wrong, but the only thing that changed is that the > cyclic-GC was added so that loops are now collected, but nothing change > with respect to cyclic collection. And everybody (including you, IIRC) > has always agreed that it would be very very hard to eradicate reference > counting from CPython and all the existing extensions; so hard that it > is probably more convenient to start a different interpreter > implementation. > And everybody knows that the past is an infallible guide to the future ... not. >> Plus, you'd be surprised how many people might want to port existing >> code (and that may include code that uses C extensions, many of which >> are also ported) to Jython or IronPython. > > I would love to be surprised, in fact! > Well both Django and Twisted are on the list of "interested in porting". > Since I fail to see any business strategy behind such a porting, I don't > see this happening very often in the business industry (and even less in > the open source community, where there are also political issues between > those versions of Python, I would say). I also never met someone that > wanted to make a cross-interpreter Python application, nor read about > someone that has a reasonable use case for wanting to do that, besides > geek fun; which is why I came to this conclusion, though I obviously > have access only to a little information compared to other people in here. > Exactly. > On the other hand, I see people using IronPython so that they can access > to the .NET framework (which can't be ported to other Python versions), > or using Java so that they can blend to existing Java programs. And > those are perfectly good use cases for the existence of such > interpreters, but not for the merits of writing cross-interpreter > portable code. > Well, they use IronPython so they can access the .NET framework FROM PYTHON. Would you willfully isolate those users from access to other Python code? > I would be pleased if you (or others) could point me to real-world use > cases of this cross-interpreter portability. > Any code that *can* be run on multiple interpreters is a candidate for use in multiple environments. I don't understand this wish to cripple your code. >> Your mistake sounds more like "nobody will ever want to run this on >> Windows, so I won't have to use the os.path module" and other >> short-sighted ideas. While you may be right in the short run, it may >> also be the death penalty for a piece of genius code that is found to >> be unportable. > > And in fact, I don't defensively code cross-OS portable code. Python is > great in that *most* of what you naturally write is portable; which > means that, the day that you need it, it's a breeze to port your code > (assuming that you have also picked up the correct extensions, which I > always try to do). But that does not mean that I have today to waste > time on something that I don't need. > Perhaps not, but just because you assess it to be a YAGNI in your own case that doesn't mean the same arguments should be applied to (for example) standard library modules. >> And, by the way, "for line in open(filename): ..." will continue to >> work. It may just not close the file right away. This is a forgivable >> sin in a small program that opens a few files only. It only becomes a >> program when this is itself inside a loop that loops over many >> filenames -- you could run out of file descriptors. > > I do understand this, but I'm sure you realize that there other similars > example where the side effects are far worse. Maybe you don't care since > you simply decided to declare that code *wrong*. But I'm unsure the > community will kindly accept such a deep change in behaviour. Especially > within the existing 2.x or 3.x release lines. The community is, I suspect, more in line with Guido's approach than with yours. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From status at bugs.python.org Fri Jan 23 18:06:42 2009 From: status at bugs.python.org (Python tracker) Date: Fri, 23 Jan 2009 18:06:42 +0100 (CET) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20090123170642.0A04578552@psf.upfronthosting.co.za> ACTIVITY SUMMARY (01/16/09 - 01/23/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2322 open (+42) / 14538 closed (+32) / 16860 total (+74) Open issues with patches: 790 Average duration of open issues: 701 days. Median duration of open issues: 6 days. Open Issues Breakdown open 2299 (+42) pending 23 ( +0) Issues Created Or Reopened (79) _______________________________ Unpickling is really slow 01/18/09 http://bugs.python.org/issue3873 reopened pitrou zipfile and winzip 01/17/09 http://bugs.python.org/issue3997 reopened pitrou patch, needs review Thread Safe Py_AddPendingCall 01/16/09 CLOSED http://bugs.python.org/issue4293 reopened marketdickinson patch, patch python3.0 -u: unbuffered stdout 01/19/09 http://bugs.python.org/issue4705 reopened pitrou patch decoding functions in _codecs module accept str arguments 01/22/09 CLOSED http://bugs.python.org/issue4874 reopened pitrou patch UTF-16 stream codec barfs on valid input 01/16/09 CLOSED http://bugs.python.org/issue4964 created gvanrossum Can doc index of html version be separately scrollable? 01/16/09 http://bugs.python.org/issue4965 created tjreedy Improving Lib Doc Sequence Types Section 01/16/09 http://bugs.python.org/issue4966 created tjreedy Bugs in _ssl object read() when a buffer is specified 01/17/09 http://bugs.python.org/issue4967 created pitrou patch Clarify inspect.is method docs 01/17/09 http://bugs.python.org/issue4968 created tjreedy mimetypes on Windows should read MIME database from registry (w/ 01/17/09 http://bugs.python.org/issue4969 created gagenellina patch test_os causes delayed failure on x86 gentoo buildbot: Unknown s 01/17/09 http://bugs.python.org/issue4970 created marketdickinson Incorrect title case 01/17/09 http://bugs.python.org/issue4971 created mrabarnett let's equip ftplib.FTP with __enter__ and __exit__ 01/17/09 http://bugs.python.org/issue4972 created tarek patch calendar formatyearpage returns bytes, not str 01/17/09 http://bugs.python.org/issue4973 created mnewman Redundant mention of lists and tuples at start of Sequence Types 01/17/09 CLOSED http://bugs.python.org/issue4974 created MLModel 3.0 base64 doc examples lack bytes 'b' indicator 01/17/09 CLOSED http://bugs.python.org/issue4975 created tjreedy Documentation of the set intersection and difference operators i 01/17/09 CLOSED http://bugs.python.org/issue4976 created MLModel test_maxint64 fails on 32-bit systems due to assumption that 64- 01/20/09 http://bugs.python.org/issue4977 reopened pitrou allow unicode keyword args 01/17/09 CLOSED http://bugs.python.org/issue4978 created benjamin.peterson patch random.uniform can return its upper limit 01/18/09 CLOSED http://bugs.python.org/issue4979 created hailperin patch Documentation for "y#" does not mention PY_SSIZE_T_CLEAN 01/18/09 CLOSED http://bugs.python.org/issue4980 created pitrou Incorrect statement regarding mutable sequences in datamodel Ref 01/18/09 CLOSED http://bugs.python.org/issue4981 created MLModel Running python 3 as Non-admin User requests the Runtime to termi 01/18/09 http://bugs.python.org/issue4982 created yhvh Spurious reference to "byte sequences" in Library stdtypes seque 01/18/09 CLOSED http://bugs.python.org/issue4983 created MLModel Inconsistent count of sequence types in Library stdtypes documen 01/18/09 CLOSED http://bugs.python.org/issue4984 created MLModel Idle hangs when given a nonexistent filename. 01/18/09 http://bugs.python.org/issue4985 created della1rv patch Augmented Assignment / Operations Confusion in Documentation 01/18/09 CLOSED http://bugs.python.org/issue4986 created acooke update distutils.README 01/18/09 http://bugs.python.org/issue4987 created tarek A link to "What???s New in Python 2.0" on "The Python Tutorial" 01/18/09 CLOSED http://bugs.python.org/issue4988 created akitada 'calendar' module is crummy and should be removed 01/18/09 CLOSED http://bugs.python.org/issue4989 created dotz test_codeccallbacks.CodecCallbackTest.test_badhandlerresult is n 01/18/09 CLOSED http://bugs.python.org/issue4990 created fijal patch os.fdopen doesn't raise on invalid file descriptors 01/18/09 CLOSED http://bugs.python.org/issue4991 created benjamin.peterson patch yield's documentation not updated 01/18/09 http://bugs.python.org/issue4992 created En-Cu-Kou Typo in importlib 01/19/09 CLOSED http://bugs.python.org/issue4993 created pitrou subprocess (Popen) doesn't works properly 01/19/09 CLOSED http://bugs.python.org/issue4994 created simonbcn sqlite3 module gives SQL logic error only in transactions 01/19/09 CLOSED http://bugs.python.org/issue4995 created alsadi io.TextIOWrapper calls buffer.read1() 01/19/09 http://bugs.python.org/issue4996 created kawai xml.sax.saxutils.XMLGenerator should write to io.RawIOBase. 01/19/09 http://bugs.python.org/issue4997 created kawai __slots__ on Fraction is useless 01/19/09 http://bugs.python.org/issue4998 created Somelauw patch multiprocessing.Queue does not order objects 01/19/09 http://bugs.python.org/issue4999 created ndfred multiprocessing - Pool.map() slower about 5 times than map() on 01/19/09 CLOSED http://bugs.python.org/issue5000 created 0x666 Remove assertion-based checking in multiprocessing 01/19/09 http://bugs.python.org/issue5001 created jnoller multiprocessing/pipe_connection.c compiler warning (conn_poll) 01/19/09 CLOSED http://bugs.python.org/issue5002 created ocean-city patch Error while installing Python-3 01/19/09 CLOSED http://bugs.python.org/issue5003 created vani socket.getfqdn() doesn't cope properly with purely DNS-based set 01/19/09 http://bugs.python.org/issue5004 created dfranke 3.0 sqlite doc: most examples refer to pysqlite2, use 2.x syntax 01/19/09 http://bugs.python.org/issue5005 created tjreedy Duplicate UTF-16 BOM if a file is open in append mode 01/19/09 http://bugs.python.org/issue5006 created haypo urllib2 HTTPS connection failure (BadStatusLine Exception) 01/19/09 http://bugs.python.org/issue5007 created ak Wrong tell() result for a file opened in append mode 01/20/09 CLOSED http://bugs.python.org/issue5008 created haypo patch multiprocessing: failure in manager._debug_info() 01/20/09 CLOSED http://bugs.python.org/issue5009 created amaury.forgeotdarc patch, needs review repoened "test_maxint64 fails on 32-bit systems due to assumptio 01/20/09 CLOSED http://bugs.python.org/issue5010 created lkcl issue4428 - make io.BufferedWriter observe max_buffer_size limit 01/20/09 CLOSED http://bugs.python.org/issue5011 created pitrou How to test python 3 was installed properly in my directory? 01/20/09 CLOSED http://bugs.python.org/issue5012 created vani Problems with delay parm of logging.handlers.RotatingFileHandler 01/20/09 CLOSED http://bugs.python.org/issue5013 created pycurry Kernel Protection Failure 01/20/09 CLOSED http://bugs.python.org/issue5014 reopened pitrou The Py_SetPythonHome C API function is undocumented 01/20/09 http://bugs.python.org/issue5015 created Kylotan FileIO.seekable() can return False 01/20/09 http://bugs.python.org/issue5016 created pitrou patch import suds help( suds ) fails 01/21/09 http://bugs.python.org/issue5017 created pjb Overly general claim about sequence unpacking in tutorial 01/21/09 http://bugs.python.org/issue5018 created MLModel Specifying common controls DLL in manifest 01/21/09 http://bugs.python.org/issue5019 created robind Regex Expression Error 01/21/09 CLOSED http://bugs.python.org/issue5020 created sleepyfish doctest.testfile should set __name__, can't use namedtuple 01/21/09 http://bugs.python.org/issue5021 created tlynn doctest should allow running tests with "python -m doctest" 01/21/09 CLOSED http://bugs.python.org/issue5022 created tlynn Segfault in datetime.time.strftime("%z") 01/21/09 http://bugs.python.org/issue5023 created eswald sndhdr.whathdr returns -1 for WAV file frame count 01/21/09 http://bugs.python.org/issue5024 created rpyle test_kqueue failure on OS X 01/21/09 http://bugs.python.org/issue5025 created marketdickinson patch [reopening] native build of python win32 using msys under both w 01/21/09 http://bugs.python.org/issue5026 created lkcl xml namespace not understood by xml.sax.saxutils.XMLGenerator 01/21/09 http://bugs.python.org/issue5027 created roug tokenize.generate_tokens doesn't always return logical line 01/22/09 http://bugs.python.org/issue5028 created duncf Odd slicing behaviour 01/22/09 CLOSED http://bugs.python.org/issue5029 created dgaletic Typo in class tkinter.filedialog.Directory prevents compilation 01/22/09 CLOSED http://bugs.python.org/issue5030 created ringhome Thread.daemon docs 01/22/09 http://bugs.python.org/issue5031 created steve21 patch, needs review itertools.count step 01/22/09 http://bugs.python.org/issue5032 created steve21 setup.py crashes if sqlite version contains 'beta' 01/22/09 http://bugs.python.org/issue5033 created blahblahwhat itertools.fixlen 01/22/09 http://bugs.python.org/issue5034 created lehmannro patch Compilation --without-threads fails 01/22/09 http://bugs.python.org/issue5035 created pitrou patch xml.parsers.expat make a dictionary which keys are broken if buf 01/23/09 http://bugs.python.org/issue5036 created tksmashiw unexpected unicode behavior for proxy objects 01/23/09 http://bugs.python.org/issue5037 created Taldor Issues Now Closed (74) ______________________ Adds the .compact() method to bsddb db.DB objects 444 days http://bugs.python.org/issue1391 jcea patch complex constructor doesn't accept string with nan and inf 341 days http://bugs.python.org/issue2121 marketdickinson patch embed manifest in windows extensions 288 days http://bugs.python.org/issue2563 mhammond patch, patch Console UnicodeDecodeError s once more 282 days http://bugs.python.org/issue2614 benjamin.peterson Multiprocessing hangs when multiprocessing.Pool methods are call 203 days http://bugs.python.org/issue3272 jnoller multiprocessing and meaningful errors 203 days http://bugs.python.org/issue3273 jnoller multiprocessing.connection doesn't import AuthenticationError, w 202 days http://bugs.python.org/issue3283 jnoller patch _multiprocessing.Connection() doesn't check handle 195 days http://bugs.python.org/issue3321 jnoller patch, needs review use of cPickle in multiprocessing 193 days http://bugs.python.org/issue3325 jnoller patch thread_nt.c update 7 days http://bugs.python.org/issue3582 krisvale patch, patch importing from UNC roots doesn't work 151 days http://bugs.python.org/issue3677 ocean-city patch _multiprocessing build fails when configure --without-threads 137 days http://bugs.python.org/issue3807 haypo patch Problem with SocketIO for closing the socket 132 days http://bugs.python.org/issue3826 gregory.p.smith patch IDLE: checksyntax() doesn't support Unicode? 109 days http://bugs.python.org/issue4008 loewis patch, needs review _multiprocessing doesn't build on macosx 10.3 104 days http://bugs.python.org/issue4065 jnoller Py_FatalError cleanup patch 101 days http://bugs.python.org/issue4077 amaury.forgeotdarc patch, easy Thread Safe Py_AddPendingCall 4 days http://bugs.python.org/issue4293 marketdickinson patch, patch incorrect and inconsistent logging in multiprocessing 68 days http://bugs.python.org/issue4301 jnoller patch, needs review Fix performance issues in xmlrpclib 63 days http://bugs.python.org/issue4336 krisvale patch, patch, easy AssertionError in Doc/includes/mp_benchmarks.py 51 days http://bugs.python.org/issue4449 jnoller patch, needs review Optimize new io library 43 days http://bugs.python.org/issue4561 pitrou patch Documentation for multiprocessing - Pool.apply() 45 days http://bugs.python.org/issue4593 jnoller easy Issue with RotatingFileHandler logging handler on Windows 25 days http://bugs.python.org/issue4749 vsajip threding, bsddb and double free or corruption (fasttop) 23 days http://bugs.python.org/issue4774 jcea idle 3.1a1 utf8 16 days http://bugs.python.org/issue4815 loewis patch, needs review md_state is not released 13 days http://bugs.python.org/issue4838 pitrou patch int('3L') still valid in Python 3.0 15 days http://bugs.python.org/issue4842 marketdickinson patch syntax: no unpacking in augassign 12 days http://bugs.python.org/issue4857 georg.brandl decoding functions in _codecs module accept str arguments 0 days http://bugs.python.org/issue4874 pitrou patch Allow buffering for HTTPResponse 12 days http://bugs.python.org/issue4879 krisvale patch, patch ast.literal_eval does not properly handled complex numbers 7 days http://bugs.python.org/issue4907 tjreedy patch, patch trunc(x) erroneously documented as built-in 7 days http://bugs.python.org/issue4914 georg.brandl set.add and set.discard are not conformant to collections.Mutabl 7 days http://bugs.python.org/issue4922 georg.brandl 26backport time.strftime documentation needs update 6 days http://bugs.python.org/issue4923 georg.brandl Inconsistent unicode repr for fileobject 3 days http://bugs.python.org/issue4927 loewis patch, patch, needs review smptlib.py can raise socket.error 6 days http://bugs.python.org/issue4929 krisvale patch, patch, needs review Small optimization in type construction 4 days http://bugs.python.org/issue4930 amaury.forgeotdarc patch native build of python win32 using msys under wine. 6 days http://bugs.python.org/issue4954 lkcl os.ftruncate raises IOError instead of OSError 4 days http://bugs.python.org/issue4957 krisvale patch, patch inspect.formatargspec fails for keyword args without defaults, a 1 days http://bugs.python.org/issue4959 benjamin.peterson patch UTF-16 stream codec barfs on valid input 0 days http://bugs.python.org/issue4964 gvanrossum Redundant mention of lists and tuples at start of Sequence Types 1 days http://bugs.python.org/issue4974 georg.brandl 3.0 base64 doc examples lack bytes 'b' indicator 1 days http://bugs.python.org/issue4975 georg.brandl Documentation of the set intersection and difference operators i 1 days http://bugs.python.org/issue4976 georg.brandl allow unicode keyword args 3 days http://bugs.python.org/issue4978 benjamin.peterson patch random.uniform can return its upper limit 1 days http://bugs.python.org/issue4979 georg.brandl patch Documentation for "y#" does not mention PY_SSIZE_T_CLEAN 1 days http://bugs.python.org/issue4980 georg.brandl Incorrect statement regarding mutable sequences in datamodel Ref 0 days http://bugs.python.org/issue4981 benjamin.peterson Spurious reference to "byte sequences" in Library stdtypes seque 0 days http://bugs.python.org/issue4983 georg.brandl Inconsistent count of sequence types in Library stdtypes documen 0 days http://bugs.python.org/issue4984 georg.brandl Augmented Assignment / Operations Confusion in Documentation 0 days http://bugs.python.org/issue4986 acooke A link to "What???s New in Python 2.0" on "The Python Tutorial" 0 days http://bugs.python.org/issue4988 georg.brandl 'calendar' module is crummy and should be removed 0 days http://bugs.python.org/issue4989 skip.montanaro test_codeccallbacks.CodecCallbackTest.test_badhandlerresult is n 0 days http://bugs.python.org/issue4990 benjamin.peterson patch os.fdopen doesn't raise on invalid file descriptors 1 days http://bugs.python.org/issue4991 benjamin.peterson patch Typo in importlib 0 days http://bugs.python.org/issue4993 brett.cannon subprocess (Popen) doesn't works properly 0 days http://bugs.python.org/issue4994 LambertDW sqlite3 module gives SQL logic error only in transactions 2 days http://bugs.python.org/issue4995 alsadi multiprocessing - Pool.map() slower about 5 times than map() on 0 days http://bugs.python.org/issue5000 0x666 multiprocessing/pipe_connection.c compiler warning (conn_poll) 0 days http://bugs.python.org/issue5002 jnoller patch Error while installing Python-3 0 days http://bugs.python.org/issue5003 pitrou Wrong tell() result for a file opened in append mode 1 days http://bugs.python.org/issue5008 pitrou patch multiprocessing: failure in manager._debug_info() 1 days http://bugs.python.org/issue5009 jnoller patch, needs review repoened "test_maxint64 fails on 32-bit systems due to assumptio 0 days http://bugs.python.org/issue5010 pitrou issue4428 - make io.BufferedWriter observe max_buffer_size limit 0 days http://bugs.python.org/issue5011 gregory.p.smith How to test python 3 was installed properly in my directory? 0 days http://bugs.python.org/issue5012 benjamin.peterson Problems with delay parm of logging.handlers.RotatingFileHandler 0 days http://bugs.python.org/issue5013 vsajip Kernel Protection Failure 0 days http://bugs.python.org/issue5014 loewis Regex Expression Error 0 days http://bugs.python.org/issue5020 amaury.forgeotdarc doctest should allow running tests with "python -m doctest" 1 days http://bugs.python.org/issue5022 benjamin.peterson Odd slicing behaviour 0 days http://bugs.python.org/issue5029 georg.brandl Typo in class tkinter.filedialog.Directory prevents compilation 0 days http://bugs.python.org/issue5030 benjamin.peterson Only "Overwrite" mode possible with curses.textpad.Textbox 1557 days http://bugs.python.org/issue1048820 dashing patch inspect.isclass() fails with custom __getattr__ 1306 days http://bugs.python.org/issue1225107 benjamin.peterson Top Issues Most Discussed (10) ______________________________ 22 IDLE won't start in custom directory. 129 days open http://bugs.python.org/issue3881 15 __slots__ on Fraction is useless 4 days open http://bugs.python.org/issue4998 14 native build of python win32 using msys under wine. 6 days closed http://bugs.python.org/issue4954 13 PyUnicode_FromWideChar incorrect for characters outside the BMP 54 days open http://bugs.python.org/issue4474 11 let's equip ftplib.FTP with __enter__ and __exit__ 6 days open http://bugs.python.org/issue4972 11 _multiprocessing.Connection() doesn't check handle 195 days closed http://bugs.python.org/issue3321 10 Compilation --without-threads fails 1 days open http://bugs.python.org/issue5035 9 test_maxint64 fails on 32-bit systems due to assumption that 64 3 days open http://bugs.python.org/issue4977 8 Wrong tell() result for a file opened in append mode 1 days closed http://bugs.python.org/issue5008 8 multiprocessing.Queue does not order objects 4 days open http://bugs.python.org/issue4999 From bcannon at gmail.com Fri Jan 23 19:15:18 2009 From: bcannon at gmail.com (Brett Cannon) Date: Fri, 23 Jan 2009 10:15:18 -0800 Subject: [Python-Dev] Should ftplib use UTF-8 instead of latin-1 encoding? In-Reply-To: <4979805B.4030107@v.loewis.de> References: <729626cc0901221813m72a9896bj30092dbfbd5cb133@mail.gmail.com> <4979805B.4030107@v.loewis.de> Message-ID: On Fri, Jan 23, 2009 at 00:31, "Martin v. L?wis" wrote: > Giampaolo Rodola' wrote: >> Hi, >> while attempting to port pyftpdlib [1] to Python 3 I have noticed that >> ftplib differs from the previous 2.x version in that it uses latin-1 >> to encode everything it's sent over the FTP command channel, but by >> reading RFC-2640 [2] it seems that UTF-8 should be preferred instead. >> I'm far from being an expert of encodings, plus the RFC is quite hard >> to understand, so sorry in advance if I have misunderstood the whole >> thing. > > I read it that a conforming client MUST issue a FEAT command, to > determine whether the server supports UTF8. One would have to go > back to the original FTP RFC, but it seams that, in the absence > of server UTF8 support, all path names must be 7-bit clean (which > means that ASCII should be the default encoding). > > In any case, Brett changed the encoding to latin1 in r58378, maybe > he can comment. > If I remember correctly something along Martin's comment about 7-bit clean is needed, but some servers don't follow the standard, so I swapped it to Latin-1. But that was so long ago I don't remember where I gleaned the details from in the RFC. If I misread the RFC and it is UTF-8 then all the better to make more of the world move over to Unicode. -Brett From brett at python.org Fri Jan 23 19:24:26 2009 From: brett at python.org (Brett Cannon) Date: Fri, 23 Jan 2009 10:24:26 -0800 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <49798235.50509@v.loewis.de> References: <49798235.50509@v.loewis.de> Message-ID: On Fri, Jan 23, 2009 at 00:39, "Martin v. L?wis" wrote: >> And I would like to thank my co-authors for their time and effort thus >> far in filling in the PEP on behalf of their favorite DVCS. Everyone >> has put in a lot of time already with I am sure more time in the >> future. > > So what will happen next? ISTM that the PEP is not complete, since it > doesn't specify a specific DVCS to migrate to (i.e. it wouldn't be > possible to implement the PEP as it stands). > Right, it isn't done. But I know from experience people care A LOT about their favorite DVCS, so I wanted to get the PEP up now so people can start sending in feedback to the proper authors instead of having it all flood in after I have done my exploratory work on all of them and be accused of having based my decision on faulty information. > Somebody will have to make a decision. That falls on my shoulders. > Ultimately, Guido will have to > approve the PEP, but it might be that he refuses to make a choice of > specific DVCS. Guido is staying out of this one. > Traditionally, it is the PEP author who makes all > choices (considering suggestions from the community, of course). So > what DVCS do the PEP authors recommend? Umm:: import random print(random.choice('svn', 'bzr', 'hg', 'git')) -Brett From brett at python.org Fri Jan 23 19:25:38 2009 From: brett at python.org (Brett Cannon) Date: Fri, 23 Jan 2009 10:25:38 -0800 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: References: Message-ID: On Fri, Jan 23, 2009 at 03:05, Antoine Pitrou wrote: > Brett Cannon python.org> writes: >> >> I have now converted PEP 374 >> (http://www.python.org/dev/peps/pep-0374/) from Google Docs to reST >> and checked it in. I am not going to paste it into an email as it is >> nearly 1500 lines in reST form. > > It seems the ">>" token is mangled into a French closing quote ("?") inside code > snippets. Yeah, the Google Docs export didn't come out well in pure text. I tried to catch as many of those characters as I could but apparently I missed a couple. -Brett From brett at python.org Fri Jan 23 19:40:02 2009 From: brett at python.org (Brett Cannon) Date: Fri, 23 Jan 2009 10:40:02 -0800 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: References: <49798235.50509@v.loewis.de> Message-ID: On Fri, Jan 23, 2009 at 07:30, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Brett, thanks for putting this PEP together! > Yep. Just make sure I don't do something like this for a LONG time. Apparently I didn't learn my lesson after the issue tracker migration. > On Jan 23, 2009, at 3:39 AM, Martin v. L?wis wrote: > >> Somebody will have to make a decision. Ultimately, Guido will have to >> approve the PEP, but it might be that he refuses to make a choice of >> specific DVCS. Traditionally, it is the PEP author who makes all >> choices (considering suggestions from the community, of course). So >> what DVCS do the PEP authors recommend? > > Brett, perhaps you should publish a tentative schedule. Milestones I'd like > to see include > > * Initial impressions section completed > * Call for rebuttals > * Second draft of impressions > * (perhaps multiple) Recommendations to Guido and python-dev > * Experimental live branches deployed for testing > * Final recommendation > * Final decision > I think you just published the schedule, Barry. =) Seriously, though, I have several other commitments that take precedent over this PEP so I don't feel comfortable locking down any dates. > My understanding is that a final decision will /not/ be made by Pycon. It's doubtful, but would be nice by then. What I will do is eliminate a contender by/at PyCon. -Brett From phd at phd.pp.ru Fri Jan 23 19:55:01 2009 From: phd at phd.pp.ru (Oleg Broytmann) Date: Fri, 23 Jan 2009 21:55:01 +0300 Subject: [Python-Dev] Should ftplib use UTF-8 instead of latin-1 encoding? In-Reply-To: References: <729626cc0901221813m72a9896bj30092dbfbd5cb133@mail.gmail.com> <4979805B.4030107@v.loewis.de> Message-ID: <20090123185501.GA24843@phd.pp.ru> On Fri, Jan 23, 2009 at 10:15:18AM -0800, Brett Cannon wrote: > If I remember correctly something along Martin's comment about 7-bit > clean is needed, but some servers don't follow the standard, so I > swapped it to Latin-1. But that was so long ago I don't remember where > I gleaned the details from in the RFC. If I misread the RFC and it is > UTF-8 then all the better to make more of the world move over to > Unicode. I don't know any server that encode file names in any way. All servers I know just pass filenames as is, 8-bit; some that implement stricter RFC-959 mangle chr(255), but that's all. One can encounter a server that stores files in a number of different encodings. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From tony at pagedna.com Fri Jan 23 20:22:55 2009 From: tony at pagedna.com (Tony Lownds) Date: Fri, 23 Jan 2009 11:22:55 -0800 Subject: [Python-Dev] Fwd: [ALERT] cbank: OldHashChecker cannot check password, uid is None References: <20090123191626.9F69BA7EB8@siteserver3.localdomain> Message-ID: Rob and/or Tim, Can you track this down? Thanks -Tony Begin forwarded message: > From: support+dev at pagedna.com > Date: January 23, 2009 11:16:26 AM PST > To: problems at pagedna.com > Subject: [ALERT] cbank: OldHashChecker cannot check password, uid is > None > > OldHashChecker cannot check password, uid is None > Script: /inet/www/clients/cbank/index.cgi > Machine: siteserver3 > Directory: /mnt/sitenfs2_clients/cbank > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdmurray at bitdance.com Fri Jan 23 20:35:01 2009 From: rdmurray at bitdance.com (rdmurray at bitdance.com) Date: Fri, 23 Jan 2009 14:35:01 -0500 (EST) Subject: [Python-Dev] Should ftplib use UTF-8 instead of latin-1 encoding? In-Reply-To: <20090123185501.GA24843@phd.pp.ru> References: <729626cc0901221813m72a9896bj30092dbfbd5cb133@mail.gmail.com> <4979805B.4030107@v.loewis.de> <20090123185501.GA24843@phd.pp.ru> Message-ID: On Fri, 23 Jan 2009 at 21:55, Oleg Broytmann wrote: > On Fri, Jan 23, 2009 at 10:15:18AM -0800, Brett Cannon wrote: >> If I remember correctly something along Martin's comment about 7-bit >> clean is needed, but some servers don't follow the standard, so I >> swapped it to Latin-1. But that was so long ago I don't remember where >> I gleaned the details from in the RFC. If I misread the RFC and it is >> UTF-8 then all the better to make more of the world move over to >> Unicode. > > I don't know any server that encode file names in any way. All servers > I know just pass filenames as is, 8-bit; some that implement stricter > RFC-959 mangle chr(255), but that's all. One can encounter a server that > stores files in a number of different encodings. Given that a Unix OS can't know what encoding a filename is in (*), I can't see that one could practically implement a Unix FTP server in any other way. --RDM (*) remember the earlier extensive discussion of this when the issue of listdir() ignoring non-encodable filesnames came up? From brett at python.org Fri Jan 23 20:39:21 2009 From: brett at python.org (Brett Cannon) Date: Fri, 23 Jan 2009 11:39:21 -0800 Subject: [Python-Dev] Fwd: [ALERT] cbank: OldHashChecker cannot check password, uid is None In-Reply-To: References: <20090123191626.9F69BA7EB8@siteserver3.localdomain> Message-ID: Uh, Tony, I think you sent this to the wrong email address. =) On Fri, Jan 23, 2009 at 11:22, Tony Lownds wrote: > Rob and/or Tim, > Can you track this down? > Thanks > -Tony > > Begin forwarded message: > > From: support+dev at pagedna.com > Date: January 23, 2009 11:16:26 AM PST > To: problems at pagedna.com > Subject: [ALERT] cbank: OldHashChecker cannot check password, uid is None > OldHashChecker cannot check password, uid is None > Script: /inet/www/clients/cbank/index.cgi > Machine: siteserver3 > Directory: /mnt/sitenfs2_clients/cbank > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > > From tony at pagedna.com Fri Jan 23 20:09:08 2009 From: tony at pagedna.com (Tony Lownds) Date: Fri, 23 Jan 2009 11:09:08 -0800 Subject: [Python-Dev] Fwd: [ALERT] cityoftoronto: problem saving to products table References: <20090123190001.9462DA7EB8@siteserver3.localdomain> Message-ID: Hi Paulus, Have you fixed these aerts before? We need a script to fix these alerts. Thanks -Tony Begin forwarded message: > From: support+dev at pagedna.com > Date: January 23, 2009 11:00:01 AM PST > To: problems at pagedna.com > Subject: [ALERT] cityoftoronto: problem saving to products table > > problem saving to products table > > Traceback (most recent call last): > File "/opt/printra/lib/python/printra/sossite/KindManager.py", line > 325, in save_to_products_table > self.save_to_products_table2(kinds) > File "/opt/printra/lib/python/printra/sossite/KindManager.py", line > 344, in save_to_products_table2 > if dbkind.update(site, kind): > File "/opt/printra/lib/python/printra/sossite/KindManager.py", line > 490, in update > v = fn(kind) > File "/opt/printra/lib/python/printra/sossite/KindManager.py", line > 474, in _buy_price > return coerce_qtyspec(kind.buy_qtyspec).price_for_qty(1) > File "/opt/printra/lib/python/printra/sossite/Sos2qtyspec.py", line > 778, in price_for_qty > q, p, l = _qtypricesplit(q) > File "/opt/printra/lib/python/printra/sossite/Sos2qtyspec.py", line > 671, in _qtypricesplit > raise ValueError, "bad tuple to qtyonly: %s" % t > ValueError: bad tuple to qtyonly: [(1000, '1000 - $67.42'), (6000, > '6000 - $356.52'), (10000, '10000 - $486.2')] > > Menu user: rolando > Script: /inet/www/clients/cityoftoronto/index.cgi > Machine: siteserver3 > Directory: /mnt/sitenfs2_clients/cityoftoronto > -------------- next part -------------- An HTML attachment was scrubbed... URL: From phd at phd.pp.ru Fri Jan 23 20:45:10 2009 From: phd at phd.pp.ru (Oleg Broytmann) Date: Fri, 23 Jan 2009 22:45:10 +0300 Subject: [Python-Dev] Should ftplib use UTF-8 instead of latin-1 encoding? In-Reply-To: References: <729626cc0901221813m72a9896bj30092dbfbd5cb133@mail.gmail.com> <4979805B.4030107@v.loewis.de> <20090123185501.GA24843@phd.pp.ru> Message-ID: <20090123194510.GB24843@phd.pp.ru> On Fri, Jan 23, 2009 at 02:35:01PM -0500, rdmurray at bitdance.com wrote: > Given that a Unix OS can't know what encoding a filename is in (*), > I can't see that one could practically implement a Unix FTP server > in any other way. Can you believe there is a well-known program that solved the issue?! It is Apache web server! One can configure different directories and different file types to have different encodings. I often do that. One (sysadmin) can even allow users to do the configuration themselves via .htaccess local files. I am pretty sure FTP servers could borrow some ideas from Apache in this area. But they don't. Pity. :( Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From guido at python.org Fri Jan 23 20:54:54 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 23 Jan 2009 11:54:54 -0800 Subject: [Python-Dev] Fwd: [ALERT] cityoftoronto: problem saving to products table In-Reply-To: References: <20090123190001.9462DA7EB8@siteserver3.localdomain> Message-ID: Tony, you are posting internal communications to python-dev!!! On Fri, Jan 23, 2009 at 11:09 AM, Tony Lownds wrote: > Hi Paulus, > Have you fixed these aerts before? We need a script to fix these alerts. > Thanks > -Tony > > Begin forwarded message: > > From: support+dev at pagedna.com > Date: January 23, 2009 11:00:01 AM PST > To: problems at pagedna.com > Subject: [ALERT] cityoftoronto: problem saving to products table > problem saving to products table > > Traceback (most recent call last): > File "/opt/printra/lib/python/printra/sossite/KindManager.py", line 325, in > save_to_products_table > self.save_to_products_table2(kinds) > File "/opt/printra/lib/python/printra/sossite/KindManager.py", line 344, in > save_to_products_table2 > if dbkind.update(site, kind): > File "/opt/printra/lib/python/printra/sossite/KindManager.py", line 490, in > update > v = fn(kind) > File "/opt/printra/lib/python/printra/sossite/KindManager.py", line 474, in > _buy_price > return coerce_qtyspec(kind.buy_qtyspec).price_for_qty(1) > File "/opt/printra/lib/python/printra/sossite/Sos2qtyspec.py", line 778, in > price_for_qty > q, p, l = _qtypricesplit(q) > File "/opt/printra/lib/python/printra/sossite/Sos2qtyspec.py", line 671, in > _qtypricesplit > raise ValueError, "bad tuple to qtyonly: %s" % t > ValueError: bad tuple to qtyonly: [(1000, '1000 - $67.42'), (6000, '6000 - > $356.52'), (10000, '10000 - $486.2')] > > Menu user: rolando > Script: /inet/www/clients/cityoftoronto/index.cgi > Machine: siteserver3 > Directory: /mnt/sitenfs2_clients/cityoftoronto > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From a.badger at gmail.com Fri Jan 23 20:54:38 2009 From: a.badger at gmail.com (Toshio Kuratomi) Date: Fri, 23 Jan 2009 11:54:38 -0800 Subject: [Python-Dev] Should ftplib use UTF-8 instead of latin-1 encoding? In-Reply-To: <20090123194510.GB24843@phd.pp.ru> References: <729626cc0901221813m72a9896bj30092dbfbd5cb133@mail.gmail.com> <4979805B.4030107@v.loewis.de> <20090123185501.GA24843@phd.pp.ru> <20090123194510.GB24843@phd.pp.ru> Message-ID: <497A207E.1040708@gmail.com> Oleg Broytmann wrote: > On Fri, Jan 23, 2009 at 02:35:01PM -0500, rdmurray at bitdance.com wrote: >> Given that a Unix OS can't know what encoding a filename is in (*), >> I can't see that one could practically implement a Unix FTP server >> in any other way. > > Can you believe there is a well-known program that solved the issue?! It > is Apache web server! One can configure different directories and different > file types to have different encodings. I often do that. One (sysadmin) can > even allow users to do the configuration themselves via .htaccess local files. > I am pretty sure FTP servers could borrow some ideas from Apache in this > area. But they don't. Pity. :( > AFAIK, Apache is in the same boat as ftp servers. You're thinking of the encoding inside of the files. The problem is with the file names themselves. -Toshio -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: OpenPGP digital signature URL: From martin at v.loewis.de Fri Jan 23 21:06:47 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Fri, 23 Jan 2009 21:06:47 +0100 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: References: <49798235.50509@v.loewis.de> Message-ID: <497A2357.7060205@v.loewis.de> > import random > print(random.choice('svn', 'bzr', 'hg', 'git')) Nice! So it's bzr, as my machine just told me (after adding the square brackets). Regards, Martin From steven.bethard at gmail.com Fri Jan 23 21:11:21 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Fri, 23 Jan 2009 12:11:21 -0800 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <497A2357.7060205@v.loewis.de> References: <49798235.50509@v.loewis.de> <497A2357.7060205@v.loewis.de> Message-ID: On Fri, Jan 23, 2009 at 12:06 PM, "Martin v. L?wis" wrote: >> import random >> print(random.choice('svn', 'bzr', 'hg', 'git')) > > Nice! So it's bzr, as my machine just told me (after adding > the square brackets). Wow, that decision was a lot easier than I thought it would be. ;-) Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From brett at python.org Fri Jan 23 21:13:52 2009 From: brett at python.org (Brett Cannon) Date: Fri, 23 Jan 2009 12:13:52 -0800 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: References: <49798235.50509@v.loewis.de> <497A2357.7060205@v.loewis.de> Message-ID: On Fri, Jan 23, 2009 at 12:11, Steven Bethard wrote: > On Fri, Jan 23, 2009 at 12:06 PM, "Martin v. L?wis" wrote: >>> import random >>> print(random.choice('svn', 'bzr', 'hg', 'git')) >> >> Nice! So it's bzr, as my machine just told me (after adding >> the square brackets). > > Wow, that decision was a lot easier than I thought it would be. ;-) But my machine just told me svn, which is even easier as that means we don't need to change anything. =) -Brett From martin at v.loewis.de Fri Jan 23 21:16:49 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 23 Jan 2009 21:16:49 +0100 Subject: [Python-Dev] Update on MS Windows CE port In-Reply-To: <200901231019.39111.eckhardt@satorlaser.com> References: <200901231019.39111.eckhardt@satorlaser.com> Message-ID: <497A25B1.6070906@v.loewis.de> > Just a short update on my porting issues. I tried various things in the > holidays/vacation, filed several bug reports, got a few patches applied and I > think the thing is taking shape now. However, I couldn't work on it for the > past two weeks since I'm just too swamped at work here. I haven't given up > though! Thanks for the update. I can only say: Ready when you are. I'm doubtful that anybody will continue to work on this (at least not in the same direction that you took). So whenever you chose to return, it's likely still where you left off (with some of the code having rot). Regards, Martin From martin at v.loewis.de Fri Jan 23 21:18:38 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Fri, 23 Jan 2009 21:18:38 +0100 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: References: <49798235.50509@v.loewis.de> Message-ID: <497A261E.5040004@v.loewis.de> > Brett mentioned in his email that he wasn't ready to make a decision yet, I > think? I also think that the PEP could still use some modifications from people > who have more experience with the DVCSs. My question really was whether it is already ready for the wider audience up for discussion (and if so, what it is that should be discussed). It seems that it's not the case, so I just sit back and wait until its ready. Regards, Martin From martin at v.loewis.de Fri Jan 23 21:23:06 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 23 Jan 2009 21:23:06 +0100 Subject: [Python-Dev] Should ftplib use UTF-8 instead of latin-1 encoding? In-Reply-To: References: <729626cc0901221813m72a9896bj30092dbfbd5cb133@mail.gmail.com> <4979805B.4030107@v.loewis.de> <20090123185501.GA24843@phd.pp.ru> Message-ID: <497A272A.4020102@v.loewis.de> > Given that a Unix OS can't know what encoding a filename is in (*), > I can't see that one could practically implement a Unix FTP server > in any other way. However, an ftp server is different. It might start up with an empty folder, and receive *all* of its files through upload. Then it can certainly know what encoding the file names have on disk. It *could* also support operation on pre-existing files, e.g. by providing a configuration directive telling the encoding of the file names, or by ignoring all file names that are not encoded in UTF-8. Regards, Martin From phd at phd.pp.ru Fri Jan 23 21:24:57 2009 From: phd at phd.pp.ru (Oleg Broytmann) Date: Fri, 23 Jan 2009 23:24:57 +0300 Subject: [Python-Dev] Should ftplib use UTF-8 instead of latin-1 encoding? In-Reply-To: <497A207E.1040708@gmail.com> References: <729626cc0901221813m72a9896bj30092dbfbd5cb133@mail.gmail.com> <4979805B.4030107@v.loewis.de> <20090123185501.GA24843@phd.pp.ru> <20090123194510.GB24843@phd.pp.ru> <497A207E.1040708@gmail.com> Message-ID: <20090123202457.GC24843@phd.pp.ru> On Fri, Jan 23, 2009 at 11:54:38AM -0800, Toshio Kuratomi wrote: > AFAIK, Apache is in the same boat as ftp servers. You're thinking of > the encoding inside of the files. The problem is with the file names > themselves. Mostly yes. But Apache is so powerful I can do (and really did) a lot of tricks - I can change LC_CTYPE with mod_env, I can map URLs to the filesystem using mod_rewrite/ScriptAlias... FTP servers don't need to be that smart, but I'd like them to be more configurable WRT filename encoding. But well, they are not, so the only thing to discuss is what to do with ftplib and pyftpd. My not so humble opinion is - either use bytes instead of strings or use latin-1 because it is the straightforward encoding that preserves all 8 bits. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From tjreedy at udel.edu Fri Jan 23 21:52:56 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 23 Jan 2009 15:52:56 -0500 Subject: [Python-Dev] __del__ and tp_dealloc in the IO lib In-Reply-To: <4979E6C5.3000502@develer.com> References: <52dc1c820901181738h53e7ecf9if254a91de2025722@mail.gmail.com> <1232708220.8900.23.camel@ozzu> <4979E6C5.3000502@develer.com> Message-ID: Giovanni Bajo wrote: > >> You are so very wrong, my son. CPython's implementation strategy >> *will* evolve. Several groups are hard at work trying to make a faster >> Python interpreter, and when they succeed, everyone, including you, >> will want to use their version (or their version may simply *be* the >> new CPython). > > I'm basing my assumption on 19 years of history of CPython. Please, > correct me if I'm wrong, but the only thing that changed is that the > cyclic-GC was added so that loops are now collected, but nothing change > with respect to cyclic collection. And everybody (including you, IIRC) > has always agreed that it would be very very hard to eradicate reference > counting from CPython and all the existing extensions; so hard that it > is probably more convenient to start a different interpreter > implementation. Your history is true, but sometimes history changes faster than most expect. [As in the last 13 months of USA.] A year ago, I might have agreed with you, but in the last 6 months, there has been more visible ferment in the area of dynamic language implementations than I remember seeing in the past decade. When Guido says "CPython's implementation strategy *will* evolve" [emphasis his], I believe him. So this is just the wrong time to ask that it be frozen ;-). While a strong argument can be made that the remaining 2.x versions should not be changed, they do not apply to 3.x. New code and ported old code should use 'with' wherever quick closing needs to be guaranteed. The 3.0 manual clearly states "An implementation is allowed to postpone garbage collection or omit it altogether " OK, it also goes on to say "(Implementation note: the current implementation uses a reference-counting scheme with (optional) delayed detection of cyclically linked garbage,...)" I think the first part should at least be amended to 'the current CPython implementation' or 'the CPython implementation currently' or even better 'one current implementation (CPython)' and a warning added "But this may change" and "is not true of all implementaions" if that is not made clear otherwise. Terry Jan Reedy From rdmurray at bitdance.com Fri Jan 23 22:10:06 2009 From: rdmurray at bitdance.com (rdmurray at bitdance.com) Date: Fri, 23 Jan 2009 16:10:06 -0500 (EST) Subject: [Python-Dev] Should ftplib use UTF-8 instead of latin-1 encoding? In-Reply-To: <497A272A.4020102@v.loewis.de> References: <729626cc0901221813m72a9896bj30092dbfbd5cb133@mail.gmail.com> <4979805B.4030107@v.loewis.de> <20090123185501.GA24843@phd.pp.ru> <497A272A.4020102@v.loewis.de> Message-ID: On Fri, 23 Jan 2009 at 21:23, "Martin v. L?wis" wrote: >> Given that a Unix OS can't know what encoding a filename is in (*), >> I can't see that one could practically implement a Unix FTP server >> in any other way. > > However, an ftp server is different. It might start up with an empty > folder, and receive *all* of its files through upload. Then it can > certainly know what encoding the file names have on disk. It *could* > also support operation on pre-existing files, e.g. by providing a > configuration directive telling the encoding of the file names, or > by ignoring all file names that are not encoded in UTF-8. I don't see how starting with an empty directory helps. The filename comes from the client, and the FTP server can't know what the actual encoding of that filename is. --RDM From p.f.moore at gmail.com Fri Jan 23 22:39:22 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 23 Jan 2009 21:39:22 +0000 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <497A261E.5040004@v.loewis.de> References: <49798235.50509@v.loewis.de> <497A261E.5040004@v.loewis.de> Message-ID: <79990c6b0901231339rd7835b6r11e5892cb8523530@mail.gmail.com> 2009/1/23 "Martin v. L?wis" : >> Brett mentioned in his email that he wasn't ready to make a decision yet, I >> think? I also think that the PEP could still use some modifications from people >> who have more experience with the DVCSs. > > My question really was whether it is already ready for the wider > audience up for discussion (and if so, what it is that should be > discussed). It seems that it's not the case, so I just sit back and wait > until its ready. That's the impression I got - until Brett reaches the point of "Call for rebuttals", he's not looking for input (other than factual corrections of the capability and scenario sections). I'm not sure I'm comfortable with sitting back and waiting to quite that extent (I'm *already* biting my tongue over some of Brett's comments with which I strongly disagree), but I'd rather not have the PEP dissolve in a flamewar before Brett has a chance to study things better. I appreciate the need for a considered, logical evaluation, but I'm not sure keeping the lid on the impassioned arguments is going to make them any less likely to explode when they finally do happen. Paul. From brett at python.org Fri Jan 23 23:10:45 2009 From: brett at python.org (Brett Cannon) Date: Fri, 23 Jan 2009 14:10:45 -0800 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <79990c6b0901231339rd7835b6r11e5892cb8523530@mail.gmail.com> References: <49798235.50509@v.loewis.de> <497A261E.5040004@v.loewis.de> <79990c6b0901231339rd7835b6r11e5892cb8523530@mail.gmail.com> Message-ID: On Fri, Jan 23, 2009 at 13:39, Paul Moore wrote: > 2009/1/23 "Martin v. L?wis" : >>> Brett mentioned in his email that he wasn't ready to make a decision yet, I >>> think? I also think that the PEP could still use some modifications from people >>> who have more experience with the DVCSs. >> >> My question really was whether it is already ready for the wider >> audience up for discussion (and if so, what it is that should be >> discussed). It seems that it's not the case, so I just sit back and wait >> until its ready. > > That's the impression I got - until Brett reaches the point of "Call > for rebuttals", he's not looking for input (other than factual > corrections of the capability and scenario sections). That's right. > I'm not sure I'm > comfortable with sitting back and waiting to quite that extent (I'm > *already* biting my tongue over some of Brett's comments with which I > strongly disagree), but I'd rather not have the PEP dissolve in a > flamewar before Brett has a chance to study things better. > It's going to dissolve anyway. I am just trying to keep it to a single situation instead of having to go over it multiple times. > I appreciate the need for a considered, logical evaluation, but I'm > not sure keeping the lid on the impassioned arguments is going to make > them any less likely to explode when they finally do happen. I know it will explode. I am just trying to save myself the time of not having to reply to the flood of emails that I know will come before I have finished even thinking things through. It's like people asking questions during a presentation that happen to be strongly affected by the next slide; you can ask, but you might as well wait until all the info is presented to start talking. -Brett From martin at v.loewis.de Fri Jan 23 23:18:37 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 23 Jan 2009 23:18:37 +0100 Subject: [Python-Dev] Should ftplib use UTF-8 instead of latin-1 encoding? In-Reply-To: References: <729626cc0901221813m72a9896bj30092dbfbd5cb133@mail.gmail.com> <4979805B.4030107@v.loewis.de> <20090123185501.GA24843@phd.pp.ru> <497A272A.4020102@v.loewis.de> Message-ID: <497A423D.9020302@v.loewis.de> > I don't see how starting with an empty directory helps. The filename > comes from the client, and the FTP server can't know what the actual > encoding of that filename is. Sure it can. If the client supports RFC 2640, it will send file names in UTF-8. If the client does not support RFC 2640, the client must restrict itself to 7-bit file names (i.e. ASCII). If the client violates the protocol, the server must respond with error 501. Regards, Martin From bugtrack at roumenpetrov.info Fri Jan 23 23:21:59 2009 From: bugtrack at roumenpetrov.info (Roumen Petrov) Date: Sat, 24 Jan 2009 00:21:59 +0200 Subject: [Python-Dev] Should ftplib use UTF-8 instead of latin-1 encoding? In-Reply-To: References: <729626cc0901221813m72a9896bj30092dbfbd5cb133@mail.gmail.com> <4979805B.4030107@v.loewis.de> <20090123185501.GA24843@phd.pp.ru> <497A272A.4020102@v.loewis.de> Message-ID: <497A4307.5000203@roumenpetrov.info> rdmurray at bitdance.com wrote: > On Fri, 23 Jan 2009 at 21:23, "Martin v. L?wis" wrote: >>> Given that a Unix OS can't know what encoding a filename is in (*), >>> I can't see that one could practically implement a Unix FTP server >>> in any other way. >> >> However, an ftp server is different. It might start up with an empty >> folder, and receive *all* of its files through upload. Then it can >> certainly know what encoding the file names have on disk. It *could* >> also support operation on pre-existing files, e.g. by providing a >> configuration directive telling the encoding of the file names, or >> by ignoring all file names that are not encoded in UTF-8. > > I don't see how starting with an empty directory helps. The filename > comes from the client, and the FTP server can't know what the actual > encoding of that filename is. Exactly, only client can do filename conversion. May be ftplib could be extended to know encoding on filenames on local and remote system based on some user settings. May be ftplib could use UTF-8 or UCS-2/4 to store internally filename but direct conversion is may be faster. It the last case filename is a byte sequence. Roumen From bugtrack at roumenpetrov.info Fri Jan 23 23:28:49 2009 From: bugtrack at roumenpetrov.info (Roumen Petrov) Date: Sat, 24 Jan 2009 00:28:49 +0200 Subject: [Python-Dev] About SCons Re: progress: compiling python2.5 under msys (specifically but not exclusively under wine) with msvcr80 In-Reply-To: References: Message-ID: <497A44A1.20907@roumenpetrov.info> anatoly techtonik wrote: > On Thu, Jan 22, 2009 at 12:51 AM, Roumen Petrov > wrote: >>> Against 2.3, rejected due to dependence on SCons. >>> Also appears to have been incomplete, needing more work. >> No it was complete but use SCons. Most of changes changes in code you will >> see again in 3871. >> > > I would better use SCons for both unix and windows builds. In case of > windows for both compilers - mingw and microsoft ones. To port curses > extension to windows I need to know what gcc options mean, what are > the rules to write Makefiles and how to repeat these rules as well as > find options in visual studio interface. Not mentioning various > platform-specific defines and warning fixes. Did you select one of existing curses library for windows ? Roumen From guido at python.org Fri Jan 23 23:28:55 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 23 Jan 2009 14:28:55 -0800 Subject: [Python-Dev] __del__ and tp_dealloc in the IO lib In-Reply-To: References: <52dc1c820901181738h53e7ecf9if254a91de2025722@mail.gmail.com> <1232708220.8900.23.camel@ozzu> <4979E6C5.3000502@develer.com> Message-ID: On Fri, Jan 23, 2009 at 12:52 PM, Terry Reedy wrote: > Giovanni Bajo wrote: >> >>> You are so very wrong, my son. CPython's implementation strategy >>> *will* evolve. Several groups are hard at work trying to make a faster >>> Python interpreter, and when they succeed, everyone, including you, >>> will want to use their version (or their version may simply *be* the >>> new CPython). >> >> I'm basing my assumption on 19 years of history of CPython. Please, >> correct me if I'm wrong, but the only thing that changed is that the >> cyclic-GC was added so that loops are now collected, but nothing change with >> respect to cyclic collection. And everybody (including you, IIRC) has always >> agreed that it would be very very hard to eradicate reference counting from >> CPython and all the existing extensions; so hard that it is probably more >> convenient to start a different interpreter implementation. > > Your history is true, but sometimes history changes faster than most expect. > [As in the last 13 months of USA.] A year ago, I might have agreed with > you, but in the last 6 months, there has been more visible ferment in the > area of dynamic language implementations than I remember seeing in the past > decade. When Guido says "CPython's implementation strategy *will* evolve" > [emphasis his], I believe him. So this is just the wrong time to ask that > it be frozen ;-). > > While a strong argument can be made that the remaining 2.x versions should > not be changed, they do not apply to 3.x. New code and ported old code > should use 'with' wherever quick closing needs to be guaranteed. The 3.0 > manual clearly states "An implementation is allowed to postpone garbage > collection or omit it altogether " I would hope the 2.x manual says the same, since that same assumption has been around explicitly ever since JPython was first introduced. I'm not sure we should exempt 2.x from these changes (though if only 3.x could be made twice as fast it would of course encourage people to upgrade... :-). We've had many changes in the past affecting the lifetime of local variables, usually due to changes in the way tracebacks were managed. > OK, it also goes on to say "(Implementation note: the current implementation > uses a reference-counting scheme with (optional) delayed detection of > cyclically linked garbage,...)" I think the first part should at least be > amended to 'the current CPython implementation' or 'the CPython > implementation currently' or even better 'one current implementation > (CPython)' and a warning added "But this may change" and "is not true of all > implementaions" if that is not made clear otherwise. True. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ziade.tarek at gmail.com Fri Jan 23 23:36:31 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 23 Jan 2009 23:36:31 +0100 Subject: [Python-Dev] distutils.mwerkscompiler and macpath deprecation Message-ID: <94bdd2610901231436y7222c130te4ec64f9da091018@mail.gmail.com> Hi, for http://bugs.python.org/issue4863 If no one objects, I am going to remove the mwerkscompiler module from distutils without any deprecation process, as Martin suggested (since it is linked into ccompiler with "mac" for the os.name) Now the question is, should I add a ticket to deprecate macpath.py as well ? Regards Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From bugtrack at roumenpetrov.info Fri Jan 23 23:48:48 2009 From: bugtrack at roumenpetrov.info (Roumen Petrov) Date: Sat, 24 Jan 2009 00:48:48 +0200 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: References: <49777A99.9010607@v.loewis.de> <49778FF6.8070608@v.loewis.de> <4978BE6D.6060000@v.loewis.de> <4978D46C.3030200@v.loewis.de> <4978E09F.4080507@v.loewis.de> Message-ID: <497A4950.80909@roumenpetrov.info> Luke Kenneth Casson Leighton wrote: [SNIP] >> python.exe (say, the official one) loads >> python25.dll. Then, an import is made of a ming-wine extension, say >> foo.pyd, which is linked with libpython2.5.dll, which then gets loaded. >> Voila, you have two interpreters in memory, with different type objects, >> memory heaps, and so on. > > ok, there's a solution for that - the gist of the solution is already > implemented in things like Apache Runtime and Apache2 (modules), and > is an extremely common standard technique implemented in OS kernels. > the "old school" name for it is "vector tables". > [SNIP] Did you think that this will escape python MSVC from "Assembly hell" ? Roumen From bugtrack at roumenpetrov.info Fri Jan 23 23:48:56 2009 From: bugtrack at roumenpetrov.info (Roumen Petrov) Date: Sat, 24 Jan 2009 00:48:56 +0200 Subject: [Python-Dev] compiling python2.5 (msys+mingw+wine) - giving up using msvcr80 assemblies for now In-Reply-To: <49797A95.6050402@v.loewis.de> References: <49777A99.9010607@v.loewis.de> <49778FF6.8070608@v.loewis.de> <4978BE6D.6060000@v.loewis.de> <4978D46C.3030200@v.loewis.de> <4978E09F.4080507@v.loewis.de> <49797A95.6050402@v.loewis.de> Message-ID: <497A4958.8050109@roumenpetrov.info> [SNIP] >>> No, Python 2.5 is linked with msvcr71.dll. >> ehn? i don't see that anywhere in any PC/* files - i do see that >> there's a dependency on .NET SDK 1.1 which uses msvcr71.dll > > Take a look at PCbuild/pythoncore.vcproj. It says > > Version="7.10" > > This is how you know VS 2003 was used to build Python 2.5, which > in turn links in msvcr71.dll. Luke, the python MSVC build assume equivalence between MSVC compiler and runtime, i.e. if compiler is version X the runtime is version Y. This is not try for GCC(mingw) build. This compiler is more flexible and allow build against different runtimes. But you know this - I already comment issue870382 in the patch. > Regards, > Martin From tjreedy at udel.edu Sat Jan 24 00:26:48 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 23 Jan 2009 18:26:48 -0500 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: References: <49798235.50509@v.loewis.de> <497A261E.5040004@v.loewis.de> <79990c6b0901231339rd7835b6r11e5892cb8523530@mail.gmail.com> Message-ID: Brett Cannon wrote: > On Fri, Jan 23, 2009 at 13:39, Paul Moore wrote: > >> I'm not sure I'm >> comfortable with sitting back and waiting to quite that extent (I'm >> *already* biting my tongue over some of Brett's comments with which I >> strongly disagree), but I'd rather not have the PEP dissolve in a >> flamewar before Brett has a chance to study things better. >> > > It's going to dissolve anyway. I am just trying to keep it to a single > situation instead of having to go over it multiple times. I have two suggestions: 1. Conduct the discussion on python-ideas rather than python-dev so as to not overwhelm the day-to-day discussions and also to provide a bit of psychological distance. 2. Have several focused threads. Some examples: Given three experimental setups a. Experience with bzr b. Experience with hg c. Experience with git later d. Comparisons based on real experience Your different comments might best be separately discussed. "In the PEP, I said 'blah, blah'. Comments?" Terry From tjreedy at udel.edu Sat Jan 24 01:19:19 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 23 Jan 2009 19:19:19 -0500 Subject: [Python-Dev] __del__ and tp_dealloc in the IO lib In-Reply-To: References: <52dc1c820901181738h53e7ecf9if254a91de2025722@mail.gmail.com> <1232708220.8900.23.camel@ozzu> <4979E6C5.3000502@develer.com> Message-ID: Guido van Rossum wrote: > On Fri, Jan 23, 2009 at 12:52 PM, Terry Reedy wrote: >> While a strong argument can be made that the remaining 2.x versions should >> not be changed, they do not apply to 3.x. New code and ported old code >> should use 'with' wherever quick closing needs to be guaranteed. The 3.0 >> manual clearly states "An implementation is allowed to postpone garbage >> collection or omit it altogether " > > I would hope the 2.x manual says the same, since that same assumption > has been around explicitly ever since JPython was first introduced. Yes. It was changed a bit when gc was added. > I'm not sure we should exempt 2.x from these changes (though if only > 3.x could be made twice as fast it would of course encourage people to > upgrade... :-). ;-) If the issue became real, one could ask 2.x users which they prefer, compatability or speed. I have no opinion since my concern is with 3.x. >> OK, it also goes on to say "(Implementation note: the current implementation >> uses a reference-counting scheme with (optional) delayed detection of >> cyclically linked garbage,...)" I think the first part should at least be >> amended to 'the current CPython implementation' or 'the CPython >> implementation currently' or even better 'one current implementation >> (CPython)' and a warning added "But this may change" and "is not true of all >> implementaions" if that is not made clear otherwise. > > True. http://bugs.python.org/issue5039 tjr From ncoghlan at gmail.com Sat Jan 24 02:44:10 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 24 Jan 2009 11:44:10 +1000 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: References: <49798235.50509@v.loewis.de> <497A2357.7060205@v.loewis.de> Message-ID: <497A726A.30204@gmail.com> Brett Cannon wrote: > On Fri, Jan 23, 2009 at 12:11, Steven Bethard wrote: >> On Fri, Jan 23, 2009 at 12:06 PM, "Martin v. L?wis" wrote: >>>> import random >>>> print(random.choice('svn', 'bzr', 'hg', 'git')) >>> Nice! So it's bzr, as my machine just told me (after adding >>> the square brackets). >> Wow, that decision was a lot easier than I thought it would be. ;-) > > But my machine just told me svn, which is even easier as that means we > don't need to change anything. =) Mine briefly flirted with git, but quickly changed its mind. It *is* a Kubuntu machine though, so it's probably biased :) Cheers, Nick. >>> import random >>> random.choice(['svn', 'bzr', 'hg', 'git']) 'git' >>> random.choice(['svn', 'bzr', 'hg', 'git']) 'bzr' >>> random.choice(['svn', 'bzr', 'hg', 'git']) 'bzr' >>> random.choice(['svn', 'bzr', 'hg', 'git']) 'bzr' -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From lkcl at lkcl.net Sat Jan 24 12:15:45 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Sat, 24 Jan 2009 11:15:45 +0000 Subject: [Python-Dev] future-proofing vector tables for python APIs: binary-module interoperability Message-ID: On Fri, Jan 23, 2009 at 10:48 PM, Roumen Petrov wrote: >>> python.exe (say, the official one) loads >>> python25.dll. Then, an import is made of a ming-wine extension, say >>> foo.pyd, which is linked with libpython2.5.dll, which then gets loaded. >>> Voila, you have two interpreters in memory, with different type objects, >>> memory heaps, and so on. >> >> ok, there's a solution for that - the gist of the solution is already >> implemented in things like Apache Runtime and Apache2 (modules), and >> is an extremely common standard technique implemented in OS kernels. >> the "old school" name for it is "vector tables". >> > [SNIP] Did you think that this will escape python MSVC from "Assembly hell" > ? let me think about that.... write some things down, i might have an answer at the end :) but it would certainly mean that there would be both a future-proof path for binary modules from either msvc-compiled _or_ mingw-compiled 2.5, 2.6, 2.7 etc. to work with 2.5, 2.6, 2.7, 2.8 etc. _without_ a recompile. [forwards-future-proof-compatibility _is_ possible, but... it's a bit more... complicated. backwards-compatibility is easy]. what you do is you make sure that the vector-table is always and only "extended" - added to - never "removed from" or altered. if one function turns out to be a screw-up (inadequate, not enough parameters), you do NOT change its function parameters, you add an "Ex" version - or an "Ex1" version. just like microsoft does. [.... now you know _why_ they do that "ridiculous" thing of adding FunctionEx1 FunctionEx2 and if you look at the MSHTML specification i think they go up to _six_ revisions of the same function in one case!] to detect revisions of the vector-table you use a "negotiation" tactic. you add a bit-field at the beginning of the struct, and each bit expresses a "new revision" indicating that the vector table has been extended (and so needs to be typecast to a different struct - exactly the same as is done with PyObject being typecast to different structs). the first _function_ in the vector-table is one which the module must call (in its initXXXX()) to pass in the "version number" of the module, to the python runtime. just in case someone needs to know. but for the most part, the initiation - of function call-out - is done _from_ modules, so each and every module will never try to call something beyond what it understands. but basically, not only is this technique nothing new - it's in use in Apache RunTime, FreeDCE, the NT Kernel, the Linux Kernel - but also it's actually _already_ in use in one form in the way that python objects are typecast from PyObject to other types of structs! the difference is that a bit-field would make detection of revisions a bit easier but to be honest you could just as easily make it an int and increase the revision number. .... ok, i've thought about your question, and i think it might [save us from assembly hell]. what you would likely have to do is compile _individual modules_ with assemblies, should they need them. for example, the msvcrt module would obviously have to be.... hey, that'd be interesting, how about having different linked versions of the msvcrt module? coool :) in the mingw builds, it's not necessary to link in PC/msvcrtmodule.o into the python dll - so (and this confused the hell out of me for a minute so i had to do find . -name "msvcrt*") you end up with a Modules/msvcrt.pyd. surely, that should be the _only_ dll which gets _specifically_ linked against msvcr71.dll (or 90, or... whatever) and it would be even _better_ if that then got _named_ msvcr71.pyd, msvcr90.pyd etc. i'll do an experiment, later, to confirm that this actually _does_ work - i.e. creating an msvcr80.pyd with "mingw gcc -specs=msvcr80". the neat thing is that if it works, you wouldn't need to _force_ people to link to the python dll or the python exe with msvcr90 or any other version. and the mingw built python.exe or python dll would be interchangeable, as it would be _specific modules_ that required specific versions of the msvc runtime. l. From techtonik at gmail.com Sat Jan 24 12:26:34 2009 From: techtonik at gmail.com (anatoly techtonik) Date: Sat, 24 Jan 2009 13:26:34 +0200 Subject: [Python-Dev] About SCons Re: progress: compiling python2.5 under msys (specifically but not exclusively under wine) with msvcr80 In-Reply-To: <497A44A1.20907@roumenpetrov.info> References: <497A44A1.20907@roumenpetrov.info> Message-ID: On Sat, Jan 24, 2009 at 12:28 AM, Roumen Petrov >> >> I would better use SCons for both unix and windows builds. In case of >> windows for both compilers - mingw and microsoft ones. To port curses >> extension to windows I need to know what gcc options mean, what are >> the rules to write Makefiles and how to repeat these rules as well as >> find options in visual studio interface. Not mentioning various >> platform-specific defines and warning fixes. > > Did you select one of existing curses library for windows ? I've selected PDCurses and successfully compiled the module and run demos manually - you may see the batch and the patch at http://bugs.python.org/issue2889 However, I was asked for VS2008 project file and this is where it all stopped for 8 months already. First I couldn't get the VS2008, then it refused to run on my W2K and now I can't get enough time to learn it (including that I have 50%/40% experience in PHP/Python and only 5%/5% C/Java). -- --anatoly t. From techtonik at gmail.com Sat Jan 24 14:07:51 2009 From: techtonik at gmail.com (anatoly techtonik) Date: Sat, 24 Jan 2009 15:07:51 +0200 Subject: [Python-Dev] subprocess crossplatformness and async communication Message-ID: Greetings, This turned out to be a rather long post that in short can be summarized as: "please-please-please, include asynchronous process communication in subprocess module and do not allow "available only on ..." functionality", because it hurts the brain". Code to speak for itself: http://code.activestate.com/recipes/440554/ The subprocess module was a great step forward to unify various spawn and system and exec and etc. calls in one module, and more importantly - in one uniform API. But this API is partly crossplatform, and I believe I've seen recent commits to docs with more unix-only differences in this module. The main point of this module is to "allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes". PEP 324 goal is also to make "make Python an even better replacement language for over-complicated shell scripts". Citing pre-subrocess PEP 324, "Currently, Python has a large number of different functions for process creation. This makes it hard for developers to choose." Now there is one class with many methods and many platform-specific comments and notices. To make thing worse people on Unix use subprocess with fcntl and people on windows tend not to use it at all, because it looks complicated and doesn't solve the problem with asynchronous communication. That I suggest is to add either support for async crossplatfrom read/write/probing of executed process or a comment at the top of module documentation that will warn that subprocess works in blocking mode. With async mode you can emulate blocking, the opposite is not possible. This will save python users a lot of time. Thanks for reading my rant. BTW, the proposed change is top 10 python recipe on ActiveState http://code.activestate.com/recipes/langs/python/ -- --anatoly t. From python-3000 at udmvt.ru Sat Jan 24 14:28:12 2009 From: python-3000 at udmvt.ru (Alexey G.) Date: Sat, 24 Jan 2009 17:28:12 +0400 Subject: [Python-Dev] Should ftplib use UTF-8 instead of latin-1 encoding? In-Reply-To: <497A423D.9020302@v.loewis.de> References: <729626cc0901221813m72a9896bj30092dbfbd5cb133@mail.gmail.com> <4979805B.4030107@v.loewis.de> <20090123185501.GA24843@phd.pp.ru> <497A272A.4020102@v.loewis.de> <497A423D.9020302@v.loewis.de> Message-ID: <20090124132812.GA15096@ruber.office.udmvt.ru> On Fri, Jan 23, 2009 at 11:18:37PM +0100, "Martin v. L?wis" wrote: > > I don't see how starting with an empty directory helps. The filename > > comes from the client, and the FTP server can't know what the actual > > encoding of that filename is. > > Sure it can. If the client supports RFC 2640, it will send file names > in UTF-8. If the client does not support RFC 2640, the client must > restrict itself to 7-bit file names (i.e. ASCII). If the client violates > the protocol, the server must respond with error 501. Perhaps, that is true, but that is in the world of standards. In my life I remember the situation when users uploaded files from Windows with names in CP866 encoding to UNIX-based ftp server, which by itself had KOI8-R as the encoding for LC_CTYPE. Since administrator was unhappy being impossible to read the names of files correctly, he found and installed specialized ("russified") version of ftp daemon, which had configuration settings, that said what is the network encoding and what is the filesystem encoding. So both ftp daemon and ftp clients violated RFC, but users and administrator were happy. I think, we should preserve the ability of ftp client to download all files he see in the listing from the server. What to do with user specified filenames when they cannot be encoded into ascii and server does not support UTF8, but violates RFC and allows 8-bit bytes in the file names? The ideal ftp client will ask the user about the encoding he thiks filenames are stored on the server side and then recode from user's encoding. It also allow the user to try several variants, if first don't work. It will allow user to download files with names in several different encodings from the same server using single ftp session. Dumb client will send filename from user as bytes, and will succeed, if user was able to specify filename verbatim. Anything between that will make the idea of using Unicode as character encoding for filenames absurd, since it will only break the i18n capabilities of the library. If python library will have file name encoding hardwired to latin1, but arguments will only be unicode strings, well, a lot of people will not even notice that, since they use only ascii part of utf-8. But then there will be again numerous "russification"-like patches to python and to modules, which are incompatible with everything, but work well in some very specific situations. This is the evil that was supposed to be defeated with i18n and with the total adoption of Unicode. Alexey G. From aahz at pythoncraft.com Sat Jan 24 16:25:08 2009 From: aahz at pythoncraft.com (Aahz) Date: Sat, 24 Jan 2009 07:25:08 -0800 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: References: Message-ID: <20090124152507.GA23994@panix.com> On Thu, Jan 22, 2009, Brett Cannon wrote: > > I have now converted PEP 374 > (http://www.python.org/dev/peps/pep-0374/) from Google Docs to reST > and checked it in. First of all, thanks for providing PEP number, URL, and short title; that makes it much easier to keep track of the discussion on list. Second, I think it would be good to explicitly mention the option of deferring this PEP. Based on previous discussion, it sounds like there are a fair number of people who think that there is a DVCS in Python's future, but not now (where "now" means over the next couple of years). -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From guido at python.org Sat Jan 24 16:58:40 2009 From: guido at python.org (Guido van Rossum) Date: Sat, 24 Jan 2009 07:58:40 -0800 Subject: [Python-Dev] subprocess crossplatformness and async communication In-Reply-To: References: Message-ID: Anatoly, I'm confused. The subprocess already allows reading/writing its stdin/stdout/stderr, and AFAIK it's a platform-neutral API. I'm sure there's something missing, but your post doesn't make it clear what exactly, and the recipe you reference is too large to digest easily. Can you explain what it is that the current subprocess does't have beyond saying "async communication" (which could mean many things to many people)? --Guido On Sat, Jan 24, 2009 at 5:07 AM, anatoly techtonik wrote: > Greetings, > > This turned out to be a rather long post that in short can be summarized as: > "please-please-please, include asynchronous process communication in > subprocess module and do not allow "available only on ..." > functionality", because it hurts the brain". > > Code to speak for itself: http://code.activestate.com/recipes/440554/ > > > The subprocess module was a great step forward to unify various spawn > and system and exec and etc. calls in one module, and more importantly > - in one uniform API. But this API is partly crossplatform, and I > believe I've seen recent commits to docs with more unix-only > differences in this module. > > The main point of this module is to "allows you to spawn new > processes, connect to their input/output/error pipes, and obtain their > return codes". PEP 324 goal is also to make "make Python an even > better replacement language for over-complicated shell scripts". > > Citing pre-subrocess PEP 324, "Currently, Python has a large number of > different functions for process creation. This makes it hard for > developers to choose." Now there is one class with many methods and > many platform-specific comments and notices. To make thing worse > people on Unix use subprocess with fcntl and people on windows tend > not to use it at all, because it looks complicated and doesn't solve > the problem with asynchronous communication. > > That I suggest is to add either support for async crossplatfrom > read/write/probing of executed process or a comment at the top of > module documentation that will warn that subprocess works in blocking > mode. With async mode you can emulate blocking, the opposite is not > possible. This will save python users a lot of time. > > Thanks for reading my rant. > > > BTW, the proposed change is top 10 python recipe on ActiveState > http://code.activestate.com/recipes/langs/python/ > > -- > --anatoly t. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Sat Jan 24 16:58:37 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 25 Jan 2009 01:58:37 +1000 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <20090124152507.GA23994@panix.com> References: <20090124152507.GA23994@panix.com> Message-ID: <497B3AAD.5020403@gmail.com> Aahz wrote: > On Thu, Jan 22, 2009, Brett Cannon wrote: >> I have now converted PEP 374 >> (http://www.python.org/dev/peps/pep-0374/) from Google Docs to reST >> and checked it in. > > First of all, thanks for providing PEP number, URL, and short title; > that makes it much easier to keep track of the discussion on list. > > Second, I think it would be good to explicitly mention the option of > deferring this PEP. Based on previous discussion, it sounds like there > are a fair number of people who think that there is a DVCS in Python's > future, but not now (where "now" means over the next couple of years). Put me in that category - the switch from CVS to SVN was simple and obvious because SVN set out to be a better CVS and achieved that goal admirably. The only major hurdle to adopting it was getting the history across, and Martin was able to handle that in the end. The benefits of atomic commits alone were well worth the migration cost. With the level of development still going on in the DVCS area, I think this is a time when dragging our feet on making a decision may actually work to our advantage. Although if Brett genuinely wants to narrow it down to a two-horse race at PyCon, then I think the one thing to keep in mind is how well the chosen tool embodies the Zen of Python (especially "Readability counts" and "One obvious way to do it"). Core devs *are* core devs at least in part because we largely like and agree with those design philosophies. I personally find the command lines for 2 of the presented options quite pleasant to read, while the examples of using the 3rd make me shudder the way I do when I'm forced to read or write a Perl script.* Performance problems can be fixed, but an antithetical design philosophy is unlikely to make for a good tool fit. Cheers, Nick. * In other words, the examples of using git in the PEP make me want to run screaming in the opposite direction. However, assuming bzr's performance issues and line feed handling limitations are addressed by the time the switch actually happens, I'm currently fairly neutral on the choice between bzr and hg. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From bugtrack at roumenpetrov.info Sat Jan 24 20:24:22 2009 From: bugtrack at roumenpetrov.info (Roumen Petrov) Date: Sat, 24 Jan 2009 21:24:22 +0200 Subject: [Python-Dev] future-proofing vector tables for python APIs: binary-module interoperability In-Reply-To: References: Message-ID: <497B6AE6.3050402@roumenpetrov.info> Luke Kenneth Casson Leighton wrote: > On Fri, Jan 23, 2009 at 10:48 PM, Roumen Petrov [SNIP] > but it would certainly mean that there would be both a future-proof > path for binary modules from either msvc-compiled _or_ mingw-compiled > 2.5, 2.6, 2.7 etc. to work with 2.5, 2.6, 2.7, 2.8 etc. _without_ a > recompile. [forwards-future-proof-compatibility _is_ possible, but... > it's a bit more... complicated. backwards-compatibility is easy]. > > what you do is you make sure that the vector-table is always and only > "extended" - added to - never "removed from" or altered. if one > function turns out to be a screw-up (inadequate, not enough > parameters), you do NOT change its function parameters, you add an > "Ex" version - or an "Ex1" version. [SNIP] > but basically, not only is this technique nothing new - it's in use in > Apache RunTime, FreeDCE, the NT Kernel, the Linux Kernel - but also > it's actually _already_ in use in one form in the way that python > objects are typecast from PyObject to other types of structs! the > difference is that a bit-field would make detection of revisions a bit > easier but to be honest you could just as easily make it an int and > increase the revision number. This look like simple RCP implemantation. If I remember well SUN-RPC assign number to program, function, version. [SNIP] > surely, that should be the _only_ dll which gets _specifically_ linked > against msvcr71.dll (or 90, or... whatever) and it would be even > _better_ if that then got _named_ msvcr71.pyd, msvcr90.pyd etc. [SNIP] Yes it is enough to encapsulate memory allocation and file functions into python shared library. The python provide memory allocation functions, but not all modules use them. File functions are hiden by posixmodule and python modules can't use them. Roumen From konryd at gmail.com Sat Jan 24 21:00:25 2009 From: konryd at gmail.com (Konrad Delong) Date: Sat, 24 Jan 2009 21:00:25 +0100 Subject: [Python-Dev] Additional behaviour for itertools.combinations Message-ID: <74401640901241200t7db8defbtb38d3f53b9ce544d@mail.gmail.com> I'm not sure if it's the right place to post it. If so - I'll be glad to learn where is one. Anyway: I think the function itertools.combinations would benefit from making the 'r' (length of the combinations) argument optionally a sequence. With that change one could call combinations(sequence, [2, 3]) in order to get all combinations of length 2 and 3. In particular, one could call combinations(sequence, range(len(sequence)) in order to get *all* combinations of given sequence. The change would be backwards compatible as it would check for sequential arguments. Is it worth the shot? best regards Konrad PS. Didn't want to spoil the beginning of the post, but I consider it to be a good practice to introduce oneself when posting the first time, so: Hello, my name is Konrad, I'm an IT student and I'm following python-dev for some time, but never posted before. From python at rcn.com Sat Jan 24 21:23:51 2009 From: python at rcn.com (Raymond Hettinger) Date: Sat, 24 Jan 2009 12:23:51 -0800 Subject: [Python-Dev] Additional behaviour for itertools.combinations References: <74401640901241200t7db8defbtb38d3f53b9ce544d@mail.gmail.com> Message-ID: From: "Konrad Delong" > I'm not sure if it's the right place to post it. If so - I'll be glad > to learn where is one. Please post a feature request on the bug tracker and assign it to me. > Anyway: > I think the function itertools.combinations would benefit from making > the 'r' (length of the combinations) argument optionally a sequence. > > With that change one could call combinations(sequence, [2, 3]) in > order to get all combinations of length 2 and 3. > In particular, one could call combinations(sequence, > range(len(sequence)) in order to get *all* combinations of given > sequence. This design is similar to the API for similar functionality in mathematica. The question is whether there are sufficient worthwhile use cases to warrant the added API complexity and algorithm complexity. The latter is a bit tricky if we want to maintain the lexicographic ordering and the notion of combinations being a subsequence of the permutations code. Since I expect students to be among the users for the comb/perm functions, there is some merit to keeping the API as simple as possible. Besides, it is not hard to use the existing tool as a primitive to get to the one you want: def mycombinations(iterable, r_seq): # mycombinations('abc', [1,2]) --> A B C AB AC BC iterable = list(iterable) return chain.from_iterable(imap(combinations, repeat(iterable), r_seq)) > PS. Didn't want to spoil the beginning of the post, but I consider it > to be a good practice to introduce oneself when posting the first > time, so: Hello, my name is Konrad, I'm an IT student and I'm > following python-dev for some time, but never posted before. Hello Konrad. Welcome to python-dev. Raymond Hettinger From skip at pobox.com Sat Jan 24 21:33:20 2009 From: skip at pobox.com (skip at pobox.com) Date: Sat, 24 Jan 2009 14:33:20 -0600 (CST) Subject: [Python-Dev] ac_sys_system == Monterey*? Message-ID: <20090124203320.3DF09D52DF4@montanaro.dyndns.org> >From configure.in: # The current (beta) Monterey compiler dies with optimizations # XXX what is Monterey? Does it still die w/ -O? Can we get rid of this? case $ac_sys_system in Monterey*) OPT="" ;; esac What is Monterey? Can this check be removed from trunk/py3k branches? Skip From eric at trueblade.com Sat Jan 24 21:47:14 2009 From: eric at trueblade.com (Eric Smith) Date: Sat, 24 Jan 2009 15:47:14 -0500 Subject: [Python-Dev] ac_sys_system == Monterey*? In-Reply-To: <20090124203320.3DF09D52DF4@montanaro.dyndns.org> References: <20090124203320.3DF09D52DF4@montanaro.dyndns.org> Message-ID: <497B7E52.9060903@trueblade.com> skip at pobox.com wrote: >>From configure.in: > > # The current (beta) Monterey compiler dies with optimizations > # XXX what is Monterey? Does it still die w/ -O? Can we get rid of this? > case $ac_sys_system in > Monterey*) > OPT="" > ;; > esac > > What is Monterey? Can this check be removed from trunk/py3k branches? This post http://mail.python.org/pipermail/patches/2000-August/001708.html would have you believe it's a 64-bit AIX compiler. From martin at v.loewis.de Sat Jan 24 21:54:57 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 24 Jan 2009 21:54:57 +0100 Subject: [Python-Dev] ac_sys_system == Monterey*? In-Reply-To: <20090124203320.3DF09D52DF4@montanaro.dyndns.org> References: <20090124203320.3DF09D52DF4@montanaro.dyndns.org> Message-ID: <497B8021.5000806@v.loewis.de> > What is Monterey? Monterey was the code name of a joint operating system project of SCO and IBM, porting AIX to 64-bit processors (apparently, IA-64 and POWER). See http://www.cs.umbc.edu/help/architecture/idfmontereylab.pdf http://en.wikipedia.org/wiki/Project_Monterey Monterey was cancelled in 2000, although parts of it were integrated into AIX 5L. Regards, Martin From skip at pobox.com Sat Jan 24 22:03:36 2009 From: skip at pobox.com (skip at pobox.com) Date: Sat, 24 Jan 2009 15:03:36 -0600 (CST) Subject: [Python-Dev] Change Makefile.pre.in based on configure? Message-ID: <20090124210336.5213AD55391@montanaro.dyndns.org> I'm working on issue 4111 which will add dtrace support to Python when requested by the builder and when supported by the platform (currently just Solaris and Mac OSX I believe). Sun and Apple have quite different ways to generate the code necessary to link into the executable. Sun's dtrace command supports a -G flag which generates a .o file from a .d file. Apple instead generates an include file using the -h flag to dtrace (-G and .o file generation are not supported). This puts a bit of a crimp in generating Makefile dependencies. In the Sun case you have a couple extra .o files to link into libpython. In the Apple case you have a couple extra .h files which define Dtrace macros. How do I work around this difference in Makefile.pre.in? I can detect Sun vs. Apple in the configure script, but I see no conditional logic in Makefile.pre.in to use as an example. It seems to only use variable expansion on the RHS of stuff. Can I do something like if @WITH_DTRACE_SUN@ = 1 then ... Sun-style dependencies here ... else ... Apple-style dependencies here ... fi where WITH_DTRACE_SUN is a macro defined in pyconfig.h by the configure script? Thanks, Skip From lkcl at lkcl.net Sat Jan 24 22:10:41 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Sat, 24 Jan 2009 21:10:41 +0000 Subject: [Python-Dev] future-proofing vector tables for python APIs: binary-module interoperability In-Reply-To: <497B6AE6.3050402@roumenpetrov.info> References: <497B6AE6.3050402@roumenpetrov.info> Message-ID: >> but basically, not only is this technique nothing new - it's in use in >> Apache RunTime, FreeDCE, the NT Kernel, the Linux Kernel - but also > This look like simple RPC implemantation. yep. > If I remember well SUN-RPC assign number to program, function, version. yep - i forgot about that one: yes, that's another example. this is pretty basic well-understood, well-documented techniques that virtually every large project that requires isolation between components (and an upgrade path) ends up using in one form or another. the only fly in the ointment will be that putting pointers to PyType_String etc. etc. into a vector table (struct), you end up with an extra de-ref overhead, which is often an argument utilised to do away with vector tables. but - tough: the decision involves getting away from "Hell" to something that makes everyone's lives that much easier, it's an easy decision to make. >> surely, that should be the _only_ dll which gets _specifically_ linked >> against msvcr71.dll (or 90, or... whatever) and it would be even >> _better_ if that then got _named_ msvcr71.pyd, msvcr90.pyd etc. > > [SNIP] > Yes it is enough to encapsulate memory allocation and file functions into > python shared library. The python provide memory allocation functions, but > not all modules use them. File functions are hiden by posixmodule and python > modules can't use them. except ... posixmodule gets renamed to ntmodule .... oh, i see what you mean: python modules aren't allowed _direct_ access to msvcrtNN's file functions, they have to go via posixmodule-renamed-to-ntmodule. so it's still ok. l. From brett at python.org Sat Jan 24 22:23:41 2009 From: brett at python.org (Brett Cannon) Date: Sat, 24 Jan 2009 13:23:41 -0800 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <20090124152507.GA23994@panix.com> References: <20090124152507.GA23994@panix.com> Message-ID: On Sat, Jan 24, 2009 at 07:25, Aahz wrote: > On Thu, Jan 22, 2009, Brett Cannon wrote: >> >> I have now converted PEP 374 >> (http://www.python.org/dev/peps/pep-0374/) from Google Docs to reST >> and checked it in. > > First of all, thanks for providing PEP number, URL, and short title; > that makes it much easier to keep track of the discussion on list. > =) Welcome. > Second, I think it would be good to explicitly mention the option of > deferring this PEP. Based on previous discussion, it sounds like there > are a fair number of people who think that there is a DVCS in Python's > future, but not now (where "now" means over the next couple of years). Sure, I can add a note somewhere that says if a clear winner doesn't come about the PEP can be revisited to a later date. -Brett From lkcl at lkcl.net Sat Jan 24 22:51:13 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Sat, 24 Jan 2009 21:51:13 +0000 Subject: [Python-Dev] mingw+msys port of python2.7a0 (svn #r68884) Message-ID: http://bugs.python.org/issue5046 mingw+msys port which was previously done against python-2.5.2 has been brought forward to latest subversion r68884. the primary reason for initially doing python 2.5.2 was to 1) "stay out of the way" of primary python development 2) provide some hope for those people still using win98 and nt. builds using msvcr90 are possible with the --enable-msvcr9build switch. install mingw, install msys, optionally install a boat-load of libraries such as sqlite3, zlib, bz2, dbm etc. (good luck, it's a pain - ask me if you'd like some prebuilt). make sure you patch the mingw runtime if you want to use --enable-msvcr9build (again, ask me if you'd like the patched headers and prebuilt libmsvcrNN.a files). also added is the msi module - http:/ /lkcl.net/msi.tgz and run make in the msi directory, to install the import libraries and header files borrowed from Wine and beaten into submission. run ./configure --enable-win32build=yes --enable-shared=yes and go do something else for about 10 minutes, then run make and make install. no proprietary compilers or tools were used or harmed [*] in the making or development of this patch. l. [*] such a shame... From python at rcn.com Sat Jan 24 23:46:52 2009 From: python at rcn.com (Raymond Hettinger) Date: Sat, 24 Jan 2009 14:46:52 -0800 Subject: [Python-Dev] Operator module deprecations Message-ID: I would like to deprecate some outdated functions in the operator module. The isSequenceType(), isMappingType(), and isNumberType() functions never worked reliably and now their intended purpose has been largely fulfilled by ABCs. The isCallable() function has long been deprecated and I think it's finally time to rip it out. The repeat() function never really corresponded to an operator. Instead, it reflected an underlying implementation detail (namely the naming of the sq_repeat slot and the abstract C API function PySequence_Repeat). That functionality is already exposed by operator.mul: operator.mul('abc', 3) --> 'abcabcabc' Raymond From skip at pobox.com Sun Jan 25 00:22:20 2009 From: skip at pobox.com (skip at pobox.com) Date: Sat, 24 Jan 2009 17:22:20 -0600 Subject: [Python-Dev] ac_sys_system == Monterey*? In-Reply-To: <497B8021.5000806@v.loewis.de> References: <20090124203320.3DF09D52DF4@montanaro.dyndns.org> <497B8021.5000806@v.loewis.de> Message-ID: <18811.41644.577224.31055@montanaro.dyndns.org> Martin> Monterey was cancelled in 2000, although parts of it were Martin> integrated into AIX 5L. Thanks... http://bugs.python.org/issue4111 It doesn't appear it's mentioned anywhere other than in the configure script. Skip From skip at pobox.com Sun Jan 25 00:23:37 2009 From: skip at pobox.com (skip at pobox.com) Date: Sat, 24 Jan 2009 17:23:37 -0600 Subject: [Python-Dev] ac_sys_system == Monterey*? In-Reply-To: <497B8021.5000806@v.loewis.de> References: <20090124203320.3DF09D52DF4@montanaro.dyndns.org> <497B8021.5000806@v.loewis.de> Message-ID: <18811.41721.380619.894328@montanaro.dyndns.org> http://bugs.python.org/issue4111 Jeez, I'm an idiot. Should be http://bugs.python.org/issue5047 Skip From martin at v.loewis.de Sun Jan 25 00:31:19 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 25 Jan 2009 00:31:19 +0100 Subject: [Python-Dev] Change Makefile.pre.in based on configure? In-Reply-To: <20090124210336.5213AD55391@montanaro.dyndns.org> References: <20090124210336.5213AD55391@montanaro.dyndns.org> Message-ID: <497BA4C7.9020807@v.loewis.de> > How do I work around this difference in Makefile.pre.in? To answer this question, I would have to see the exact fragment that you want to see in the Solaris case, and the exact fragment that you want to have in the OSX case. Can you provide these? Regards, Martin From martin at v.loewis.de Sun Jan 25 00:34:40 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 25 Jan 2009 00:34:40 +0100 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: References: <20090124152507.GA23994@panix.com> Message-ID: <497BA590.7060406@v.loewis.de> >> Second, I think it would be good to explicitly mention the option of >> deferring this PEP. Based on previous discussion, it sounds like there >> are a fair number of people who think that there is a DVCS in Python's >> future, but not now (where "now" means over the next couple of years). > > Sure, I can add a note somewhere that says if a clear winner doesn't > come about the PEP can be revisited to a later date. > I think the request is slightly different: consider that a potential outcome should be "svn for the next five years, then reconsider" - not because none of the DVCS is a clear winner, but because there is too much resistance to DVCSes in general, at the moment. Regards, Martin From skip at pobox.com Sun Jan 25 00:37:21 2009 From: skip at pobox.com (skip at pobox.com) Date: Sat, 24 Jan 2009 17:37:21 -0600 Subject: [Python-Dev] Change Makefile.pre.in based on configure? In-Reply-To: <497BA4C7.9020807@v.loewis.de> References: <20090124210336.5213AD55391@montanaro.dyndns.org> <497BA4C7.9020807@v.loewis.de> Message-ID: <18811.42545.569745.170812@montanaro.dyndns.org> >> How do I work around this difference in Makefile.pre.in? Martin> To answer this question, I would have to see the exact fragment Martin> that you want to see in the Solaris case, and the exact fragment Martin> that you want to have in the OSX case. Can you provide these? I'll work on it and get back to you if I get completely stuck. I think I've figured a way out of my current dilemma, but a little experimentation will be required to see if I'm correct. Thx, Skip From brett at python.org Sun Jan 25 00:40:38 2009 From: brett at python.org (Brett Cannon) Date: Sat, 24 Jan 2009 15:40:38 -0800 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <497BA590.7060406@v.loewis.de> References: <20090124152507.GA23994@panix.com> <497BA590.7060406@v.loewis.de> Message-ID: On Sat, Jan 24, 2009 at 15:34, "Martin v. L?wis" wrote: >>> Second, I think it would be good to explicitly mention the option of >>> deferring this PEP. Based on previous discussion, it sounds like there >>> are a fair number of people who think that there is a DVCS in Python's >>> future, but not now (where "now" means over the next couple of years). >> >> Sure, I can add a note somewhere that says if a clear winner doesn't >> come about the PEP can be revisited to a later date. >> > > I think the request is slightly different: consider that a potential > outcome should be "svn for the next five years, then reconsider" - not > because none of the DVCS is a clear winner, but because there is too > much resistance to DVCSes in general, at the moment. I already put a note in that no DVCS might be chosen once the PEP is finished. Whether it is because no DVCS is a clear improvement over svn or people just don't like a DVCS seems like a minor thing to worry about to spell out in the PEP. -Brett From ncoghlan at gmail.com Sun Jan 25 01:44:23 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 25 Jan 2009 10:44:23 +1000 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: References: <20090124152507.GA23994@panix.com> <497BA590.7060406@v.loewis.de> Message-ID: <497BB5E7.4080606@gmail.com> Brett Cannon wrote: > On Sat, Jan 24, 2009 at 15:34, "Martin v. L?wis" wrote: >>>> Second, I think it would be good to explicitly mention the option of >>>> deferring this PEP. Based on previous discussion, it sounds like there >>>> are a fair number of people who think that there is a DVCS in Python's >>>> future, but not now (where "now" means over the next couple of years). >>> Sure, I can add a note somewhere that says if a clear winner doesn't >>> come about the PEP can be revisited to a later date. >>> >> I think the request is slightly different: consider that a potential >> outcome should be "svn for the next five years, then reconsider" - not >> because none of the DVCS is a clear winner, but because there is too >> much resistance to DVCSes in general, at the moment. > > I already put a note in that no DVCS might be chosen once the PEP is > finished. Whether it is because no DVCS is a clear improvement over > svn or people just don't like a DVCS seems like a minor thing to worry > about to spell out in the PEP. I suspect the reactions will be more nuanced than that anyway - e.g. my current position is that while I like the idea of a DVCS in principle and agree there are definite gains to be had in switching to one, I don't think the contenders have had enough time to shake out their competing feature sets and relative performance. We don't seem to lose a lot by sticking with SVN at least until after 2.7/3.1 are out the door and then revisiting the DVCS question (this is particularly so given that the current plan is go for a fairly short turnaround on those two releases). As the zen says, now is better than never, but never is often better than *right* now :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From brett at python.org Sun Jan 25 01:48:29 2009 From: brett at python.org (Brett Cannon) Date: Sat, 24 Jan 2009 16:48:29 -0800 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <497BB5E7.4080606@gmail.com> References: <20090124152507.GA23994@panix.com> <497BA590.7060406@v.loewis.de> <497BB5E7.4080606@gmail.com> Message-ID: On Sat, Jan 24, 2009 at 16:44, Nick Coghlan wrote: > Brett Cannon wrote: >> On Sat, Jan 24, 2009 at 15:34, "Martin v. L?wis" wrote: >>>>> Second, I think it would be good to explicitly mention the option of >>>>> deferring this PEP. Based on previous discussion, it sounds like there >>>>> are a fair number of people who think that there is a DVCS in Python's >>>>> future, but not now (where "now" means over the next couple of years). >>>> Sure, I can add a note somewhere that says if a clear winner doesn't >>>> come about the PEP can be revisited to a later date. >>>> >>> I think the request is slightly different: consider that a potential >>> outcome should be "svn for the next five years, then reconsider" - not >>> because none of the DVCS is a clear winner, but because there is too >>> much resistance to DVCSes in general, at the moment. >> >> I already put a note in that no DVCS might be chosen once the PEP is >> finished. Whether it is because no DVCS is a clear improvement over >> svn or people just don't like a DVCS seems like a minor thing to worry >> about to spell out in the PEP. > > I suspect the reactions will be more nuanced than that anyway - e.g. my > current position is that while I like the idea of a DVCS in principle > and agree there are definite gains to be had in switching to one, I > don't think the contenders have had enough time to shake out their > competing feature sets and relative performance. We don't seem to lose a > lot by sticking with SVN at least until after 2.7/3.1 are out the door > and then revisiting the DVCS question (this is particularly so given > that the current plan is go for a fairly short turnaround on those two > releases). > As part of my impressions I plan to also look at usage on top of svn as a viable alternative if no clear winner comes about. That way if they work well directly on top of svn we can write up very clear documentation on how to use any of them directly on top of svn and still gain the benefits of offline checkins and cheap branching. Maintenance then becomes simply keeping a read-only mirror going on code.python.org. > As the zen says, now is better than never, but never is often better > than *right* now :) Don't worry, I am not going to push something down anyone's throats if I don't feel secure that it is the best choice. From brett at python.org Sun Jan 25 01:49:51 2009 From: brett at python.org (Brett Cannon) Date: Sat, 24 Jan 2009 16:49:51 -0800 Subject: [Python-Dev] Operator module deprecations In-Reply-To: References: Message-ID: On Sat, Jan 24, 2009 at 14:46, Raymond Hettinger wrote: > I would like to deprecate some outdated functions in the operator module. > > The isSequenceType(), isMappingType(), and isNumberType() > functions never worked reliably and now their > intended purpose has been largely fulfilled by > ABCs. > > The isCallable() function has long been deprecated > and I think it's finally time to rip it out. > > The repeat() function never really corresponded to an > operator. Instead, it reflected an underlying implementation detail (namely > the naming of the sq_repeat slot and the abstract C API function > PySequence_Repeat). That functionality is already exposed by operator.mul: > > operator.mul('abc', 3) --> 'abcabcabc' +1 to all of it. From ncoghlan at gmail.com Sun Jan 25 02:18:23 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 25 Jan 2009 11:18:23 +1000 Subject: [Python-Dev] Additional behaviour for itertools.combinations In-Reply-To: References: <74401640901241200t7db8defbtb38d3f53b9ce544d@mail.gmail.com> Message-ID: <497BBDDF.3060406@gmail.com> Raymond Hettinger wrote: > Since I expect students to be among the users for the comb/perm > functions, there is some merit to keeping the API as simple as possible. > Besides, it is not hard to use the existing tool as a primitive to get to > the one you want: > > def mycombinations(iterable, r_seq): > # mycombinations('abc', [1,2]) --> A B C AB AC BC > iterable = list(iterable) > return chain.from_iterable(imap(combinations, repeat(iterable), > r_seq)) Perhaps a reasonable starting point would be to include this as one of the example itertools recipes in the documentation? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Sun Jan 25 02:22:28 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 25 Jan 2009 11:22:28 +1000 Subject: [Python-Dev] Operator module deprecations In-Reply-To: References: Message-ID: <497BBED4.1090005@gmail.com> Brett Cannon wrote: > On Sat, Jan 24, 2009 at 14:46, Raymond Hettinger wrote: >> I would like to deprecate some outdated functions in the operator module. >> >> The isSequenceType(), isMappingType(), and isNumberType() >> functions never worked reliably and now their >> intended purpose has been largely fulfilled by >> ABCs. >> >> The isCallable() function has long been deprecated >> and I think it's finally time to rip it out. >> >> The repeat() function never really corresponded to an >> operator. Instead, it reflected an underlying implementation detail (namely >> the naming of the sq_repeat slot and the abstract C API function >> PySequence_Repeat). That functionality is already exposed by operator.mul: >> >> operator.mul('abc', 3) --> 'abcabcabc' > > +1 to all of it. What Brett said. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From python at rcn.com Sun Jan 25 04:33:37 2009 From: python at rcn.com (Raymond Hettinger) Date: Sat, 24 Jan 2009 19:33:37 -0800 Subject: [Python-Dev] Additional behaviour for itertools.combinations References: <74401640901241200t7db8defbtb38d3f53b9ce544d@mail.gmail.com> <497BBDDF.3060406@gmail.com> Message-ID: <43209A9EB88F49E59730F5BBC63CAF05@RaymondLaptop1> > Raymond Hettinger wrote: >> Since I expect students to be among the users for the comb/perm >> functions, there is some merit to keeping the API as simple as possible. >> Besides, it is not hard to use the existing tool as a primitive to get to >> the one you want: >> >> def mycombinations(iterable, r_seq): >> # mycombinations('abc', [1,2]) --> A B C AB AC BC >> iterable = list(iterable) >> return chain.from_iterable(imap(combinations, repeat(iterable), >> r_seq)) [Nick Coglan] > Perhaps a reasonable starting point would be to include this as one of > the example itertools recipes in the documentation? I would have suggested that but recipe itself is use case challenged. The OP did not mention any compelling use cases or motivations. Essentially, he just pointed-out that it is possible, not that it is desirable. I can't the of a case where I've wanted to loop over variable length subsequences. Having for-loops with tuple unpacking won't work because the combos have more than one possible size. This seems like a hypergeneralization to me. Raymond From ronaldoussoren at mac.com Sun Jan 25 11:11:51 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sun, 25 Jan 2009 11:11:51 +0100 Subject: [Python-Dev] Change Makefile.pre.in based on configure? In-Reply-To: <20090124210336.5213AD55391@montanaro.dyndns.org> References: <20090124210336.5213AD55391@montanaro.dyndns.org> Message-ID: On 24 Jan, 2009, at 22:03, skip at pobox.com wrote: > I'm working on issue 4111 which will add dtrace support to Python when > requested by the builder and when supported by the platform > (currently just > Solaris and Mac OSX I believe). > > Sun and Apple have quite different ways to generate the code > necessary to > link into the executable. Sun's dtrace command supports a -G flag > which > generates a .o file from a .d file. Apple instead generates an > include file > using the -h flag to dtrace (-G and .o file generation are not > supported). > This puts a bit of a crimp in generating Makefile dependencies. In > the Sun > case you have a couple extra .o files to link into libpython. In > the Apple > case you have a couple extra .h files which define Dtrace macros. > > How do I work around this difference in Makefile.pre.in? I can > detect Sun > vs. Apple in the configure script, but I see no conditional logic in > Makefile.pre.in to use as an example. It seems to only use variable > expansion on the RHS of stuff. Can I do something like > > if @WITH_DTRACE_SUN@ = 1 > then > ... Sun-style dependencies here ... > else > ... Apple-style dependencies here ... > fi > > where WITH_DTRACE_SUN is a macro defined in pyconfig.h by the > configure > script? I use configure to paste bits into Makefile.pre.in for the OSX framework support. In Makefile.pre.in: install: @FRAMEWORKINSTALLFIRST@ altinstall bininstall maninstall @FRAMEWORKINSTALLLAST@ FRAMEWORKINSTALLFIRST and FRAMEWORKINSTALLLAST are calculated in configure.in. This should work for dtrace as well. That is, in the configure script define DTRACE_HEADER_DEPS and DTRACE_OBJECT_DEPS and add @DTRACE_HEADER_DEPS@ and @DTRACE_OBJECT_DEPS@ to the proper targets in Makefile.pre.in. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available URL: From steve at pearwood.info Sun Jan 25 11:49:43 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 25 Jan 2009 21:49:43 +1100 Subject: [Python-Dev] Additional behaviour for itertools.combinations In-Reply-To: <43209A9EB88F49E59730F5BBC63CAF05@RaymondLaptop1> References: <74401640901241200t7db8defbtb38d3f53b9ce544d@mail.gmail.com> <497BBDDF.3060406@gmail.com> <43209A9EB88F49E59730F5BBC63CAF05@RaymondLaptop1> Message-ID: <200901252149.43750.steve@pearwood.info> On Sun, 25 Jan 2009 02:33:37 pm Raymond Hettinger wrote: > > Raymond Hettinger wrote: > >> Since I expect students to be among the users for the comb/perm > >> functions, there is some merit to keeping the API as simple as > >> possible. Besides, it is not hard to use the existing tool as a > >> primitive to get to the one you want: > >> > >> def mycombinations(iterable, r_seq): > >> # mycombinations('abc', [1,2]) --> A B C AB AC BC > >> iterable = list(iterable) > >> return chain.from_iterable(imap(combinations, > >> repeat(iterable), r_seq)) > > [Nick Coglan] > > > Perhaps a reasonable starting point would be to include this as one > > of the example itertools recipes in the documentation? > > I would have suggested that but recipe itself is use case challenged. > The OP did not mention any compelling use cases or motivations. > Essentially, he just pointed-out that it is possible, not that it is > desirable. > > I can't the of a case where I've wanted to loop over variable length > subsequences. Having for-loops with tuple unpacking won't work > because the combos have more than one possible size. > > This seems like a hypergeneralization to me. Does answering homework questions count as a use-case? http://mathforum.org/library/drmath/view/56121.html Also calculating the odds of winning Powerball: http://mathforum.org/library/drmath/view/56122.html The number of combinations taken (1, 2, 3, ..., n) at a time is closely related to the Bell Numbers. And according to Wikipedia, the oldest known reference to combinatrics included such a question. http://en.wikipedia.org/wiki/History_of_combinatorics Having said all that, I'm inclined to agree that this is an over-generalisation. As far as I can tell, there's no name for this in mathematics, which suggests that useful applications and theorems are both rare. In any case, it's not that difficult to create a generator to yield all the combinations: (comb for k in ks for comb in itertools.combinations(seq, k)) I'm with Nick that this would make a good example for the documentation. I don't object to combinations growing the extra functionality, but if it does, people will ask why permutations doesn't as well. -- Steven D'Aprano From lkcl at lkcl.net Sun Jan 25 13:34:58 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Sun, 25 Jan 2009 12:34:58 +0000 Subject: [Python-Dev] future-proofing vector tables for python APIs: binary-module interoperability In-Reply-To: References: <497B6AE6.3050402@roumenpetrov.info> Message-ID: >> [SNIP] >> Yes it is enough to encapsulate memory allocation and file functions into >> python shared library. The python provide memory allocation functions, but >> not all modules use them. File functions are hiden by posixmodule and python >> modules can't use them. > > except ... posixmodule gets renamed to ntmodule .... oh, i see what > you mean: python modules aren't allowed _direct_ access to msvcrtNN's > file functions, they have to go via posixmodule-renamed-to-ntmodule. .... thinking about this some more... posixmodule.c is linked (by default) into pythonNN.dll, thus making pythonNN.dll totally dependent on a version of msvcrt. decoupling posixmodule.c from pythonNN.dll leaves the possibility to make python independent of msvcrt versioning. it would need to be a custom-compiled .pyd module, due to the early dependency. i'll see if this is possible. l. From barry at python.org Sun Jan 25 15:52:36 2009 From: barry at python.org (Barry Warsaw) Date: Sun, 25 Jan 2009 09:52:36 -0500 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: References: <20090124152507.GA23994@panix.com> <497BA590.7060406@v.loewis.de> <497BB5E7.4080606@gmail.com> Message-ID: <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 24, 2009, at 7:48 PM, Brett Cannon wrote: > As part of my impressions I plan to also look at usage on top of svn > as a viable alternative if no clear winner comes about. That way if > they work well directly on top of svn we can write up very clear > documentation on how to use any of them directly on top of svn and > still gain the benefits of offline checkins and cheap branching. > Maintenance then becomes simply keeping a read-only mirror going on > code.python.org. There's a possible third way. I've heard (though haven't investigated) that some people are working on supporting the svn wire protocol in the bzr server. This would mean that anybody who's still comfortable with svn and feels no need to change their current habits can continue to work the way they always have. Those that want the extra benefits of a DVCS, or who do not have commit access to the code.python.org branches would have viable alternatives. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSXx8tHEjvBPtnXfVAQK1CgQAoDlHr9KthVr9sA6DfeXE3D35mYUop01X TD06OggbayFDGQYA0Zae+zU050R9UvuTpaF7XtSiSgBlI6n0Bb/rLAgVGskwbMHD LU8BAljNq6FpRp8QY2IHVRWKgOqzSHtz8CvCdlD1yw5CbA/pEvigoLzR0AWAeQJl tzOAetiud2c= =5qIJ -----END PGP SIGNATURE----- From lkcl at lkcl.net Sun Jan 25 16:37:47 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Sun, 25 Jan 2009 15:37:47 +0000 Subject: [Python-Dev] future-proofing vector tables for python APIs: binary-module interoperability In-Reply-To: References: <497B6AE6.3050402@roumenpetrov.info> Message-ID: > decoupling posixmodule.c from pythonNN.dll leaves the possibility to > make python independent of msvcrt versioning. > > it would need to be a custom-compiled .pyd module, due to the early dependency. > > i'll see if this is possible. i'd added PyExc_OSError, for example, as data exported from dlls. i'm finding that this causes.... problems :) so when posixmodule.c is a module (nt.pyd), doing this works: PyAPI_FUNC(PyObject *) PyErr_GetPyExc_OSError(void) { return (PyObject*)PyExc_OSError; } and thus oserr = PyErr_GetPyExc_OSError(); Py_INCREF(oserr); PyModule_AddObject(m, "error", oserr) but doing _direct_ access to PyExc_OSError fails miserably. i'll try to track down why (am adding __cdecl to PyAPI_DATA to see if that helps). l. From lkcl at lkcl.net Sun Jan 25 16:44:02 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Sun, 25 Jan 2009 15:44:02 +0000 Subject: [Python-Dev] progress: compiling python2.5 under msys (specifically but not exclusively under wine) with msvcr80 In-Reply-To: References: Message-ID: > Have you made some benchmarks like pystone? > Cheers, > Cesare Cesare, hi, thanks for responding: unfortunately, there's absolutely no point in making any benchmark figures under an emulated environment which does things like take 2 billion instruction cycles to start up a program named "c:/msys/bin/sh.exe", due to it inexplicably loading 200 GUI-only truetype fonts. and to do benchmarks on say windows would require that i install ... windows! so if somebody else would like to make some benchmarks, and publish them, they are most welcome to do so. l. From bugtrack at roumenpetrov.info Sun Jan 25 16:55:47 2009 From: bugtrack at roumenpetrov.info (Roumen Petrov) Date: Sun, 25 Jan 2009 17:55:47 +0200 Subject: [Python-Dev] future-proofing vector tables for python APIs: binary-module interoperability In-Reply-To: References: <497B6AE6.3050402@roumenpetrov.info> Message-ID: <497C8B83.7080604@roumenpetrov.info> Luke Kenneth Casson Leighton wrote: >>> [SNIP] >>> Yes it is enough to encapsulate memory allocation and file functions into >>> python shared library. The python provide memory allocation functions, but >>> not all modules use them. File functions are hiden by posixmodule and python >>> modules can't use them. >> except ... posixmodule gets renamed to ntmodule .... oh, i see what >> you mean: python modules aren't allowed _direct_ access to msvcrtNN's >> file functions, they have to go via posixmodule-renamed-to-ntmodule. > > .... thinking about this some more... posixmodule.c is linked (by > default) into pythonNN.dll, thus making pythonNN.dll totally dependent > on a version of msvcrt. This is not problem. If python*.dll hide msvcrt and other modules depend directly from python*.dll I expect issue to be resolved. i.e. python*.dll to be "portable runtime interface". > decoupling posixmodule.c from pythonNN.dll leaves the possibility to > make python independent of msvcrt versioning. > > it would need to be a custom-compiled .pyd module, due to the early dependency. > > i'll see if this is possible. ????? From lkcl at lkcl.net Sun Jan 25 16:58:48 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Sun, 25 Jan 2009 15:58:48 +0000 Subject: [Python-Dev] microsoft dlls apparently don't support data. implications: PyAPI functions required to access data across modules. Message-ID: according to the wikipedia entry on dlls, dlls do not support data, only functions. which would explain two things: 1) why certain modules are forcibly linked into pythonNN.dll 2) why attempts to move them out of pythonNN.dll cause runtime crashes. so i will continue the experiment, and remove all the "data" references from the pythonNN.def that i added, and deal with the knock-on consequences, which will involve adding "get" functions. for example, PyAPI_FUNC(char*) _PyStructSequence_Get_UnnamedField(void) use of such functions will allow various bits and pieces - such as PyStructSequence_UnnamedField - to be converted back to static in their respective c files. any objections, speak now, because this will involve quite a bit of work. l. From lkcl at lkcl.net Sun Jan 25 17:15:28 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Sun, 25 Jan 2009 16:15:28 +0000 Subject: [Python-Dev] microsoft dlls apparently don't support data. implications: PyAPI functions required to access data across modules. In-Reply-To: References: Message-ID: On Sun, Jan 25, 2009 at 3:58 PM, Luke Kenneth Casson Leighton wrote: > according to the wikipedia entry on dlls, dlls do not support data, > only functions. which would explain two things: > 1) why certain modules are forcibly linked into pythonNN.dll > 2) why attempts to move them out of pythonNN.dll cause runtime crashes. > so i will continue the experiment, and remove all the "data" > references from the pythonNN.def that i added, and deal with the > knock-on consequences, which will involve adding "get" functions. > > for example, PyAPI_FUNC(char*) _PyStructSequence_Get_UnnamedField(void) > > use of such functions will allow various bits and pieces - such as > PyStructSequence_UnnamedField - to be converted back to static in > their respective c files. > > any objections, speak now, because this will involve quite a bit of work. here is a starting list of data items which will require "getter" functions, found just by creating a posixmodule.pyd: Info: resolving __Py_NoneStruct by linking to __imp___Py_NoneStruct (auto-import) Info: resolving _Py_FileSystemDefaultEncoding by linking to __imp__Py_FileSystemDefaultEncoding (auto-import) Info: resolving _PyExc_OSError by linking to __imp__PyExc_OSError (auto-import) Info: resolving _PyUnicode_Type by linking to __imp__PyUnicode_Type (auto-import) Info: resolving _PyFloat_Type by linking to __imp__PyFloat_Type (auto-import) Info: resolving _PyExc_TypeError by linking to __imp__PyExc_TypeError (auto-impoModules/posixmodule.ort) Info: resolving _PyExc_RuntimeError by linking to __imp__PyExc_RuntimeError (auto-import) Info: resolving _PyExc_ValueError by linking to __imp__PyExc_ValueError (auto-import) Info: resolving _PyExc_RuntimeWarning by linking to __imp__PyExc_RuntimeWarning (auto-import) Info: resolving _PyExc_NotImplementedError by linking to __imp__PyExc_NotImplementedError (auto-import) obviously, auto-import can't happen. so getter-functions it is. l. From lkcl at lkcl.net Sun Jan 25 17:33:54 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Sun, 25 Jan 2009 16:33:54 +0000 Subject: [Python-Dev] microsoft dlls apparently don't support data. implications: PyAPI functions required to access data across modules. In-Reply-To: References: Message-ID: > here is a starting list of data items which will require "getter" > functions, found just by creating a posixmodule.pyd: > > Info: resolving __Py_NoneStruct by linking to __imp___Py_NoneStruct > (auto-import) by no small coincidence, every single module with which we've had difficulties in the mingw port - _sre, thread, operator, locale, winreg, signal and have been forced to put them into python2N.dll - all of them _happen_ to _directly_ reference the _PyNone_Struct data variable. surpriiise. that means that the Py_None macro must call the "getter" function. l. From solipsis at pitrou.net Sun Jan 25 17:41:54 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 25 Jan 2009 16:41:54 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?microsoft_dlls_apparently_don=27t_support_?= =?utf-8?q?data=2E=09implications=3A_PyAPI_functions_required_to_ac?= =?utf-8?q?cess_data_across=09modules=2E?= References: Message-ID: Luke Kenneth Casson Leighton lkcl.net> writes: > > that means that the Py_None macro must call the "getter" function. Given the negative performance implications that it would have, chances are it is out of question. Also, while not a Windows expert *at all*, I'd question your interpretation of the problem. If data could not be found in a DLL, how could Windows builds of Python (and third-party extensions) work at all? From lkcl at lkcl.net Sun Jan 25 17:56:05 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Sun, 25 Jan 2009 16:56:05 +0000 Subject: [Python-Dev] microsoft dlls apparently don't support data. implications: PyAPI functions required to access data across modules. In-Reply-To: References: Message-ID: On Sun, Jan 25, 2009 at 4:33 PM, Luke Kenneth Casson Leighton wrote: >> Info: resolving __Py_NoneStruct by linking to __imp___Py_NoneStruct >> (auto-import) > > by no small coincidence, every single module with which we've had > difficulties in the mingw port - _sre, thread, operator, locale, > winreg, signal and have been forced to put them into python2N.dll - > all of them _happen_ to _directly_ reference the _PyNone_Struct data > variable. > > surpriiise. > > that means that the Py_None macro must call the "getter" function. btw - if anyone has any objections, think about this: how is anyone - third party or otherwise - meant to return Py_None from c code in a dynamic module (or any other type) - and expect their code to work on windows?? i mean... has anyone _written_ a third party module that returns Py_None on a c-code module and had it compiled on windows? it wouldn't surprise me in the least if this is one of the severe issues (unresolved and unexplained) that people encounter on win32. l. From matthieu.brucher at gmail.com Sun Jan 25 18:01:31 2009 From: matthieu.brucher at gmail.com (Matthieu Brucher) Date: Sun, 25 Jan 2009 18:01:31 +0100 Subject: [Python-Dev] microsoft dlls apparently don't support data. implications: PyAPI functions required to access data across modules. In-Reply-To: References: Message-ID: 2009/1/25 Luke Kenneth Casson Leighton : > according to the wikipedia entry on dlls, dlls do not support data, > only functions. What do you mean by "not support data"? Having global data variables in a dll? In wikipedia, it is explicitely told that this is possible to have data (http://en.wikipedia.org/wiki/Dynamic-link_library). Without them, shared library cannot be used. Matthieu -- Information System Engineer, Ph.D. Website: http://matthieu-brucher.developpez.com/ Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn: http://www.linkedin.com/in/matthieubrucher From curt at hagenlocher.org Sun Jan 25 18:03:31 2009 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Sun, 25 Jan 2009 09:03:31 -0800 Subject: [Python-Dev] microsoft dlls apparently don't support data. implications: PyAPI functions required to access data across modules. In-Reply-To: References: Message-ID: On Sun, Jan 25, 2009 at 9:01 AM, Matthieu Brucher wrote: > 2009/1/25 Luke Kenneth Casson Leighton : >> according to the wikipedia entry on dlls, dlls do not support data, >> only functions. > > What do you mean by "not support data"? Having global data variables in a dll? > In wikipedia, it is explicitely told that this is possible to have > data (http://en.wikipedia.org/wiki/Dynamic-link_library). Without > them, shared library cannot be used. Indeed. That's why the header files contain define PyAPI_DATA(RTYPE) extern __declspec(dllexport) RTYPE define PyAPI_DATA(RTYPE) extern __declspec(dllimport) RTYPE -- Curt Hagenlocher curt at hagenlocher.org From lists at cheimes.de Sun Jan 25 18:12:15 2009 From: lists at cheimes.de (Christian Heimes) Date: Sun, 25 Jan 2009 18:12:15 +0100 Subject: [Python-Dev] microsoft dlls apparently don't support data. implications: PyAPI functions required to access data across modules. In-Reply-To: References: Message-ID: Luke Kenneth Casson Leighton schrieb: > i mean... has anyone _written_ a third party module that returns > Py_None on a c-code module and had it compiled on windows? Lot's of people have written 3rd party extensions that work on Windows and can return a Py_None object. Please stop spamming the Python developer list with irrelevant, wrong, confusing and sometimes offensive messages. To be perfectly honest it's annoying. If you want to propose a new feature then python-ideas is the right mailing list. For everything else you should stick to the python-general or capi-sig list. Christian From lkcl at lkcl.net Sun Jan 25 18:34:42 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Sun, 25 Jan 2009 17:34:42 +0000 Subject: [Python-Dev] future-proofing vector tables for python APIs: binary-module interoperability In-Reply-To: <497C8B83.7080604@roumenpetrov.info> References: <497B6AE6.3050402@roumenpetrov.info> <497C8B83.7080604@roumenpetrov.info> Message-ID: On Sun, Jan 25, 2009 at 3:55 PM, Roumen Petrov wrote: > Luke Kenneth Casson Leighton wrote: >>>> >>>> [SNIP] >>>> Yes it is enough to encapsulate memory allocation and file functions >>>> into >>>> python shared library. The python provide memory allocation functions, >>>> but >>>> not all modules use them. File functions are hiden by posixmodule and >>>> python >>>> modules can't use them. >>> >>> except ... posixmodule gets renamed to ntmodule .... oh, i see what >>> you mean: python modules aren't allowed _direct_ access to msvcrtNN's >>> file functions, they have to go via posixmodule-renamed-to-ntmodule. >> >> .... thinking about this some more... posixmodule.c is linked (by >> default) into pythonNN.dll, thus making pythonNN.dll totally dependent >> on a version of msvcrt. > > This is not problem. If python*.dll hide msvcrt and other modules depend > directly from python*.dll I expect issue to be resolved. i.e. python*.dll to > be "portable runtime interface". yehhhh :) well - it looks like i am having success with removing all references to data e.g. Py_NoneStruct, all of the PyExc_*Warning and PyExc_*Error, all of the Py*_Types and more. i'm making sure that macros are heavily used - so that on systems where data _can_ be accessed through dynamic shared objects, it's done so. #if defined(MS_WINDOWS) || defined(__MINGW32__) /* Define macros for conveniently creating "getter" functions, * to avoid restrictions on dlls being unable to access data. * see #5056 */ /* use these for data that is already a pointer */ #define PyAPI_GETHDR(type, obj) \ PyAPI_FUNC(type) _Py_Get_##obj(void); #define PyAPI_GETIMPL(type, obj) \ PyAPI_FUNC(type) _Py_Get_##obj(void) { return (type)obj; } #define _PYGET(obj) _Py_Get_##obj() /* use these for data where a pointer (&) to the object is returned * e.g. no longer #define Py_None (&Py_NoneStruct) but * #define Py_None _PTGETPTR(Py_NoneStruct) */ #define PyAPI_GETHDRPTR(type, obj) \ PyAPI_FUNC(type) _Py_Get_##obj(void); #define PyAPI_GETIMPLPTR(type, obj) \ PyAPI_FUNC(type) _Py_Get_##obj(void) { return (type)&obj; } #define _PYGETPTR(obj) _Py_Get_##obj() #else /* on systems where data can be accessed directly in shared modules, * as an optimisation, return the object itself, directly. */ #define PyAPI_GETFNIMPL(obj) ; #define PyAPI_GETHDR(type, obj) ; #define PyAPI_GETIMPL(type, obj) ; #define PyAPI_GETIMPLPTR(type, obj) ; #define _PYGET(obj) (obj) #define _PYGETPTR(obj) (&obj) #endif /* defined(MS_WINDOWS) || defined(__MINGW32__)*/ as you can see from the Py_None example, on non-dll-based systems, you get... wow, a macro which returns... exactly what was returned before. zero impact. on windows, you end up defining, creating and using a "getter" function. two types. one which returns the object, the other returns a pointer to the object. l. From barry at python.org Sun Jan 25 18:37:29 2009 From: barry at python.org (Barry Warsaw) Date: Sun, 25 Jan 2009 12:37:29 -0500 Subject: [Python-Dev] [Python-checkins] PEP 374 In-Reply-To: References: <20090124230633.9B1801E4026@bag.python.org> <6247AFD31A7A43E7AAB7109012EFBF7C@RaymondLaptop1> <6FC2E18B-F3FD-4B3F-90C0-4EA611AF93ED@python.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 25, 2009, at 11:49 AM, Antoine Pitrou wrote: > Barry Warsaw python.org> writes: >> >> Besides, certain developments like support for the svn wire protocol >> in bzr would make the WFC (we fear change :) argument moot. > > This is an argument *against* the usefulness of switching! Why? This simply allows those who are happy with the status quo to live peacefully with those who want the benefits of the new capabilities. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSXyjW3EjvBPtnXfVAQLZHwP/UAwA/fcfaDDoaQI1Qa0F50u57AESc/GN bPIgUe6I93fwgAHx/+9jQWxgVJCjIOWlSavZqtLOr7nPR6gN4B27d4XpntLE7O47 3JXkV5QEZL0YDob0M33qAPSgPZsMv1++fWo9FDrk0o9SVzmsrP4OytsUsykRiOkC gkMtAPnzeAQ= =j2t2 -----END PGP SIGNATURE----- From solipsis at pitrou.net Sun Jan 25 18:44:05 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 25 Jan 2009 17:44:05 +0000 (UTC) Subject: [Python-Dev] [Python-checkins] PEP 374 References: <20090124230633.9B1801E4026@bag.python.org> <6247AFD31A7A43E7AAB7109012EFBF7C@RaymondLaptop1> <6FC2E18B-F3FD-4B3F-90C0-4EA611AF93ED@python.org> Message-ID: Barry Warsaw python.org> writes: > >> > >> Besides, certain developments like support for the svn wire protocol > >> in bzr would make the WFC (we fear change :) argument moot. > > > > This is an argument *against* the usefulness of switching! > > Why? > > This simply allows those who are happy with the status quo to live > peacefully with those who want the benefits of the new capabilities. Hmm, perhaps I misunderstood what you were saying... Would the bzr client allow accessing an svn server, or the reverse? (please note that we already have bzr, hg and git mirrors. So it's not like people wanting to use a DVCS are out of solutions) From barry at python.org Sun Jan 25 18:51:17 2009 From: barry at python.org (Barry Warsaw) Date: Sun, 25 Jan 2009 12:51:17 -0500 Subject: [Python-Dev] [Python-checkins] PEP 374 In-Reply-To: References: <20090124230633.9B1801E4026@bag.python.org> <6247AFD31A7A43E7AAB7109012EFBF7C@RaymondLaptop1> <6FC2E18B-F3FD-4B3F-90C0-4EA611AF93ED@python.org> Message-ID: <7EB43365-E8A5-444B-87FC-4AF26EE0B058@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 25, 2009, at 12:44 PM, Antoine Pitrou wrote: > Barry Warsaw python.org> writes: >>>> >>>> Besides, certain developments like support for the svn wire >>>> protocol >>>> in bzr would make the WFC (we fear change :) argument moot. >>> >>> This is an argument *against* the usefulness of switching! >> >> Why? >> >> This simply allows those who are happy with the status quo to live >> peacefully with those who want the benefits of the new capabilities. > > Hmm, perhaps I misunderstood what you were saying... Would the bzr > client allow > accessing an svn server, or the reverse? The reverse. IIUC, the bzr server would be able to speak to svn clients. bzr supports a centralized model perfectly fine along side a decentralized model, so any current developers who want to continue using their svn client can do so, and everyone else could use the bzr client to work decentralized. > (please note that we already have bzr, hg and git mirrors. So it's > not like > people wanting to use a DVCS are out of solutions) Right, but ideally the server would support the full distributed model and users wold chose which client and models they are most comfortable with. One option would be to promote the mirrors to official status, but this is clunky because svn has a less complete model than a dvcs so it's more difficult to go in that direction. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSXymlXEjvBPtnXfVAQJAoAP/SAqlCiEB883wLH1eXrkiMxc5MlChodqt BHBdaO2kZgs0rJCfGfoVD/ly65tuheahP5lwMsoa6don6uKD7lkzJkvBSNjtg1ZL 4U/MTIQWtg8WbDJUPaPT8ArV9Xo6/Y+B1yeFz+Ge5hY29+PGEop9pAYOXKUl/Jyk hTlhbuXQqkA= =haid -----END PGP SIGNATURE----- From martin at v.loewis.de Sun Jan 25 19:37:57 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 25 Jan 2009 19:37:57 +0100 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> References: <20090124152507.GA23994@panix.com> <497BA590.7060406@v.loewis.de> <497BB5E7.4080606@gmail.com> <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> Message-ID: <497CB185.3010601@v.loewis.de> > There's a possible third way. I've heard (though haven't investigated) > that some people are working on supporting the svn wire protocol in the > bzr server. This would mean that anybody who's still comfortable with > svn and feels no need to change their current habits can continue to > work the way they always have. Those that want the extra benefits of a > DVCS, or who do not have commit access to the code.python.org branches > would have viable alternatives. Of course, those without commit access *already* have viable alternatives, IIUC, by means of the automatic ongoing conversion of the svn repository to bzr and hg (and, IIUC, git - or perhaps you can use git-svn without the need for server-side conversion). So a conversion to a DVCS would only benefit those committers who see a benefit in using a DVCS (*) (and would put a burden on those committers who see a DVCS as a burden). It would also put a burden on contributors who are uncomfortable with using a DVCS. Regards, Martin (*) I'm probably missing something, but ISTM that committers can already use the DVCS - they only need to create a patch just before committing. This, of course, is somewhat more complicated than directly pushing the changes to the server, but it still gives them most of what is often reported as the advantage of a DVCS (local commits, ability to have many branches simultaneously, ability to share work-in-progress, etc). In essence, committers wanting to use a DVCS can do so today, by acting as if they were non-committers, and only using svn for actual changes to the master repository. From lkcl at lkcl.net Sun Jan 25 19:45:13 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Sun, 25 Jan 2009 18:45:13 +0000 Subject: [Python-Dev] microsoft dlls apparently don't support data. implications: PyAPI functions required to access data across modules. In-Reply-To: References: Message-ID: > On Sun, Jan 25, 2009 at 9:01 AM, Matthieu Brucher > wrote: > > 2009/1/25 Luke Kenneth Casson Leighton : > >> according to the wikipedia entry on dlls, dlls do not support data, > >> only functions. > > > > What do you mean by "not support data"? Having global data variables in a dll? > > In wikipedia, it is explicitely told that this is possible to have > > data (http://en.wikipedia.org/wiki/Dynamic-link_library). Without > > them, shared library cannot be used. matthieu, thank you for responding. from http://en.wikipedia.org/wiki/Dynamic-link_library: "Third, dynamic linking is inherently the wrong model for paged memory managed systems. Such systems work best with the idea that code is invariant from the time of assembly/compilation on. ........... Data references do not need to be so vectored because DLLs do not share data." ^^^^^^^^^^^^^^^^^^^^ does anyone happen to know what this means? also, what do you mean by "without data, shared library cannot be used"? you can _always_ call a function which returns a pointer to the data, rather than access the data directly. > Indeed. That's why the header files contain > define PyAPI_DATA(RTYPE) extern __declspec(dllexport) RTYPE > define PyAPI_DATA(RTYPE) extern __declspec(dllimport) RTYPE curt, thank you for responding. i'd seen this: i understood it - and... yet... mingw happily segfaults when asked to access _any_ data in _any_ object file of the python2N dll. Py_NoneStruct, PyExc_* (of which there are about 50), Py*_Type - all of them. solutions so far involve ensuring that anything declared with PyAPI_DATA is *NEVER* accessed [across a dll boundary] - by for example moving the module into python2N.dll. also, yes i had looked up how to do .def files, and how __declspec(dllexport) etc. work. of all the examples that you find about dlltool, mingw, dlls, defs, etc. they _all_ say "function." to declare a _function_ you do X, Y and Z. not one of them says "to place data in a dll, you do X Y and Z". then, looking at the wine dlls and .defs, i haven't yet found a _single_ one which returns data - they're _all_ functions (from looking so far. e.g. i expected MSVCRT.DLL errno to be an int - it's not: it's a function). *sigh*. if this turns out to be yet another gcc / mingw bug i'm going to be slightly annoyed. only slightly, because this _is_ free software, after all :) l. From solipsis at pitrou.net Sun Jan 25 19:45:21 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 25 Jan 2009 18:45:21 +0000 (UTC) Subject: [Python-Dev] PEP 374 (DVCS) now in reST References: <20090124152507.GA23994@panix.com> <497BA590.7060406@v.loewis.de> <497BB5E7.4080606@gmail.com> <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> <497CB185.3010601@v.loewis.de> Message-ID: Martin v. L?wis v.loewis.de> writes: > In > essence, committers wanting to use a DVCS can do so today, by acting > as if they were non-committers, and only using svn for actual changes > to the master repository. Indeed. It is how I work. Regards Antoine. From rlight2 at gmail.com Sun Jan 25 19:42:06 2009 From: rlight2 at gmail.com (Ross Light) Date: Sun, 25 Jan 2009 10:42:06 -0800 Subject: [Python-Dev] Issue 4285 Patch Message-ID: <4553f0901251042j12a63434g6b4c56cceb665a98@mail.gmail.com> Hello, python-dev! My name is Ross Light. I was a documentation contributor at GHOP a couple years back and I wanted to start contributing to the core interpreter. I found Issue 4285 and decided to write a patch for it. This is my first patch, so I'd like someone to review my patch and make sure I'm doing things right. http://bugs.python.org/issue4285 Thanks! Cheers, Ross Light From lkcl at lkcl.net Sun Jan 25 19:57:19 2009 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Sun, 25 Jan 2009 18:57:19 +0000 Subject: [Python-Dev] microsoft dlls apparently don't support data. implications: PyAPI functions required to access data across modules. In-Reply-To: References: Message-ID: > Luke Kenneth Casson Leighton schrieb: > > i mean... has anyone _written_ a third party module that returns > > Py_None on a c-code module and had it compiled on windows? > Lot's of people have written 3rd party extensions that work on Windows > and can return a Py_None object. ahh - ok. so... i will have to find out what the heck is going on... ohno, it couldn't be as simple as i left out "DATA" on the lines of the export files, could it?? :) > Please stop spamming the Python developer list with irrelevant, wrong, > confusing and sometimes offensive messages. To be perfectly honest it's > annoying. i'm sorry to hear that you believe my messages to be sometimes offensive. i'm sorry that you are annoyed. i'm sorry that i am learning about things and that i believe that people would like to help cooperate on the development of python as a free software project, by helping point me in the right directions. i'm sorry that i am unable to get things perfect the first time, so that i have to ask people for help and advice, and i'm sorry that you are annoyed by my asking. > If you want to propose a new feature then python-ideas is the right > mailing list. thank you for informing me of that - i was not aware of that list: i believed that the python-dev mailing list would be the location for discussion of development and ports of python. l. From rlight2 at gmail.com Sun Jan 25 20:11:55 2009 From: rlight2 at gmail.com (Ross Light) Date: Sun, 25 Jan 2009 11:11:55 -0800 Subject: [Python-Dev] Issue 4285 Patch Message-ID: <4553f0901251111w262bb595h7cc1edc1f69d8048@mail.gmail.com> Hello, python-dev! My name is Ross Light. I was a documentation contributor at GHOP a couple years back and I wanted to start contributing to the core interpreter. I found Issue 4285 and decided to write a patch for it. This is my first patch, so I'd like someone to review my patch and make sure I'm doing things right. http://bugs.python.org/issue4285 Thanks! Cheers, Ross Light From brett at python.org Sun Jan 25 22:00:04 2009 From: brett at python.org (Brett Cannon) Date: Sun, 25 Jan 2009 13:00:04 -0800 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <497CB185.3010601@v.loewis.de> References: <20090124152507.GA23994@panix.com> <497BA590.7060406@v.loewis.de> <497BB5E7.4080606@gmail.com> <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> <497CB185.3010601@v.loewis.de> Message-ID: On Sun, Jan 25, 2009 at 10:37, "Martin v. L?wis" wrote: >> There's a possible third way. I've heard (though haven't investigated) >> that some people are working on supporting the svn wire protocol in the >> bzr server. This would mean that anybody who's still comfortable with >> svn and feels no need to change their current habits can continue to >> work the way they always have. Those that want the extra benefits of a >> DVCS, or who do not have commit access to the code.python.org branches >> would have viable alternatives. > > Of course, those without commit access *already* have viable > alternatives, IIUC, by means of the automatic ongoing conversion of > the svn repository to bzr and hg (and, IIUC, git - or perhaps you > can use git-svn without the need for server-side conversion). > > So a conversion to a DVCS would only benefit those committers who > see a benefit in using a DVCS (*) (and would put a burden on those > committers who see a DVCS as a burden). It would also put a burden > on contributors who are uncomfortable with using a DVCS. > > Regards, > Martin > > (*) I'm probably missing something, but ISTM that committers can already > use the DVCS - they only need to create a patch just before committing. > This, of course, is somewhat more complicated than directly pushing the > changes to the server, but it still gives them most of what is often > reported as the advantage of a DVCS (local commits, ability to have many > branches simultaneously, ability to share work-in-progress, etc). In > essence, committers wanting to use a DVCS can do so today, by acting > as if they were non-committers, and only using svn for actual changes > to the master repository. > If I can't choose a clear winner I am going to look into what it take to run directly on top of svn to avoid the extra step for committers. Otherwise I will get standardized instructions for the three DVCSs and maybe write a script or three to make it dead-simple to work with the DVCSs but have our official repository be svn so we can all use the DVCSs as we see fit until a clear winner springs up. -Brett From fuzzyman at voidspace.org.uk Sun Jan 25 22:03:26 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 25 Jan 2009 21:03:26 +0000 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: References: <20090124152507.GA23994@panix.com> <497BA590.7060406@v.loewis.de> <497BB5E7.4080606@gmail.com> <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> <497CB185.3010601@v.loewis.de> Message-ID: <497CD39E.6010600@voidspace.org.uk> Brett Cannon wrote: > On Sun, Jan 25, 2009 at 10:37, "Martin v. L?wis" wrote: > >>> There's a possible third way. I've heard (though haven't investigated) >>> that some people are working on supporting the svn wire protocol in the >>> bzr server. This would mean that anybody who's still comfortable with >>> svn and feels no need to change their current habits can continue to >>> work the way they always have. Those that want the extra benefits of a >>> DVCS, or who do not have commit access to the code.python.org branches >>> would have viable alternatives. >>> >> Of course, those without commit access *already* have viable >> alternatives, IIUC, by means of the automatic ongoing conversion of >> the svn repository to bzr and hg (and, IIUC, git - or perhaps you >> can use git-svn without the need for server-side conversion). >> >> So a conversion to a DVCS would only benefit those committers who >> see a benefit in using a DVCS (*) (and would put a burden on those >> committers who see a DVCS as a burden). It would also put a burden >> on contributors who are uncomfortable with using a DVCS. >> >> Regards, >> Martin >> >> (*) I'm probably missing something, but ISTM that committers can already >> use the DVCS - they only need to create a patch just before committing. >> This, of course, is somewhat more complicated than directly pushing the >> changes to the server, but it still gives them most of what is often >> reported as the advantage of a DVCS (local commits, ability to have many >> branches simultaneously, ability to share work-in-progress, etc). In >> essence, committers wanting to use a DVCS can do so today, by acting >> as if they were non-committers, and only using svn for actual changes >> to the master repository. >> >> > > If I can't choose a clear winner I am going to look into what it take > to run directly on top of svn to avoid the extra step for committers. > Otherwise I will get standardized instructions for the three DVCSs and > maybe write a script or three to make it dead-simple to work with the > DVCSs but have our official repository be svn so we can all use the > DVCSs as we see fit until a clear winner springs up. > Well, that sounds like an ideal situation to end up in. Is there a downside other than the work it creates for you? Michael > -Brett > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From bugtrack at roumenpetrov.info Sun Jan 25 22:05:31 2009 From: bugtrack at roumenpetrov.info (Roumen Petrov) Date: Sun, 25 Jan 2009 23:05:31 +0200 Subject: [Python-Dev] progress: compiling python2.5 under msys (specifically but not exclusively under wine) with msvcr80 In-Reply-To: <4978FDA4.8050101@roumenpetrov.info> References: <50019.151.53.150.247.1232570935.squirrel@webmail1.pair.com> <4978FDA4.8050101@roumenpetrov.info> Message-ID: <497CD41B.4000900@roumenpetrov.info> Roumen Petrov wrote: > Cesare Di Mauro wrote: >> Have you made some benchmarks like pystone? Its seems to me version 2.6.1 is not optimized build so I remove(uninstall) it. I repeat the pystone tests with an optimized GCC(mingw32) build. - python-trunk-GCC(mingw32, local, native, optimized) -- shell=cmd.exe 35453,3; 35700,4; 35747,3; 35615,5; 35632,3; 35661,8; 35547,1 average=35622,5 deviation=98,0 -- shell=bash.exe(msys) 36002,1; 35884,4; 35961,7; 35859,5; 35997,3; 36062,9; 35747,1 average=35930,7 deviation=107,2 - python-2.6-MSVC -- shell=cmd.exe 35891,3; 35827,9; 35791,3; 35901,7; 35876,5; 36081,1; 36149,2 average=35931,3 deviation=132,7 -- shell=bash.exe(msys) 35532,9; 35621,1; 35526,8; 35639,4; 35671,2; 35702,4; 35633,0; average=35618,1 deviation=66,1 I don't have idea why performance of official python 2.6 goes down(see previous results below). It is same PC. Every tested program load own files. The result show unexpected behaviour: - the MSVC build is faster by ~0.9% if it is run under cmd.exe then msys bash; - the GCC build is faster by ~0.9% if it is run under msys bash. Otherwise results lock similar but note that builds use different source base and in this case we may can't compare. The old results: > There is result from pystone test test run an old PC (NT 5.1): > - 2.6(official build): > 42194,6; 42302,4; 41990,8; 42658,0; 42660,6; 42770,1 > average=42429,4 > deviation=311,6 > - 2.6.1(official build): > 35612,1; 35778,8; 35666,7; 35697,9; 35514,9; 35654,0 > average=35654,1 > deviation=88,1 > - trunk(my mingw based build): > 35256,7; 35272,5; 35247,2; 35270,7; 35225,6; 35233,5 > average=35251,0 > deviation=19,2 > > There is problem with python performance between 2.6 and 2.6.1 ~ 19% :(. > Also the test for GCC-mingw is not with same source base. > > Roumen Roumen From brett at python.org Sun Jan 25 22:23:20 2009 From: brett at python.org (Brett Cannon) Date: Sun, 25 Jan 2009 13:23:20 -0800 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <497CD39E.6010600@voidspace.org.uk> References: <497BA590.7060406@v.loewis.de> <497BB5E7.4080606@gmail.com> <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> <497CB185.3010601@v.loewis.de> <497CD39E.6010600@voidspace.org.uk> Message-ID: On Sun, Jan 25, 2009 at 13:03, Michael Foord wrote: > Brett Cannon wrote: >> >> On Sun, Jan 25, 2009 at 10:37, "Martin v. L?wis" >> wrote: >> >>>> >>>> There's a possible third way. I've heard (though haven't investigated) >>>> that some people are working on supporting the svn wire protocol in the >>>> bzr server. This would mean that anybody who's still comfortable with >>>> svn and feels no need to change their current habits can continue to >>>> work the way they always have. Those that want the extra benefits of a >>>> DVCS, or who do not have commit access to the code.python.org branches >>>> would have viable alternatives. >>>> >>> >>> Of course, those without commit access *already* have viable >>> alternatives, IIUC, by means of the automatic ongoing conversion of >>> the svn repository to bzr and hg (and, IIUC, git - or perhaps you >>> can use git-svn without the need for server-side conversion). >>> >>> So a conversion to a DVCS would only benefit those committers who >>> see a benefit in using a DVCS (*) (and would put a burden on those >>> committers who see a DVCS as a burden). It would also put a burden >>> on contributors who are uncomfortable with using a DVCS. >>> >>> Regards, >>> Martin >>> >>> (*) I'm probably missing something, but ISTM that committers can already >>> use the DVCS - they only need to create a patch just before committing. >>> This, of course, is somewhat more complicated than directly pushing the >>> changes to the server, but it still gives them most of what is often >>> reported as the advantage of a DVCS (local commits, ability to have many >>> branches simultaneously, ability to share work-in-progress, etc). In >>> essence, committers wanting to use a DVCS can do so today, by acting >>> as if they were non-committers, and only using svn for actual changes >>> to the master repository. >>> >>> >> >> If I can't choose a clear winner I am going to look into what it take >> to run directly on top of svn to avoid the extra step for committers. >> Otherwise I will get standardized instructions for the three DVCSs and >> maybe write a script or three to make it dead-simple to work with the >> DVCSs but have our official repository be svn so we can all use the >> DVCSs as we see fit until a clear winner springs up. >> > > Well, that sounds like an ideal situation to end up in. Is there a downside > other than the work it creates for you? What, isn't creating even more work from me enough of a downside? =) There is also the issue of support. If we as a development team start using four different VCSs then that will severely cut down on who can help whom. The only reason I have been able to keep the dev FAQ full of such key svn commands is because inevitably someone on this list knew how to do something if I didn't. Spread that across three more DVCSs and the chances of someone knowing the best solution for something dwindles. It also means three more VCSs to keep up and running on code.python.org. While it has worked so far, that's just because we have been going with what Debian stable has. If you want some new-fangled thing, e.g. bzr's 1.9 tree support which apparently makes a huge performance difference (Barry is on vacation but I am prodding him to put the details in the PEP when he gets back) someone will then need to step up that we trust to stay on top of security patches, etc. So yes, it's a nice solution if a winner cannot be chosen, but I don't think that should necessarily be the end of this quite yet. -Brett From barry at python.org Sun Jan 25 23:15:41 2009 From: barry at python.org (Barry Warsaw) Date: Sun, 25 Jan 2009 17:15:41 -0500 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <497CB185.3010601@v.loewis.de> References: <20090124152507.GA23994@panix.com> <497BA590.7060406@v.loewis.de> <497BB5E7.4080606@gmail.com> <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> <497CB185.3010601@v.loewis.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 25, 2009, at 1:37 PM, Martin v. L?wis wrote: > (*) I'm probably missing something, but ISTM that committers can > already > use the DVCS - they only need to create a patch just before > committing. > This, of course, is somewhat more complicated than directly pushing > the > changes to the server, but it still gives them most of what is often > reported as the advantage of a DVCS (local commits, ability to have > many > branches simultaneously, ability to share work-in-progress, etc). In > essence, committers wanting to use a DVCS can do so today, by acting > as if they were non-committers, and only using svn for actual changes > to the master repository. The approach you outline also has the disadvantages of losing history at the point of patch generation, and causing a discontinuity in the chain of revisions leading up to that point. Depending on the specific changes being merged, this may or may not be important. You're right that we can do this today, but I still believe there are advantages to supporting a DVCS for the official branches. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSXzkjnEjvBPtnXfVAQIUOAP/SLkPAkIqDKNpoIpbaCJTsoLwFsSKj58P ISKqF7QkMgjl+cnw4YngHHwJr+OniX4cR1Wc5S9LPB3xMgsoOtxqYWmvfG1ReJRs fbmI1iOOCmOY1MltRlPErihS3wk7+37pc4lIkEvClvZMRcoLq3JjborIQjiy0ORY pqmovGlx/AI= =wXVD -----END PGP SIGNATURE----- From guido at python.org Sun Jan 25 23:50:59 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 25 Jan 2009 14:50:59 -0800 Subject: [Python-Dev] Operator module deprecations In-Reply-To: <497BBED4.1090005@gmail.com> References: <497BBED4.1090005@gmail.com> Message-ID: +1 indeedy. On Sat, Jan 24, 2009 at 5:22 PM, Nick Coghlan wrote: > Brett Cannon wrote: >> On Sat, Jan 24, 2009 at 14:46, Raymond Hettinger wrote: >>> I would like to deprecate some outdated functions in the operator module. >>> >>> The isSequenceType(), isMappingType(), and isNumberType() >>> functions never worked reliably and now their >>> intended purpose has been largely fulfilled by >>> ABCs. >>> >>> The isCallable() function has long been deprecated >>> and I think it's finally time to rip it out. >>> >>> The repeat() function never really corresponded to an >>> operator. Instead, it reflected an underlying implementation detail (namely >>> the naming of the sq_repeat slot and the abstract C API function >>> PySequence_Repeat). That functionality is already exposed by operator.mul: >>> >>> operator.mul('abc', 3) --> 'abcabcabc' >> >> +1 to all of it. > > What Brett said. > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ptoal at takeflight.ca Sun Jan 25 23:05:41 2009 From: ptoal at takeflight.ca (Patrick Toal) Date: Sun, 25 Jan 2009 17:05:41 -0500 Subject: [Python-Dev] Changes needed for python-2.6.spec to build successfully Message-ID: Hello, I'm not subscribed to this list, but this is the best place I could think of to send this. Please feel free to ignore if this has already been addressed, or if I've approached it completely wrong. When trying to perform an rpmbuild of the python-2.6.1 tarball on my CentOS 5.1 system using the included python-2.6.spec file, I ran into a bunch of vexing problems. My solution to them is included in the diff to the specfile attached. Some of these fixes are probably not appropriate for everyone (eg: my need for shared libs, vs static). I hope this saves someone else a bit of time. :) -------------- next part -------------- A non-text attachment was scrubbed... Name: python-2.6.spec.diff Type: application/octet-stream Size: 5726 bytes Desc: not available URL: -------------- next part -------------- Cheers, Pat ---- Patrick Toal ptoal at takeflight.ca From lists at cheimes.de Mon Jan 26 00:16:35 2009 From: lists at cheimes.de (Christian Heimes) Date: Mon, 26 Jan 2009 00:16:35 +0100 Subject: [Python-Dev] Changes needed for python-2.6.spec to build successfully In-Reply-To: References: Message-ID: Patrick Toal schrieb: > Hello, > > I'm not subscribed to this list, but this is the best place I could > think of to send this. Please feel free to ignore if this has already > been addressed, or if I've approached it completely wrong. > > When trying to perform an rpmbuild of the python-2.6.1 tarball on my > CentOS 5.1 system using the included python-2.6.spec file, I ran into a > bunch of vexing problems. My solution to them is included in the diff > to the specfile attached. Some of these fixes are probably not > appropriate for everyone (eg: my need for shared libs, vs static). > > I hope this saves someone else a bit of time. :) Thanks Patrick! Please open a ticket in our tracker and attach your patch. Patches in mails tend to get lost. Christian From martin at v.loewis.de Mon Jan 26 00:27:42 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 26 Jan 2009 00:27:42 +0100 Subject: [Python-Dev] Changes needed for python-2.6.spec to build successfully In-Reply-To: References: Message-ID: <497CF56E.10802@v.loewis.de> > I hope this saves someone else a bit of time. :) Please submit the parts that you consider of general use to the bug tracker, so we can include it in future releases. Regards, Martin From python at rcn.com Mon Jan 26 00:50:19 2009 From: python at rcn.com (Raymond Hettinger) Date: Sun, 25 Jan 2009 15:50:19 -0800 Subject: [Python-Dev] Operator module deprecations References: <497BBED4.1090005@gmail.com> Message-ID: <3F6A09444036498DA949DA776D20726A@RaymondLaptop1> For Py3.0.1, can we just rip these out and skip deprecation? I don't think they will be missed at all. Raymond ----- Original Message ----- From: "Guido van Rossum" To: "Nick Coghlan" Cc: Sent: Sunday, January 25, 2009 2:50 PM Subject: Re: [Python-Dev] Operator module deprecations > +1 indeedy. > > On Sat, Jan 24, 2009 at 5:22 PM, Nick Coghlan wrote: >> Brett Cannon wrote: >>> On Sat, Jan 24, 2009 at 14:46, Raymond Hettinger wrote: >>>> I would like to deprecate some outdated functions in the operator module. >>>> >>>> The isSequenceType(), isMappingType(), and isNumberType() >>>> functions never worked reliably and now their >>>> intended purpose has been largely fulfilled by >>>> ABCs. >>>> >>>> The isCallable() function has long been deprecated >>>> and I think it's finally time to rip it out. >>>> >>>> The repeat() function never really corresponded to an >>>> operator. Instead, it reflected an underlying implementation detail (namely >>>> the naming of the sq_repeat slot and the abstract C API function >>>> PySequence_Repeat). That functionality is already exposed by operator.mul: >>>> >>>> operator.mul('abc', 3) --> 'abcabcabc' >>> >>> +1 to all of it. >> >> What Brett said. >> >> Cheers, >> Nick. >> >> -- >> Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia >> --------------------------------------------------------------- >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org >> > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python%40rcn.com From guido at python.org Mon Jan 26 00:55:24 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 25 Jan 2009 15:55:24 -0800 Subject: [Python-Dev] Operator module deprecations In-Reply-To: <3F6A09444036498DA949DA776D20726A@RaymondLaptop1> References: <497BBED4.1090005@gmail.com> <3F6A09444036498DA949DA776D20726A@RaymondLaptop1> Message-ID: Since 3.0.1 is going to do a couple of these already, I think that's fine. On Sun, Jan 25, 2009 at 3:50 PM, Raymond Hettinger wrote: > For Py3.0.1, can we just rip these out and skip deprecation? > I don't think they will be missed at all. > > Raymond > > ----- Original Message ----- From: "Guido van Rossum" > To: "Nick Coghlan" > Cc: > Sent: Sunday, January 25, 2009 2:50 PM > Subject: Re: [Python-Dev] Operator module deprecations > > >> +1 indeedy. >> >> On Sat, Jan 24, 2009 at 5:22 PM, Nick Coghlan wrote: >>> >>> Brett Cannon wrote: >>>> >>>> On Sat, Jan 24, 2009 at 14:46, Raymond Hettinger wrote: >>>>> >>>>> I would like to deprecate some outdated functions in the operator >>>>> module. >>>>> >>>>> The isSequenceType(), isMappingType(), and isNumberType() >>>>> functions never worked reliably and now their >>>>> intended purpose has been largely fulfilled by >>>>> ABCs. >>>>> >>>>> The isCallable() function has long been deprecated >>>>> and I think it's finally time to rip it out. >>>>> >>>>> The repeat() function never really corresponded to an >>>>> operator. Instead, it reflected an underlying implementation detail >>>>> (namely >>>>> the naming of the sq_repeat slot and the abstract C API function >>>>> PySequence_Repeat). That functionality is already exposed by >>>>> operator.mul: >>>>> >>>>> operator.mul('abc', 3) --> 'abcabcabc' >>>> >>>> +1 to all of it. >>> >>> What Brett said. >>> >>> Cheers, >>> Nick. >>> >>> -- >>> Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia >>> --------------------------------------------------------------- >>> _______________________________________________ >>> Python-Dev mailing list >>> Python-Dev at python.org >>> http://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: >>> http://mail.python.org/mailman/options/python-dev/guido%40python.org >>> >> >> >> >> -- >> --Guido van Rossum (home page: http://www.python.org/~guido/) >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/python%40rcn.com > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From carl at carlsensei.com Mon Jan 26 01:53:02 2009 From: carl at carlsensei.com (Carl Johnson) Date: Sun, 25 Jan 2009 14:53:02 -1000 Subject: [Python-Dev] Incorrect documentation (and possibly implementation) for rlcompleter.Completer? Message-ID: The documentation at http://docs.python.org/library/rlcompleter.html claims that > Completer.complete(text, state)? > > Return the state*th completion for *text. > > If called for text that doesn?t include a period character > ('.'), it will complete from names currently defined in __main__, > __builtin__ and keywords (as defined by the keyword module). > > If called for a dotted name, it will try to evaluate anything > without obvious side-effects (functions will not be evaluated, but > it can generate calls to __getattr__()) up to the last part, and > find matches for the rest via the dir() function. Any exception > raised during the evaluation of the expression is caught, silenced > and None is returned. In other words, it claims to use dir(obj) as part of the tab completion process. This is not true (using Python 2.6.1 on OS X): >>> class B(object): ... def __dir__(self): return dir(u"") #Makes B objects look like strings ... >>> b = B() >>> dir(b) ['__add__', '__class__', '__contains__', '__delattr__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getnewargs__', '__getslice__', '__gt__', '__hash__', '__init__', '__le__', '__len__', '__lt__', '__mod__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmod__', '__rmul__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '_formatter_field_name_split', '_formatter_parser', 'capitalize', 'center', 'count', 'decode', 'encode', 'endswith', 'expandtabs', 'find', 'format', 'index', 'isalnum', 'isalpha', 'isdecimal', 'isdigit', 'islower', 'isnumeric', 'isspace', 'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip', 'partition', 'replace', 'rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines', 'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill'] >>> c = rlcompleter.Completer() >>> c.complete("b.", 0) #Notice that it does NOT return __add__ u'b.__class__(' >>> c.matches #Notice that this list is completely different from the list given by dir(b) [u'b.__class__(', u'b.__delattr__(', u'b.__doc__', u'b.__format__(', u'b.__getattribute__(', u'b.__hash__(', u'b.__init__(', u'b.__new__(', u'b.__reduce__(', u'b.__reduce_ex__(', u'b.__repr__(', u'b.__setattr__(', u'b.__sizeof__(', u'b.__str__(', u'b.__subclasshook__(', u'b.__class__(', u'b.__class__(', u'b.__delattr__(', u'b.__dict__', u'b.__dir__(', u'b.__doc__', u'b.__format__(', u'b.__getattribute__(', u'b.__hash__(', u'b.__init__(', u'b.__module__', u'b.__new__(', u'b.__reduce__(', u'b.__reduce_ex__(', u'b.__repr__(', u'b.__setattr__(', u'b.__sizeof__(', u'b.__str__(', u'b.__subclasshook__(', u'b.__weakref__', u'b.__class__(', u'b.__delattr__(', u'b.__doc__', u'b.__format__(', u'b.__getattribute__(', u'b.__hash__(', u'b.__init__(', u'b.__new__(', u'b.__reduce__(', u'b.__reduce_ex__(', u'b.__repr__(', u'b.__setattr__(', u'b.__sizeof__(', u'b.__str__(', u'b.__subclasshook__('] As I see it, there are two ways to fix the problem: Change the documentation or change rlcompleter.Complete. I think the latter option is preferable (although it might have to wait for Python 2.7/3.1), but I thought I would ask other people if I'm missing something and if not which fix is preferred. If other people agree that it's a bug, I'll file it. -- Carl Johnson From ncoghlan at gmail.com Mon Jan 26 02:22:53 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 26 Jan 2009 11:22:53 +1000 Subject: [Python-Dev] Incorrect documentation (and possibly implementation) for rlcompleter.Completer? In-Reply-To: References: Message-ID: <497D106D.6070401@gmail.com> Carl Johnson wrote: > As I see it, there are two ways to fix the problem: Change the > documentation or change rlcompleter.Complete. I think the latter option > is preferable (although it might have to wait for Python 2.7/3.1), but I > thought I would ask other people if I'm missing something and if not > which fix is preferred. If other people agree that it's a bug, I'll file > it. It needs to go on the tracker regardless of whether the problem is in the documentation or in the implementation, so file away. Given that rlcompleter already evaluates the expression preceding the last "." when asked to perform a completion, you're probably right that actually invoking dir() on the result as the documentation claims is the way to go. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From carl at carlsensei.com Mon Jan 26 03:02:18 2009 From: carl at carlsensei.com (Carl Johnson) Date: Sun, 25 Jan 2009 16:02:18 -1000 Subject: [Python-Dev] Incorrect documentation (and possibly implementation) for rlcompleter.Completer? In-Reply-To: <497D106D.6070401@gmail.com> References: <497D106D.6070401@gmail.com> Message-ID: <35D480C0-17FB-4E05-A3F7-EF00442335A3@carlsensei.com> On 2009/01/25, at 3:22 pm, Nick Coghlan wrote: > It needs to go on the tracker regardless of whether the problem is in > the documentation or in the implementation, so file away. Issue #5062: http://bugs.python.org/issue5062 -- Carl Johnson From scott+python-dev at scottdial.com Mon Jan 26 03:08:13 2009 From: scott+python-dev at scottdial.com (Scott Dial) Date: Sun, 25 Jan 2009 21:08:13 -0500 Subject: [Python-Dev] microsoft dlls apparently don't support data. implications: PyAPI functions required to access data across modules. In-Reply-To: References: Message-ID: <497D1B0D.5090206@scottdial.com> Luke Kenneth Casson Leighton wrote: > i'm sorry to hear that you believe my messages to be sometimes > offensive. i'm sorry that you are annoyed. i'm sorry that i am > learning about things and that i believe that people would like to > help cooperate on the development of python as a free software > project, by helping point me in the right directions. i'm sorry that i > am unable to get things perfect the first time, so that i have to ask > people for help and advice, and i'm sorry that you are annoyed by my > asking. Nice job with the fake-apology-that-is-actually-an-attack maneuver there. I believe the main complaint is that you clearly have not exercised enough due diligence to find the answers yourself before asking on the list. There are a number of examples in this very thread of you replying to yourself because you just figured out something new that you didn't in the previous email. You should not consider this list an open forum for stream-of-thought style emails. Pointed, well-explored questions are the only sort that will be useful to you (in getting people to read your questions and answer them) and the community (in not having an enormous amount of low-SNR emails to sort through). These threads with obviously disprovable "facts" are not useful to anyone at-large, whether they are (dubiously) educational to you or not -- it would be more educational for you to explore your assumption and find out it's wrong all on your own. >> If you want to propose a new feature then python-ideas is the right >> mailing list. > > thank you for informing me of that - i was not aware of that list: i > believed that the python-dev mailing list would be the location for > discussion of development and ports of python. As far as I can tell, you have replied to your own threads more than anyone else on the mailing list, and you should interpret that as a general lack of interest from the developers reading this list. I think it's been made clear that nobody is opposed to having an all-free build of Python for Win32, however it is not the focus of anyone's interest here because it's "free enough" for our purposes. I believe Martin wrote you a reply that explained that quite well. -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From jared.grubb at gmail.com Mon Jan 26 04:27:09 2009 From: jared.grubb at gmail.com (Jared Grubb) Date: Sun, 25 Jan 2009 19:27:09 -0800 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <497CB185.3010601@v.loewis.de> References: <20090124152507.GA23994@panix.com> <497BA590.7060406@v.loewis.de> <497BB5E7.4080606@gmail.com> <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> <497CB185.3010601@v.loewis.de> Message-ID: <573EFD3B-2215-4747-B4B0-45C35A9F9F86@gmail.com> Regardless of the outcome, those that want to use SVN can use SVN, and those that want to use "chosen DVCS" can use that. In the end, which is the more "lossy" repository? It seems like if the change is transparent to everyone who is using it, then the only thing that we care about is that the chosen backend will preserve all the information to make it truly transparent to everyone involved. Jared On 25 Jan 2009, at 10:37, Martin v. L?wis wrote: >> There's a possible third way. I've heard (though haven't >> investigated) >> that some people are working on supporting the svn wire protocol in >> the >> bzr server. This would mean that anybody who's still comfortable >> with >> svn and feels no need to change their current habits can continue to >> work the way they always have. Those that want the extra benefits >> of a >> DVCS, or who do not have commit access to the code.python.org >> branches >> would have viable alternatives. > > Of course, those without commit access *already* have viable > alternatives, IIUC, by means of the automatic ongoing conversion of > the svn repository to bzr and hg (and, IIUC, git - or perhaps you > can use git-svn without the need for server-side conversion). > > So a conversion to a DVCS would only benefit those committers who > see a benefit in using a DVCS (*) (and would put a burden on those > committers who see a DVCS as a burden). It would also put a burden > on contributors who are uncomfortable with using a DVCS. > > Regards, > Martin > > (*) I'm probably missing something, but ISTM that committers can > already > use the DVCS - they only need to create a patch just before > committing. > This, of course, is somewhat more complicated than directly pushing > the > changes to the server, but it still gives them most of what is often > reported as the advantage of a DVCS (local commits, ability to have > many > branches simultaneously, ability to share work-in-progress, etc). In > essence, committers wanting to use a DVCS can do so today, by acting > as if they were non-committers, and only using svn for actual changes > to the master repository. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jared.grubb%40gmail.com From stephen at xemacs.org Mon Jan 26 05:38:28 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 26 Jan 2009 13:38:28 +0900 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <497CB185.3010601@v.loewis.de> References: <20090124152507.GA23994@panix.com> <497BA590.7060406@v.loewis.de> <497BB5E7.4080606@gmail.com> <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> <497CB185.3010601@v.loewis.de> Message-ID: <87eiyqfzqz.fsf@xemacs.org> "Martin v. L?wis" writes: > So a conversion to a DVCS would only benefit those committers who > see a benefit in using a DVCS (*) (and would put a burden on those > committers who see a DVCS as a burden). That's false. Especially with bzr, they would see improved log formats by default, and with git they'd have access to some very advanced and powerful history querying, even if the update-hack- commit side of the workflow doesn't change. Not having used these tools in most cases, the fact that they don't currently *perceive* a benefit from switching doesn't mean there won't be one. There's also the very high likelihood that if Python does switch to a DVCS, many such committers will start to use the distributed VCS features actively. That doesn't mean that it will fully offset the costs of the switch for them; it does mean that the net cost of the switch for them is probably a lot lower than it would appear from your description. > It would also put a burden on contributors who are uncomfortable > with using a DVCS. "Discomfort" is not something I take lightly, but I have to wonder how long that discomfort would persist. All of the DVCSes support exactly the same workflow as CVS/Subversion, with some change in naming, and possibly a need to have a trivial commit+push script to give a single command with the semantics of "svn commit". This is crystal clear if you look at the draft PEP 374, which deliberately emulates the Subversion workflow in a variety of scenarios. It's true that the svn workflow looks more efficient and/or natural in many places, but please remember that the current workflow has evolved to deal with the specific features of Subversion compared to the DVCSes, and has had years to evolve best practices, where the PEP authors have only had a few days. I suspect that (if a DVCS is adopted) both the workflow and the best practices will evolve naturally and without much additional cost of learning to users, and arrive at a similarly efficient, but more powerful, workflow, well within a year after adoption. So there might be some rough places around the edges, especially in coming up with a way to get the functionality of "svnmerge.py block", but as far as I can see, unless the project decides that it wants to adopt a decentralized workflow, the full cost of DVCS is simply learning a new VCS; the "D" has nothing to do with it. It won't be much harder than switching from CVS to Subversion. Again, I don't take the cost of learning a new tool lightly, but please let's call that cost by its name, and not bring "distributed" into it. > (*) I'm probably missing something, but ISTM that committers can already > use the DVCS - they only need to create a patch just before committing. That's true. It's also true that to have the benefit of distributed version control with Subversion, "they only need to run a Subversion server locally". In both cases, it amounts to a fair amount of extra effort, and misses out on all the benefits of improved merging, automatic propagation of history from a merged local branch to the master repo, etc., etc. > In essence, committers wanting to use a DVCS can do so today, by > acting as if they were non-committers, and only using svn for > actual changes to the master repository. That's false. Again, those people who want to use a DVCS as a DVCS will benefit from having the master repository(s) coordinate history for them. This doesn't work very well across VCS systems, essentially forcing all committers who want to use the distributed features to coordinate with each other directly, and only with those who use the same DVCS. The mental models used by git users, hg users, and bzr users differ significantly, though they probably differ more from the model that's appropriate for Subversion. Nevertheless, there is a lot of potential benefit to be gained from having a common DVCS for all developers. Whether the benefits available *today* and in the near future outweigh the costs of an early transition, I make no claims. But those benefits *are* fairly large, and much of the cost is subjective (based on the bad reputation of the DVCSes for UI awkwardness, especially that of git) and *may* (at this stage, I don't say "will") dissipate quickly with a rather small amount of experience. From stephen at xemacs.org Mon Jan 26 05:46:35 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 26 Jan 2009 13:46:35 +0900 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <497CD39E.6010600@voidspace.org.uk> References: <20090124152507.GA23994@panix.com> <497BA590.7060406@v.loewis.de> <497BB5E7.4080606@gmail.com> <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> <497CB185.3010601@v.loewis.de> <497CD39E.6010600@voidspace.org.uk> Message-ID: <87d4eafzdg.fsf@xemacs.org> Michael Foord writes: > > If I can't choose a clear winner I am going to look into what it take > > to run directly on top of svn to avoid the extra step for committers. > Well, that sounds like an ideal situation to end up in. Is there a > downside other than the work it creates for you? I'm with Brett, the extra work for him is more than downside enough. But over and above that, the various DVCSes have different strengths and weaknesses, and their proponents have different mental models of how DVCS is "supposed" to work. I believe this is reflected to a great extent in their capabilities, creating great friction to trying to work with a different VCS from the "native" one of the master repositories. You just end up using a buttload of extra CPU cycles to achieve a Subversion-based workflow. The big advantage, IMO, to going to a DVCS for the master repo is that you can start with the same workflow currently used, and gradually adapt it to the capabilities of the more powerful tools. If we don't do that, the workflow will never really change, and the project-wide advantages of the tools will be lost. From curt at hagenlocher.org Mon Jan 26 08:15:01 2009 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Sun, 25 Jan 2009 23:15:01 -0800 Subject: [Python-Dev] microsoft dlls apparently don't support data. implications: PyAPI functions required to access data across modules. In-Reply-To: References: Message-ID: On Sun, Jan 25, 2009 at 10:45 AM, Luke Kenneth Casson Leighton wrote: > > from http://en.wikipedia.org/wiki/Dynamic-link_library: > > "Third, dynamic linking is inherently the wrong model for paged memory > managed systems. Such systems work best with the idea that code is > invariant from the time of assembly/compilation on. > ........... Data references do not need to be so vectored because > DLLs do not share data." That section ("The case against DLLs") should probably be ignored. It appears to have been written by a single individual with a particular axe to grind. Much of what it contains is opinion rather than fact, and some of its facts are downright inaccurate -- at least by my recollection. I haven't thought much about any of this in well over ten years, but here's what I remember: The reason for the vectored importing of function addresses is strictly performance -- it means that you only need to fixup one location with the address of the target function instead of each location in the code. This also has obvious advantages for paging. But this may very well be a feature of the linker rather than the operating system; I imagine the loader will happily fixup the same address multiple times if you ask it to. There are differences between importing code and importing data: the code produced by the compiler for calling a function does not depend on whether or not that function is defined in the current module or in a different one -- under x86, they're both just CALL instructions. But when accessing data, addresses in the current module can be used directly while those in a different module must be indirected -- which means that different opcodes must be generated. I don't know if it's up-to-date, but the page at http://sourceware.org/binutils/docs/ld/WIN32.html suggests some ways of dealing with this for cygwin/mingw. Look for the section entitled "automatic data imports". If you have specific questions related to DLL or loader behavior under Windows, feel free to ping me off-list. I can't guarantee that I can provide an answer, but I may be able to point you in a particular direction. -- Curt Hagenlocher curt at hagenlocher.org From techtonik at gmail.com Mon Jan 26 09:44:01 2009 From: techtonik at gmail.com (anatoly techtonik) Date: Mon, 26 Jan 2009 10:44:01 +0200 Subject: [Python-Dev] subprocess crossplatformness and async communication In-Reply-To: References: Message-ID: Hi, Guido You can't launch a process and communicate with it without blocking at some point. The scenario where you can periodically check output and error pipes without blocking and send input is not supported. Scenario One: control console program. This implies reading from input and writing replies asyncronously - Popen.stdin, stdout, stderr - stdin.write(), stdout.read() or stderr.read() block script execution if any of the other OS pipe buffers fills up and blocks the child process. Advice is to use communicate(), but it waits for process to exit and hence doesn't allow to send the input based on reply from previous send. Scenario Two: waiting for process that terminates with extensive output blocks - Popen.wait() blocks if the child process generates enough output to a stdout or stderr pipe, advice is to use Popen.communicate() http://docs.python.org/library/subprocess.html#subprocess.Popen.wait - Popen.communicate() - Waits for process to terminate, reads data from stdout and stderr until end-of-file. Cashes data in memory - should not be used for if the data size is large or unlimited. Solution - read(maxsize) and write(maxsize) functions that will return immediately. -- anatoly t. On Sat, Jan 24, 2009 at 5:58 PM, Guido van Rossum wrote: > Anatoly, > > I'm confused. The subprocess already allows reading/writing its > stdin/stdout/stderr, and AFAIK it's a platform-neutral API. I'm sure > there's something missing, but your post doesn't make it clear what > exactly, and the recipe you reference is too large to digest easily. > Can you explain what it is that the current subprocess does't have > beyond saying "async communication" (which could mean many things to > many people)? > > --Guido > > On Sat, Jan 24, 2009 at 5:07 AM, anatoly techtonik wrote: >> Greetings, >> >> This turned out to be a rather long post that in short can be summarized as: >> "please-please-please, include asynchronous process communication in >> subprocess module and do not allow "available only on ..." >> functionality", because it hurts the brain". >> >> Code to speak for itself: http://code.activestate.com/recipes/440554/ >> >> >> The subprocess module was a great step forward to unify various spawn >> and system and exec and etc. calls in one module, and more importantly >> - in one uniform API. But this API is partly crossplatform, and I >> believe I've seen recent commits to docs with more unix-only >> differences in this module. >> >> The main point of this module is to "allows you to spawn new >> processes, connect to their input/output/error pipes, and obtain their >> return codes". PEP 324 goal is also to make "make Python an even >> better replacement language for over-complicated shell scripts". >> >> Citing pre-subrocess PEP 324, "Currently, Python has a large number of >> different functions for process creation. This makes it hard for >> developers to choose." Now there is one class with many methods and >> many platform-specific comments and notices. To make thing worse >> people on Unix use subprocess with fcntl and people on windows tend >> not to use it at all, because it looks complicated and doesn't solve >> the problem with asynchronous communication. >> >> That I suggest is to add either support for async crossplatfrom >> read/write/probing of executed process or a comment at the top of >> module documentation that will warn that subprocess works in blocking >> mode. With async mode you can emulate blocking, the opposite is not >> possible. This will save python users a lot of time. >> >> Thanks for reading my rant. >> >> >> BTW, the proposed change is top 10 python recipe on ActiveState >> http://code.activestate.com/recipes/langs/python/ >> >> -- >> --anatoly t. >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org >> > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -- --anatoly t. From cournape at gmail.com Mon Jan 26 09:44:15 2009 From: cournape at gmail.com (David Cournapeau) Date: Mon, 26 Jan 2009 17:44:15 +0900 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <87eiyqfzqz.fsf@xemacs.org> References: <20090124152507.GA23994@panix.com> <497BA590.7060406@v.loewis.de> <497BB5E7.4080606@gmail.com> <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> <497CB185.3010601@v.loewis.de> <87eiyqfzqz.fsf@xemacs.org> Message-ID: <5b8d13220901260044p633fd312i77bed5152059cde5@mail.gmail.com> On Mon, Jan 26, 2009 at 1:38 PM, Stephen J. Turnbull wrote: > > Again, I don't take the cost of learning a new tool lightly, but > please let's call that cost by its name, and not bring "distributed" > into it. I can only strongly agree on this point - most people asserting that DVCS are much more complicated than CVS/SVN/etc..., forget their long experience with the later. I had little experience with svn before using bzr, and I find bzr much simpler than svn in almost every way, both for personal projects and more significant open source projects. > > That's false. Again, those people who want to use a DVCS as a DVCS > will benefit from having the master repository(s) coordinate history > for them. This doesn't work very well across VCS systems, essentially > forcing all committers who want to use the distributed features to > coordinate with each other directly, and only with those who use the > same DVCS. The mental models used by git users, hg users, and bzr > users differ significantly, though they probably differ more from the > model that's appropriate for Subversion. Nevertheless, there is a lot > of potential benefit to be gained from having a common DVCS for all > developers. Agreed. A point shared by all svn-to-bzr/git/whatever in my experience is the pain of merging. In particular, although git-svn on top of svn is very useful, and brings some power of git without forcing git pain on other users, merging between branches is not really doable without going back to svn. And that's certainly a big plus of DVCS compared to svn: since svn is inherently incapable of tracking merge (at least until recently, I have no experience with 1.5), you can't use svn as a backend and benefeting from all the DVCS advantages at the same time. cheers, David From nick at craig-wood.com Mon Jan 26 13:15:30 2009 From: nick at craig-wood.com (Nick Craig-Wood) Date: Mon, 26 Jan 2009 12:15:30 +0000 Subject: [Python-Dev] subprocess crossplatformness and async communication In-Reply-To: References: Message-ID: <20090126121529.GA29551@craig-wood.com> On Sat, Jan 24, 2009 at 07:58:40AM -0800, Guido van Rossum wrote: > I'm confused. The subprocess already allows reading/writing its > stdin/stdout/stderr, and AFAIK it's a platform-neutral API. I'm sure > there's something missing, but your post doesn't make it clear what > exactly, and the recipe you reference is too large to digest easily. > Can you explain what it is that the current subprocess does't have > beyond saying "async communication" (which could mean many things to > many people)? The main problem with subprocess is that it doesn't work if you want to have a conversation / interact with your child process. subprocess works very well indeed for this case :- run child send stuff to stdin child reads stdin and writes stdout child exits read stuff from stdout But for the conversational case (eg using it to do a remote login) it doesn't work at all :- run child send stuff to stdin child reads stdin and writes stdout read stuff from stdout send stuff to stdin child reads stdin and writes stdout read stuff from stdout send stuff to stdin child reads stdin and writes stdout read stuff from stdout child exits In subprocess "read stuff from stdout" means read stdout until the other end closes it, not read what is available and return it, so it blocks on reading the first reply and never returns. Hence Anatoly's request for "async communication" and the existence of that recipe. http://code.activestate.com/recipes/440554/ I've spend quite a lot of time explaning this to people on comp.lang.python. I usually suggest either the recipe as suggested by Anatoly or if on unix the pexpect module. There are other solutions I know of, eg in twisted and wxPython. I heard rumours of a pexpect port to Windows but I don't know how far that has progressed. A cross platform async subprocess would indeed be a boon! -- Nick Craig-Wood -- http://www.craig-wood.com/nick From ncoghlan at gmail.com Mon Jan 26 13:31:56 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 26 Jan 2009 22:31:56 +1000 Subject: [Python-Dev] microsoft dlls apparently don't support data. implications: PyAPI functions required to access data across modules. In-Reply-To: <497D1B0D.5090206@scottdial.com> References: <497D1B0D.5090206@scottdial.com> Message-ID: <497DAD3C.2070904@gmail.com> Scott Dial wrote: > I think > it's been made clear that nobody is opposed to having an all-free build > of Python for Win32, however it is not the focus of anyone's interest > here because it's "free enough" for our purposes. I believe Martin wrote > you a reply that explained that quite well. One thing to keep in mind is the fact that CPython uses a BSD-style licensing model and hence will tend to attract developers that have no problem with the idea of someone making a proprietary fork of our code. One consequence of this self-selection process is that the Python core developers aren't likely to see anything inherently wrong with the idea of closed source proprietary software (it may be an inefficient and wasteful method of development when it comes to commodity software, but it isn't actually morally *wrong* in any way). Visual Studio is the best available tool for native Windows C/C++ development and these days it even comes with the free-as-in-beer Express edition. The fact that VS is itself a non-free closed source application may bother developers out there with a stronger philosophical preference for free software, but it doesn't really bother me or, I believe, most of the core committers in the slightest. I have no problem with anyone that dislikes non-free software and chooses to opt out of the Windows world altogether (I myself use my Windows machine almost solely to play games, as I prefer Linux for development and general computing tasks). But if a developer decides (for whatever reason) to opt into that world and support the platform, it doesn't make any sense to me to complain that the recommended tools for developing in a non-free environment are themselves non-free (at least in the software libre sense). Going "Oh, I may be targeting a non-free platform, but at least I used free software tools to do it" strikes me as sheer sophistry and a fairly pointless waste of time. If a developer can't even find someone to either build Windows binaries for them or else to donate the cash for a single Windows license to run Visual Studio Express in a virtual machine, then it seems to me that any supposed demand for Windows support must be pretty damn tenuous. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From hrvoje.niksic at avl.com Mon Jan 26 14:10:04 2009 From: hrvoje.niksic at avl.com (Hrvoje Niksic) Date: Mon, 26 Jan 2009 14:10:04 +0100 Subject: [Python-Dev] subprocess crossplatformness and async communication In-Reply-To: <20407628.490931.1232972271521.JavaMail.xicrypt@atgrzls001> References: <20407628.490931.1232972271521.JavaMail.xicrypt@atgrzls001> Message-ID: <497DB62C.4050108@avl.com> Nick Craig-Wood wrote: > But for the conversational case (eg using it to do a remote login) it > doesn't work at all :- > > run child > send stuff to stdin > child reads stdin and writes stdout Can this really be made safe without an explicit flow control protocol, such as a pseudo-TTY? stdio reads data from pipes such as stdin in 4K or so chunks. I can easily imagine the child blocking while it waits for its stdin buffer to fill, while the parent in turn blocks waiting for the child's output arrive. Shell pipelines (and the subprocess module as it stands) don't have this problem because they're unidirectional: you read input from one process and write output to another, but you typically don't feed data back to the process you've read it from. From daniel at stutzbachenterprises.com Mon Jan 26 16:19:01 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 26 Jan 2009 09:19:01 -0600 Subject: [Python-Dev] subprocess crossplatformness and async communication In-Reply-To: <497DB62C.4050108@avl.com> References: <20407628.490931.1232972271521.JavaMail.xicrypt@atgrzls001> <497DB62C.4050108@avl.com> Message-ID: I'm confused. What's wrong with the following? p = Popen('do_something', stdin=PIPE, stdout=PIPE) p.stdin.write('la la la\n') p.stdin.flush() line = p.stdout.readline() p.stdin.write(process(line)) p.stdin.flush() If you want to see if data is available on p.stdout, use the select module (unless you're on Windows). The child process has to flush its output buffer for this to work, but that isn't Python's problem. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul at eventuallyanyway.com Mon Jan 26 17:08:46 2009 From: paul at eventuallyanyway.com (Paul Hummer) Date: Mon, 26 Jan 2009 09:08:46 -0700 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <5b8d13220901260044p633fd312i77bed5152059cde5@mail.gmail.com> References: <20090124152507.GA23994@panix.com> <497BA590.7060406@v.loewis.de> <497BB5E7.4080606@gmail.com> <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> <497CB185.3010601@v.loewis.de> <87eiyqfzqz.fsf@xemacs.org> <5b8d13220901260044p633fd312i77bed5152059cde5@mail.gmail.com> Message-ID: <20090126090846.6ad1895c@megatron> On Mon, 26 Jan 2009 17:44:15 +0900, David Cournapeau wrote: > > > > Again, I don't take the cost of learning a new tool lightly, but > > please let's call that cost by its name, and not bring "distributed" > > into it. > > I can only strongly agree on this point - most people asserting that > DVCS are much more complicated than CVS/SVN/etc..., forget their long > experience with the later. I had little experience with svn before > using bzr, and I find bzr much simpler than svn in almost every way, > both for personal projects and more significant open source projects. > > > > > That's false. Again, those people who want to use a DVCS as a DVCS > > will benefit from having the master repository(s) coordinate history > > for them. This doesn't work very well across VCS systems, essentially > > forcing all committers who want to use the distributed features to > > coordinate with each other directly, and only with those who use the > > same DVCS. The mental models used by git users, hg users, and bzr > > users differ significantly, though they probably differ more from the > > model that's appropriate for Subversion. Nevertheless, there is a lot > > of potential benefit to be gained from having a common DVCS for all > > developers. > > Agreed. A point shared by all svn-to-bzr/git/whatever in my experience > is the pain of merging. In particular, although git-svn on top of svn > is very useful, and brings some power of git without forcing git pain > on other users, merging between branches is not really doable without > going back to svn. And that's certainly a big plus of DVCS compared to > svn: since svn is inherently incapable of tracking merge (at least > until recently, I have no experience with 1.5), you can't use svn as a > backend and benefeting from all the DVCS advantages at the same time. > At a previous employer, we had this same discussion about switching to a DVCS, and the time and cost required to learn the new tool. We switched to bzr, and while there were days where someone got lost in the DVCS, the overall advantages with merging allowed that cost to be offset by the fact that merging was so cheap (and we merged a lot). That's a big consideration to be made when you're considering a DVCS. Merges in SVN and CVS can be painful, where merging well is a core feature of any DVCS. -- Paul Hummer http://theironlion.net 1024/862FF08F C921 E962 58F8 5547 6723 0E8C 1C4D 8AC5 862F F08F From eckhardt at satorlaser.com Mon Jan 26 17:31:37 2009 From: eckhardt at satorlaser.com (Ulrich Eckhardt) Date: Mon, 26 Jan 2009 17:31:37 +0100 Subject: [Python-Dev] microsoft dlls apparently don't support data. implications: PyAPI functions required to access data across modules. In-Reply-To: References: Message-ID: <200901261731.37142.eckhardt@satorlaser.com> On Sunday 25 January 2009, Luke Kenneth Casson Leighton wrote: > matthieu, thank you for responding. from > http://en.wikipedia.org/wiki/Dynamic-link_library: > > "Third, dynamic linking is inherently the wrong model for paged memory > managed systems. Such systems work best with the idea that code is > invariant from the time of assembly/compilation on. > ........... Data references do not need to be so vectored because > DLLs do not share data." > ^^^^^^^^^^^^^^^^^^^^ > > does anyone happen to know what this means? I can only guess: The difference between code and data is that code can be loaded into a process by simply mapping it into the virtual memory. For data that is constant, the same applies. For non-const data, you absolutely must not do that though, because it would make processes interfere with each other, and that is what the above text probably means. So, the important difference is rather that read-only stuff can be memory-mapped while read-write stuff can't. Since code is read-only (barring self-modifying code and trampolines etc), it is automatically always sharable. > curt, thank you for responding. i'd seen this: i understood it - > and... yet... mingw happily segfaults when asked to access _any_ data > in _any_ object file of the python2N dll. Dump the address of said data and its size from inside that DLL and from outside just to see if they differ, both from the same process. I'd also dump the size, in case different compiler settings messed up padding or something like that. > from looking so far. e.g. i expected MSVCRT.DLL errno to be an > int - it's not: it's a function). 'errno' can't be an int, because it needs to be thread-local. Also, note the important difference between "errno is an int" and "errno yields an lvalue of type int". The latter is how the standard defines it. > *sigh*. if this turns out to be yet another gcc / mingw bug i'm going > to be slightly annoyed. only slightly, because this _is_ free > software, after all :) Can you reproduce this with a separate example? Uli -- Sator Laser GmbH Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Sator Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at ************************************************************************************** Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder anderweitig benutzt werden. E-Mails k?nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte ?nderungen enthalten. Sator Laser GmbH ist f?r diese Folgen nicht verantwortlich. ************************************************************************************** From eckhardt at satorlaser.com Mon Jan 26 19:04:17 2009 From: eckhardt at satorlaser.com (Ulrich Eckhardt) Date: Mon, 26 Jan 2009 19:04:17 +0100 Subject: [Python-Dev] FormatError() in callproc.c under win32 Message-ID: <200901261904.17978.eckhardt@satorlaser.com> Hi! In callproc.c from trunk is a function called SetException(), which calls FormatError() only to discard the contents. Can anyone enlighten me to the reasons thereof? Is it just to check if the errorcode is registered in the stringtables? The reason I ask is the CE port. The FormatMessage API exists there (or, rather, the FormatMessageW API), but the tables containing the error messages are optional for the OS and for space reasons many vendors chose not to include them. That means that the function there regularly fails to retrieve the requested string. My first approach was to fall back to simply providing a sting with a numeric representation of the errorcode, but that would change the meaning of above function, because then it could never fails. My second approach was to enhance PyErr_SetFromWindowsErr() to handle the additional error codes that are checked in SetException(). However, those require more context than just the error code, they use the EXCEPTION_RECORD passed to SetException() for that. My third approach would be to filter out the special error codes first and delegate all others to PyErr_SetFromWindowsErr(). The latter already handles the lack of a string for the code by formatting it numerically. This would also improve consistency, since the two functions use different ways to format unrecognised errors numerically. This approach would change where and how a completely unrecognised error code is formatted, but would otherwise be pretty equivalent. Suggestions? Uli -- Sator Laser GmbH Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Sator Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at ************************************************************************************** Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder anderweitig benutzt werden. E-Mails k?nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte ?nderungen enthalten. Sator Laser GmbH ist f?r diese Folgen nicht verantwortlich. ************************************************************************************** From guido at python.org Mon Jan 26 19:31:55 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 26 Jan 2009 10:31:55 -0800 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <20090126090846.6ad1895c@megatron> References: <497BA590.7060406@v.loewis.de> <497BB5E7.4080606@gmail.com> <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> <497CB185.3010601@v.loewis.de> <87eiyqfzqz.fsf@xemacs.org> <5b8d13220901260044p633fd312i77bed5152059cde5@mail.gmail.com> <20090126090846.6ad1895c@megatron> Message-ID: On Mon, Jan 26, 2009 at 8:08 AM, Paul Hummer wrote: > At a previous employer, we had this same discussion about switching to a DVCS, > and the time and cost required to learn the new tool. We switched to bzr, and > while there were days where someone got lost in the DVCS, the overall > advantages with merging allowed that cost to be offset by the fact that merging > was so cheap (and we merged a lot). > > That's a big consideration to be made when you're considering a DVCS. Merges > in SVN and CVS can be painful, where merging well is a core feature of any > DVCS. I hear you. I for one have been frustrated (now that you mention it) by the inability to track changes across merges. We do lots of merges from the trunk into the py3k branch, and the merge revisions in the branch quotes the full text of the changes merged from the trunk, but not the list of affected files for each revision merged. Since merges typically combine a lot of revisions, it is very painful to find out more about a particular change to a file when that change came from such a merge -- often even after reading through the entire list of descriptions you still have no idea which merged revision is responsible for a particular change. Assuming this problem does not exist in DVCS, that would be a huge bonus from switching to a DVCS! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Mon Jan 26 19:32:02 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 26 Jan 2009 19:32:02 +0100 Subject: [Python-Dev] IDLE docs at python.org/idle Message-ID: (re #5066) Is that documentation maintained in some way? Shouldn't it be merged into the main docs? Georg From guido at python.org Mon Jan 26 19:36:42 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 26 Jan 2009 10:36:42 -0800 Subject: [Python-Dev] subprocess crossplatformness and async communication In-Reply-To: References: Message-ID: On Mon, Jan 26, 2009 at 12:44 AM, anatoly techtonik wrote: > You can't launch a process and communicate with it without blocking at > some point. The scenario where you can periodically check output and > error pipes without blocking and send input is not supported. > > Scenario One: control console program. This implies reading from input > and writing replies asyncronously > - Popen.stdin, stdout, stderr - stdin.write(), stdout.read() or > stderr.read() block script execution if any of the other OS pipe > buffers fills up and blocks the child process. Advice is to use > communicate(), but it waits for process to exit and hence doesn't > allow to send the input based on reply from previous send. > > Scenario Two: waiting for process that terminates with extensive output blocks > - Popen.wait() blocks if the child process generates enough output to > a stdout or stderr pipe, advice is to use Popen.communicate() > http://docs.python.org/library/subprocess.html#subprocess.Popen.wait > - Popen.communicate() - Waits for process to terminate, reads data > from stdout and stderr until end-of-file. Cashes data in memory - > should not be used for if the data size is large or unlimited. > > Solution - read(maxsize) and write(maxsize) functions that will return > immediately. Hi Anatoly -- thanks for clarifying your issues. I hope other developers more familiar with subprocess.py will chime in and either help you figure out a way to do this without changes to the subprocess module, or, if that would be too painful, help you develop additional APIs. You could start by proposing a set of changes to subprocess.py and submit them as a patch to bugs.python.org; that is easier to deal with than pointing to a recipe on the Activestate site. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Mon Jan 26 20:56:06 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 26 Jan 2009 14:56:06 -0500 Subject: [Python-Dev] IDLE docs at python.org/idle In-Reply-To: References: Message-ID: Georg Brandl wrote: > re http://bugs.python.org/issue5066 > Is that documentation maintained in some way? Not currently, pretty obviously. Screenshots are 1.5.2. Windows was 95/98. > Shouldn't it be merged into the main docs? If and only if updated. As noted on the issue, I am willing to help. Terry Jan Reedy From theller at ctypes.org Mon Jan 26 21:12:18 2009 From: theller at ctypes.org (Thomas Heller) Date: Mon, 26 Jan 2009 21:12:18 +0100 Subject: [Python-Dev] FormatError() in callproc.c under win32 In-Reply-To: <200901261904.17978.eckhardt@satorlaser.com> References: <200901261904.17978.eckhardt@satorlaser.com> Message-ID: Ulrich Eckhardt schrieb: > Hi! > > In callproc.c from trunk is a function called SetException(), which calls > FormatError() only to discard the contents. Can anyone enlighten me to the > reasons thereof? Is it just to check if the errorcode is registered in the > stringtables? I think that your guess is somewhat similar to what I thought when I wrote the code. > > The reason I ask is the CE port. The FormatMessage API exists there (or, > rather, the FormatMessageW API), but the tables containing the error messages > are optional for the OS and for space reasons many vendors chose not to > include them. That means that the function there regularly fails to retrieve > the requested string. > > My first approach was to fall back to simply providing a sting with a numeric > representation of the errorcode, but that would change the meaning of above > function, because then it could never fails. > > My second approach was to enhance PyErr_SetFromWindowsErr() to handle the > additional error codes that are checked in SetException(). However, those > require more context than just the error code, they use the EXCEPTION_RECORD > passed to SetException() for that. > > My third approach would be to filter out the special error codes first and > delegate all others to PyErr_SetFromWindowsErr(). The latter already handles > the lack of a string for the code by formatting it numerically. This would > also improve consistency, since the two functions use different ways to > format unrecognised errors numerically. This approach would change where and > how a completely unrecognised error code is formatted, but would otherwise be > pretty equivalent. The third approach is fine with me. Sidenote: The only error codes that I remember having seen in practice are 'access violation reading...' and 'access violation writing...', although it may be that on WinCE 'datatype misalignment' may also be possible. -- Thanks, Thomas From solipsis at pitrou.net Mon Jan 26 22:54:14 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 26 Jan 2009 21:54:14 +0000 (UTC) Subject: [Python-Dev] enabling a configure option Message-ID: Hello python-dev, r68924 in py3k introduced a new configure option named --with-computed-gotos. It would be nice if one of the buildbots could exercise this option, so that the code doesn't rot (the buildbot has to use gcc). Whom should I ask for this? Speaking of which, there are only five buildbots remaining in the "stable" bunch... What has happened to the others? Regards Antoine. From rasky at develer.com Mon Jan 26 22:57:36 2009 From: rasky at develer.com (Giovanni Bajo) Date: Mon, 26 Jan 2009 21:57:36 +0000 (UTC) Subject: [Python-Dev] PEP 374 (DVCS) now in reST References: <497BA590.7060406@v.loewis.de> <497BB5E7.4080606@gmail.com> <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> <497CB185.3010601@v.loewis.de> <87eiyqfzqz.fsf@xemacs.org> <5b8d13220901260044p633fd312i77bed5152059cde5@mail.gmail.com> <20090126090846.6ad1895c@megatron> Message-ID: On Mon, 26 Jan 2009 10:31:55 -0800, Guido van Rossum wrote: > On Mon, Jan 26, 2009 at 8:08 AM, Paul Hummer > wrote: >> At a previous employer, we had this same discussion about switching to >> a DVCS, and the time and cost required to learn the new tool. We >> switched to bzr, and while there were days where someone got lost in >> the DVCS, the overall advantages with merging allowed that cost to be >> offset by the fact that merging was so cheap (and we merged a lot). >> >> That's a big consideration to be made when you're considering a DVCS. >> Merges in SVN and CVS can be painful, where merging well is a core >> feature of any DVCS. > > I hear you. I for one have been frustrated (now that you mention it) by > the inability to track changes across merges. We do lots of merges from > the trunk into the py3k branch, and the merge revisions in the branch > quotes the full text of the changes merged from the trunk, but not the > list of affected files for each revision merged. Since merges typically > combine a lot of revisions, it is very painful to find out more about a > particular change to a file when that change came from such a merge -- > often even after reading through the entire list of descriptions you > still have no idea which merged revision is responsible for a particular > change. Assuming this problem does not exist in DVCS, that would be a > huge bonus from switching to a DVCS! Well, not only it does not exist by design in any DVCS, but I have a better news: it does not exist anymore in Subversion 1.5. You just need to upgrade your SVN server to 1.5, migrate your merge history from the format of svnmerge to the new builtin format (using the official script), and you're done: say hello to "-g/--use-merge-history", to be use with svn log and svn blame. This is a good writeup of the new features: http://chestofbooks.com/computers/revision-control/subversion-svn/Merge- Sensitive-Logs-And-Annotations-Branchmerge-Advanced-Lo.html -- Giovanni Bajo Develer S.r.l. http://www.develer.com From solipsis at pitrou.net Mon Jan 26 22:59:29 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 26 Jan 2009 21:59:29 +0000 (UTC) Subject: [Python-Dev] enabling a configure option on a buildbot References: Message-ID: (Apologies for the incomplete title! I sometimes eat my words...) Antoine Pitrou pitrou.net> writes: > > Hello python-dev, > [snip] From guido at python.org Mon Jan 26 23:00:19 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 26 Jan 2009 14:00:19 -0800 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: References: <497BB5E7.4080606@gmail.com> <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> <497CB185.3010601@v.loewis.de> <87eiyqfzqz.fsf@xemacs.org> <5b8d13220901260044p633fd312i77bed5152059cde5@mail.gmail.com> <20090126090846.6ad1895c@megatron> Message-ID: On Mon, Jan 26, 2009 at 1:57 PM, Giovanni Bajo wrote: > On Mon, 26 Jan 2009 10:31:55 -0800, Guido van Rossum wrote: > >> On Mon, Jan 26, 2009 at 8:08 AM, Paul Hummer >> wrote: >>> At a previous employer, we had this same discussion about switching to >>> a DVCS, and the time and cost required to learn the new tool. We >>> switched to bzr, and while there were days where someone got lost in >>> the DVCS, the overall advantages with merging allowed that cost to be >>> offset by the fact that merging was so cheap (and we merged a lot). >>> >>> That's a big consideration to be made when you're considering a DVCS. >>> Merges in SVN and CVS can be painful, where merging well is a core >>> feature of any DVCS. >> >> I hear you. I for one have been frustrated (now that you mention it) by >> the inability to track changes across merges. We do lots of merges from >> the trunk into the py3k branch, and the merge revisions in the branch >> quotes the full text of the changes merged from the trunk, but not the >> list of affected files for each revision merged. Since merges typically >> combine a lot of revisions, it is very painful to find out more about a >> particular change to a file when that change came from such a merge -- >> often even after reading through the entire list of descriptions you >> still have no idea which merged revision is responsible for a particular >> change. Assuming this problem does not exist in DVCS, that would be a >> huge bonus from switching to a DVCS! > > Well, not only it does not exist by design in any DVCS, but I have a > better news: it does not exist anymore in Subversion 1.5. You just need > to upgrade your SVN server to 1.5, migrate your merge history from the > format of svnmerge to the new builtin format (using the official script), > and you're done: say hello to "-g/--use-merge-history", to be use with > svn log and svn blame. > > This is a good writeup of the new features: > http://chestofbooks.com/computers/revision-control/subversion-svn/Merge- > Sensitive-Logs-And-Annotations-Branchmerge-Advanced-Lo.html Unfortunately I've heard we shouldn't upgrade to svn 1.5 until more Linux distributions ship with it by default. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Mon Jan 26 23:12:22 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 26 Jan 2009 23:12:22 +0100 Subject: [Python-Dev] FormatError() in callproc.c under win32 In-Reply-To: <200901261904.17978.eckhardt@satorlaser.com> References: <200901261904.17978.eckhardt@satorlaser.com> Message-ID: <497E3546.8000408@v.loewis.de> > In callproc.c from trunk is a function called SetException(), which calls > FormatError() only to discard the contents. Can anyone enlighten me to the > reasons thereof? Interestingly enough, the code used to say PyErr_SetString(PyExc_WindowsError, lpMsgBuf); Then it was changed to its current form, with a log message of Changes for windows CE, contributed by Luke Dunstan. Thanks a lot! See http://ctypes.cvs.sourceforge.net/viewvc/ctypes/ctypes/source/callproc.c?hideattic=0&r1=1.127.2.15&r2=1.127.2.16 I suggest you ask Thomas Heller and Luke Dunstan (if available) what the rationale for this partial change was. Regards, Martin From martin at v.loewis.de Mon Jan 26 23:15:22 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 26 Jan 2009 23:15:22 +0100 Subject: [Python-Dev] IDLE docs at python.org/idle In-Reply-To: References: Message-ID: <497E35FA.9050706@v.loewis.de> > Is that documentation maintained in some way? I don't think so. It isn't in the pydotorg repository, and the files were last touched in 2005. Regards, Martin From jyasskin at gmail.com Mon Jan 26 23:18:15 2009 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Mon, 26 Jan 2009 14:18:15 -0800 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: References: <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> <497CB185.3010601@v.loewis.de> <87eiyqfzqz.fsf@xemacs.org> <5b8d13220901260044p633fd312i77bed5152059cde5@mail.gmail.com> <20090126090846.6ad1895c@megatron> Message-ID: <5d44f72f0901261418y8de7bc0o3165e4bf28e01c16@mail.gmail.com> On Mon, Jan 26, 2009 at 2:00 PM, Guido van Rossum wrote: > On Mon, Jan 26, 2009 at 1:57 PM, Giovanni Bajo wrote: >> On Mon, 26 Jan 2009 10:31:55 -0800, Guido van Rossum wrote: >> >>> On Mon, Jan 26, 2009 at 8:08 AM, Paul Hummer >>> wrote: >>>> At a previous employer, we had this same discussion about switching to >>>> a DVCS, and the time and cost required to learn the new tool. We >>>> switched to bzr, and while there were days where someone got lost in >>>> the DVCS, the overall advantages with merging allowed that cost to be >>>> offset by the fact that merging was so cheap (and we merged a lot). >>>> >>>> That's a big consideration to be made when you're considering a DVCS. >>>> Merges in SVN and CVS can be painful, where merging well is a core >>>> feature of any DVCS. >>> >>> I hear you. I for one have been frustrated (now that you mention it) by >>> the inability to track changes across merges. We do lots of merges from >>> the trunk into the py3k branch, and the merge revisions in the branch >>> quotes the full text of the changes merged from the trunk, but not the >>> list of affected files for each revision merged. Since merges typically >>> combine a lot of revisions, it is very painful to find out more about a >>> particular change to a file when that change came from such a merge -- >>> often even after reading through the entire list of descriptions you >>> still have no idea which merged revision is responsible for a particular >>> change. Assuming this problem does not exist in DVCS, that would be a >>> huge bonus from switching to a DVCS! >> >> Well, not only it does not exist by design in any DVCS, but I have a >> better news: it does not exist anymore in Subversion 1.5. You just need >> to upgrade your SVN server to 1.5, migrate your merge history from the >> format of svnmerge to the new builtin format (using the official script), >> and you're done: say hello to "-g/--use-merge-history", to be use with >> svn log and svn blame. >> >> This is a good writeup of the new features: >> http://chestofbooks.com/computers/revision-control/subversion-svn/Merge- >> Sensitive-Logs-And-Annotations-Branchmerge-Advanced-Lo.html > > Unfortunately I've heard we shouldn't upgrade to svn 1.5 until more > Linux distributions ship with it by default. Besides that, `svn merge` cannot handle parallel branches like trunk/py3k without lots of handholding. Unlike svnmerge.py, when you merge to and then from a branch, it tries to merge changes that came from trunk and produces lots of conflicts. (Before you point me at --reintegrate, note "In Subversion 1.5, once a --reintegrate merge is done from branch to trunk, the branch is no longer usable for further work." from the book.) In principle, the svn devs could fix this, but they didn't in svn 1.5. To keep this slighly on topic ... maybe the abilities and limits of svnmerge.py and `svn merge` should be mentioned in the PEP? From martin at v.loewis.de Mon Jan 26 23:21:54 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 26 Jan 2009 23:21:54 +0100 Subject: [Python-Dev] enabling a configure option In-Reply-To: References: Message-ID: <497E3782.5080307@v.loewis.de> Antoine Pitrou wrote: > Hello python-dev, > > r68924 in py3k introduced a new configure option named --with-computed-gotos. It > would be nice if one of the buildbots could exercise this option, so that the > code doesn't rot (the buildbot has to use gcc). Whom should I ask for this? Me. Does it have to be a configure option? It is difficult to invoke different commands in different branches; better if the configures in all branches get the same options. Of course, the configure command doesn't have to be "configure"; any other script available in all branches would work (there is already Tools/buildbot for such scripts). > Speaking of which, there are only five buildbots remaining in the "stable" > bunch... What has happened to the others? I've removed all slaves that were down and where the owners either didn't respond, or indicated that they can't bring the slaves up anytime soon. Regards, Martin From jnoller at gmail.com Mon Jan 26 23:22:07 2009 From: jnoller at gmail.com (Jesse Noller) Date: Mon, 26 Jan 2009 17:22:07 -0500 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <5d44f72f0901261418y8de7bc0o3165e4bf28e01c16@mail.gmail.com> References: <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> <497CB185.3010601@v.loewis.de> <87eiyqfzqz.fsf@xemacs.org> <5b8d13220901260044p633fd312i77bed5152059cde5@mail.gmail.com> <20090126090846.6ad1895c@megatron> <5d44f72f0901261418y8de7bc0o3165e4bf28e01c16@mail.gmail.com> Message-ID: <6C0E7BE0-3F18-423F-890C-BDB29B48E64D@gmail.com> On Jan 26, 2009, at 5:18 PM, Jeffrey Yasskin wrote: > On Mon, Jan 26, 2009 at 2:00 PM, Guido van Rossum > wrote: >> On Mon, Jan 26, 2009 at 1:57 PM, Giovanni Bajo >> wrote: >>> On Mon, 26 Jan 2009 10:31:55 -0800, Guido van Rossum wrote: >>> >>>> On Mon, Jan 26, 2009 at 8:08 AM, Paul Hummer >>> > >>>> wrote: >>>>> At a previous employer, we had this same discussion about >>>>> switching to >>>>> a DVCS, and the time and cost required to learn the new tool. We >>>>> switched to bzr, and while there were days where someone got >>>>> lost in >>>>> the DVCS, the overall advantages with merging allowed that cost >>>>> to be >>>>> offset by the fact that merging was so cheap (and we merged a >>>>> lot). >>>>> >>>>> That's a big consideration to be made when you're considering a >>>>> DVCS. >>>>> Merges in SVN and CVS can be painful, where merging well is a core >>>>> feature of any DVCS. >>>> >>>> I hear you. I for one have been frustrated (now that you mention >>>> it) by >>>> the inability to track changes across merges. We do lots of >>>> merges from >>>> the trunk into the py3k branch, and the merge revisions in the >>>> branch >>>> quotes the full text of the changes merged from the trunk, but >>>> not the >>>> list of affected files for each revision merged. Since merges >>>> typically >>>> combine a lot of revisions, it is very painful to find out more >>>> about a >>>> particular change to a file when that change came from such a >>>> merge -- >>>> often even after reading through the entire list of descriptions >>>> you >>>> still have no idea which merged revision is responsible for a >>>> particular >>>> change. Assuming this problem does not exist in DVCS, that would >>>> be a >>>> huge bonus from switching to a DVCS! >>> >>> Well, not only it does not exist by design in any DVCS, but I have a >>> better news: it does not exist anymore in Subversion 1.5. You just >>> need >>> to upgrade your SVN server to 1.5, migrate your merge history from >>> the >>> format of svnmerge to the new builtin format (using the official >>> script), >>> and you're done: say hello to "-g/--use-merge-history", to be use >>> with >>> svn log and svn blame. >>> >>> This is a good writeup of the new features: >>> http://chestofbooks.com/computers/revision-control/subversion-svn/Merge- >>> Sensitive-Logs-And-Annotations-Branchmerge-Advanced-Lo.html >> >> Unfortunately I've heard we shouldn't upgrade to svn 1.5 until more >> Linux distributions ship with it by default. > > Besides that, `svn merge` cannot handle parallel branches like > trunk/py3k without lots of handholding. Unlike svnmerge.py, when you > merge to and then from a branch, it tries to merge changes that came > from trunk and produces lots of conflicts. (Before you point me at > --reintegrate, note "In Subversion 1.5, once a --reintegrate merge is > done from branch to trunk, the branch is no longer usable for further > work." from the book.) In principle, the svn devs could fix this, but > they didn't in svn 1.5. > > To keep this slighly on topic ... maybe the abilities and limits of > svnmerge.py and `svn merge` should be mentioned in the PEP? Everytime I merge with subversion, it makes me appreciate perforce's branching and merging that much more. Jesse From martin at v.loewis.de Mon Jan 26 23:24:42 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 26 Jan 2009 23:24:42 +0100 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: References: <497BB5E7.4080606@gmail.com> <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> <497CB185.3010601@v.loewis.de> <87eiyqfzqz.fsf@xemacs.org> <5b8d13220901260044p633fd312i77bed5152059cde5@mail.gmail.com> <20090126090846.6ad1895c@megatron> Message-ID: <497E382A.80809@v.loewis.de> > Unfortunately I've heard we shouldn't upgrade to svn 1.5 until more > Linux distributions ship with it by default. We *could* upgrade to subversion 1.5 on the server (if only Debian would get their ... together and release the version they promised for last September). The question is then whether we would drop svnmerge, in favour of the 1.5 merge tracking. IIUC, that would require all committers to use 1.5 - I'm not sure whether this poses a challenge to any committer. Regards, Martin From solipsis at pitrou.net Mon Jan 26 23:31:12 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 26 Jan 2009 22:31:12 +0000 (UTC) Subject: [Python-Dev] enabling a configure option References: <497E3782.5080307@v.loewis.de> Message-ID: Martin v. L?wis v.loewis.de> writes: > > Me. Does it have to be a configure option? It is difficult to invoke > different commands in different branches; better if the configures in > all branches get the same options. Well, after a quick test, it seems that configure doesn't complain if you pass it an unknown option (at least one that begins with '--with'). So we can still use the same options on all branches. (as for the need for it to be a configure option, it was the consensus which emerged after discussion in the tracker entry, both to provide some flexibility and for fear that enabling it by default could trigger some compiler bugs -- although the latter is of course unlikely) Regards Antoine. From ncoghlan at gmail.com Mon Jan 26 23:43:24 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 27 Jan 2009 08:43:24 +1000 Subject: [Python-Dev] subprocess crossplatformness and async communication In-Reply-To: References: <20407628.490931.1232972271521.JavaMail.xicrypt@atgrzls001> <497DB62C.4050108@avl.com> Message-ID: <497E3C8C.2030303@gmail.com> Daniel Stutzbach wrote: > I'm confused. What's wrong with the following? > > p = Popen('do_something', stdin=PIPE, stdout=PIPE) > p.stdin.write('la la la\n') > p.stdin.flush() > line = p.stdout.readline() > p.stdin.write(process(line)) > p.stdin.flush() > > If you want to see if data is available on p.stdout, use the select > module (unless you're on Windows). > > The child process has to flush its output buffer for this to work, but > that isn't Python's problem. Anatoly covered that in his response to Guido: "You can't launch a process and communicate with it without blocking at some point. The scenario where you can periodically check output and error pipes without blocking and send input is not supported." With the vanilla subprocess Popen implmentation, the stdin.write calls can actually both block if the stdin buffer is full (waiting for the child process to clear space) and the stdout.readline call can definitely block (waiting for the child process to end the line). So it isn't async communication in general that is the concern: it is *non-blocking* async communication. (Since I'm happy with the idea of threaded programs, I'd personally just farm the task off to some worker threads and make it non-blocking that way, but that approach effectively abandons the whole concept of non-blocking IO and it's ability to avoid using threads). As Guido said though, while there doesn't seem to be anything fundamentally wrong with the idea of adding better support for non-blocking IO to subprocess, it's difficult to have an opinion without a concrete proposal for API changes to subprocess.Popen. The linked recipe certainly can't be adopted as is (e.g. due to the dependency on pywin32) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From daniel at stutzbachenterprises.com Tue Jan 27 00:36:37 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 26 Jan 2009 17:36:37 -0600 Subject: [Python-Dev] subprocess crossplatformness and async communication In-Reply-To: <497E3C8C.2030303@gmail.com> References: <20407628.490931.1232972271521.JavaMail.xicrypt@atgrzls001> <497DB62C.4050108@avl.com> <497E3C8C.2030303@gmail.com> Message-ID: On Mon, Jan 26, 2009 at 4:43 PM, Nick Coghlan wrote: > With the vanilla subprocess Popen implmentation, the stdin.write calls > can actually both block if the stdin buffer is full (waiting for the > child process to clear space) and the stdout.readline call can > definitely block (waiting for the child process to end the line). That's true, but for the use cases presented so far, .readline() is satisfactory, unless you have an unusual application that will fill the pipe without sending a single newline (in which case, see below). > So it isn't async communication in general that is the concern: it is > *non-blocking* async communication. (Since I'm happy with the idea of > threaded programs, I'd personally just farm the task off to some worker > threads and make it non-blocking that way, but that approach effectively > abandons the whole concept of non-blocking IO and it's ability to avoid > using threads). If you really need to communicate with multiple subprocesses (which so far has not been suggested as a motivating example), then you can use select(). You don't need non-blocking IO to avoid using threads (although that is a common misconception). If a program never blocks, then it uses 100% of CPU by definition, which is undesirable. ;-) A program just needs select() so it knows which file descriptors it can call os.read() or os.write() on without blocking. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew-pythondev at puzzling.org Tue Jan 27 00:45:02 2009 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Tue, 27 Jan 2009 10:45:02 +1100 Subject: [Python-Dev] subprocess crossplatformness and async communication In-Reply-To: References: <20407628.490931.1232972271521.JavaMail.xicrypt@atgrzls001> <497DB62C.4050108@avl.com> <497E3C8C.2030303@gmail.com> Message-ID: <20090126234502.GA8567@steerpike.home.puzzling.org> Daniel Stutzbach wrote: [...] > > If you really need to communicate with multiple subprocesses (which so far has > not been suggested as a motivating example), then you can use select(). Not portably. select() on windows only works on sockets. -Andrew. From ncoghlan at gmail.com Tue Jan 27 00:58:42 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 27 Jan 2009 09:58:42 +1000 Subject: [Python-Dev] subprocess crossplatformness and async communication In-Reply-To: <20090126234502.GA8567@steerpike.home.puzzling.org> References: <20407628.490931.1232972271521.JavaMail.xicrypt@atgrzls001> <497DB62C.4050108@avl.com> <497E3C8C.2030303@gmail.com> <20090126234502.GA8567@steerpike.home.puzzling.org> Message-ID: <497E4E32.7020200@gmail.com> Andrew Bennetts wrote: > Daniel Stutzbach wrote: > [...] >> If you really need to communicate with multiple subprocesses (which so far has >> not been suggested as a motivating example), then you can use select(). > > Not portably. select() on windows only works on sockets. In addition, select() is exactly what the linked recipe uses to implement non-blocking I/O for subprocesses on non-Windows platforms. I agree the actual use cases need to be better articulated in any RFE that is actually posted, but a cleanly encapsulated approach to non-blocking communication with subprocesses over stdin/out/err certainly sounds like something that could reasonably be added to the subprocess module. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From loewis at informatik.hu-berlin.de Mon Jan 26 22:55:29 2009 From: loewis at informatik.hu-berlin.de (Martin v. Loewis) Date: Mon, 26 Jan 2009 22:55:29 +0100 Subject: [Python-Dev] subprocess crossplatformness and async communication In-Reply-To: <497DB62C.4050108@avl.com> References: <20407628.490931.1232972271521.JavaMail.xicrypt@atgrzls001> <497DB62C.4050108@avl.com> Message-ID: <497E3151.9090703@informatik.hu-berlin.de> > Can this really be made safe without an explicit flow control protocol, > such as a pseudo-TTY? stdio reads data from pipes such as stdin in 4K > or so chunks. I don't think the subprocess module uses stdio. > I can easily imagine the child blocking while it waits > for its stdin buffer to fill, while the parent in turn blocks waiting > for the child's output arrive. That would be a bug in the parent process, for expecting output when none will arrive. As a consequence, some child programs might not be suitable for this kind of operation. This is no surprise - some programs are not suitable for automated operation at all, because they bring up windows, and communicate with their environment by means other than stdin and stdout (or, if you want to operate them automatically, you have to use AppleScript, or COM). Regards, Martin From barry at python.org Tue Jan 27 03:20:22 2009 From: barry at python.org (Barry Warsaw) Date: Mon, 26 Jan 2009 21:20:22 -0500 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: <573EFD3B-2215-4747-B4B0-45C35A9F9F86@gmail.com> References: <20090124152507.GA23994@panix.com> <497BA590.7060406@v.loewis.de> <497BB5E7.4080606@gmail.com> <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> <497CB185.3010601@v.loewis.de> <573EFD3B-2215-4747-B4B0-45C35A9F9F86@gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 25, 2009, at 10:27 PM, Jared Grubb wrote: > Regardless of the outcome, those that want to use SVN can use SVN, > and those that want to use "chosen DVCS" can use that. In the end, > which is the more "lossy" repository? It seems like if the change is > transparent to everyone who is using it, then the only thing that we > care about is that the chosen backend will preserve all the > information to make it truly transparent to everyone involved. svn is the more lossy repository format. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSX5vZnEjvBPtnXfVAQJzcgP/SweUwXoECPJpO5BEkmdTLDoEfPP1X1Lg m4AALSFZ3cfRUPX3UgGmT7anY604o5oaElFR8b0HkIScJvhF56nzs9oAR0Yqi8jN zThG1rizDHh+RSqUZ0yXKHVF6ScNf8aRg/cLoVtV+J6KGpYtTCSGTQWGnvSQxCj9 I+BY75DHOI8= =9A3a -----END PGP SIGNATURE----- From nnorwitz at gmail.com Tue Jan 27 06:17:58 2009 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 26 Jan 2009 21:17:58 -0800 Subject: [Python-Dev] enabling a configure option In-Reply-To: References: <497E3782.5080307@v.loewis.de> Message-ID: If you only care about this running on a single machine to get some coverage and don't care about all architectures, you can change Misc/build.sh to add the configure option. n On Mon, Jan 26, 2009 at 2:31 PM, Antoine Pitrou wrote: > Martin v. L?wis v.loewis.de> writes: >> >> Me. Does it have to be a configure option? It is difficult to invoke >> different commands in different branches; better if the configures in >> all branches get the same options. > > Well, after a quick test, it seems that configure doesn't complain if you pass > it an unknown option (at least one that begins with '--with'). So we can still > use the same options on all branches. > > (as for the need for it to be a configure option, it was the consensus which > emerged after discussion in the tracker entry, both to provide some flexibility > and for fear that enabling it by default could trigger some compiler bugs -- > although the latter is of course unlikely) > > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > From eckhardt at satorlaser.com Tue Jan 27 11:36:40 2009 From: eckhardt at satorlaser.com (Ulrich Eckhardt) Date: Tue, 27 Jan 2009 11:36:40 +0100 Subject: [Python-Dev] FormatError() in callproc.c under win32 In-Reply-To: References: <200901261904.17978.eckhardt@satorlaser.com> Message-ID: <200901271136.40940.eckhardt@satorlaser.com> On Monday 26 January 2009, Thomas Heller wrote: > Ulrich Eckhardt schrieb: > > In callproc.c from trunk is a function called SetException(), which calls [...] > > My third approach would be to filter out the special error codes first > > and delegate all others to PyErr_SetFromWindowsErr(). The latter already > > handles the lack of a string for the code by formatting it numerically. > > This would also improve consistency, since the two functions use > > different ways to format unrecognised errors numerically. This approach > > would change where and how a completely unrecognised error code is > > formatted, but would otherwise be pretty equivalent. > > The third approach is fine with me. Sidenote: The only error codes that I > remember having seen in practice are 'access violation reading...' and > 'access violation writing...', although it may be that on WinCE 'datatype > misalignment' may also be possible. Submitted as patch for issue #5078. Note: under CE, you can actually encounter datatype misalignments, since it runs on CPUs that don't emulate them. I wonder if the same doesn't also apply to win64.... Uli -- Sator Laser GmbH Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Sator Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at ************************************************************************************** Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder anderweitig benutzt werden. E-Mails k?nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte ?nderungen enthalten. Sator Laser GmbH ist f?r diese Folgen nicht verantwortlich. ************************************************************************************** From eckhardt at satorlaser.com Tue Jan 27 12:16:01 2009 From: eckhardt at satorlaser.com (Ulrich Eckhardt) Date: Tue, 27 Jan 2009 12:16:01 +0100 Subject: [Python-Dev] FormatError() in callproc.c under win32 In-Reply-To: <497E3546.8000408@v.loewis.de> References: <200901261904.17978.eckhardt@satorlaser.com> <497E3546.8000408@v.loewis.de> Message-ID: <200901271216.01726.eckhardt@satorlaser.com> On Monday 26 January 2009, Martin v. L?wis wrote: > > In callproc.c from trunk is a function called SetException(), which calls > > FormatError() only to discard the contents. Can anyone enlighten me to > > the reasons thereof? > > Interestingly enough, the code used to say > > PyErr_SetString(PyExc_WindowsError, lpMsgBuf); > > Then it was changed to its current form, with a log message of > > Changes for windows CE, contributed by Luke Dunstan. Thanks a lot! > > See > > http://ctypes.cvs.sourceforge.net/viewvc/ctypes/ctypes/source/callproc.c?hideattic=0&r1=1.127.2.15&r2=1.127.2.16 > > I suggest you ask Thomas Heller and Luke Dunstan (if available) what the > rationale for this partial change was. I can only guess: 1. Those changes seem to generate TCHAR strings. This is necessary to compile it on both win9x (TCHAR=char) and CE (TCHAR=wchar_t). Since win9x was dropped from the supported platforms, that isn't necessary any more and all the code could use WCHAR directly. 2. Those changes also seem to change a few byte-strings to Unicode-strings, see format_error(). This is a questionable step, since those are changes that are visible to Python code. Worse, even on the same platform it could return different string types when the lookup of the errorcode fails. I wonder if that is intentional. In any case, CCing Luke on the issue, maybe he can clarify things. cheers Uli ************************************************************************************** Sator Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at ************************************************************************************** Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder anderweitig benutzt werden. E-Mails k?nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte ?nderungen enthalten. Sator Laser GmbH ist f?r diese Folgen nicht verantwortlich. ************************************************************************************** From steve at holdenweb.com Tue Jan 27 13:08:46 2009 From: steve at holdenweb.com (Steve Holden) Date: Tue, 27 Jan 2009 07:08:46 -0500 Subject: [Python-Dev] [PSF-Board] I've got a surprise for you! In-Reply-To: <497E9320.2030404@sun.com> References: <20090126233246.GA37662@wind.teleri.net> <497E9320.2030404@sun.com> Message-ID: <497EF94E.3050302@holdenweb.com> Jim Walker wrote: [Trent's announcement] > > Great stuff Trent! I was wondering how you were doing. > > I really appreciate what it takes to put these open resources > together ;) There's a lot of moving parts :) > > Cheers, > Jim > > BTW. > > We now have zone servers in the OpenSolaris test farm, and > I plan to add guest os servers in the next few weeks using > ldoms (sparc) and xvm (x64). The zone servers provide whole > root zones, which should be a good development environment > for most projects. Check it out: > > http://test.opensolaris.org/testfarm > http://www.opensolaris.org/os/community/testing/testfarm/zones/ > > Let me know if there is interest from the python community to > manage one of the test farm servers for python development. > Besides the general use machines, the php community is already > managing a T2000 server. Jim: Thanks, this is a terrific offer. I am copying it to the Python developers list so they can discuss it - I know that Solaris is one of the platforms we do get quite a few build questions about. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Chairman, PSF http://www.holdenweb.com/ From jake at youtube.com Tue Jan 27 10:49:29 2009 From: jake at youtube.com (Jake McGuire) Date: Tue, 27 Jan 2009 01:49:29 -0800 Subject: [Python-Dev] undesireable unpickle behavior, proposed fix Message-ID: <2003160D-7A1C-4567-87B4-10E329E169E1@youtube.com> Instance attribute names are normally interned - this is done in PyObject_SetAttr (among other places). Unpickling (in pickle and cPickle) directly updates __dict__ on the instance object. This bypasses the interning so you end up with many copies of the strings representing your attribute names, which wastes a lot of space, both in RAM and in pickles of sequences of objects created from pickles. Note that the native python memcached client uses pickle to serialize objects. >>> import pickle >>> class C(object): ... def __init__(self, x): ... self.long_attribute_name = x ... >>> len(pickle.dumps([pickle.loads(pickle.dumps(C(None), pickle.HIGHEST_PROTOCOL)) for i in range(100)], pickle.HIGHEST_PROTOCOL)) 3658 >>> len(pickle.dumps([C(None) for i in range(100)], pickle.HIGHEST_PROTOCOL)) 1441 >>> Interning the strings on unpickling makes the pickles smaller, and at least for cPickle actually makes unpickling sequences of many objects slightly faster. I have included proposed patches to cPickle.c and pickle.py, and would appreciate any feedback. dhcp-172-31-170-32:~ mcguire$ diff -u Downloads/Python-2.4.3/Modules/ cPickle.c cPickle.c --- Downloads/Python-2.4.3/Modules/cPickle.c 2004-07-26 22:22:33.000000000 -0700 +++ cPickle.c 2009-01-26 23:30:31.000000000 -0800 @@ -4258,6 +4258,8 @@ PyObject *state, *inst, *slotstate; PyObject *__setstate__; PyObject *d_key, *d_value; + PyObject *name; + char * key_str; int i; int res = -1; @@ -4319,8 +4321,24 @@ i = 0; while (PyDict_Next(state, &i, &d_key, &d_value)) { - if (PyObject_SetItem(dict, d_key, d_value) < 0) - goto finally; + /* normally the keys for instance attributes are + interned. we should try to do that here. */ + if (PyString_CheckExact(d_key)) { + key_str = PyString_AsString(d_key); + name = PyString_FromString(key_str); + if (! name) + goto finally; + + PyString_InternInPlace(&name); + if (PyObject_SetItem(dict, name, d_value) < 0) { + Py_DECREF(name); + goto finally; + } + Py_DECREF(name); + } else { + if (PyObject_SetItem(dict, d_key, d_value) < 0) + goto finally; + } } Py_DECREF(dict); } dhcp-172-31-170-32:~ mcguire$ diff -u Downloads/Python-2.4.3/Lib/ pickle.py pickle.py --- Downloads/Python-2.4.3/Lib/pickle.py 2009-01-27 01:41:43.000000000 -0800 +++ pickle.py 2009-01-27 01:41:31.000000000 -0800 @@ -1241,7 +1241,15 @@ state, slotstate = state if state: try: - inst.__dict__.update(state) + d = inst.__dict__ + try: + for k,v in state.items(): + d[intern(k)] = v + # keys in state don't have to be strings + # don't blow up, but don't go out of our way + except TypeError: + d.update(state) + except RuntimeError: # XXX In restricted execution, the instance's __dict__ # is not accessible. Use the old way of unpickling From jnoller at gmail.com Tue Jan 27 15:23:06 2009 From: jnoller at gmail.com (Jesse Noller) Date: Tue, 27 Jan 2009 09:23:06 -0500 Subject: [Python-Dev] undesireable unpickle behavior, proposed fix In-Reply-To: <2003160D-7A1C-4567-87B4-10E329E169E1@youtube.com> References: <2003160D-7A1C-4567-87B4-10E329E169E1@youtube.com> Message-ID: <4222a8490901270623l765523e8gc9bdef65287fd010@mail.gmail.com> On Tue, Jan 27, 2009 at 4:49 AM, Jake McGuire wrote: > Instance attribute names are normally interned - this is done in > PyObject_SetAttr (among other places). Unpickling (in pickle and cPickle) > directly updates __dict__ on the instance object. This bypasses the > interning so you end up with many copies of the strings representing your > attribute names, which wastes a lot of space, both in RAM and in pickles of > sequences of objects created from pickles. Note that the native python > memcached client uses pickle to serialize objects. > >>>> import pickle >>>> class C(object): > ... def __init__(self, x): > ... self.long_attribute_name = x > ... >>>> len(pickle.dumps([pickle.loads(pickle.dumps(C(None), >>>> pickle.HIGHEST_PROTOCOL)) for i in range(100)], pickle.HIGHEST_PROTOCOL)) > 3658 >>>> len(pickle.dumps([C(None) for i in range(100)], >>>> pickle.HIGHEST_PROTOCOL)) > 1441 >>>> > > Interning the strings on unpickling makes the pickles smaller, and at least > for cPickle actually makes unpickling sequences of many objects slightly > faster. I have included proposed patches to cPickle.c and pickle.py, and > would appreciate any feedback. > > dhcp-172-31-170-32:~ mcguire$ diff -u > Downloads/Python-2.4.3/Modules/cPickle.c cPickle.c > --- Downloads/Python-2.4.3/Modules/cPickle.c 2004-07-26 > 22:22:33.000000000 -0700 > +++ cPickle.c 2009-01-26 23:30:31.000000000 -0800 > @@ -4258,6 +4258,8 @@ > PyObject *state, *inst, *slotstate; > PyObject *__setstate__; > PyObject *d_key, *d_value; > + PyObject *name; > + char * key_str; > int i; > int res = -1; > > @@ -4319,8 +4321,24 @@ > > i = 0; > while (PyDict_Next(state, &i, &d_key, &d_value)) { > - if (PyObject_SetItem(dict, d_key, d_value) < 0) > - goto finally; > + /* normally the keys for instance attributes are > + interned. we should try to do that here. */ > + if (PyString_CheckExact(d_key)) { > + key_str = PyString_AsString(d_key); > + name = PyString_FromString(key_str); > + if (! name) > + goto finally; > + > + PyString_InternInPlace(&name); > + if (PyObject_SetItem(dict, name, d_value) < > 0) { > + Py_DECREF(name); > + goto finally; > + } > + Py_DECREF(name); > + } else { > + if (PyObject_SetItem(dict, d_key, d_value) < > 0) > + goto finally; > + } > } > Py_DECREF(dict); > } > > dhcp-172-31-170-32:~ mcguire$ diff -u Downloads/Python-2.4.3/Lib/pickle.py > pickle.py > --- Downloads/Python-2.4.3/Lib/pickle.py 2009-01-27 > 01:41:43.000000000 -0800 > +++ pickle.py 2009-01-27 01:41:31.000000000 -0800 > @@ -1241,7 +1241,15 @@ > state, slotstate = state > if state: > try: > - inst.__dict__.update(state) > + d = inst.__dict__ > + try: > + for k,v in state.items(): > + d[intern(k)] = v > + # keys in state don't have to be strings > + # don't blow up, but don't go out of our way > + except TypeError: > + d.update(state) > + > except RuntimeError: > # XXX In restricted execution, the instance's __dict__ > # is not accessible. Use the old way of unpickling > Hi Jake, You should really post this to bugs.python.org as an enhancement so we can track the discussion there. -jesse From mrts.pydev at gmail.com Tue Jan 27 15:50:12 2009 From: mrts.pydev at gmail.com (=?ISO-8859-1?Q?Mart_S=F5mermaa?=) Date: Tue, 27 Jan 2009 16:50:12 +0200 Subject: [Python-Dev] V8, TraceMonkey, SquirrelFish and Python Message-ID: As most of you know there's constant struggle on the JavaScript front to get even faster performance out of interpreters. V8, TraceMonkey and SquirrelFish have brought novel ideas to interpreter design, wouldn't it make sense to reap the best bits and bring them to Python? Has anyone delved into the designs and considered their applicability to Python? Hoping-to-see-some-V8-and-Python-teams-collaboration-in-Mountain-View-ly yours, Mart S?mermaa -------------- next part -------------- An HTML attachment was scrubbed... URL: From jnoller at gmail.com Tue Jan 27 16:04:36 2009 From: jnoller at gmail.com (Jesse Noller) Date: Tue, 27 Jan 2009 10:04:36 -0500 Subject: [Python-Dev] V8, TraceMonkey, SquirrelFish and Python In-Reply-To: References: Message-ID: <4222a8490901270704y41308595i2bdb65dab7bc4b07@mail.gmail.com> On Tue, Jan 27, 2009 at 9:50 AM, Mart S?mermaa wrote: > As most of you know there's constant struggle on the JavaScript front to get > even faster performance out of interpreters. > V8, TraceMonkey and SquirrelFish have brought novel ideas to interpreter > design, wouldn't it make sense to reap the best bits and bring them to > Python? > > Has anyone delved into the designs and considered their applicability to > Python? > > Hoping-to-see-some-V8-and-Python-teams-collaboration-in-Mountain-View-ly > yours, > Mart S?mermaa > Hi Mart, This is a better discussion for the python-ideas list. That being said, there was a thread discussing this last year, see: http://mail.python.org/pipermail/python-dev/2008-October/083176.html -jesse From steve at holdenweb.com Tue Jan 27 16:17:08 2009 From: steve at holdenweb.com (Steve Holden) Date: Tue, 27 Jan 2009 10:17:08 -0500 Subject: [Python-Dev] V8, TraceMonkey, SquirrelFish and Python In-Reply-To: <4222a8490901270704y41308595i2bdb65dab7bc4b07@mail.gmail.com> References: <4222a8490901270704y41308595i2bdb65dab7bc4b07@mail.gmail.com> Message-ID: <497F2574.1090007@holdenweb.com> Jesse Noller wrote: > On Tue, Jan 27, 2009 at 9:50 AM, Mart S?mermaa wrote: >> As most of you know there's constant struggle on the JavaScript front to get >> even faster performance out of interpreters. >> V8, TraceMonkey and SquirrelFish have brought novel ideas to interpreter >> design, wouldn't it make sense to reap the best bits and bring them to >> Python? >> >> Has anyone delved into the designs and considered their applicability to >> Python? >> >> Hoping-to-see-some-V8-and-Python-teams-collaboration-in-Mountain-View-ly >> yours, >> Mart S?mermaa >> > > Hi Mart, > > This is a better discussion for the python-ideas list. That being > said, there was a thread discussing this last year, see: > > http://mail.python.org/pipermail/python-dev/2008-October/083176.html > I am sure this will be included as a part of the discussion at the VM summit that's taking place as a part of the pre-PyCon activity. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From coder_infidel at hotmail.com Tue Jan 27 16:29:25 2009 From: coder_infidel at hotmail.com (Luke Dunstan) Date: Wed, 28 Jan 2009 00:29:25 +0900 Subject: [Python-Dev] FormatError() in callproc.c under win32 In-Reply-To: <200901271216.01726.eckhardt@satorlaser.com> References: <200901261904.17978.eckhardt@satorlaser.com> <497E3546.8000408@v.loewis.de> <200901271216.01726.eckhardt@satorlaser.com> Message-ID: > From: eckhardt at satorlaser.com > To: python-dev at python.org > Subject: Re: [Python-Dev] FormatError() in callproc.c under win32 > Date: Tue, 27 Jan 2009 12:16:01 +0100 > CC: coder_infidel at hotmail.com > > On Monday 26 January 2009, Martin v. L?wis wrote: > > > In callproc.c from trunk is a function called SetException(), which calls > > > FormatError() only to discard the contents. Can anyone enlighten me to > > > the reasons thereof? The left over call to FormatError() looks like a mistake to me. > > > > Interestingly enough, the code used to say > > > > PyErr_SetString(PyExc_WindowsError, lpMsgBuf); > > > > Then it was changed to its current form, with a log message of > > > > Changes for windows CE, contributed by Luke Dunstan. Thanks a lot! > > > > See > > > > > http://ctypes.cvs.sourceforge.net/viewvc/ctypes/ctypes/source/callproc.c?hideattic=0&r1=1.127.2.15&r2=1.127.2.16 > > > > I suggest you ask Thomas Heller and Luke Dunstan (if available) what the > > rationale for this partial change was. > > I can only guess: > 1. Those changes seem to generate TCHAR strings. This is necessary to compile > it on both win9x (TCHAR=char) and CE (TCHAR=wchar_t). Since win9x was dropped > from the supported platforms, that isn't necessary any more and all the code > could use WCHAR directly. As far as I remember TCHAR was char for Windows NT/2K/XP Python builds too, at least at that time, but yes it would be clearer to use WCHAR instead now. > 2. Those changes also seem to change a few byte-strings to Unicode-strings, > see format_error(). This is a questionable step, since those are changes that > are visible to Python code. Worse, even on the same platform it could return > different string types when the lookup of the errorcode fails. I wonder if > that is intentional. Probably not intentional. Yes, it would be better if the return value was either always char or always WCHAR. > > In any case, CCing Luke on the issue, maybe he can clarify things. > > cheers > > Uli Good luck, Luke -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Tue Jan 27 18:32:57 2009 From: steve at holdenweb.com (Steve Holden) Date: Tue, 27 Jan 2009 12:32:57 -0500 Subject: [Python-Dev] Sun resources [Was: I've got a surprise for you!] Message-ID: <497F4549.3030106@holdenweb.com> So, if anyone wants to run a Sun buildbot or whatever, Jim would be the person to contact. Synchronize on this list to ensure Jim doesn't get multiple approaches, please. regards Steve -------- Original Message -------- Subject: Re: [PSF-Board] I've got a surprise for you! Date: Tue, 27 Jan 2009 10:49:29 -0700 From: Jim Walker Reply-To: James.Walker at Sun.COM Organization: Sun Microsystems, Inc. To: Steve Holden References: <20090126233246.GA37662 at wind.teleri.net> <497E9320.2030404 at sun.com> <497EF94E.3050302 at holdenweb.com> Steve Holden wrote: > > Thanks, this is a terrific offer. I am copying it to the Python > developers list so they can discuss it - I know that Solaris is one of > the platforms we do get quite a few build questions about. > Sounds good. Depending on what you want to do, I can assign a system to your group within a week or two. Cheers, Jim -- Jim Walker, http://blogs.sun.com/jwalker Sun Microsystems, Software, Solaris QE x77744, 500 Eldorado Blvd, Broomfield CO 80021 -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From guido at python.org Tue Jan 27 18:57:40 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 27 Jan 2009 09:57:40 -0800 Subject: [Python-Dev] undesireable unpickle behavior, proposed fix In-Reply-To: <4222a8490901270623l765523e8gc9bdef65287fd010@mail.gmail.com> References: <2003160D-7A1C-4567-87B4-10E329E169E1@youtube.com> <4222a8490901270623l765523e8gc9bdef65287fd010@mail.gmail.com> Message-ID: On Tue, Jan 27, 2009 at 6:23 AM, Jesse Noller wrote: > On Tue, Jan 27, 2009 at 4:49 AM, Jake McGuire wrote: >> Instance attribute names are normally interned - this is done in >> PyObject_SetAttr (among other places). Unpickling (in pickle and cPickle) >> directly updates __dict__ on the instance object. This bypasses the >> interning so you end up with many copies of the strings representing your >> attribute names, which wastes a lot of space, both in RAM and in pickles of >> sequences of objects created from pickles. Note that the native python >> memcached client uses pickle to serialize objects. >> >>>>> import pickle >>>>> class C(object): >> ... def __init__(self, x): >> ... self.long_attribute_name = x >> ... >>>>> len(pickle.dumps([pickle.loads(pickle.dumps(C(None), >>>>> pickle.HIGHEST_PROTOCOL)) for i in range(100)], pickle.HIGHEST_PROTOCOL)) >> 3658 >>>>> len(pickle.dumps([C(None) for i in range(100)], >>>>> pickle.HIGHEST_PROTOCOL)) >> 1441 >>>>> >> >> Interning the strings on unpickling makes the pickles smaller, and at least >> for cPickle actually makes unpickling sequences of many objects slightly >> faster. I have included proposed patches to cPickle.c and pickle.py, and >> would appreciate any feedback. >> >> dhcp-172-31-170-32:~ mcguire$ diff -u >> Downloads/Python-2.4.3/Modules/cPickle.c cPickle.c >> --- Downloads/Python-2.4.3/Modules/cPickle.c 2004-07-26 >> 22:22:33.000000000 -0700 >> +++ cPickle.c 2009-01-26 23:30:31.000000000 -0800 >> @@ -4258,6 +4258,8 @@ >> PyObject *state, *inst, *slotstate; >> PyObject *__setstate__; >> PyObject *d_key, *d_value; >> + PyObject *name; >> + char * key_str; >> int i; >> int res = -1; >> >> @@ -4319,8 +4321,24 @@ >> >> i = 0; >> while (PyDict_Next(state, &i, &d_key, &d_value)) { >> - if (PyObject_SetItem(dict, d_key, d_value) < 0) >> - goto finally; >> + /* normally the keys for instance attributes are >> + interned. we should try to do that here. */ >> + if (PyString_CheckExact(d_key)) { >> + key_str = PyString_AsString(d_key); >> + name = PyString_FromString(key_str); >> + if (! name) >> + goto finally; >> + >> + PyString_InternInPlace(&name); >> + if (PyObject_SetItem(dict, name, d_value) < >> 0) { >> + Py_DECREF(name); >> + goto finally; >> + } >> + Py_DECREF(name); >> + } else { >> + if (PyObject_SetItem(dict, d_key, d_value) < >> 0) >> + goto finally; >> + } >> } >> Py_DECREF(dict); >> } >> >> dhcp-172-31-170-32:~ mcguire$ diff -u Downloads/Python-2.4.3/Lib/pickle.py >> pickle.py >> --- Downloads/Python-2.4.3/Lib/pickle.py 2009-01-27 >> 01:41:43.000000000 -0800 >> +++ pickle.py 2009-01-27 01:41:31.000000000 -0800 >> @@ -1241,7 +1241,15 @@ >> state, slotstate = state >> if state: >> try: >> - inst.__dict__.update(state) >> + d = inst.__dict__ >> + try: >> + for k,v in state.items(): >> + d[intern(k)] = v >> + # keys in state don't have to be strings >> + # don't blow up, but don't go out of our way >> + except TypeError: >> + d.update(state) >> + >> except RuntimeError: >> # XXX In restricted execution, the instance's __dict__ >> # is not accessible. Use the old way of unpickling >> > > Hi Jake, > > You should really post this to bugs.python.org as an enhancement so we > can track the discussion there. > > -jesse Seconded, with eagerness -- interning attribute names when unpickling makes a lot of sense! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Tue Jan 27 19:43:30 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 27 Jan 2009 19:43:30 +0100 Subject: [Python-Dev] undesireable unpickle behavior, proposed fix In-Reply-To: <2003160D-7A1C-4567-87B4-10E329E169E1@youtube.com> References: <2003160D-7A1C-4567-87B4-10E329E169E1@youtube.com> Message-ID: <497F55D2.1020805@v.loewis.de> > Interning the strings on unpickling makes the pickles smaller, and at > least for cPickle actually makes unpickling sequences of many objects > slightly faster. I have included proposed patches to cPickle.c and > pickle.py, and would appreciate any feedback. Please submit patches always to the bug tracker. On the proposed change: While it is fairly unintrusive, I would like to propose a different approach - pickle interned strings special. The marshal module already uses this approach, and it should extend to pickle (although it would probably require a new protocol). On pickling, inspect each string and check whether it is interned. If so, emit a different code, and record it into the object id dictionary. On a second occurrence of the string, only pickle a backward reference. (Alternatively, check whether pickling the same string a second time would be more compact). On unpickling, support the new code to intern the result strings; subsequent references to it will go to the standard backreferencing algorithm. Regards, Martin From martin at v.loewis.de Tue Jan 27 19:45:27 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 27 Jan 2009 19:45:27 +0100 Subject: [Python-Dev] FormatError() in callproc.c under win32 In-Reply-To: <200901271136.40940.eckhardt@satorlaser.com> References: <200901261904.17978.eckhardt@satorlaser.com> <200901271136.40940.eckhardt@satorlaser.com> Message-ID: <497F5647.4070806@v.loewis.de> > Note: under CE, you can actually encounter datatype misalignments, since it > runs on CPUs that don't emulate them. I wonder if the same doesn't also apply > to win64.... I don't think you can get misalignment traps on AMD64. Not sure about IA-64: I know that the processor will trap on misaligned accesses, but the operating system might silently fix the access. Regards, Martin From guido at python.org Tue Jan 27 19:57:22 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 27 Jan 2009 10:57:22 -0800 Subject: [Python-Dev] undesireable unpickle behavior, proposed fix In-Reply-To: <497F55D2.1020805@v.loewis.de> References: <2003160D-7A1C-4567-87B4-10E329E169E1@youtube.com> <497F55D2.1020805@v.loewis.de> Message-ID: On Tue, Jan 27, 2009 at 10:43 AM, "Martin v. L?wis" wrote: >> Interning the strings on unpickling makes the pickles smaller, and at >> least for cPickle actually makes unpickling sequences of many objects >> slightly faster. I have included proposed patches to cPickle.c and >> pickle.py, and would appreciate any feedback. > > Please submit patches always to the bug tracker. > > On the proposed change: While it is fairly unintrusive, I would like to > propose a different approach - pickle interned strings special. The > marshal module already uses this approach, and it should extend to > pickle (although it would probably require a new protocol). > > On pickling, inspect each string and check whether it is interned. If > so, emit a different code, and record it into the object id dictionary. > On a second occurrence of the string, only pickle a backward reference. > (Alternatively, check whether pickling the same string a second time > would be more compact). > > On unpickling, support the new code to intern the result strings; > subsequent references to it will go to the standard backreferencing > algorithm. Hm. This would change the pickling format though. Wouldn't just interning (short) strings on unpickling be simpler? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Tue Jan 27 20:00:11 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 27 Jan 2009 11:00:11 -0800 Subject: [Python-Dev] Python 3.0.1 Message-ID: With the extensive changes in the works, Python 3.0.1 is shaping-up to be a complete rerelease of 3.0 with API changes and major usability fixes. It will fully supplant the original 3.0 release which was hobbled by poor IO performance. I propose to make the new release more attractive by backporting several module improvements already in 3.1, including two new itertools and one collections class. These are already fully documented, tested, and checked-in to 3.1 and it would be ashamed to let them sit idle for a year or so, when the module updates are already ready-to-ship. Raymond From guido at python.org Tue Jan 27 20:05:03 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 27 Jan 2009 11:05:03 -0800 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: Message-ID: On Tue, Jan 27, 2009 at 11:00 AM, Raymond Hettinger wrote: > With the extensive changes in the works, Python 3.0.1 is shaping-up to be a > complete rerelease of 3.0 with API changes and major usability fixes. It > will fully supplant the original 3.0 release which was hobbled by poor IO > performance. > > I propose to make the new release more attractive by backporting several > module improvements already in 3.1, including two new itertools and one > collections class. These are already fully documented, tested, and > checked-in to 3.1 and it would be ashamed to let them sit idle for a year or > so, when the module updates are already ready-to-ship. In that case, I recommend just releasing it as 3.1. I had always anticipated a 3.1 release much sooner than the typical release schedule. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Tue Jan 27 20:09:30 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 27 Jan 2009 11:09:30 -0800 Subject: [Python-Dev] Python 3.0.1 References: Message-ID: <2D3E0C7651454837ACF976E1D0563116@RaymondLaptop1> From: "Guido van Rossum" > In that case, I recommend just releasing it as 3.1. I had always > anticipated a 3.1 release much sooner than the typical release > schedule. That is great idea. It's a strong cue that there is a somewhat major break with 3.0 (removed functions, API fixes, huge performance fixes, and whatnot). Raymond From barry at python.org Tue Jan 27 20:29:03 2009 From: barry at python.org (Barry Warsaw) Date: Tue, 27 Jan 2009 14:29:03 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: Message-ID: <66CA6609-A9B6-4544-BDB4-9B5E0F618446@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 27, 2009, at 2:05 PM, Guido van Rossum wrote: > On Tue, Jan 27, 2009 at 11:00 AM, Raymond Hettinger > wrote: >> With the extensive changes in the works, Python 3.0.1 is shaping-up >> to be a >> complete rerelease of 3.0 with API changes and major usability >> fixes. It >> will fully supplant the original 3.0 release which was hobbled by >> poor IO >> performance. >> >> I propose to make the new release more attractive by backporting >> several >> module improvements already in 3.1, including two new itertools and >> one >> collections class. These are already fully documented, tested, and >> checked-in to 3.1 and it would be ashamed to let them sit idle for >> a year or >> so, when the module updates are already ready-to-ship. > > In that case, I recommend just releasing it as 3.1. I had always > anticipated a 3.1 release much sooner than the typical release > schedule. I was going to object on principle to Raymond's suggestion to rip out the operator module functions in Python 3.0.1. I have no objection to ripping them out for 3.1. If you really think we need a Python 3.1 soon, then I won't worry about trying to get a 3.0.1 out soon. 3.1 is Benjamin's baby :). If OTOH we do intend to get a 3.0.1 out, say by the end of February, then please be careful to adhere to our guidelines for which version various changes can go in. For example, the operator methods needs to be restored to the 3.0 maintenance branch, and any other API changes added to 3.0 need to be backed out and applied only to the python3 trunk. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSX9ggHEjvBPtnXfVAQJkTwQAmpKLlXwiIdgHANxlj85wNko4kB7o8Xv8 8wKT6/ZZeU8t09eelchklhw9rAB4I/BQcoQYPg9jiUydbFWdPd/0/G8xrr+F+dTO J2fkGEK1GVorcAZ3iWywpLQXPnHgfrelUBhKT5KzIu5xWzuEnLBDT3c+r2fwNZia hNpAu1Ihj+s= =g69v -----END PGP SIGNATURE----- From brett at python.org Tue Jan 27 20:39:13 2009 From: brett at python.org (Brett Cannon) Date: Tue, 27 Jan 2009 11:39:13 -0800 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <66CA6609-A9B6-4544-BDB4-9B5E0F618446@python.org> References: <66CA6609-A9B6-4544-BDB4-9B5E0F618446@python.org> Message-ID: On Tue, Jan 27, 2009 at 11:29, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Jan 27, 2009, at 2:05 PM, Guido van Rossum wrote: > >> On Tue, Jan 27, 2009 at 11:00 AM, Raymond Hettinger >> wrote: >>> >>> With the extensive changes in the works, Python 3.0.1 is shaping-up to be >>> a >>> complete rerelease of 3.0 with API changes and major usability fixes. It >>> will fully supplant the original 3.0 release which was hobbled by poor IO >>> performance. >>> >>> I propose to make the new release more attractive by backporting several >>> module improvements already in 3.1, including two new itertools and one >>> collections class. These are already fully documented, tested, and >>> checked-in to 3.1 and it would be ashamed to let them sit idle for a year >>> or >>> so, when the module updates are already ready-to-ship. >> >> In that case, I recommend just releasing it as 3.1. I had always >> anticipated a 3.1 release much sooner than the typical release >> schedule. > A quick 3.1 release also shows how committed we are to 3.x and that we realize that 3.0 had some initial growing pains that needed to be worked out. > I was going to object on principle to Raymond's suggestion to rip out the > operator module functions in Python 3.0.1. I thought it was for 3.1? > I have no objection to ripping > them out for 3.1. > > If you really think we need a Python 3.1 soon, then I won't worry about > trying to get a 3.0.1 out soon. 3.1 is Benjamin's baby :). > Depending on what Benjamin wants to do we could try for something like a release by PyCon or at PyCon during the sprints. Actually the sprint one is a rather nice idea if Benjamin is willing to spend sprint time on it (and he is sticking around for the sprints) as I assume you, Barry, will be there to be able to help in person and we can squash last minute issues really quickly. > If OTOH we do intend to get a 3.0.1 out, say by the end of February, then > please be careful to adhere to our guidelines for which version various > changes can go in. For example, the operator methods needs to be restored > to the 3.0 maintenance branch, and any other API changes added to 3.0 need > to be backed out and applied only to the python3 trunk. If you have the time for it, Barry, I am +1 on an end of February 3.0.1 with a March/April 3.1 if that works for Benjamin. -Brett From martin at v.loewis.de Tue Jan 27 20:40:06 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 27 Jan 2009 20:40:06 +0100 Subject: [Python-Dev] undesireable unpickle behavior, proposed fix In-Reply-To: References: <2003160D-7A1C-4567-87B4-10E329E169E1@youtube.com> <497F55D2.1020805@v.loewis.de> Message-ID: <497F6316.60902@v.loewis.de> > Hm. This would change the pickling format though. Wouldn't just > interning (short) strings on unpickling be simpler? Sure - that's what Jake had proposed. However, it is always difficult to select which strings to intern - his heuristics (IIUC) is to intern all strings that appear as dictionary keys. Whether this is good enough, I don't know. In particular, it might intern very large strings that aren't identifiers at all. Regards, Martin From barry at python.org Tue Jan 27 20:52:57 2009 From: barry at python.org (Barry Warsaw) Date: Tue, 27 Jan 2009 14:52:57 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <66CA6609-A9B6-4544-BDB4-9B5E0F618446@python.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 27, 2009, at 2:39 PM, Brett Cannon wrote: >> I was going to object on principle to Raymond's suggestion to rip >> out the >> operator module functions in Python 3.0.1. > > I thought it was for 3.1? Sorry, I probably misread Raymond's suggestion. >> I have no objection to ripping >> them out for 3.1. >> >> If you really think we need a Python 3.1 soon, then I won't worry >> about >> trying to get a 3.0.1 out soon. 3.1 is Benjamin's baby :). >> > > Depending on what Benjamin wants to do we could try for something like > a release by PyCon or at PyCon during the sprints. Actually the sprint > one is a rather nice idea if Benjamin is willing to spend sprint time > on it (and he is sticking around for the sprints) as I assume you, > Barry, will be there to be able to help in person and we can squash > last minute issues really quickly. Yep, I'm planning on sticking around, so that's a great idea. >> If OTOH we do intend to get a 3.0.1 out, say by the end of >> February, then >> please be careful to adhere to our guidelines for which version >> various >> changes can go in. For example, the operator methods needs to be >> restored >> to the 3.0 maintenance branch, and any other API changes added to >> 3.0 need >> to be backed out and applied only to the python3 trunk. > > If you have the time for it, Barry, I am +1 on an end of February > 3.0.1 with a March/April 3.1 if that works for Benjamin. Or at least a 3.1alpha/beta/whatever during Pycon. I'm sure I can find the time to do a 3.0.1 before Pycon. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSX9mGXEjvBPtnXfVAQL5BgP+JXX43hbNlrjeV9YBFBbCB9SfnFlImTTx ZHhilw12yH13Ha2RLbre+sWlBDQFdTeAJkjUWg2/iZ7Ti8g9eD7sp1KRRuLkbTx0 83h+ciTd9Fdp+sv4JRKfP609X0dlAfbrjjVU/NzXCHePXb++Tr2liHRtHwnr3DgL kZNp1jOTG8Q= =nVHs -----END PGP SIGNATURE----- From guido at python.org Tue Jan 27 20:57:46 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 27 Jan 2009 11:57:46 -0800 Subject: [Python-Dev] undesireable unpickle behavior, proposed fix In-Reply-To: <497F6316.60902@v.loewis.de> References: <2003160D-7A1C-4567-87B4-10E329E169E1@youtube.com> <497F55D2.1020805@v.loewis.de> <497F6316.60902@v.loewis.de> Message-ID: On Tue, Jan 27, 2009 at 11:40 AM, "Martin v. L?wis" wrote: >> Hm. This would change the pickling format though. Wouldn't just >> interning (short) strings on unpickling be simpler? > > Sure - that's what Jake had proposed. However, it is always difficult > to select which strings to intern - his heuristics (IIUC) is to intern > all strings that appear as dictionary keys. Whether this is good enough, > I don't know. In particular, it might intern very large strings that > aren't identifiers at all. Just set a size limit, e.g. 30 or 100. It's just a heuristic. I believe somewhere in Python itself I intern string literals if they are reasonably short and fit the pattern of an identifier; I'd worry that the pattern matching would slow down unpickling more than the expected benefit though, so perhaps just a size test would be better. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Tue Jan 27 21:07:53 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 27 Jan 2009 21:07:53 +0100 Subject: [Python-Dev] undesireable unpickle behavior, proposed fix In-Reply-To: References: <2003160D-7A1C-4567-87B4-10E329E169E1@youtube.com> <497F55D2.1020805@v.loewis.de> <497F6316.60902@v.loewis.de> Message-ID: <497F6999.1050403@v.loewis.de> > Just set a size limit, e.g. 30 or 100. It's just a heuristic. I > believe somewhere in Python itself I intern string literals if they > are reasonably short and fit the pattern of an identifier; I'd worry > that the pattern matching would slow down unpickling more than the > expected benefit though, so perhaps just a size test would be better. Ok. So, Jake, it's back to my original request - please submit this to the tracker (preferably along with test cases). Regards, Martin From benjamin at python.org Tue Jan 27 21:22:19 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 27 Jan 2009 14:22:19 -0600 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: Message-ID: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> On Tue, Jan 27, 2009 at 1:00 PM, Raymond Hettinger wrote: > With the extensive changes in the works, Python 3.0.1 is shaping-up to be a > complete rerelease of 3.0 with API changes and major usability fixes. It > will fully supplant the original 3.0 release which was hobbled by poor IO > performance. > > I propose to make the new release more attractive by backporting several > module improvements already in 3.1, including two new itertools and one > collections class. These are already fully documented, tested, and > checked-in to 3.1 and it would be ashamed to let them sit idle for a year or > so, when the module updates are already ready-to-ship. At the moment, there are 4 release blockers for 3.0.1. I'd like to see 3.0.1 released soon (within the next month.) It would fix the hugest mistakes in the initial release most of which have been done committed since December. I'm sure it would be attractive enough with the nasty bugs fixed in it! Let's not completely open the flood gates. Releasing 3.1 in March or April also sounds good. I will be at least at the first day of sprints. -- Regards, Benjamin From jake at youtube.com Tue Jan 27 21:25:02 2009 From: jake at youtube.com (Jake McGuire) Date: Tue, 27 Jan 2009 12:25:02 -0800 Subject: [Python-Dev] undesireable unpickle behavior, proposed fix In-Reply-To: <497F6316.60902@v.loewis.de> References: <2003160D-7A1C-4567-87B4-10E329E169E1@youtube.com> <497F55D2.1020805@v.loewis.de> <497F6316.60902@v.loewis.de> Message-ID: On Jan 27, 2009, at 11:40 AM, Martin v. L?wis wrote: >> Hm. This would change the pickling format though. Wouldn't just >> interning (short) strings on unpickling be simpler? > > Sure - that's what Jake had proposed. However, it is always difficult > to select which strings to intern - his heuristics (IIUC) is to intern > all strings that appear as dictionary keys. Whether this is good > enough, > I don't know. In particular, it might intern very large strings that > aren't identifiers at all. I may have misunderstood how unpickling works, but I believe that my path only interns strings that are keys in a dictionary used to populate an instance. This is very similar to how instance creation and modification works in Python now. The only difference is if you set an attribute via "inst.__dict__['attribute_name'] = value" then 'attribute_name' will not be automatically interned, but if you pickle the instance, 'attribute_name' will be interned on unpickling. There may be cases where users specifically go through __dict__ to avoid interning attribute names, but I would be surprised to hear about it and very interested in talking to the person who did that. Creating a new pickle protocol to handle this case seems excessive... -jake From martin at v.loewis.de Tue Jan 27 21:28:05 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 27 Jan 2009 21:28:05 +0100 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> Message-ID: <497F6E55.6090608@v.loewis.de> > At the moment, there are 4 release blockers for 3.0.1. I'd like to see > 3.0.1 released soon (within the next month.) I agree. In December, there was a huge sense of urgency that we absolutely must have a 3.0.1 last year - and now people talk about giving up 3.0 entirely. Releasing 3.1 6 months after 3.0 sounds reasonable; I don't think it should be released earlier (else 3.0 looks fairly ridiculous). Regards, Martin From bioinformed at gmail.com Tue Jan 27 21:32:00 2009 From: bioinformed at gmail.com (Kevin Jacobs ) Date: Tue, 27 Jan 2009 15:32:00 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> Message-ID: <2e1434c10901271232l139018q360e662b006c0580@mail.gmail.com> On Tue, Jan 27, 2009 at 3:22 PM, Benjamin Peterson wrote: > > At the moment, there are 4 release blockers for 3.0.1. I'd like to see > 3.0.1 released soon (within the next month.) It would fix the hugest > mistakes in the initial release most of which have been done committed > since December. I'm sure it would be attractive enough with the nasty > bugs fixed in it! Let's not completely open the flood gates. > > Releasing 3.1 in March or April also sounds good. I will be at least > at the first day of sprints. > As an interested observer, but not yet user of the 3.x series, I was wondering about progress on restoring io performance and what release those improvements were slated for. This is the major blocker for me to begin porting my non-numpy/scipy dependent code. Much of my current work is in bioinformatics, often dealing with multi-gigabyte datasets, so file io fast is critical. Otherwise, I'll have to live with 2.x for the indefinite future. Thanks, ~Kevin -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Tue Jan 27 21:39:40 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 27 Jan 2009 21:39:40 +0100 Subject: [Python-Dev] undesireable unpickle behavior, proposed fix In-Reply-To: References: <2003160D-7A1C-4567-87B4-10E329E169E1@youtube.com> <497F55D2.1020805@v.loewis.de> <497F6316.60902@v.loewis.de> Message-ID: <497F710C.8080406@v.loewis.de> > I may have misunderstood how unpickling works Perhaps I have misunderstood your patch. Posting it to Rietveld might also be useful. Regards, Martin From guido at python.org Tue Jan 27 21:40:20 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 27 Jan 2009 12:40:20 -0800 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <497F6E55.6090608@v.loewis.de> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> Message-ID: On Tue, Jan 27, 2009 at 12:28 PM, "Martin v. L?wis" wrote: >> At the moment, there are 4 release blockers for 3.0.1. I'd like to see >> 3.0.1 released soon (within the next month.) > > I agree. In December, there was a huge sense of urgency that we > absolutely must have a 3.0.1 last year - and now people talk about > giving up 3.0 entirely. > > Releasing 3.1 6 months after 3.0 sounds reasonable; I don't think > it should be released earlier (else 3.0 looks fairly ridiculous). It sounds like my approval of Raymond's removal of certain (admittedly obsolete) operators from the 3.0 branch was premature. Barry at least thinks those should be rolled back. Others? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Tue Jan 27 21:48:37 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 27 Jan 2009 21:48:37 +0100 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> Message-ID: <497F7325.7070802@v.loewis.de> >> Releasing 3.1 6 months after 3.0 sounds reasonable; I don't think >> it should be released earlier (else 3.0 looks fairly ridiculous). > > It sounds like my approval of Raymond's removal of certain (admittedly > obsolete) operators from the 3.0 branch was premature. Barry at least > thinks those should be rolled back. Others? I agree that not too much harm is done by removing stuff in 3.0.1 that erroneously had been left in the 3.0 release - in particular if 3.0.1 gets released quickly (e.g. within two months of the original release). If that is an acceptable policy, then those changes would fall under the policy. If the policy is *not* acceptable, a lot of changes to 3.0.1 need to be rolled back (e.g. the ongoing removal of __cmp__ fragments) Regards, Martin From python at rcn.com Tue Jan 27 22:19:21 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 27 Jan 2009 13:19:21 -0800 Subject: [Python-Dev] Python 3.0.1 References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> Message-ID: From: ""Martin v. L?wis"" > Releasing 3.1 6 months after 3.0 sounds reasonable; I don't think > it should be released earlier (else 3.0 looks fairly ridiculous). I think it should be released earlier and completely supplant 3.0 before more third-party developers spend time migrating code. We needed 3.0 to get released so we could get the feedback necessary to shake it out. Now, it is time for it to fade into history and take advantage of the lessons learned. The principles for the 2.x series don't really apply here. In 2.x, there was always a useful, stable, clean release already fielded and there were tons of third-party apps that needed a slow rate of change. In contrast, 3.0 has a near zero installed user base (at least in terms of being used in production). It has very few migrated apps. It is not particularly clean and some of the work for it was incomplete when it was released. My preference is to drop 3.0 entirely (no incompatable bugfix release) and in early February release 3.1 as the real 3.x that migrators ought to aim for and that won't have incompatable bugfix releases. Then at PyCon, we can have a real bug day and fix-up any chips in the paint. If 3.1 goes out right away, then it doesn't matter if 3.0 looks ridiculous. All eyes go to the latest release. Better to get this done before more people download 3.0 to kick the tires. Raymond From mrts.pydev at gmail.com Tue Jan 27 22:21:44 2009 From: mrts.pydev at gmail.com (=?ISO-8859-1?Q?Mart_S=F5mermaa?=) Date: Tue, 27 Jan 2009 23:21:44 +0200 Subject: [Python-Dev] V8, TraceMonkey, SquirrelFish and Python In-Reply-To: <4222a8490901270704y41308595i2bdb65dab7bc4b07@mail.gmail.com> References: <4222a8490901270704y41308595i2bdb65dab7bc4b07@mail.gmail.com> Message-ID: On Tue, Jan 27, 2009 at 5:04 PM, Jesse Noller wrote: > Hi Mart, > > This is a better discussion for the python-ideas list. That being > said, there was a thread discussing this last year, see: > > http://mail.python.org/pipermail/python-dev/2008-October/083176.html > > -jesse > Indeed, sorry. Incidentally, there is a similar discussion going on just now. -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Tue Jan 27 22:44:35 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 27 Jan 2009 21:44:35 +0000 (UTC) Subject: [Python-Dev] IO performance References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <2e1434c10901271232l139018q360e662b006c0580@mail.gmail.com> Message-ID: Hello Kevin, > As an interested observer, but not yet user of the 3.x series, I was wondering about progress on restoring io performance and what release those improvements were slated for. There is an SVN branch with a complete rewrite (in C) of the IO stack. You can find it in branches/io-c. Apart from a problem in _ssl.c, it should be quite usable. Your tests and observations are welcome! Regards Antoine. From python at rcn.com Tue Jan 27 22:46:35 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 27 Jan 2009 13:46:35 -0800 Subject: [Python-Dev] pprint(iterator) Message-ID: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> It is becoming the norm in 3.x for functions to return iterators, generators, or views whereever possible. I had a thought that pprint() ought to be taught to print iterators: pprint(enumerate(seq)) pprint(map(somefunc, somedata)) pprint(permutations(elements)) pprint(mydict.items()) Currently, all four of those will print something like: >>> pprint(d.items()) >>> pprint(enumerate(d)) If pprint() is to give a more useful result, the question is how best to represent the iterators. In the examples for itertools, I adopted the convention of displaying results like a collection with no commas or enclosing delimiters: # chain('ABC', 'DEF') --> A B C D E F The equivalent for pprint would be the same for items, using space for items on one row or using linefeeds for output too long for one row. Another idea is to make-up an angle-bracket style to provide a visual cue for iterator output: <'A' 'B' 'C' 'D' 'E' 'F'> Perhaps with commas: <'A', 'B', 'C', 'D', 'E', 'F'> None of those ideas can be run through eval, nor do they identify the type of iterator. Perhaps these would be better: or iter(['A', 'B', 'C', 'D', 'E', 'F']) Do you guys have any thoughts on the subject? Raymond From benjamin at python.org Tue Jan 27 22:48:08 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 27 Jan 2009 15:48:08 -0600 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> Message-ID: <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> On Tue, Jan 27, 2009 at 3:19 PM, Raymond Hettinger wrote: > If 3.1 goes out right away, then it doesn't matter if 3.0 looks ridiculous. > All eyes go to the latest release. Better to get this done before more > people download 3.0 to kick the tires. It seems like we are arguing over the version number of basically the same thing. I would like to see 3.0.1 released in early February for nearly the reasons you name. However, it seems to me that there are two kinds of issues: those like __cmp__ removal and some silly IO bugs that have been fixed for a while and our waiting to be released. There's also projects like io in c which are important, but would not make the schedule you and I want for 3.0.1/3.1. It's for those longer term features that I want 3.0.1 and 3.1. If we immedatly released 3.1, when would those longer term projects that are important for migration make it to stable? 3.2 is probably a while off. -- Regards, Benjamin From guido at python.org Tue Jan 27 22:49:22 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 27 Jan 2009 13:49:22 -0800 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> Message-ID: My only thought is that whatever you do, target Python 3.1, not 3.0.1. On Tue, Jan 27, 2009 at 1:46 PM, Raymond Hettinger wrote: > It is becoming the norm in 3.x for functions to return iterators, > generators, or views whereever possible. > > I had a thought that pprint() ought to be taught to print iterators: > > pprint(enumerate(seq)) > pprint(map(somefunc, somedata)) > pprint(permutations(elements)) > pprint(mydict.items()) > > Currently, all four of those will print something like: > > >>> pprint(d.items()) > > >>> pprint(enumerate(d)) > > > If pprint() is to give a more useful result, the question is how best to > represent the iterators. > > In the examples for itertools, I adopted the convention of displaying > results > like a collection with no commas or enclosing delimiters: > > # chain('ABC', 'DEF') --> A B C D E F > > The equivalent for pprint would be the same for items, using space for items > on one row or using linefeeds for output too long for one row. > > Another idea is to make-up an angle-bracket style to provide a visual cue > for iterator output: > > <'A' 'B' 'C' 'D' 'E' 'F'> > > Perhaps with commas: > > <'A', 'B', 'C', 'D', 'E', 'F'> > > None of those ideas can be run through eval, nor do they identify the type > of iterator. Perhaps these would be better: > > > > or > > iter(['A', 'B', 'C', 'D', 'E', 'F']) > > > Do you guys have any thoughts on the subject? > > > Raymond > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From solipsis at pitrou.net Tue Jan 27 22:49:58 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 27 Jan 2009 21:49:58 +0000 (UTC) Subject: [Python-Dev] Python 3.0.1 References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> Message-ID: Benjamin Peterson python.org> writes: > > At the moment, there are 4 release blockers for 3.0.1. I'd like to see > 3.0.1 released soon (within the next month.) It would fix the hugest > mistakes in the initial release most of which have been done committed > since December. I'm sure it would be attractive enough with the nasty > bugs fixed in it! Let's not completely open the flood gates. > > Releasing 3.1 in March or April also sounds good. I will be at least > at the first day of sprints. +1 on all Benjamin said. The IO-in-C branch cannot be reasonably pulled in release30-maint, but it will be ready for 3.1. Speaking of which, testers are welcome (the branch is in branches/io-c). Also, I need someone to update the Windows build files. Regards Antoine. From phd at phd.pp.ru Tue Jan 27 23:06:34 2009 From: phd at phd.pp.ru (Oleg Broytmann) Date: Wed, 28 Jan 2009 01:06:34 +0300 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> Message-ID: <20090127220634.GA16885@phd.pp.ru> On Tue, Jan 27, 2009 at 01:46:35PM -0800, Raymond Hettinger wrote: > I like the idea, and I prefer this formatting. Also bear in mind there are infinite generators, and there are iterators that cannot be reset. For infinite generators pprint() must have a parameter, say, 'max_items', and print . The situation with iterators that cannot be reset should be documented. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From benjamin at python.org Tue Jan 27 23:12:46 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 27 Jan 2009 16:12:46 -0600 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> Message-ID: <1afaf6160901271412p45d5e33dw14d487c7183870a1@mail.gmail.com> On Tue, Jan 27, 2009 at 3:46 PM, Raymond Hettinger wrote: > It is becoming the norm in 3.x for functions to return iterators, > generators, or views whereever possible. > Do you guys have any thoughts on the subject? Maybe a solution like this could help with bugs like #2610? -- Regards, Benjamin From guido at python.org Tue Jan 27 23:14:13 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 27 Jan 2009 14:14:13 -0800 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <20090127220634.GA16885@phd.pp.ru> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090127220634.GA16885@phd.pp.ru> Message-ID: On Tue, Jan 27, 2009 at 2:06 PM, Oleg Broytmann wrote: > On Tue, Jan 27, 2009 at 01:46:35PM -0800, Raymond Hettinger wrote: >> > > I like the idea, and I prefer this formatting. Also bear in mind there > are infinite generators, and there are iterators that cannot be reset. For > infinite generators pprint() must have a parameter, say, 'max_items', and > print . The situation with > iterators that cannot be reset should be documented. This pretty much kills the proposal. Calling a "print" function like pprint() should not have a side effect on the object being printed. I'd be okay of pprint() special-cased the views returned by e.g. dict.keys(), but if all we know is that the argument has a __next__ method, pprint() should *not* be calling that. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Tue Jan 27 23:14:37 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 28 Jan 2009 08:14:37 +1000 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> Message-ID: <497F874D.9090202@gmail.com> Raymond Hettinger wrote: > I quite like the idea of something along those lines. For example: try: itr = iter(obj) except TypeError: pass else: return " " % (obj.__class__.__name__, )) Doing this only in pprint also reduces the chances of accidentally consuming an iterator (which was a reasonable objection when I suggested changing the __str__ implementation on some of the standard iterators some time ago). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From guido at python.org Tue Jan 27 23:15:07 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 27 Jan 2009 14:15:07 -0800 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <1afaf6160901271412p45d5e33dw14d487c7183870a1@mail.gmail.com> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <1afaf6160901271412p45d5e33dw14d487c7183870a1@mail.gmail.com> Message-ID: On Tue, Jan 27, 2009 at 2:12 PM, Benjamin Peterson wrote: > On Tue, Jan 27, 2009 at 3:46 PM, Raymond Hettinger wrote: >> It is becoming the norm in 3.x for functions to return iterators, >> generators, or views whereever possible. > >> Do you guys have any thoughts on the subject? > > Maybe a solution like this could help with bugs like #2610? It would have to special-case range() objects. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jake at youtube.com Tue Jan 27 23:16:29 2009 From: jake at youtube.com (Jake McGuire) Date: Tue, 27 Jan 2009 14:16:29 -0800 Subject: [Python-Dev] undesireable unpickle behavior, proposed fix In-Reply-To: <497F710C.8080406@v.loewis.de> References: <2003160D-7A1C-4567-87B4-10E329E169E1@youtube.com> <497F55D2.1020805@v.loewis.de> <497F6316.60902@v.loewis.de> <497F710C.8080406@v.loewis.de> Message-ID: <11495282-A381-4C98-85B0-F811ECD9A072@youtube.com> On Jan 27, 2009, at 12:39 PM, Martin v. L?wis wrote: >> I may have misunderstood how unpickling works > > Perhaps I have misunderstood your patch. Posting it to Rietveld might > also be useful. It is not immediately clear to me how Rietveld works. But I have created an issue on tracker: http://bugs.python.org/issue5084 Another vaguely related change would be to store string and unicode objects in the pickler memo keyed as themselves rather than their object ids. Depending on the data set, you can have many copies of the same string, e.g. "application/octet-stream". This may marginally increase memory usage during pickling, depending on the data being pickled and the way in which the code was written. I'm happy to write this up if people are interested... -jake From python at rcn.com Tue Jan 27 23:24:50 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 27 Jan 2009 14:24:50 -0800 Subject: [Python-Dev] pprint(iterator) References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> Message-ID: <0850D72339234C9F9CB7BF17FCAE1D78@RaymondLaptop1> [Guido van Rossum] > My only thought is that whatever you do, target Python 3.1, not 3.0.1. Of course. Do you have any thoughts on the most useful display format? What do you want to see from pprint(mydict.items())? Raymond From python at rcn.com Tue Jan 27 23:28:44 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 27 Jan 2009 14:28:44 -0800 Subject: [Python-Dev] Python 3.0.1 References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> Message-ID: [Benjamin Peterson] > It seems like we are arguing over the version number of basically the > same thing. I would like to see 3.0.1 released in early February for > nearly the reasons you name. However, it seems to me that there are > two kinds of issues: those like __cmp__ removal and some silly IO bugs > that have been fixed for a while and our waiting to be released. > There's also projects like io in c which are important, but would not > make the schedule you and I want for 3.0.1/3.1. What is involved in finishing io-in-c? ISTM, that is critical and that its absence is a serious barrier to adoption in a production environment. How far away is it? Raymond From martin at v.loewis.de Tue Jan 27 23:31:05 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 27 Jan 2009 23:31:05 +0100 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> Message-ID: <497F8B29.5030206@v.loewis.de> > My preference is to drop 3.0 entirely (no incompatable bugfix release) > and in early February release 3.1 as the real 3.x that migrators ought > to aim for and that won't have incompatable bugfix releases. Then at > PyCon, we can have a real bug day and fix-up any chips in the paint. I would fear that than 3.1 gets the same fate as 3.0. In May, we will all think "what piece of junk was that 3.1 release, let's put it to history", and replace it with 3.2. By then, users will wonder if there is ever a 3.x release that is any good. Regards, Martin From martin at v.loewis.de Tue Jan 27 23:34:52 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 27 Jan 2009 23:34:52 +0100 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> Message-ID: <497F8C0C.7090502@v.loewis.de> > The IO-in-C branch cannot be reasonably pulled in release30-maint, but it will > be ready for 3.1. Even if 3.1 is released in February? Regards, Martin From brett at python.org Tue Jan 27 23:36:57 2009 From: brett at python.org (Brett Cannon) Date: Tue, 27 Jan 2009 14:36:57 -0800 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <497F8B29.5030206@v.loewis.de> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <497F8B29.5030206@v.loewis.de> Message-ID: On Tue, Jan 27, 2009 at 14:31, "Martin v. L?wis" wrote: >> My preference is to drop 3.0 entirely (no incompatable bugfix release) >> and in early February release 3.1 as the real 3.x that migrators ought >> to aim for and that won't have incompatable bugfix releases. Then at >> PyCon, we can have a real bug day and fix-up any chips in the paint. > > I would fear that than 3.1 gets the same fate as 3.0. In May, we will > all think "what piece of junk was that 3.1 release, let's put it to > history", and replace it with 3.2. By then, users will wonder if there > is ever a 3.x release that is any good. That's my fear as well. I have no problem doing a quick 3.0.1 release any time between now and the end of February and start with the first alpha or beta of 3.1 at PyCon. -Brett From guido at python.org Tue Jan 27 23:42:59 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 27 Jan 2009 14:42:59 -0800 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <0850D72339234C9F9CB7BF17FCAE1D78@RaymondLaptop1> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <0850D72339234C9F9CB7BF17FCAE1D78@RaymondLaptop1> Message-ID: On Tue, Jan 27, 2009 at 2:24 PM, Raymond Hettinger wrote: > > [Guido van Rossum] >> >> My only thought is that whatever you do, target Python 3.1, not 3.0.1. > > Of course. > Do you have any thoughts on the most useful display format? > What do you want to see from pprint(mydict.items())? Perhaps <['a', 'b', ...]> ? The list display is familiar to everyone; the surrounding <> make it clear that it's not really a list without adding much noise. Another idea would be which helpfully includes the name of the type of the object that was passed into pprint(). Regarding range(), I wonder if we really need to show more than 'range(0, 10)' -- anything besides that would be wasteful IMO. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From janssen at parc.com Tue Jan 27 23:41:49 2009 From: janssen at parc.com (Bill Janssen) Date: Tue, 27 Jan 2009 14:41:49 PST Subject: [Python-Dev] IO performance In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <2e1434c10901271232l139018q360e662b006c0580@mail.gmail.com> Message-ID: <35189.1233096109@pippin.parc.xerox.com> Antoine Pitrou wrote: > There is an SVN branch with a complete rewrite (in C) of the IO stack. You can > find it in branches/io-c. Apart from a problem in _ssl.c, it should be quite > usable. Your tests and observations are welcome! And I'll look at that _ssl.c problem. Bill From solipsis at pitrou.net Tue Jan 27 23:44:49 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 27 Jan 2009 22:44:49 +0000 (UTC) Subject: [Python-Dev] Python 3.0.1 (io-in-c) References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> Message-ID: Raymond Hettinger rcn.com> writes: > > What is involved in finishing io-in-c? Off the top of my head: - fix the _ssl bug which prevents some tests from passing (issue #4967) - clean up io.py (and decide what to do with the remaining Python code: basically, the parts of StringIO which are implemented in Python) - of course, test in various situations, review the code, suggest possible improvements... Now here are some performance figures. Text I/O is done in utf-8 with universal newlines enabled: === I/O in C === ** Binary input ** [ 400KB ] read one unit at a time... 1.64 MB/s [ 400KB ] read 20 units at a time... 27.2 MB/s [ 400KB ] read 4096 units at a time... 845 MB/s [ 20KB ] read whole contents at once... 924 MB/s [ 400KB ] read whole contents at once... 883 MB/s [ 10MB ] read whole contents at once... 980 MB/s [ 400KB ] seek forward one unit at a time... 0.528 MB/s [ 400KB ] seek forward 1000 units at a time... 516 MB/s [ 400KB ] alternate read & seek one unit... 1.33 MB/s [ 400KB ] alternate read & seek 1000 units... 490 MB/s ** Text input ** [ 400KB ] read one unit at a time... 2.28 MB/s [ 400KB ] read 20 units at a time... 29.2 MB/s [ 400KB ] read one line at a time... 71.7 MB/s [ 400KB ] read 4096 units at a time... 97.4 MB/s [ 20KB ] read whole contents at once... 108 MB/s [ 400KB ] read whole contents at once... 112 MB/s [ 10MB ] read whole contents at once... 89.7 MB/s [ 400KB ] seek forward one unit at a time... 0.0904 MB/s [ 400KB ] seek forward 1000 units at a time... 87.4 MB/s ** Binary append ** [ 20KB ] write one unit at a time... 0.668 MB/s [ 400KB ] write 20 units at a time... 12.2 MB/s [ 400KB ] write 4096 units at a time... 722 MB/s [ 10MB ] write 1e6 units at a time... 1529 MB/s ** Text append ** [ 20KB ] write one unit at a time... 0.983 MB/s [ 400KB ] write 20 units at a time... 16 MB/s [ 400KB ] write 4096 units at a time... 236 MB/s [ 10MB ] write 1e6 units at a time... 261 MB/s ** Binary overwrite ** [ 20KB ] modify one unit at a time... 0.677 MB/s [ 400KB ] modify 20 units at a time... 12.1 MB/s [ 400KB ] modify 4096 units at a time... 382 MB/s [ 400KB ] alternate write & seek one unit... 0.212 MB/s [ 400KB ] alternate write & seek 1000 units... 173 MB/s [ 400KB ] alternate read & write one unit... 0.827 MB/s [ 400KB ] alternate read & write 1000 units... 276 MB/s ** Text overwrite ** [ 20KB ] modify one unit at a time... 0.296 MB/s [ 400KB ] modify 20 units at a time... 5.69 MB/s [ 400KB ] modify 4096 units at a time... 151 MB/s === I/O in Python (branches/py3k) === ** Binary input ** [ 400KB ] read one unit at a time... 0.174 MB/s [ 400KB ] read 20 units at a time... 3.44 MB/s [ 400KB ] read 4096 units at a time... 246 MB/s [ 20KB ] read whole contents at once... 443 MB/s [ 400KB ] read whole contents at once... 216 MB/s [ 10MB ] read whole contents at once... 274 MB/s [ 400KB ] seek forward one unit at a time... 0.188 MB/s [ 400KB ] seek forward 1000 units at a time... 182 MB/s [ 400KB ] alternate read & seek one unit... 0.0821 MB/s [ 400KB ] alternate read & seek 1000 units... 81.2 MB/s ** Text input ** [ 400KB ] read one unit at a time... 0.218 MB/s [ 400KB ] read 20 units at a time... 3.8 MB/s [ 400KB ] read one line at a time... 3.69 MB/s [ 400KB ] read 4096 units at a time... 34.9 MB/s [ 20KB ] read whole contents at once... 70.5 MB/s [ 400KB ] read whole contents at once... 81 MB/s [ 10MB ] read whole contents at once... 68.7 MB/s [ 400KB ] seek forward one unit at a time... 0.0709 MB/s [ 400KB ] seek forward 1000 units at a time... 67.3 MB/s ** Binary append ** [ 20KB ] write one unit at a time... 0.15 MB/s [ 400KB ] write 20 units at a time... 2.88 MB/s [ 400KB ] write 4096 units at a time... 346 MB/s [ 10MB ] write 1e6 units at a time... 728 MB/s ** Text append ** [ 20KB ] write one unit at a time... 0.0814 MB/s [ 400KB ] write 20 units at a time... 1.51 MB/s [ 400KB ] write 4096 units at a time... 118 MB/s [ 10MB ] write 1e6 units at a time... 218 MB/s ** Binary overwrite ** [ 20KB ] modify one unit at a time... 0.123 MB/s [ 400KB ] modify 20 units at a time... 2.34 MB/s [ 400KB ] modify 4096 units at a time... 213 MB/s [ 400KB ] alternate write & seek one unit... 0.0816 MB/s [ 400KB ] alternate write & seek 1000 units... 71.4 MB/s [ 400KB ] alternate read & write one unit... 0.0448 MB/s [ 400KB ] alternate read & write 1000 units... 41.1 MB/s ** Text overwrite ** [ 20KB ] modify one unit at a time... 0.0723 MB/s [ 400KB ] modify 20 units at a time... 1.36 MB/s [ 400KB ] modify 4096 units at a time... 88.3 MB/s Regards Antoine. From benjamin at python.org Tue Jan 27 23:48:33 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 27 Jan 2009 16:48:33 -0600 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> Message-ID: <1afaf6160901271448x2a80d172rd9b7c814e5c510c6@mail.gmail.com> On Tue, Jan 27, 2009 at 4:44 PM, Antoine Pitrou wrote: > Raymond Hettinger rcn.com> writes: >> >> What is involved in finishing io-in-c? > > Off the top of my head: > - fix the _ssl bug which prevents some tests from passing (issue #4967) > - clean up io.py (and decide what to do with the remaining Python code: > basically, the parts of StringIO which are implemented in Python) > - of course, test in various situations, review the code, suggest possible > improvements... There are also several IO bugs that should be fixed before it becomes official like #5006. > > Now here are some performance figures. Text I/O is done in utf-8 with universal > newlines enabled: -- Regards, Benjamin From daniel at stutzbachenterprises.com Tue Jan 27 23:48:38 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 27 Jan 2009 16:48:38 -0600 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> Message-ID: On Tue, Jan 27, 2009 at 4:44 PM, Antoine Pitrou wrote: > Now here are some performance figures. Text I/O is done in utf-8 with > universal > newlines enabled: > Would it be much trouble to also compare performance with Python 2.6? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Tue Jan 27 23:46:43 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 27 Jan 2009 22:46:43 +0000 (UTC) Subject: [Python-Dev] Python 3.0.1 References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F8C0C.7090502@v.loewis.de> Message-ID: Martin v. L?wis v.loewis.de> writes: > > > The IO-in-C branch cannot be reasonably pulled in release30-maint, but it will > > be ready for 3.1. > > Even if 3.1 is released in February? No, unless we take some risks and rush it in. (technically, it seems to work, but it's such a critical piece of code that it would be nice to let it rest a little) Regards Antoine. From solipsis at pitrou.net Tue Jan 27 23:54:32 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 27 Jan 2009 22:54:32 +0000 (UTC) Subject: [Python-Dev] Python 3.0.1 (io-in-c) References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> Message-ID: Daniel Stutzbach stutzbachenterprises.com> writes: > > Would it be much trouble to also compare performance with Python 2.6? Here are the results on trunk. Keep in mind Text IO, while it's still `open("r", filename)`, does not mean the same thing. === 2.7 I/O (trunk) === ** Binary input ** [ 400KB ] read one unit at a time... 1.48 MB/s [ 400KB ] read 20 units at a time... 29.2 MB/s [ 400KB ] read 4096 units at a time... 1038 MB/s [ 20KB ] read whole contents at once... 1145 MB/s [ 400KB ] read whole contents at once... 891 MB/s [ 10MB ] read whole contents at once... 966 MB/s [ 400KB ] seek forward one unit at a time... 0.893 MB/s [ 400KB ] seek forward 1000 units at a time... 568 MB/s [ 400KB ] alternate read & seek one unit... 1.11 MB/s [ 400KB ] alternate read & seek 1000 units... 563 MB/s ** Text input ** [ 400KB ] read one unit at a time... 1.41 MB/s [ 400KB ] read 20 units at a time... 28.4 MB/s [ 400KB ] read one line at a time... 207 MB/s [ 400KB ] read 4096 units at a time... 1060 MB/s [ 20KB ] read whole contents at once... 1196 MB/s [ 400KB ] read whole contents at once... 841 MB/s [ 10MB ] read whole contents at once... 966 MB/s [ 400KB ] seek forward one unit at a time... 0.873 MB/s [ 400KB ] seek forward 1000 units at a time... 589 MB/s ** Binary append ** [ 20KB ] write one unit at a time... 0.887 MB/s [ 400KB ] write 20 units at a time... 15.8 MB/s [ 400KB ] write 4096 units at a time... 1071 MB/s [ 10MB ] write 1e6 units at a time... 1523 MB/s ** Text append ** [ 20KB ] write one unit at a time... 1.33 MB/s [ 400KB ] write 20 units at a time... 22.9 MB/s [ 400KB ] write 4096 units at a time... 1244 MB/s [ 10MB ] write 1e6 units at a time... 1540 MB/s ** Binary overwrite ** [ 20KB ] modify one unit at a time... 0.867 MB/s [ 400KB ] modify 20 units at a time... 15.3 MB/s [ 400KB ] modify 4096 units at a time... 446 MB/s [ 400KB ] alternate write & seek one unit... 0.237 MB/s [ 400KB ] alternate write & seek 1000 units... 151 MB/s [ 400KB ] alternate read & write one unit... 0.221 MB/s [ 400KB ] alternate read & write 1000 units... 153 MB/s ** Text overwrite ** [ 20KB ] modify one unit at a time... 1.32 MB/s [ 400KB ] modify 20 units at a time... 22.5 MB/s [ 400KB ] modify 4096 units at a time... 509 MB/s From python at rcn.com Tue Jan 27 23:57:48 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 27 Jan 2009 14:57:48 -0800 Subject: [Python-Dev] Python 3.0.1 (io-in-c) References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com><497F6E55.6090608@v.loewis.de> <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> Message-ID: <9EEA3AE4EA4646519D89DFF7BB65C3A7@RaymondLaptop1> [Antoine Pitrou] > Now here are some performance figures. Text I/O is done in utf-8 with universal > newlines enabled: That's a substantial boost. How does it compare to Py2.x equivalents? Raymond From barry at python.org Tue Jan 27 23:59:52 2009 From: barry at python.org (Barry Warsaw) Date: Tue, 27 Jan 2009 17:59:52 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <497F8B29.5030206@v.loewis.de> Message-ID: <98A7C5B2-45BE-4335-BAA2-4698A4E971CD@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 27, 2009, at 5:36 PM, Brett Cannon wrote: > On Tue, Jan 27, 2009 at 14:31, "Martin v. L?wis" > wrote: >>> My preference is to drop 3.0 entirely (no incompatable bugfix >>> release) >>> and in early February release 3.1 as the real 3.x that migrators >>> ought >>> to aim for and that won't have incompatable bugfix releases. Then >>> at >>> PyCon, we can have a real bug day and fix-up any chips in the paint. >> >> I would fear that than 3.1 gets the same fate as 3.0. In May, we will >> all think "what piece of junk was that 3.1 release, let's put it to >> history", and replace it with 3.2. By then, users will wonder if >> there >> is ever a 3.x release that is any good. > > That's my fear as well. I have no problem doing a quick 3.0.1 release > any time between now and the end of February and start with the first > alpha or beta of 3.1 at PyCon. +1 Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSX+R6HEjvBPtnXfVAQLQBwQAuJfVHtKQRqptjl1Hlkz37RSqMnCGNE/f Fm2JmulfWbtlZgeZ+YgBMyPw2jGpmkSp/zB0aThuBNRrtcEPOnO0nFKxWwcFwBa/ ZddlM9RJvb+GgBPNOjnSXNSJcYmNLwea7GuKPkTVmkb9nH0JLOnk2dLVTGjJ89Q4 F3qsGz5coEc= =gUH4 -----END PGP SIGNATURE----- From brett at python.org Wed Jan 28 00:02:13 2009 From: brett at python.org (Brett Cannon) Date: Tue, 27 Jan 2009 15:02:13 -0800 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> Message-ID: On Tue, Jan 27, 2009 at 14:44, Antoine Pitrou wrote: > Raymond Hettinger rcn.com> writes: >> >> What is involved in finishing io-in-c? > > Off the top of my head: > - fix the _ssl bug which prevents some tests from passing (issue #4967) > - clean up io.py (and decide what to do with the remaining Python code: > basically, the parts of StringIO which are implemented in Python) The other VMs might appreciate the code being available and used if _io is not available for import. If you need help on how to have the tests run twice, once on the Python code and again on the C code, you can look at test_heapq and test_warnings for approaches. > - of course, test in various situations, review the code, suggest possible > improvements... > > Now here are some performance figures. Text I/O is done in utf-8 with universal > newlines enabled: > That is impressive! Congrats to you and (I think) Amaury for all the hard work you guys have put in. -Brett > > === I/O in C === > > ** Binary input ** > > [ 400KB ] read one unit at a time... 1.64 MB/s > [ 400KB ] read 20 units at a time... 27.2 MB/s > [ 400KB ] read 4096 units at a time... 845 MB/s > > [ 20KB ] read whole contents at once... 924 MB/s > [ 400KB ] read whole contents at once... 883 MB/s > [ 10MB ] read whole contents at once... 980 MB/s > > [ 400KB ] seek forward one unit at a time... 0.528 MB/s > [ 400KB ] seek forward 1000 units at a time... 516 MB/s > [ 400KB ] alternate read & seek one unit... 1.33 MB/s > [ 400KB ] alternate read & seek 1000 units... 490 MB/s > > ** Text input ** > > [ 400KB ] read one unit at a time... 2.28 MB/s > [ 400KB ] read 20 units at a time... 29.2 MB/s > [ 400KB ] read one line at a time... 71.7 MB/s > [ 400KB ] read 4096 units at a time... 97.4 MB/s > > [ 20KB ] read whole contents at once... 108 MB/s > [ 400KB ] read whole contents at once... 112 MB/s > [ 10MB ] read whole contents at once... 89.7 MB/s > > [ 400KB ] seek forward one unit at a time... 0.0904 MB/s > [ 400KB ] seek forward 1000 units at a time... 87.4 MB/s > > ** Binary append ** > > [ 20KB ] write one unit at a time... 0.668 MB/s > [ 400KB ] write 20 units at a time... 12.2 MB/s > [ 400KB ] write 4096 units at a time... 722 MB/s > [ 10MB ] write 1e6 units at a time... 1529 MB/s > > ** Text append ** > > [ 20KB ] write one unit at a time... 0.983 MB/s > [ 400KB ] write 20 units at a time... 16 MB/s > [ 400KB ] write 4096 units at a time... 236 MB/s > [ 10MB ] write 1e6 units at a time... 261 MB/s > > ** Binary overwrite ** > > [ 20KB ] modify one unit at a time... 0.677 MB/s > [ 400KB ] modify 20 units at a time... 12.1 MB/s > [ 400KB ] modify 4096 units at a time... 382 MB/s > > [ 400KB ] alternate write & seek one unit... 0.212 MB/s > [ 400KB ] alternate write & seek 1000 units... 173 MB/s > [ 400KB ] alternate read & write one unit... 0.827 MB/s > [ 400KB ] alternate read & write 1000 units... 276 MB/s > > ** Text overwrite ** > > [ 20KB ] modify one unit at a time... 0.296 MB/s > [ 400KB ] modify 20 units at a time... 5.69 MB/s > [ 400KB ] modify 4096 units at a time... 151 MB/s > > > === I/O in Python (branches/py3k) === > > ** Binary input ** > > [ 400KB ] read one unit at a time... 0.174 MB/s > [ 400KB ] read 20 units at a time... 3.44 MB/s > [ 400KB ] read 4096 units at a time... 246 MB/s > > [ 20KB ] read whole contents at once... 443 MB/s > [ 400KB ] read whole contents at once... 216 MB/s > [ 10MB ] read whole contents at once... 274 MB/s > > [ 400KB ] seek forward one unit at a time... 0.188 MB/s > [ 400KB ] seek forward 1000 units at a time... 182 MB/s > [ 400KB ] alternate read & seek one unit... 0.0821 MB/s > [ 400KB ] alternate read & seek 1000 units... 81.2 MB/s > > ** Text input ** > > [ 400KB ] read one unit at a time... 0.218 MB/s > [ 400KB ] read 20 units at a time... 3.8 MB/s > [ 400KB ] read one line at a time... 3.69 MB/s > [ 400KB ] read 4096 units at a time... 34.9 MB/s > > [ 20KB ] read whole contents at once... 70.5 MB/s > [ 400KB ] read whole contents at once... 81 MB/s > [ 10MB ] read whole contents at once... 68.7 MB/s > > [ 400KB ] seek forward one unit at a time... 0.0709 MB/s > [ 400KB ] seek forward 1000 units at a time... 67.3 MB/s > > ** Binary append ** > > [ 20KB ] write one unit at a time... 0.15 MB/s > [ 400KB ] write 20 units at a time... 2.88 MB/s > [ 400KB ] write 4096 units at a time... 346 MB/s > [ 10MB ] write 1e6 units at a time... 728 MB/s > > ** Text append ** > > [ 20KB ] write one unit at a time... 0.0814 MB/s > [ 400KB ] write 20 units at a time... 1.51 MB/s > [ 400KB ] write 4096 units at a time... 118 MB/s > [ 10MB ] write 1e6 units at a time... 218 MB/s > > ** Binary overwrite ** > > [ 20KB ] modify one unit at a time... 0.123 MB/s > [ 400KB ] modify 20 units at a time... 2.34 MB/s > [ 400KB ] modify 4096 units at a time... 213 MB/s > > [ 400KB ] alternate write & seek one unit... 0.0816 MB/s > [ 400KB ] alternate write & seek 1000 units... 71.4 MB/s > [ 400KB ] alternate read & write one unit... 0.0448 MB/s > [ 400KB ] alternate read & write 1000 units... 41.1 MB/s > > ** Text overwrite ** > > [ 20KB ] modify one unit at a time... 0.0723 MB/s > [ 400KB ] modify 20 units at a time... 1.36 MB/s > [ 400KB ] modify 4096 units at a time... 88.3 MB/s > > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org > From barry at python.org Wed Jan 28 00:04:01 2009 From: barry at python.org (Barry Warsaw) Date: Tue, 27 Jan 2009 18:04:01 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <497F7325.7070802@v.loewis.de> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <497F7325.7070802@v.loewis.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 27, 2009, at 3:48 PM, Martin v. L?wis wrote: >>> Releasing 3.1 6 months after 3.0 sounds reasonable; I don't think >>> it should be released earlier (else 3.0 looks fairly ridiculous). >> >> It sounds like my approval of Raymond's removal of certain >> (admittedly >> obsolete) operators from the 3.0 branch was premature. Barry at least >> thinks those should be rolled back. Others? > > I agree that not too much harm is done by removing stuff in 3.0.1 that > erroneously had been left in the 3.0 release - in particular if 3.0.1 > gets released quickly (e.g. within two months of the original > release). > > If that is an acceptable policy, then those changes would fall under > the policy. If the policy is *not* acceptable, a lot of changes to > 3.0.1 need to be rolled back (e.g. the ongoing removal of __cmp__ > fragments) I have no problem with removing things that were advertised and/or documented to be removed in 3.0 but accidentally were not. That seems like a reasonable policy to me. However, if we did not tell people that something was going to be removed, then I don't think we can really remove it in 3.0. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSX+S4nEjvBPtnXfVAQIjuQQAucsAp79ZtlcOq1GPiwDaEoYMKTEgkkNp hLgdDW85ktmFf0xHl/KAU8lcxeaiWGepefsRxsx7c5fX6UIVZPUHDvkDkf5rImx6 wg7Nin2MirLT/lXY7a8//N+5TwLqIBTLLEfAIAFvDhrQT/CuMfZej7leB7BAd7Ti puLWYYYUL+M= =pK8E -----END PGP SIGNATURE----- From janssen at parc.com Wed Jan 28 00:14:46 2009 From: janssen at parc.com (Bill Janssen) Date: Tue, 27 Jan 2009 15:14:46 PST Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> Message-ID: <35670.1233098086@pippin.parc.xerox.com> > - fix the _ssl bug which prevents some tests from passing (issue #4967) I see you've already got a patch for this. I'll try it out. Bill From python at rcn.com Wed Jan 28 00:16:48 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 27 Jan 2009 15:16:48 -0800 Subject: [Python-Dev] Python 3.0.1 References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <497F8B29.5030206@v.loewis.de> Message-ID: [Martin] > I would fear that than 3.1 gets the same fate as 3.0. In May, we will > all think "what piece of junk was that 3.1 release, let's put it to > history", and replace it with 3.2. By then, users will wonder if there > is ever a 3.x release that is any good. I thought the gist of Guido's idea was to label 3.0.1 as 3.1 to emphasize the magnitude of differences from 3.0. That seemed like a good idea to me. But I'm happy no matter what you want to call it. The important thing is that the bugfixes go in and the half-started removals get finished. I would like the next release (whatever it is called) to include the IO speedups which will help remove a barrier to adoption for serious use. I do hope the next release goes out as soon as possible. I use 3.0 daily and my impression is that the current version needs to be replaced as soon as possible. If it gets called 3.1, the nice side effect for me is that my itertools updates get fielded a bit sooner. But that is a somewhat unimportant consideration. I really have no opinion on what the next release gets called. Raymond From python at rcn.com Wed Jan 28 00:21:05 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 27 Jan 2009 15:21:05 -0800 Subject: [Python-Dev] Python 3.0.1 References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <497F7325.7070802@v.loewis.de> Message-ID: If something gets left in 3.0.1 and then ripped-out in 3.1, I think we're doing more harm than good. Very little code has been ported to 3.0 so far. One there is a base, all changes become more difficult. In the interests of our users, I vote for sooner than later. Also, 3.0 is a special case because it is IMO a broken release. AFAICT, it is not in any distro yet. Hopefully, no one will keep it around and it will vanish silently. Raymond ----- Original Message ----- I have no problem with removing things that were advertised and/or documented to be removed in 3.0 but accidentally were not. That seems like a reasonable policy to me. However, if we did not tell people that something was going to be removed, then I don't think we can really remove it in 3.0. Barry From solipsis at pitrou.net Wed Jan 28 00:25:38 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 27 Jan 2009 23:25:38 +0000 (UTC) Subject: [Python-Dev] Python 3.0.1 References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <497F7325.7070802@v.loewis.de> Message-ID: Raymond Hettinger rcn.com> writes: > > Also, 3.0 is a special case because it is IMO a broken release. > AFAICT, it is not in any distro yet. I have access to an Ubuntu 8.10 box and: $ apt-cache search python3.0 idle-python3.0 - An IDE for Python (v3.0) using Tkinter libpython3.0 - Shared Python runtime library (version 3.0) python3-all - Package depending on all supported Python runtime versions python3-all-dbg - Package depending on all supported Python debugging packages python3-all-dev - Package depending on all supported Python development packages python3-dbg - Debug Build of the Python Interpreter (version 3.0) python3.0 - An interactive high-level object-oriented language (version 3.0) python3.0-dbg - Debug Build of the Python Interpreter (version 3.0) python3.0-dev - Header files and a static library for Python (v3.0) python3.0-doc - Documentation for the high-level object-oriented language Python (v3.0) python3.0-examples - Examples for the Python language (v3.0) python3.0-minimal - A minimal subset of the Python language (version 3.0) But it's not installed by default. Regards Antoine. From daniel at stutzbachenterprises.com Wed Jan 28 00:28:53 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 27 Jan 2009 17:28:53 -0600 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> Message-ID: On Tue, Jan 27, 2009 at 4:54 PM, Antoine Pitrou wrote: > Daniel Stutzbach stutzbachenterprises.com> writes: > > Would it be much trouble to also compare performance with Python 2.6? > > Here are the results on trunk. > Thanks, Antoine! To make comparison easier, I put together the results into a Google Spreadsheet: http://spreadsheets.google.com/pub?key=pbqSxQEo4UXwPlifXmvPHGQ Keep in mind Text IO, while it's still `open("r", > filename)`, does not mean the same thing. That's because in Python 3, the Text IO has to convert to Unicode, correct? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.brandl at gmx.net Wed Jan 28 00:35:48 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 28 Jan 2009 00:35:48 +0100 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <497F8B29.5030206@v.loewis.de> Message-ID: Raymond Hettinger schrieb: > [Martin] >> I would fear that than 3.1 gets the same fate as 3.0. In May, we will >> all think "what piece of junk was that 3.1 release, let's put it to >> history", and replace it with 3.2. By then, users will wonder if there >> is ever a 3.x release that is any good. > > I thought the gist of Guido's idea was to label 3.0.1 as 3.1 to emphasize > the magnitude of differences from 3.0. That seemed like a good idea > to me. But I'm happy no matter what you want to call it. The important > thing is that the bugfixes go in and the half-started removals get finished. > I would like the next release (whatever it is called) to include the IO > speedups which will help remove a barrier to adoption for serious use. FWIW, I completely agree here. > I do hope the next release goes out as soon as possible. I use 3.0 daily > and my impression is that the current version needs to be replaced as soon > as possible. That's important to note: I do not use Python 3.x productively in any way, other than trying to port a bit of a library every now and then, and I expect that many others here are in the same position. In these matters, we should give more weight to what *actual users* like Raymond think. It's a great thing that we actually got 3.0 out, and didn't stall somewhere along the way, but the next step is to make sure it gets accepted and used, and doesn't get abandoned for a long time because of policies that come from the 2.x branch but might not be healthy for 3.x. Georg From benjamin at python.org Wed Jan 28 00:37:52 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 27 Jan 2009 17:37:52 -0600 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <497F7325.7070802@v.loewis.de> Message-ID: <1afaf6160901271537t5017f65u57c3ce09db18f83c@mail.gmail.com> On Tue, Jan 27, 2009 at 5:04 PM, Barry Warsaw wrote: > I have no problem with removing things that were advertised and/or > documented to be removed in 3.0 but accidentally were not. That seems like > a reasonable policy to me. However, if we did not tell people that > something was going to be removed, then I don't think we can really remove > it in 3.0. As others have said, this would technically include cmp() removal. In the 2.x docs, there are big warnings by the operator functions and a suggestion to use ABCs. We also already have a 2to3 fixer for the module. -- Regards, Benjamin From solipsis at pitrou.net Wed Jan 28 00:44:55 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 27 Jan 2009 23:44:55 +0000 (UTC) Subject: [Python-Dev] Python 3.0.1 (io-in-c) References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> Message-ID: Daniel Stutzbach stutzbachenterprises.com> writes: > > Thanks, Antoine!? To make comparison easier, I put together the results into a Google Spreadsheet:http://spreadsheets.google.com/pub?key=pbqSxQEo4UXwPlifXmvPHGQ Thanks, that's much more readable indeed. > That's because in Python 3, the Text IO has to convert to Unicode, correct?? Yes, exactly. Regards Antoine. From barry at python.org Wed Jan 28 00:56:36 2009 From: barry at python.org (Barry Warsaw) Date: Tue, 27 Jan 2009 18:56:36 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <497F7325.7070802@v.loewis.de> Message-ID: <2E3BD230-BE90-4E31-88D3-7844E5FFC3BC@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 27, 2009, at 6:21 PM, Raymond Hettinger wrote: > If something gets left in 3.0.1 and then ripped-out in 3.1, I think > we're > doing more harm than good. Very little code has been ported to 3.0 > so far. One there is a base, all changes become more difficult. > > In the interests of our users, I vote for sooner than later. > > Also, 3.0 is a special case because it is IMO a broken release. > AFAICT, it is not in any distro yet. Hopefully, no one will keep it > around > and it will vanish silently. I stand by my opinion about the right way to do this. I also think that a 3.1 release 6 months after 3.0 is perfectly fine and serves our users just as well. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSX+fNnEjvBPtnXfVAQJO1QQAmRVH0tslNfRfpQsC+2jlJu5uljOVvuvN uE3/HFktxLUr6NPdOk+Ir1r2p4mQ5iXFlZbJvOSNckM3UYSFkeKmS/T0nVJzqx89 +23sv7UC2Qf8zJRJBEhzuePT1iAE8OybRH1Vxql9ka8FVzCrZHt2JhnRZUmHNblT Y2d92iL7eqE= =Qzdr -----END PGP SIGNATURE----- From goodger at python.org Wed Jan 28 01:04:05 2009 From: goodger at python.org (David Goodger) Date: Tue, 27 Jan 2009 19:04:05 -0500 Subject: [Python-Dev] PyCon 2009 registration is now open! Message-ID: <4335d2c40901271604j1d86cce0yb07ff8ac7e04f8f4@mail.gmail.com> Register here: http://us.pycon.org/2009/register/ Information (rates etc.): http://us.pycon.org/2009/registration/ Hotel information & reservations: http://us.pycon.org/2009/about/hotel/ Early bird registration ends February 21, so don't delay! -- David Goodger, PyCon 2009 Chair From daniel at stutzbachenterprises.com Wed Jan 28 01:04:30 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 27 Jan 2009 18:04:30 -0600 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: References: <497F6E55.6090608@v.loewis.de> <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> Message-ID: On Tue, Jan 27, 2009 at 5:44 PM, Antoine Pitrou wrote: > Daniel Stutzbach stutzbachenterprises.com> writes: > > That's because in Python 3, the Text IO has to convert to Unicode, > correct? > > Yes, exactly. > What kind of input are you using for the Text tests? I'm kind of surprised that the conversion to Unicode results in such a dramatic slowdown, if you're feeding it plain text (characters 0x00 through 0x7f). -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Wed Jan 28 01:15:27 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 28 Jan 2009 00:15:27 +0000 (UTC) Subject: [Python-Dev] Python 3.0.1 (io-in-c) References: <497F6E55.6090608@v.loewis.de> <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> Message-ID: Daniel Stutzbach stutzbachenterprises.com> writes: > > What kind of input are you using for the Text tests?? I'm kind of surprised that the conversion to Unicode results in such a dramatic slowdown, if you're feeding it plain text (characters 0x00 through 0x7f). It's some arbitrary text composed of 95% ASCII characters and 5% non-ASCII. On this specific example, utf8 decodes at around 250 MB/s, latin1 at almost 1 GB/s (on the same machine on which I ran the benchmarks). You can find the test here: http://svn.python.org/view/sandbox/trunk/iobench/ From daniel at stutzbachenterprises.com Wed Jan 28 01:30:08 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 27 Jan 2009 18:30:08 -0600 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: References: <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> Message-ID: On Tue, Jan 27, 2009 at 6:15 PM, Antoine Pitrou wrote: > It's some arbitrary text composed of 95% ASCII characters and 5% non-ASCII. > On > this specific example, utf8 decodes at around 250 MB/s, latin1 at almost 1 > GB/s > (on the same machine on which I ran the benchmarks). > For the "10MB whole contents at once" test, we then have: (assuming the code does no pipelining of disk I/O with decoding) 10MB / 980MB/s to read from disk = 10 ms 10MB / 250MB/s to decode to utf8 = 40 ms 10MB / (10ms + 40ms) = 200 MB/s In practice, your results shows around 90 MB/s. That's at least vaguely in the same ballpark. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Wed Jan 28 01:39:37 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 28 Jan 2009 00:39:37 +0000 (UTC) Subject: [Python-Dev] Python 3.0.1 (io-in-c) References: <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> Message-ID: Daniel Stutzbach stutzbachenterprises.com> writes: > For the "10MB whole contents at once" test, we then have: > (assuming the code does no pipelining of disk I/O with decoding) > > 10MB / 980MB/s to read from disk = 10 ms > 10MB / 250MB/s to decode to utf8 = 40 ms > 10MB / (10ms + 40ms) = 200 MB/s > > In practice, your results shows around 90 MB/s. That's at least vaguely in > the same ballpark. Yes, the remaining CPU time is spent in the IncrementalNewlineDecoder (which does universal newline translation). Antoine. From steve at holdenweb.com Wed Jan 28 01:45:22 2009 From: steve at holdenweb.com (Steve Holden) Date: Tue, 27 Jan 2009 19:45:22 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <2E3BD230-BE90-4E31-88D3-7844E5FFC3BC@python.org> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <497F7325.7070802@v.loewis.de> <2E3BD230-BE90-4E31-88D3-7844E5FFC3BC@python.org> Message-ID: Barry Warsaw wrote: > On Jan 27, 2009, at 6:21 PM, Raymond Hettinger wrote: > >> If something gets left in 3.0.1 and then ripped-out in 3.1, I think we're >> doing more harm than good. Very little code has been ported to 3.0 >> so far. One there is a base, all changes become more difficult. > >> In the interests of our users, I vote for sooner than later. > >> Also, 3.0 is a special case because it is IMO a broken release. >> AFAICT, it is not in any distro yet. Hopefully, no one will keep it >> around >> and it will vanish silently. > > I stand by my opinion about the right way to do this. I also think that > a 3.1 release 6 months after 3.0 is perfectly fine and serves our users > just as well. > +1 regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From victor.stinner at haypocalc.com Wed Jan 28 01:57:15 2009 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Wed, 28 Jan 2009 01:57:15 +0100 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: <1afaf6160901271448x2a80d172rd9b7c814e5c510c6@mail.gmail.com> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> <1afaf6160901271448x2a80d172rd9b7c814e5c510c6@mail.gmail.com> Message-ID: <497FAD6B.7020608@haypocalc.com> Benjamin Peterson a ?crit : > There are also several IO bugs that should be fixed before it becomes > official like #5006. > I looked at this one, but I discovered another a bug with f.tell(): it's now issue #5008. This issue is now closed, that I will look again to #5006. See also #5016 (f.seekable() bug). Victor From matthew at matthewwilkes.co.uk Wed Jan 28 02:50:20 2009 From: matthew at matthewwilkes.co.uk (Matthew Wilkes) Date: Wed, 28 Jan 2009 01:50:20 +0000 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <2E3BD230-BE90-4E31-88D3-7844E5FFC3BC@python.org> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <497F7325.7070802@v.loewis.de> <2E3BD230-BE90-4E31-88D3-7844E5FFC3BC@python.org> Message-ID: <0959C095-CD9B-460A-B240-907A9B79696C@matthewwilkes.co.uk> On 27 Jan 2009, at 23:56, Barry Warsaw wrote: >> Also, 3.0 is a special case because it is IMO a broken release. >> AFAICT, it is not in any distro yet. Hopefully, no one will keep >> it around >> and it will vanish silently. > > I stand by my opinion about the right way to do this. I also think > that a 3.1 release 6 months after 3.0 is perfectly fine and serves > our users just as well. I'm lurking here, as I usually have nothing to contribute, but here's my take on this: I'm generally a Python 2.4 user, but have recently been able to tinker in 2.6. I hope to be using 2.6 as my main language within a year. I anticipate dropping all 2.4 projects within 5 years. We have not yet dropped 2.3. I didn't know 3.0 is considered a broken release, but teething troubles are to be expected. Knowing this, I would be reluctant to use 3.0.1, it sounds like too small a change. If you put a lot of things into a minor point release you risk setting expectations about future ones. From the 2.x series I 2.x.{y,y+1) to be seemless, but 2. {x,x+1} to be more performant, include new features and potentially break comlpex code. I personally would see a 3.1 with C based IO support as being more sensible than a 3.0.1 with lots of changes. I wouldn't worry about 3.x being seen as a dead duck, as you say it's not in wide use yet. We trust you guys, if there's been big fixes there should be a big version update. Broadcast what's been made better and it'll encourage us to try it. Matt From alexandre at peadrop.com Wed Jan 28 03:51:04 2009 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Tue, 27 Jan 2009 21:51:04 -0500 Subject: [Python-Dev] undesireable unpickle behavior, proposed fix In-Reply-To: <11495282-A381-4C98-85B0-F811ECD9A072@youtube.com> References: <2003160D-7A1C-4567-87B4-10E329E169E1@youtube.com> <497F55D2.1020805@v.loewis.de> <497F6316.60902@v.loewis.de> <497F710C.8080406@v.loewis.de> <11495282-A381-4C98-85B0-F811ECD9A072@youtube.com> Message-ID: On Tue, Jan 27, 2009 at 5:16 PM, Jake McGuire wrote: > Another vaguely related change would be to store string and unicode objects > in the pickler memo keyed as themselves rather than their object ids. That wouldn't be difficult to do--i.e., simply add a type check in Pickler.memoize and another in Pickler.save(). But I am not sure if that would be a good idea, since you would end up hashing every string pickled. And, that would probably be expensive if you are pickling for long strings. -- Alexandre From python at rcn.com Wed Jan 28 04:04:54 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 27 Jan 2009 19:04:54 -0800 Subject: [Python-Dev] Python 3.0.1 References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <497F7325.7070802@v.loewis.de> <2E3BD230-BE90-4E31-88D3-7844E5FFC3BC@python.org> <0959C095-CD9B-460A-B240-907A9B79696C@matthewwilkes.co.uk> Message-ID: [Matthew Wilkes] > I didn't know 3.0 is considered a broken release, but teething > troubles are to be expected. Knowing this, I would be reluctant to > use 3.0.1, it sounds like too small a change. Not to worry. Many of the major language features are stable and many of the rough edges are quickly getting ironed-out. Over time, anything that's slow will get optimized and all will be well. What we're discussing are subtlies of major vs minor releases. When the tp_compare change goes in, will it affect third-party C extensions enough to warrant a 3.1 name instead of 3.0.1? Are users better served by removing operator.isSequenceType() in 3.0.1 while there are still few early adopers and few converted third-party modules or will we help them more by warning them in advance and waiting for 3.1. The nice thing about the IO speedups is that the API is already set and won't change. So, the speedup doesn't really affect whether the release gets named 3.0.1 or 3.1. The important part is that we get it out as soon as it's solid so that we don't preclude adoption by users who need fast IO. Raymond From steve at holdenweb.com Wed Jan 28 04:32:04 2009 From: steve at holdenweb.com (Steve Holden) Date: Tue, 27 Jan 2009 22:32:04 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <497F7325.7070802@v.loewis.de> <2E3BD230-BE90-4E31-88D3-7844E5FFC3BC@python.org> Message-ID: Steve Holden wrote: > Barry Warsaw wrote: [...] >> I stand by my opinion about the right way to do this. I also think that >> a 3.1 release 6 months after 3.0 is perfectly fine and serves our users >> just as well. >> > +1 > I should have been more explicit. I think that stuff that was slated for removal in 3.0 should be removed as soon as possible, and a micro release is fine for that. ISTM that if we really cared about our users we would have got this right before we released 3.0. Since we clearly didn't, it behooves us make sure that any 3.1 release isn't a( repeat performance. There are changes that should clearly have been made before 3.0 saw the light of day, which are now being discussed for incorporation. If those changes were *supposed* to be made before 3.0 came out then they should be made as soon as possible. Waiting for a major release only encourages people to use them, and once they get use further changes will be seen as introducing incompatibilities that we have promised would not occur. So it seems that the operator functions should stand not on the order of their going, but depart. While a quick 3.1 release might look like the best compromise for now, it cannot then be followed with a quick 3.2 release, and then we are in the territory Martin warned about. Quality is crucial after a poor initial release: we have to engender confidence in the user base that we are not dicking them around with ill-thought-out changes. So on balance I think it might be better to live with the known inadequacies of 3.0, making small changes for 3.0.1 and possibly ignoring the policy that says we don't remove features in point releases (since they apparently should have been taken out of 3.0 but weren't). But this is only going to work if the quality of 3.1 is considerably higher than 3.0, making it worth the wait. I think that both 3.0 and 2.6 were rushed releases. 2.6 showed it in the inclusion (later recognizable as somewhat ill-advised so late in the day) of multiprocessing; 3.0 shows it in the very fact that this discussion has become necessary. So we face an important turning point: is 3.1 going to be serious production quality or not? Given that we have just been presented with a fabulous resource that could help improve delivered quality (I am talking about snakebite.org, of course) we might be well-advised to use the 3.1 release as a demonstration of how much it is going to improve the quality of delivered releases. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From Scott.Daniels at Acm.Org Wed Jan 28 08:05:32 2009 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Tue, 27 Jan 2009 23:05:32 -0800 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: <9EEA3AE4EA4646519D89DFF7BB65C3A7@RaymondLaptop1> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com><497F6E55.6090608@v.loewis.de> <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> <9EEA3AE4EA4646519D89DFF7BB65C3A7@RaymondLaptop1> Message-ID: Raymond Hettinger wrote: > > [Antoine Pitrou] >> Now here are some performance figures. Text I/O is done in utf-8 with >> universal >> newlines enabled: > > That's a substantial boost. > How does it compare to Py2.x equivalents? Comparison of three cases (including performance rations): MB/S MB/S MB/S in C in py3k in 2.7 C/3k 2.7/3k ** Binary append ** 10M write 1e6 units at a time 1529.00 728.000 1523.000 2.10 2.09 20K write one unit at a time 0.668 0.150 0.887 4.45 5.91 400K write 20 units at a time 12.200 2.880 15.800 4.24 5.49 400K write 4096 units at a time 722.00 346.000 1071.000 2.09 3.10 ** Binary input ** 10M read whole contents at once 980.00 274.000 966.000 3.58 3.53 20K read whole contents at once 924.00 443.000 1145.000 2.09 2.58 400K alternate read & seek 1000 units 490.000 81.200 563.000 6.03 6.93 400K alternate read & seek one unit 1.330 0.082 1.11 16.20 13.52 400K read 20 units at a time 27.200 3.440 29.200 7.91 8.49 400K read 4096 units at a time 845.00 246.000 1038.000 3.43 4.22 400K read one unit at a time 1.64 0.174 1.480 9.43 8.51 400K read whole contents at once 883.00 216.000 891.000 4.09 4.13 400K seek forward 1000 units a time 516.00 182.000 568.000 2.84 3.12 400K seek forward one unit at a time 0.528 0.188 0.893 2.81 4.75 ** Binary overwrite ** 20K modify one unit at a time 0.677 0.123 0.867 5.50 7.05 400K alternate read & write 1000 unit 276.000 41.100 153.000 6.72 3.72 400K alternate read & write one unit 0.827 0.045 0.22 18.46 4.93 400K alternate write & seek 1000 unit 173.000 71.400 151.000 2.42 2.11 400K alternate write & seek one unit 0.212 0.082 0.237 2.60 2.90 400K modify 20 units at a time 12.100 2.340 15.300 5.17 6.54 400K modify 4096 units at a time 382.00 213.000 446.000 1.79 2.09 ** Text append ** 10M write 1e6 units at a time 261.00 218.000 1540.000 1.20 7.06 20K write one unit at a time 0.983 0.081 1.33 12.08 16.34 400K write 20 units at a time 16.000 1.510 22.90 10.60 15.17 400K write 4096 units at a time 236.00 118.000 1244.000 2.00 10.54 ** Text input ** 10M read whole contents at once 89.700 68.700 966.000 1.31 14.06 20K read whole contents at once 108.000 70.500 1196.000 1.53 16.96 400K read 20 units at a time 29.200 3.800 28.400 7.68 7.47 400K read 4096 units at a time 97.400 34.900 1060.000 2.79 30.37 400K read one line at a time 71.700 3.690 207.00 19.43 56.10 400K read one unit at a time 2.280 0.218 1.41 10.46 6.47 400K read whole contents at once 112.000 81.000 841.000 1.38 10.38 400K seek forward 1000 units at a time 87.400 67.300 589.000 1.30 8.75 400K seek forward one unit at a time 0.090 0.071 0.873 1.28 12.31 ** Text overwrite ** 20K modify one unit at a time 0.296 0.072 1.320 4.09 18.26 400K modify 20 units at a time 5.690 1.360 22.500 4.18 16.54 400K modify 4096 units at a time 151.000 88.300 509.000 1.71 5.76 --Scott David Daniels Scott.Daniels at Acm.Org From asmodai at in-nomine.org Wed Jan 28 08:22:20 2009 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Wed, 28 Jan 2009 08:22:20 +0100 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> Message-ID: <20090128072220.GO99614@nexus.in-nomine.org> -On [20090128 00:21], Raymond Hettinger (python at rcn.com) wrote: >Also, 3.0 is a special case because it is IMO a broken release. >AFAICT, it is not in any distro yet. Hopefully, no one will keep it around >and it will vanish silently. It is in FreeBSD's ports since December. Fairly good chance it is in pkgsrc also by now. Might even be that it is part of FreeBSD's 7.1-RELEASE. So I reckon with 'distro' you were speaking of Linux only? -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Earth to earth, ashes to ashes, dust to dust... From asmodai at in-nomine.org Wed Jan 28 08:24:25 2009 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Wed, 28 Jan 2009 08:24:25 +0100 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <2E3BD230-BE90-4E31-88D3-7844E5FFC3BC@python.org> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <2E3BD230-BE90-4E31-88D3-7844E5FFC3BC@python.org> Message-ID: <20090128072425.GP99614@nexus.in-nomine.org> -On [20090128 00:57], Barry Warsaw (barry at python.org) wrote: >I stand by my opinion about the right way to do this. I also think >that a 3.1 release 6 months after 3.0 is perfectly fine and serves our >users just as well. When API fixes were mentioned, does that mean changes in the API which influence the C extension? If so, then I think a minor number update (3.1) is more warranted than a revision number update (3.0.1). -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Earth to earth, ashes to ashes, dust to dust... From tjreedy at udel.edu Wed Jan 28 08:47:54 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 28 Jan 2009 02:47:54 -0500 Subject: [Python-Dev] [PSF-Board] I've got a surprise for you! In-Reply-To: <497EF94E.3050302@holdenweb.com> References: <20090126233246.GA37662@wind.teleri.net> <497E9320.2030404@sun.com> <497EF94E.3050302@holdenweb.com> Message-ID: Steve Holden wrote: >> We now have zone servers in the OpenSolaris test farm, and >> I plan to add guest os servers in the next few weeks using >> ldoms (sparc) and xvm (x64). The zone servers provide whole >> root zones, which should be a good development environment >> for most projects. Check it out: >> >> http://test.opensolaris.org/testfarm Requires sign-in. >> http://www.opensolaris.org/os/community/testing/testfarm/zones/ Freely readable. From python at rcn.com Wed Jan 28 11:03:48 2009 From: python at rcn.com (Raymond Hettinger) Date: Wed, 28 Jan 2009 02:03:48 -0800 Subject: [Python-Dev] Python 3.0.1 (io-in-c) References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com><497F6E55.6090608@v.loewis.de> <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> <9EEA3AE4EA4646519D89DFF7BB65C3A7@RaymondLaptop1> Message-ID: <37FB8BA2BF694F2CA28843AA86FDE6D5@RaymondLaptop1> [Scott David Daniels] > Comparison of three cases (including performance rations): > MB/S MB/S MB/S > in C in py3k in 2.7 C/3k 2.7/3k > ** Text append ** > 10M write 1e6 units at a time 261.00 218.000 1540.000 1.20 7.06 > 20K write one unit at a time 0.983 0.081 1.33 12.08 16.34 > 400K write 20 units at a time 16.000 1.510 22.90 10.60 15.17 > 400K write 4096 units at a time 236.00 118.000 1244.000 2.00 10.54 Do you know why the text-appends fell off so much in the 1st and last cases? > ** Text input ** > 10M read whole contents at once 89.700 68.700 966.000 1.31 14.06 > 20K read whole contents at once 108.000 70.500 1196.000 1.53 16.96 ... > 400K read one line at a time 71.700 3.690 207.00 19.43 56.10 ... > 400K read whole contents at once 112.000 81.000 841.000 1.38 10.38 > 400K seek forward 1000 units at a time 87.400 67.300 589.000 1.30 8.75 > 400K seek forward one unit at a time 0.090 0.071 0.873 1.28 12.31 Looks like most of these still have substantial falloffs in performance. Is this part still a work in progress or is this as good as its going to get? > ** Text overwrite ** > 20K modify one unit at a time 0.296 0.072 1.320 4.09 18.26 > 400K modify 20 units at a time 5.690 1.360 22.500 4.18 16.54 > 400K modify 4096 units at a time 151.000 88.300 509.000 1.71 5.76 Same question on this batch. Raymond From solipsis at pitrou.net Wed Jan 28 11:55:16 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 28 Jan 2009 10:55:16 +0000 (UTC) Subject: [Python-Dev] Python 3.0.1 (io-in-c) References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com><497F6E55.6090608@v.loewis.de> <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> <9EEA3AE4EA4646519D89DFF7BB65C3A7@RaymondLaptop1> <37FB8BA2BF694F2CA28843AA86FDE6D5@RaymondLaptop1> Message-ID: Hello, Raymond Hettinger rcn.com> writes: > > > MB/S MB/S MB/S > > in C in py3k in 2.7 C/3k 2.7/3k > > ** Text append ** > > 10M write 1e6 units at a time 261.00 218.000 1540.000 1.20 7.06 > > 20K write one unit at a time 0.983 0.081 1.33 12.08 16.34 > > 400K write 20 units at a time 16.000 1.510 22.90 10.60 15.17 > > 400K write 4096 units at a time 236.00 118.000 1244.000 2.00 10.54 > > Do you know why the text-appends fell off so much in the 1st and last cases? When writing large chunks of text (4096, 1e6), bookkeeping costs become marginal and encoding costs dominate. 2.x has no encoding costs, which explains why it's so much faster. A quick test tells me utf-8 encoding runs at 280 MB/s. on this dataset (the 400KB text file). You see that there is not much left to optimize on large writes. > > ** Text input ** > > 10M read whole contents at once 89.700 68.700 966.000 1.31 14.06 > > 20K read whole contents at once 108.000 70.500 1196.000 1.53 16.96 > ... > > 400K read one line at a time 71.700 3.690 207.00 19.43 56.10 > ... > > 400K read whole contents at once 112.000 81.000 841.000 1.38 10.38 > > 400K seek forward 1000 units at a time 87.400 67.300 589.000 1.30 8.75 > > 400K seek forward one unit at a time 0.090 0.071 0.873 1.28 12.31 > > Looks like most of these still have substantial falloffs in performance. > Is this part still a work in progress or is this as good as its going to get? There is nothing left obvious to optimize in the read() department. Decoding and newline translation costs dominate. Decoding has already been optimized for the most popular encodings in py3k: http://mail.python.org/pipermail/python-checkins/2009-January/077024.html Newline translation follows a fast path depending on various heuristics. I also took particular care of the "read one line at a time" scenario because it's the most likely idiom when reading a text file. I think there is hardly anything left to optimize on this one. Your eyes are welcome, though. Note that the benchmark is run with the following default settings for text I/O: utf-8 encoding, universal newlines enabled, text containing only "\n" newlines. You can play with settings here: http://svn.python.org/view/sandbox/trunk/iobench/ Text seek() and tell(), on the other hand, is known to be slow, and it could perhaps be improved. It is assumed, however, that they won't be used a lot for text files. > > ** Text overwrite ** > > 20K modify one unit at a time 0.296 0.072 1.320 4.09 18.26 > > 400K modify 20 units at a time 5.690 1.360 22.500 4.18 16.54 > > 400K modify 4096 units at a time 151.000 88.300 509.000 1.71 5.76 > > Same question on this batch. There seems to be some additional overhead in this case. Perhaps it could be improved, I'll have to take a look... But I doubt overwriting chunks of text is a common scenario. Regards Antoine. From p.f.moore at gmail.com Wed Jan 28 12:19:50 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 28 Jan 2009 11:19:50 +0000 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: References: <497F6E55.6090608@v.loewis.de> <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> <9EEA3AE4EA4646519D89DFF7BB65C3A7@RaymondLaptop1> <37FB8BA2BF694F2CA28843AA86FDE6D5@RaymondLaptop1> Message-ID: <79990c6b0901280319q7a9ee669w758d4d1ae765b3d3@mail.gmail.com> 2009/1/28 Antoine Pitrou : > When writing large chunks of text (4096, 1e6), bookkeeping costs become > marginal and encoding costs dominate. 2.x has no encoding costs, which > explains why it's so much faster. Interesting. However, it's still "slower" in terms of perception. In 2.x, I regularly do the equivalent of f = open("filename", "r") ... read strings from f ... Yes, I know this is byte I/O in reality, but for everything I do (Latin-1 on input and output, and for most practical purposes ASCII-only) it simply isn't relevant to me. If Python 3.x makes this substantially slower (working in a naive mode where I ignore encoding issues), claiming it's "encoding costs" doesn't make any difference - in a practical sense, I don't get any benefits and yet I pay the cost. (You can say my approach is wrong, but so what? I'll just say that 2.x is faster for me, and not migrate. Ultimately, this is about "marketing" 3.x...) It would be helpful to limit this cost as much as possible - maybe that's simply ensuring that the default encoding for open is (in the majority of cases) a highly-optimised one whose costs *don't* dominate in the way you describe (although if you're using UTF-8, I'd guess that would be the usual default on Linux, so it looks like there's some work needed there). Hmm, I just checked and on Windows, it appears that sys.getdefaultencoding() is UTF-8. That seems odd - I would have thought the majority of Windows systems were NOT set to use UTF-8 by default... Paul. From victor.stinner at haypocalc.com Wed Jan 28 12:22:21 2009 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Wed, 28 Jan 2009 12:22:21 +0100 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: References: <37FB8BA2BF694F2CA28843AA86FDE6D5@RaymondLaptop1> Message-ID: <200901281222.21611.victor.stinner@haypocalc.com> Le Wednesday 28 January 2009 11:55:16 Antoine Pitrou, vous avez ?crit?: > 2.x has no encoding costs, which explains why it's so much faster. Why not testing io.open() or codecs.open() which create unicode strings? -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ From solipsis at pitrou.net Wed Jan 28 12:39:22 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 28 Jan 2009 11:39:22 +0000 (UTC) Subject: [Python-Dev] Python 3.0.1 (io-in-c) References: <497F6E55.6090608@v.loewis.de> <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> <9EEA3AE4EA4646519D89DFF7BB65C3A7@RaymondLaptop1> <37FB8BA2BF694F2CA28843AA86FDE6D5@RaymondLaptop1> <79990c6b0901280319q7a9ee669w758d4d1ae765b3d3@mail.gmail.com> Message-ID: Paul Moore gmail.com> writes: > > It would be helpful to limit this cost as much as possible - maybe > that's simply ensuring that the default encoding for open is (in the > majority of cases) a highly-optimised one whose costs *don't* dominate > in the way you describe As I pointed out, utf-8, utf-16 and latin1 decoders have already been optimized in py3k. For *pure ASCII* input, utf-8 decoding is blazingly fast (1GB/s here). The dataset for iobench isn't pure ASCII though, and that's why it's not as fast. People are invited to test their own workloads with the io-c branch and report performance figures (and possible bugs). There are so many possibilities that the benchmark figures given by a generic tool can only be indicative. Regards Antoine. From solipsis at pitrou.net Wed Jan 28 12:41:07 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 28 Jan 2009 11:41:07 +0000 (UTC) Subject: [Python-Dev] Python 3.0.1 (io-in-c) References: <37FB8BA2BF694F2CA28843AA86FDE6D5@RaymondLaptop1> <200901281222.21611.victor.stinner@haypocalc.com> Message-ID: Victor Stinner haypocalc.com> writes: > > Le Wednesday 28 January 2009 11:55:16 Antoine Pitrou, vous avez ?crit?: > > 2.x has no encoding costs, which explains why it's so much faster. > > Why not testing io.open() or codecs.open() which create unicode strings? The goal is to test the idiomatic way of opening text files (the "one obvious way to do it", if you want). There is no doubt that io.open() and codecs.open() in 2.x are much slower than the io-c branch. However, nobody is expecting very good performance from io.open() and codecs.open() in 2.x either. Regards Antoine. From p.f.moore at gmail.com Wed Jan 28 13:10:29 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 28 Jan 2009 12:10:29 +0000 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: References: <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> <9EEA3AE4EA4646519D89DFF7BB65C3A7@RaymondLaptop1> <37FB8BA2BF694F2CA28843AA86FDE6D5@RaymondLaptop1> <79990c6b0901280319q7a9ee669w758d4d1ae765b3d3@mail.gmail.com> Message-ID: <79990c6b0901280410p68a77baakca66dce118912df3@mail.gmail.com> 2009/1/28 Antoine Pitrou : > Paul Moore gmail.com> writes: >> >> It would be helpful to limit this cost as much as possible - maybe >> that's simply ensuring that the default encoding for open is (in the >> majority of cases) a highly-optimised one whose costs *don't* dominate >> in the way you describe > > As I pointed out, utf-8, utf-16 and latin1 decoders have already been optimized > in py3k. For *pure ASCII* input, utf-8 decoding is blazingly fast (1GB/s here). > The dataset for iobench isn't pure ASCII though, and that's why it's not as fast. Ah, thanks. Although you said your data was 95% ASCII, and you're getting decode speeds of 250MB/s. That's 75% slowdown for 5% of the data! Surely that's not right??? > People are invited to test their own workloads with the io-c branch and report > performance figures (and possible bugs). There are so many possibilities that > the benchmark figures given by a generic tool can only be indicative. At the moment, I don't have the time to download and build the branch, and in any case as I only have Visual Studio Express, I don't get the PGO optimisations, making any tests I do highly suspect. Paul. PS Can anyone comment on why Python defaults to utf-8 on Windows? That seems like a highly suspect default... From victor.stinner at haypocalc.com Wed Jan 28 13:14:37 2009 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Wed, 28 Jan 2009 13:14:37 +0100 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: References: <200901281222.21611.victor.stinner@haypocalc.com> Message-ID: <200901281314.37816.victor.stinner@haypocalc.com> Le Wednesday 28 January 2009 12:41:07 Antoine Pitrou, vous avez ?crit?: > > Why not testing io.open() or codecs.open() which create unicode strings? > > There is no doubt that io.open() and codecs.open() in 2.x are much slower > than the io-c branch. However, nobody is expecting very good performance > from io.open() and codecs.open() in 2.x either. I use codecs.open() in my programs and so I'm interested by the benchmark on this function ;-) But if I understand correctly, Python (3.1 ?) will be faster (or much faster) to read/write files in unicode, and that's a great news ;-) -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ From l.oluyede at gmail.com Wed Jan 28 16:46:41 2009 From: l.oluyede at gmail.com (Lawrence Oluyede) Date: Wed, 28 Jan 2009 16:46:41 +0100 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <497F7325.7070802@v.loewis.de> <2E3BD230-BE90-4E31-88D3-7844E5FFC3BC@python.org> Message-ID: <9eebf5740901280746p2af7b15ci4d37fef40b57bf1f@mail.gmail.com> On Wed, Jan 28, 2009 at 4:32 AM, Steve Holden wrote: > I think that both 3.0 and 2.6 were rushed releases. 2.6 showed it in the > inclusion (later recognizable as somewhat ill-advised so late in the > day) of multiprocessing; 3.0 shows it in the very fact that this > discussion has become necessary. What about some kine of mechanism to "triage" 3rd party modules? Something like: module gains popularity -> the core team decides it's worthy -> the module is included in the library in some kind of "contrib"/"ext" package (like the future mechanism) and for one major release stays in that package (so developers don't have to rush fixing _all_ the bugs they can while making a major release) -> after (at least) one major release the module moves up one level and it's considered stable and rock solid. Meanwhile the documentation must say that the 3rd party contributed module is not considered production ready, though usable, until the release current + 1 I don't know if it feasible, if it's insane or what, it's just an idea I had. -- Lawrence, http://oluyede.org - http://twitter.com/lawrenceoluyede "It is difficult to get a man to understand something when his salary depends on not understanding it" - Upton Sinclair From solipsis at pitrou.net Wed Jan 28 17:23:19 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 28 Jan 2009 16:23:19 +0000 (UTC) Subject: [Python-Dev] Python 3.0.1 (io-in-c) References: <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> <9EEA3AE4EA4646519D89DFF7BB65C3A7@RaymondLaptop1> <37FB8BA2BF694F2CA28843AA86FDE6D5@RaymondLaptop1> <79990c6b0901280319q7a9ee669w758d4d1ae765b3d3@mail.gmail.com> <79990c6b0901280410p68a77baakca66dce118912df3@mail.gmail.com> Message-ID: Paul Moore gmail.com> writes: > > > > As I pointed out, utf-8, utf-16 and latin1 decoders have already been optimized > > in py3k. For *pure ASCII* input, utf-8 decoding is blazingly fast (1GB/s here). > > The dataset for iobench isn't pure ASCII though, and that's why it's not as fast. > > Ah, thanks. Although you said your data was 95% ASCII, and you're > getting decode speeds of 250MB/s. That's 75% slowdown for 5% of the > data! Surely that's not right??? If you look at how utf-8 decoding is implemented (in unicodeobject.c), it's quite obvious why it is so :-) There is a (very) fast path for chunks of pure ASCII data, and (fast but not blazingly fast) fallback for non ASCII data. Please don't think of it as a slowdown... It's still much faster than 2.x, which manages 130MB/s on the same data. Regards Antoine. From p.f.moore at gmail.com Wed Jan 28 17:54:49 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 28 Jan 2009 16:54:49 +0000 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: References: <9EEA3AE4EA4646519D89DFF7BB65C3A7@RaymondLaptop1> <37FB8BA2BF694F2CA28843AA86FDE6D5@RaymondLaptop1> <79990c6b0901280319q7a9ee669w758d4d1ae765b3d3@mail.gmail.com> <79990c6b0901280410p68a77baakca66dce118912df3@mail.gmail.com> Message-ID: <79990c6b0901280854s17ad8f06ldf9c894b5d00e455@mail.gmail.com> 2009/1/28 Antoine Pitrou : > If you look at how utf-8 decoding is implemented (in unicodeobject.c), it's > quite obvious why it is so :-) There is a (very) fast path for chunks of pure > ASCII data, and (fast but not blazingly fast) fallback for non ASCII data. Thanks for the explanation. > Please don't think of it as a slowdown... It's still much faster than 2.x, which > manages 130MB/s on the same data. Don't get me wrong - I'm hugely grateful for this work. And personally, I don't expect that I/O speed is ever likely to be a real bottleneck in the type of program I write. But I'm concerned that (much as with the whole "Python 3.0 is incompatible, and it will be hard to port to" meme) people will pick up on raw benchmark figures - no matter how much they aren't comparing like with like - and start making it sound like "Python 3.0 I/O is slower than 2.x" - which is a great disservice to the good work that's been done. I do think it's worth taking care over the default encoding, though. Quite apart from performance, getting "correct" behaviour is important. I can't speak for Unix, but on Windows, the following behaviour feels like a bug to me: >echo a?b >a1 >python Python 2.6.1 (r261:67517, Dec 4 2008, 16:51:00) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> print open("a1").read() a?b >>> ^Z >\Apps\Python30\python.exe Python 3.0 (r30:67507, Dec 3 2008, 20:14:27) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> print(open("a1").read()) Traceback (most recent call last): File " ", line 1, in File "D:\Apps\Python30\lib\io.py", line 1491, in write b = encoder.encode(s) File "D:\Apps\Python30\lib\encodings\cp850.py", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_map)[0] UnicodeEncodeError: 'charmap' codec can't encode character '\u0153' in position 1: character maps to >>> ^Z >chcp Active code page: 850 Paul. From solipsis at pitrou.net Wed Jan 28 18:04:18 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 28 Jan 2009 18:04:18 +0100 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: <79990c6b0901280854s17ad8f06ldf9c894b5d00e455@mail.gmail.com> References: <9EEA3AE4EA4646519D89DFF7BB65C3A7@RaymondLaptop1> <37FB8BA2BF694F2CA28843AA86FDE6D5@RaymondLaptop1> <79990c6b0901280319q7a9ee669w758d4d1ae765b3d3@mail.gmail.com> <79990c6b0901280410p68a77baakca66dce118912df3@mail.gmail.com> <79990c6b0901280854s17ad8f06ldf9c894b5d00e455@mail.gmail.com> Message-ID: <1233162258.6489.4.camel@fsol> Le mercredi 28 janvier 2009 ? 16:54 +0000, Paul Moore a ?crit : > I do think it's worth taking care over the default encoding, though. > Quite apart from performance, getting "correct" behaviour is > important. I can't speak for Unix, but on Windows, the following > behaviour feels like a bug to me: [...] Please open a bug :) cheers Antoine. From mal at egenix.com Wed Jan 28 18:51:08 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 28 Jan 2009 18:51:08 +0100 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> Message-ID: <49809B0C.4020905@egenix.com> On 2009-01-27 22:19, Raymond Hettinger wrote: > From: ""Martin v. L?wis"" >> Releasing 3.1 6 months after 3.0 sounds reasonable; I don't think >> it should be released earlier (else 3.0 looks fairly ridiculous). > > I think it should be released earlier and completely supplant 3.0 > before more third-party developers spend time migrating code. > We needed 3.0 to get released so we could get the feedback > necessary to shake it out. Now, it is time for it to fade into history > and take advantage of the lessons learned. > > The principles for the 2.x series don't really apply here. In 2.x, there > was always a useful, stable, clean release already fielded and there > were tons of third-party apps that needed a slow rate of change. > > In contrast, 3.0 has a near zero installed user base (at least in terms > of being used in production). It has very few migrated apps. It is > not particularly clean and some of the work for it was incomplete > when it was released. > > My preference is to drop 3.0 entirely (no incompatable bugfix release) > and in early February release 3.1 as the real 3.x that migrators ought > to aim for and that won't have incompatable bugfix releases. Then at > PyCon, we can have a real bug day and fix-up any chips in the paint. > > If 3.1 goes out right away, then it doesn't matter if 3.0 looks ridiculous. > All eyes go to the latest release. Better to get this done before more > people download 3.0 to kick the tires. Why don't we just mark 3.0.x as experimental branch and keep updating/ fixing things that were not sorted out for the 3.0.0 release ?! I think that's a fair approach, given that the only way to get field testing for new open-source software is to release early and often. A 3.1 release should then be the first stable release of the 3.x series and mark the start of the usual deprecation mechanisms we have in the 2.x series. Needless to say, that rushing 3.1 out now would only cause yet another experimental release... major releases do take time to stabilize. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 28 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From martin at v.loewis.de Wed Jan 28 18:55:31 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 28 Jan 2009 18:55:31 +0100 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: <79990c6b0901280410p68a77baakca66dce118912df3@mail.gmail.com> References: <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> <9EEA3AE4EA4646519D89DFF7BB65C3A7@RaymondLaptop1> <37FB8BA2BF694F2CA28843AA86FDE6D5@RaymondLaptop1> <79990c6b0901280319q7a9ee669w758d4d1ae765b3d3@mail.gmail.com> <79990c6b0901280410p68a77baakca66dce118912df3@mail.gmail.com> Message-ID: <49809C13.7070208@v.loewis.de> > PS Can anyone comment on why Python defaults to utf-8 on Windows? Don't panic. It doesn't, and you are misinterpreting what you are seeing. Regards, Martin From fuzzyman at voidspace.org.uk Wed Jan 28 18:55:39 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 28 Jan 2009 17:55:39 +0000 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <49809B0C.4020905@egenix.com> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> Message-ID: <49809C1B.3090805@voidspace.org.uk> M.-A. Lemburg wrote: > On 2009-01-27 22:19, Raymond Hettinger wrote: > >> From: ""Martin v. L?wis"" >> >>> Releasing 3.1 6 months after 3.0 sounds reasonable; I don't think >>> it should be released earlier (else 3.0 looks fairly ridiculous). >>> >> I think it should be released earlier and completely supplant 3.0 >> before more third-party developers spend time migrating code. >> We needed 3.0 to get released so we could get the feedback >> necessary to shake it out. Now, it is time for it to fade into history >> and take advantage of the lessons learned. >> >> The principles for the 2.x series don't really apply here. In 2.x, there >> was always a useful, stable, clean release already fielded and there >> were tons of third-party apps that needed a slow rate of change. >> >> In contrast, 3.0 has a near zero installed user base (at least in terms >> of being used in production). It has very few migrated apps. It is >> not particularly clean and some of the work for it was incomplete >> when it was released. >> >> My preference is to drop 3.0 entirely (no incompatable bugfix release) >> and in early February release 3.1 as the real 3.x that migrators ought >> to aim for and that won't have incompatable bugfix releases. Then at >> PyCon, we can have a real bug day and fix-up any chips in the paint. >> >> If 3.1 goes out right away, then it doesn't matter if 3.0 looks ridiculous. >> All eyes go to the latest release. Better to get this done before more >> people download 3.0 to kick the tires. >> > > Why don't we just mark 3.0.x as experimental branch and keep updating/ > fixing things that were not sorted out for the 3.0.0 release ?! I think > that's a fair approach, given that the only way to get field testing > for new open-source software is to release early and often. > > A 3.1 release should then be the first stable release of the 3.x series > and mark the start of the usual deprecation mechanisms we have > in the 2.x series. Needless to say, that rushing 3.1 out now would > only cause yet another experimental release... major releases do take > time to stabilize. > > +1 I don't think we do users any favours by being cautious in removing / fixing things in the 3.0 releases. Michael Foord -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From martin at v.loewis.de Wed Jan 28 18:59:33 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 28 Jan 2009 18:59:33 +0100 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: <79990c6b0901280319q7a9ee669w758d4d1ae765b3d3@mail.gmail.com> References: <497F6E55.6090608@v.loewis.de> <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> <9EEA3AE4EA4646519D89DFF7BB65C3A7@RaymondLaptop1> <37FB8BA2BF694F2CA28843AA86FDE6D5@RaymondLaptop1> <79990c6b0901280319q7a9ee669w758d4d1ae765b3d3@mail.gmail.com> Message-ID: <49809D05.4040005@v.loewis.de> Paul Moore wrote: > Hmm, I just checked and on Windows, it > appears that sys.getdefaultencoding() is UTF-8. That seems odd - I > would have thought the majority of Windows systems were NOT set to use > UTF-8 by default... In Python 3, sys.getdefaultencoding() is "utf-8" on all platforms, just as it was "ascii" in 2.x, on all platforms. The default encoding isn't used for I/O; check f.encoding to find out what encoding is used to read the file you are reading. Regards, Martin From martin at v.loewis.de Wed Jan 28 19:01:27 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 28 Jan 2009 19:01:27 +0100 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: <79990c6b0901280854s17ad8f06ldf9c894b5d00e455@mail.gmail.com> References: <9EEA3AE4EA4646519D89DFF7BB65C3A7@RaymondLaptop1> <37FB8BA2BF694F2CA28843AA86FDE6D5@RaymondLaptop1> <79990c6b0901280319q7a9ee669w758d4d1ae765b3d3@mail.gmail.com> <79990c6b0901280410p68a77baakca66dce118912df3@mail.gmail.com> <79990c6b0901280854s17ad8f06ldf9c894b5d00e455@mail.gmail.com> Message-ID: <49809D77.7010703@v.loewis.de> >>>> print(open("a1").read()) > Traceback (most recent call last): > File " ", line 1, in > File "D:\Apps\Python30\lib\io.py", line 1491, in write > b = encoder.encode(s) > File "D:\Apps\Python30\lib\encodings\cp850.py", line 19, in encode > return codecs.charmap_encode(input,self.errors,encoding_map)[0] > UnicodeEncodeError: 'charmap' codec can't encode character '\u0153' in > position 1: character maps to Looks right to me. Martin From p.f.moore at gmail.com Wed Jan 28 19:17:16 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 28 Jan 2009 18:17:16 +0000 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: <49809D05.4040005@v.loewis.de> References: <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> <9EEA3AE4EA4646519D89DFF7BB65C3A7@RaymondLaptop1> <37FB8BA2BF694F2CA28843AA86FDE6D5@RaymondLaptop1> <79990c6b0901280319q7a9ee669w758d4d1ae765b3d3@mail.gmail.com> <49809D05.4040005@v.loewis.de> Message-ID: <79990c6b0901281017t569bf594l2f66e5aee6cff4e2@mail.gmail.com> 2009/1/28 "Martin v. L?wis" : > Paul Moore wrote: >> Hmm, I just checked and on Windows, it >> appears that sys.getdefaultencoding() is UTF-8. That seems odd - I >> would have thought the majority of Windows systems were NOT set to use >> UTF-8 by default... > > In Python 3, sys.getdefaultencoding() is "utf-8" on all platforms, just > as it was "ascii" in 2.x, on all platforms. The default encoding isn't > used for I/O; check f.encoding to find out what encoding is used to > read the file you are reading. Thanks for the explanation. It might be clearer to document this a little more explicitly in the docs for open() (on the basis that people using open() are the most likely to be naive about encodings). I'll see if I can come up with an appropriate doc patch. Paul. From p.f.moore at gmail.com Wed Jan 28 19:20:08 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 28 Jan 2009 18:20:08 +0000 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: <49809D77.7010703@v.loewis.de> References: <37FB8BA2BF694F2CA28843AA86FDE6D5@RaymondLaptop1> <79990c6b0901280319q7a9ee669w758d4d1ae765b3d3@mail.gmail.com> <79990c6b0901280410p68a77baakca66dce118912df3@mail.gmail.com> <79990c6b0901280854s17ad8f06ldf9c894b5d00e455@mail.gmail.com> <49809D77.7010703@v.loewis.de> Message-ID: <79990c6b0901281020ue5c609br3cd7f84ffdf44411@mail.gmail.com> 2009/1/28 "Martin v. L?wis" : >>>>> print(open("a1").read()) >> Traceback (most recent call last): >> File " ", line 1, in >> File "D:\Apps\Python30\lib\io.py", line 1491, in write >> b = encoder.encode(s) >> File "D:\Apps\Python30\lib\encodings\cp850.py", line 19, in encode >> return codecs.charmap_encode(input,self.errors,encoding_map)[0] >> UnicodeEncodeError: 'charmap' codec can't encode character '\u0153' in >> position 1: character maps to > > Looks right to me. I don't see why. I wrote the file from the console (cp850), read it in Python using the default encoding (which I would expect to match the console encoding), wrote it to sys.stdout (which I would expect to use the console encoding). How did the character end up not being encodable, when I've only used one encoding throughout? (And if my assumptions about the encodings used are wrong at some point, that's what I'm suggesting is the error). Paul. From martin at v.loewis.de Wed Jan 28 19:29:07 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 28 Jan 2009 19:29:07 +0100 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: <79990c6b0901281017t569bf594l2f66e5aee6cff4e2@mail.gmail.com> References: <1afaf6160901271348g48d3bc72t546821ad2feeed76@mail.gmail.com> <9EEA3AE4EA4646519D89DFF7BB65C3A7@RaymondLaptop1> <37FB8BA2BF694F2CA28843AA86FDE6D5@RaymondLaptop1> <79990c6b0901280319q7a9ee669w758d4d1ae765b3d3@mail.gmail.com> <49809D05.4040005@v.loewis.de> <79990c6b0901281017t569bf594l2f66e5aee6cff4e2@mail.gmail.com> Message-ID: <4980A3F3.6060005@v.loewis.de> > Thanks for the explanation. It might be clearer to document this a > little more explicitly in the docs for open() (on the basis that > people using open() are the most likely to be naive about encodings). > I'll see if I can come up with an appropriate doc patch. Notice that the determination of the specific encoding used is fairly elaborate: - if IO is to a terminal, Python tries to determine the encoding of the terminal. This is mostly relevant for Windows (which uses, by default, the "OEM code page" in the terminal). - if IO is to a file, Python tries to guess the "common" encoding for the system. On Unix, it queries the locale, and falls back to "ascii" if no locale is set. On Windows, it uses the "ANSI code page". On OSX, it uses the "system encoding". - if IO is binary, (clearly) no encoding is used. Network IO is always binary. - for file names, yet different algorithms apply. On Windows, it uses the Unicode API, so no need for an encoding. On Unix, it (again) uses the locale encoding. On OSX, it uses UTF-8 (just to be clear: this applies to the first argument of open(), not to the resulting file object) Regards, Martin From steven.bethard at gmail.com Wed Jan 28 19:40:22 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 28 Jan 2009 10:40:22 -0800 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: <4980A3F3.6060005@v.loewis.de> References: <9EEA3AE4EA4646519D89DFF7BB65C3A7@RaymondLaptop1> <37FB8BA2BF694F2CA28843AA86FDE6D5@RaymondLaptop1> <79990c6b0901280319q7a9ee669w758d4d1ae765b3d3@mail.gmail.com> <49809D05.4040005@v.loewis.de> <79990c6b0901281017t569bf594l2f66e5aee6cff4e2@mail.gmail.com> <4980A3F3.6060005@v.loewis.de> Message-ID: On Wed, Jan 28, 2009 at 10:29 AM, "Martin v. L?wis" wrote: > Notice that the determination of the specific encoding used is fairly > elaborate: > - if IO is to a terminal, Python tries to determine the encoding of > the terminal. This is mostly relevant for Windows (which uses, > by default, the "OEM code page" in the terminal). > - if IO is to a file, Python tries to guess the "common" encoding > for the system. On Unix, it queries the locale, and falls back > to "ascii" if no locale is set. On Windows, it uses the "ANSI > code page". On OSX, it uses the "system encoding". > - if IO is binary, (clearly) no encoding is used. Network IO is > always binary. > - for file names, yet different algorithms apply. On Windows, it > uses the Unicode API, so no need for an encoding. On Unix, it > (again) uses the locale encoding. On OSX, it uses UTF-8 > (just to be clear: this applies to the first argument of open(), > not to the resulting file object) This a very helpful explanation. Is it in the docs somewhere, or if it isn't, could it be? Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From martin at v.loewis.de Wed Jan 28 19:43:03 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Wed, 28 Jan 2009 19:43:03 +0100 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: <79990c6b0901281020ue5c609br3cd7f84ffdf44411@mail.gmail.com> References: <37FB8BA2BF694F2CA28843AA86FDE6D5@RaymondLaptop1> <79990c6b0901280319q7a9ee669w758d4d1ae765b3d3@mail.gmail.com> <79990c6b0901280410p68a77baakca66dce118912df3@mail.gmail.com> <79990c6b0901280854s17ad8f06ldf9c894b5d00e455@mail.gmail.com> <49809D77.7010703@v.loewis.de> <79990c6b0901281020ue5c609br3cd7f84ffdf44411@mail.gmail.com> Message-ID: <4980A737.9080509@v.loewis.de> Paul Moore wrote: > 2009/1/28 "Martin v. L?wis" : >>>>>> print(open("a1").read()) >>> Traceback (most recent call last): >>> File " ", line 1, in >>> File "D:\Apps\Python30\lib\io.py", line 1491, in write >>> b = encoder.encode(s) >>> File "D:\Apps\Python30\lib\encodings\cp850.py", line 19, in encode >>> return codecs.charmap_encode(input,self.errors,encoding_map)[0] >>> UnicodeEncodeError: 'charmap' codec can't encode character '\u0153' in >>> position 1: character maps to >> Looks right to me. > > I don't see why. I wrote the file from the console (cp850), read it in > Python using the default encoding (which I would expect to match the > console encoding), wrote it to sys.stdout (which I would expect to use > the console encoding). > > How did the character end up not being encodable, when I've only used > one encoding throughout? (And if my assumptions about the encodings > used are wrong at some point, that's what I'm suggesting is the > error). Well, first try to understand what the error *is*: py> unicodedata.name('\u0153') 'LATIN SMALL LIGATURE OE' py> unicodedata.name('?') 'POUND SIGN' py> ascii('?') "'\\xa3'" py> ascii('?'.encode('cp850').decode('cp1252')) "'\\u0153'" So when Python reads the file, it uses cp1252. This is sensible - just that the console uses cp850 doesn't change the fact that the "common" encoding of files on your system is cp1252. It is an unfortunate fact of Windows that the console window uses a different encoding from the rest of the system (namely, the console uses the OEM code page, and everything else uses the ANSI code page). Furthermore, U+0153 does not exist in cp850 (i.e. the terminal doesn't support ?), hence the exception. Regards, Martin From martin at v.loewis.de Wed Jan 28 19:46:41 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Wed, 28 Jan 2009 19:46:41 +0100 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: References: <9EEA3AE4EA4646519D89DFF7BB65C3A7@RaymondLaptop1> <37FB8BA2BF694F2CA28843AA86FDE6D5@RaymondLaptop1> <79990c6b0901280319q7a9ee669w758d4d1ae765b3d3@mail.gmail.com> <49809D05.4040005@v.loewis.de> <79990c6b0901281017t569bf594l2f66e5aee6cff4e2@mail.gmail.com> <4980A3F3.6060005@v.loewis.de> Message-ID: <4980A811.4070900@v.loewis.de> > This a very helpful explanation. Is it in the docs somewhere, or if it > isn't, could it be? I actually don't know. Regards, Martin From p.f.moore at gmail.com Wed Jan 28 19:52:41 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 28 Jan 2009 18:52:41 +0000 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: <4980A737.9080509@v.loewis.de> References: <79990c6b0901280319q7a9ee669w758d4d1ae765b3d3@mail.gmail.com> <79990c6b0901280410p68a77baakca66dce118912df3@mail.gmail.com> <79990c6b0901280854s17ad8f06ldf9c894b5d00e455@mail.gmail.com> <49809D77.7010703@v.loewis.de> <79990c6b0901281020ue5c609br3cd7f84ffdf44411@mail.gmail.com> <4980A737.9080509@v.loewis.de> Message-ID: <79990c6b0901281052q23634382y7fffa767ac32d81b@mail.gmail.com> 2009/1/28 "Martin v. L?wis" : > Well, first try to understand what the error *is*: > > py> unicodedata.name('\u0153') > 'LATIN SMALL LIGATURE OE' > py> unicodedata.name('?') > 'POUND SIGN' > py> ascii('?') > "'\\xa3'" > py> ascii('?'.encode('cp850').decode('cp1252')) > "'\\u0153'" > > So when Python reads the file, it uses cp1252. This is sensible - just > that the console uses cp850 doesn't change the fact that the "common" > encoding of files on your system is cp1252. It is an unfortunate fact > of Windows that the console window uses a different encoding from the > rest of the system (namely, the console uses the OEM code page, and > everything else uses the ANSI code page). Ah, I see. That is entirely obvious. The key bit of information is that the default io encoding is cp1252, not cp850. I know that in theory, I see the consequences often enough (:-)), but it isn't "instinctive" for me. And the simple "default encoding is system dependent" comment is not very helpful in terms of warning me that there could be an issue. I do think that more wording around encoding defaults would be useful - as I said, I'll think about how best it could be made into a doc patch. I suspect the best approach would be to have a section (maybe in the docs for the codecs module) explaining all the details, and then a cross-reference to that from the various places (open, io) where default encodings are mentioned. Paul. > > Furthermore, U+0153 does not exist in cp850 (i.e. the terminal doesn't > support ?), hence the exception. > > Regards, > Martin > From tjreedy at udel.edu Wed Jan 28 20:03:31 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 28 Jan 2009 14:03:31 -0500 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: References: <9EEA3AE4EA4646519D89DFF7BB65C3A7@RaymondLaptop1> <37FB8BA2BF694F2CA28843AA86FDE6D5@RaymondLaptop1> <79990c6b0901280319q7a9ee669w758d4d1ae765b3d3@mail.gmail.com> <49809D05.4040005@v.loewis.de> <79990c6b0901281017t569bf594l2f66e5aee6cff4e2@mail.gmail.com> <4980A3F3.6060005@v.loewis.de> Message-ID: Steven Bethard wrote: > On Wed, Jan 28, 2009 at 10:29 AM, "Martin v. L?wis" wrote: >> Notice that the determination of the specific encoding used is fairly >> elaborate: >> - if IO is to a terminal, Python tries to determine the encoding of >> the terminal. This is mostly relevant for Windows (which uses, >> by default, the "OEM code page" in the terminal). >> - if IO is to a file, Python tries to guess the "common" encoding >> for the system. On Unix, it queries the locale, and falls back >> to "ascii" if no locale is set. On Windows, it uses the "ANSI >> code page". On OSX, it uses the "system encoding". >> - if IO is binary, (clearly) no encoding is used. Network IO is >> always binary. >> - for file names, yet different algorithms apply. On Windows, it >> uses the Unicode API, so no need for an encoding. On Unix, it >> (again) uses the locale encoding. On OSX, it uses UTF-8 >> (just to be clear: this applies to the first argument of open(), >> not to the resulting file object) > > This a very helpful explanation. Is it in the docs somewhere, or if it > isn't, could it be? Here is the current entry on encodings in the Lib ref, built-in types, file objects. file.encoding The encoding that this file uses. When strings are written to a file, they will be converted to byte strings using this encoding. In addition, when the file is connected to a terminal, the attribute gives the encoding that the terminal is likely to use (that information might be incorrect if the user has misconfigured the terminal). The attribute is read-only and may not be present on all file-like objects. It may also be None, in which case the file uses the system default encoding for converting strings. From exarkun at divmod.com Wed Jan 28 20:17:46 2009 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Wed, 28 Jan 2009 14:17:46 -0500 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: <79990c6b0901281052q23634382y7fffa767ac32d81b@mail.gmail.com> Message-ID: <20090128191746.24460.683659615.divmod.quotient.3337@henry.divmod.com> On Wed, 28 Jan 2009 18:52:41 +0000, Paul Moore wrote: >2009/1/28 "Martin v. L?wis" : >> Well, first try to understand what the error *is*: >> >> py> unicodedata.name('\u0153') >> 'LATIN SMALL LIGATURE OE' >> py> unicodedata.name('?') >> 'POUND SIGN' >> py> ascii('?') >> "'\\xa3'" >> py> ascii('?'.encode('cp850').decode('cp1252')) >> "'\\u0153'" >> >> So when Python reads the file, it uses cp1252. This is sensible - just >> that the console uses cp850 doesn't change the fact that the "common" >> encoding of files on your system is cp1252. It is an unfortunate fact >> of Windows that the console window uses a different encoding from the >> rest of the system (namely, the console uses the OEM code page, and >> everything else uses the ANSI code page). > >Ah, I see. That is entirely obvious. The key bit of information is >that the default io encoding is cp1252, not cp850. I know that in >theory, I see the consequences often enough (:-)), but it isn't >"instinctive" for me. And the simple "default encoding is system >dependent" comment is not very helpful in terms of warning me that >there could be an issue. It probably didn't help that the exception raised told you that the error was in the "charmap" codec. This should have said "cp850" instead. The fact that cp850 is implemented in terms of "charmap" isn't very interesting. The fact that while encoding some text using "cp850" is. Jean-Paul From tjreedy at udel.edu Wed Jan 28 20:21:46 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 28 Jan 2009 14:21:46 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <49809C1B.3090805@voidspace.org.uk> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <49809C1B.3090805@voidspace.org.uk> Message-ID: Michael Foord wrote: > M.-A. Lemburg wrote: >> Why don't we just mark 3.0.x as experimental branch and keep updating/ >> fixing things that were not sorted out for the 3.0.0 release ?! I think >> that's a fair approach, given that the only way to get field testing >> for new open-source software is to release early and often. >> >> A 3.1 release should then be the first stable release of the 3.x series >> and mark the start of the usual deprecation mechanisms we have >> in the 2.x series. Needless to say, that rushing 3.1 out now would >> only cause yet another experimental release... major releases do take >> time to stabilize. >> >> > +1 > > I don't think we do users any favours by being cautious in removing / > fixing things in the 3.0 releases. I have two main reactions to 3.0. 1. It is great for my purpose -- coding algorithms. The cleaner object and text models are a mental relief for me. So it was a service to me to release it. I look forward to it becoming standard Python and have made my small contribution by helping clean up the 3.0 version of the docs. 2. It is something of a trial run that it should be fixed as soon as possible. I seem to remember sometning from Shakespear(?) "If it twer done, tis best it twer done quickly". Guido said something over a year ago to the effect that he did not expect 3.0 to be used as a production release, so I do not think it should to treated as one. Label it developmental and people will not try to keep in use for years and years in the way that, say, 2.4 still is. tjr From rhamph at gmail.com Wed Jan 28 20:42:30 2009 From: rhamph at gmail.com (Adam Olsen) Date: Wed, 28 Jan 2009 12:42:30 -0700 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: <79990c6b0901281052q23634382y7fffa767ac32d81b@mail.gmail.com> References: <79990c6b0901280319q7a9ee669w758d4d1ae765b3d3@mail.gmail.com> <79990c6b0901280410p68a77baakca66dce118912df3@mail.gmail.com> <79990c6b0901280854s17ad8f06ldf9c894b5d00e455@mail.gmail.com> <49809D77.7010703@v.loewis.de> <79990c6b0901281020ue5c609br3cd7f84ffdf44411@mail.gmail.com> <4980A737.9080509@v.loewis.de> <79990c6b0901281052q23634382y7fffa767ac32d81b@mail.gmail.com> Message-ID: On Wed, Jan 28, 2009 at 11:52 AM, Paul Moore wrote: > Ah, I see. That is entirely obvious. The key bit of information is > that the default io encoding is cp1252, not cp850. I know that in > theory, I see the consequences often enough (:-)), but it isn't > "instinctive" for me. And the simple "default encoding is system > dependent" comment is not very helpful in terms of warning me that > there could be an issue. > > I do think that more wording around encoding defaults would be useful > - as I said, I'll think about how best it could be made into a doc > patch. I suspect the best approach would be to have a section (maybe > in the docs for the codecs module) explaining all the details, and > then a cross-reference to that from the various places (open, io) > where default encodings are mentioned. It'd also help if the file repr gave the encoding: >>> f = open('/dev/null') >>> f >>> import sys >>> sys.stdout Of course I can check .encoding manually, but it needs to be more intuitive. -- Adam Olsen, aka Rhamphoryncus From daniel at stutzbachenterprises.com Wed Jan 28 20:59:13 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Wed, 28 Jan 2009 13:59:13 -0600 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: References: <79990c6b0901280410p68a77baakca66dce118912df3@mail.gmail.com> <79990c6b0901280854s17ad8f06ldf9c894b5d00e455@mail.gmail.com> <49809D77.7010703@v.loewis.de> <79990c6b0901281020ue5c609br3cd7f84ffdf44411@mail.gmail.com> <4980A737.9080509@v.loewis.de> <79990c6b0901281052q23634382y7fffa767ac32d81b@mail.gmail.com> Message-ID: On Wed, Jan 28, 2009 at 1:42 PM, Adam Olsen wrote: > It'd also help if the file repr gave the encoding: > +1 -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at rcn.com Wed Jan 28 21:08:43 2009 From: python at rcn.com (Raymond Hettinger) Date: Wed, 28 Jan 2009 12:08:43 -0800 Subject: [Python-Dev] Python 3.0.1 (io-in-c) References: <79990c6b0901280410p68a77baakca66dce118912df3@mail.gmail.com> <79990c6b0901280854s17ad8f06ldf9c894b5d00e455@mail.gmail.com><49809D77.7010703@v.loewis.de><79990c6b0901281020ue5c609br3cd7f84ffdf44411@mail.gmail.com><4980A737.9080509@v.loewis.de><79990c6b0901281052q23634382y7fffa767ac32d81b@mail.gmail.com> Message-ID: [Adam Olsen] > It'd also help if the file repr gave the encoding: +1 from me too. That will be a big help. Raymond From steve at holdenweb.com Wed Jan 28 22:19:32 2009 From: steve at holdenweb.com (Steve Holden) Date: Wed, 28 Jan 2009 16:19:32 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <49809C1B.3090805@voidspace.org.uk> Message-ID: Terry Reedy wrote: > Michael Foord wrote: >> M.-A. Lemburg wrote: > >>> Why don't we just mark 3.0.x as experimental branch and keep updating/ >>> fixing things that were not sorted out for the 3.0.0 release ?! I think >>> that's a fair approach, given that the only way to get field testing >>> for new open-source software is to release early and often. >>> >>> A 3.1 release should then be the first stable release of the 3.x series >>> and mark the start of the usual deprecation mechanisms we have >>> in the 2.x series. Needless to say, that rushing 3.1 out now would >>> only cause yet another experimental release... major releases do take >>> time to stabilize. >>> >>> >> +1 >> >> I don't think we do users any favours by being cautious in removing / >> fixing things in the 3.0 releases. > > I have two main reactions to 3.0. > > 1. It is great for my purpose -- coding algorithms. > The cleaner object and text models are a mental relief for me. > So it was a service to me to release it. > I look forward to it becoming standard Python and have made my small > contribution by helping clean up the 3.0 version of the docs. > > 2. It is something of a trial run that it should be fixed as soon as > possible. I seem to remember sometning from Shakespear(?) "If it twer > done, tis best it twer done quickly". > > Guido said something over a year ago to the effect that he did not > expect 3.0 to be used as a production release, so I do not think it > should to treated as one. Label it developmental and people will not > try to keep in use for years and years in the way that, say, 2.4 still is. > It might also be a good idea to take the download link off the front page of python.org: until that happens newbies are going to keep coming along and downloading it "because it's the newest". regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From ncoghlan at gmail.com Wed Jan 28 22:24:13 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 29 Jan 2009 07:24:13 +1000 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <497F7325.7070802@v.loewis.de> <2E3BD230-BE90-4E31-88D3-7844E5FFC3BC@python.org> Message-ID: <4980CCFD.2050605@gmail.com> Steve Holden wrote: > 2.6 showed it in the > inclusion (later recognizable as somewhat ill-advised so late in the > day) of multiprocessing; Given the longstanding fork() bugs that were fixed as a result of that inclusion, I think that ill-advised is too strong... could it have done with a little more time to bed down multiprocessing in particular? Possibly. Was it worth holding up the whole release just for that? I don't think so - we'd already fixed up the problems that the test suite and python-dev were likely to find, so the cost/benefit ratio on a delay would have been pretty poor. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From martin at v.loewis.de Wed Jan 28 22:26:18 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 28 Jan 2009 22:26:18 +0100 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <49809C1B.3090805@voidspace.org.uk> Message-ID: <4980CD7A.6000506@v.loewis.de> > It might also be a good idea to take the download link off the front > page of python.org: until that happens newbies are going to keep coming > along and downloading it "because it's the newest". It was (and probably still is) Guido's position that 3.0 *is* the version that newbies should be using. Regards, Martin From p.f.moore at gmail.com Thu Jan 29 00:33:59 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 28 Jan 2009 23:33:59 +0000 Subject: [Python-Dev] Python 3.0.1 (io-in-c) In-Reply-To: References: <79990c6b0901280854s17ad8f06ldf9c894b5d00e455@mail.gmail.com> <49809D77.7010703@v.loewis.de> <79990c6b0901281020ue5c609br3cd7f84ffdf44411@mail.gmail.com> <4980A737.9080509@v.loewis.de> <79990c6b0901281052q23634382y7fffa767ac32d81b@mail.gmail.com> Message-ID: <79990c6b0901281533x3918052wbb2a1e866338def4@mail.gmail.com> 2009/1/28 Raymond Hettinger : > [Adam Olsen] >> >> It'd also help if the file repr gave the encoding: > > +1 from me too. That will be a big help. Definitely. People *are* going to get confused by encoding errors - let's give them all the help we can. Paul From stephen at xemacs.org Thu Jan 29 01:59:34 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 29 Jan 2009 09:59:34 +0900 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <4980CD7A.6000506@v.loewis.de> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <49809C1B.3090805@voidspace.org.uk> <4980CD7A.6000506@v.loewis.de> Message-ID: <87d4e7dj0p.fsf@xemacs.org> "Martin v. L?wis" writes: > > It might also be a good idea to take the download link off the front > > page of python.org: until that happens newbies are going to keep coming > > along and downloading it "because it's the newest". > > It was (and probably still is) Guido's position that 3.0 *is* the > version that newbies should be using. Indeed. See Terry Reedy's post. Somebody who is looking for a platform for a production application is not going to download something "because it's the newest". Sure, those advocating other platforms will carp about Python 3.0, but hey, where is Perl 6? "The amazing thing about a dancing bear is *not* how well it dances." Let's not get too worried about the PR aspects; just fixing the bugs as we go along will fix that to the extent that people are not totally prejudiced anyway. I think there is definitely something to the notion that the 3.x vs. 3.0.y distinction should signal something, and I personally like MAL's suggestion that 3.0.x should be marked some sort of beta in perpetuity, or at least until 3.1 is ready to ship as stable and production-ready. (That's AIUI, MAL's intent may be somewhat different.) From tjreedy at udel.edu Thu Jan 29 04:22:46 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 28 Jan 2009 22:22:46 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <87d4e7dj0p.fsf@xemacs.org> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <49809C1B.3090805@voidspace.org.uk> <4980CD7A.6000506@v.loewis.de> <87d4e7dj0p.fsf@xemacs.org> Message-ID: Stephen J. Turnbull wrote: > "Martin v. L?wis" writes: > > > It might also be a good idea to take the download link off the front > > > page of python.org: until that happens newbies are going to keep coming > > > along and downloading it "because it's the newest". By that logic, I would suggest removing 2.6 ;-) See below. > > > > It was (and probably still is) Guido's position that 3.0 *is* the > > version that newbies should be using. > > Indeed. See Terry Reedy's post. When people ask on c.l.p, I recommend either 3.0 for the relative cleanliness or 2.5 (until now, at least) for the 3rd-party add-on availability (that will gradually improve for both 2.6 and more slowly, for 3.x). I expect that some newbies would find 2.6 a somewhat confusing mix of old and new. tjr From steve at holdenweb.com Thu Jan 29 04:38:31 2009 From: steve at holdenweb.com (Steve Holden) Date: Wed, 28 Jan 2009 22:38:31 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <49809C1B.3090805@voidspace.org.uk> <4980CD7A.6000506@v.loewis.de> <87d4e7dj0p.fsf@xemacs.org> Message-ID: <498124B7.1010204@holdenweb.com> Terry Reedy wrote: > Stephen J. Turnbull wrote: >> "Martin v. L?wis" writes: >> > > It might also be a good idea to take the download link off the front >> > > page of python.org: until that happens newbies are going to keep >> coming >> > > along and downloading it "because it's the newest". > > By that logic, I would suggest removing 2.6 ;-) > See below. > >> > > It was (and probably still is) Guido's position that 3.0 *is* the >> > version that newbies should be using. >> >> Indeed. See Terry Reedy's post. > > When people ask on c.l.p, I recommend either 3.0 for the relative > cleanliness or 2.5 (until now, at least) for the 3rd-party add-on > availability (that will gradually improve for both 2.6 and more slowly, > for 3.x). I expect that some newbies would find 2.6 a somewhat > confusing mix of old and new. > Fair point. At least we both agree that the current site doesn't best serve the punters. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Thu Jan 29 04:38:31 2009 From: steve at holdenweb.com (Steve Holden) Date: Wed, 28 Jan 2009 22:38:31 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <49809C1B.3090805@voidspace.org.uk> <4980CD7A.6000506@v.loewis.de> <87d4e7dj0p.fsf@xemacs.org> Message-ID: <498124B7.1010204@holdenweb.com> Terry Reedy wrote: > Stephen J. Turnbull wrote: >> "Martin v. L?wis" writes: >> > > It might also be a good idea to take the download link off the front >> > > page of python.org: until that happens newbies are going to keep >> coming >> > > along and downloading it "because it's the newest". > > By that logic, I would suggest removing 2.6 ;-) > See below. > >> > > It was (and probably still is) Guido's position that 3.0 *is* the >> > version that newbies should be using. >> >> Indeed. See Terry Reedy's post. > > When people ask on c.l.p, I recommend either 3.0 for the relative > cleanliness or 2.5 (until now, at least) for the 3rd-party add-on > availability (that will gradually improve for both 2.6 and more slowly, > for 3.x). I expect that some newbies would find 2.6 a somewhat > confusing mix of old and new. > Fair point. At least we both agree that the current site doesn't best serve the punters. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From mal at egenix.com Thu Jan 29 09:56:52 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 29 Jan 2009 09:56:52 +0100 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <87d4e7dj0p.fsf@xemacs.org> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <49809C1B.3090805@voidspace.org.uk> <4980CD7A.6000506@v.loewis.de> <87d4e7dj0p.fsf@xemacs.org> Message-ID: <49816F54.20501@egenix.com> On 2009-01-29 01:59, Stephen J. Turnbull wrote: > I think there is definitely something to the notion that the 3.x > vs. 3.0.y distinction should signal something, and I personally like > MAL's suggestion that 3.0.x should be marked some sort of beta in > perpetuity, or at least until 3.1 is ready to ship as stable and > production-ready. (That's AIUI, MAL's intent may be somewhat > different.) That's basically it, yes. I don't think that marking 3.0 as experimental is bad in any way, as long as we're clear about it. Having lots of incompatible changes in a patch level release will start to get users worrying about the stability of the 3.x branch anyway, so a heads-up message and clear perspective for the 3.1 release is a lot better than dumping 3.0 altogether or not providing such a perspective at all. That said, we should stick to the statement already made for 3.0 (too early as it now appears), ie. that the same development and releases processes will apply to the 3.x branch as we have for 2.x - starting with 3.1. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 29 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From ajp at eutechnyx.com Thu Jan 29 11:38:26 2009 From: ajp at eutechnyx.com (Dr Andrew Perella) Date: Thu, 29 Jan 2009 10:38:26 -0000 Subject: [Python-Dev] python breakpoint opcode Message-ID: <014f01c981fd$b7bb3c60$2731b520$@com> Hi, I was thinking of adding a breakpoint opcode to python to enable less invasive debugging. I came across posts from 1999 by Vladimir Marangozov and Christian Tismer discussing this issue but the links to the code are all out of date. Did anything come of this? Is this a good approach to take? - if so why was this never incorporated? Cheers, Andrew Dr. Andrew Perella Chief Software Architect Eutechnyx Limited. Metro Centre East Business Park, Waterside Drive, Gateshead, Tyne & Wear NE11 9HU UK Co.Reg.No. 2172322 T +44 (0) 191 460 6060 F +44 (0) 191 460 2266 E ajp at eutechnyx.com W www.eutechnyx.com This e-mail is confidential and may be privileged. It may be read, copied and used only by the intended recipient. No communication sent by e-mail to or from Eutechnyx is intended to give rise to contractual or other legal liability, apart from liability which cannot be excluded under English law. This e-mail is confidential and may be privileged. It may be read, copied and used only by the intended recipient. No communication sent by e-mail to or from Eutechnyx is intended to give rise to contractual or other legal liability, apart from liability which cannot be excluded under English law. This email has been scanned for all known viruses by the Email Protection Agency. http://www.epagency.net www.eutechnyx.com Eutechnyx Limited. Registered in England No: 2172322 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcea at jcea.es Thu Jan 29 13:38:29 2009 From: jcea at jcea.es (Jesus Cea) Date: Thu, 29 Jan 2009 13:38:29 +0100 Subject: [Python-Dev] mlockall() in Python? In-Reply-To: References: Message-ID: <4981A345.5030004@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Evgeni Golov wrote: > I'd like to write a small daemon in Python, which should never be > swapped out (on Linux, this daemon will be Linux specific, so no need > in a platform-independent solution). > > In C I'd do: > #include > mlockall(MCL_FUTURE); > //do stuff here > munlockall(); > > Is there anything similar in Python? I would like things like this added to core python, but since you are restricting yourself to linux, you can use a (trivial) ctypes wrapper. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSYGjQZlgi5GaxT1NAQIUMAP/Tl7SWFgVkeeEdRHbkrtlOX4eERbfny7A xBkUVO72lPB1XnRxZT0+Vo2ggYh/6IHN6SQriEZZPe9Wwn3cZzirjjAqpdvb70TJ 1BezGtLKsoDp4cf6QqDwfITecMaGjfaNhKvvSvPFzaKlpbjsdQjyGCI0dOvxzY5J 6BUxE2yYJdc= =N0dO -----END PGP SIGNATURE----- From facundobatista at gmail.com Thu Jan 29 13:50:14 2009 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 29 Jan 2009 10:50:14 -0200 Subject: [Python-Dev] Examples in Py2 or Py3 Message-ID: Hi! In the Python Argentina mail list there's already people passing examples and asking help about Python 3. This introduces the problem that some examples are in Py2 and others are in Py3. Sometimes this is not explicit, and gets confusing. I'm trying to avoid this confusion when preparing my own examples. So far, I use (py3) as a prefix for any example block, like: (Py3k) >>> (some example) (some result) Is there any recommended way to avoid confusion in these cases? (I'm thinking about changing the prompt in my Python installation, to something like ">2>>" and ">3>>", to be explicit about it... but I wanted to know if there's another better way) Thanks. -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From orsenthil at gmail.com Thu Jan 29 13:57:24 2009 From: orsenthil at gmail.com (Senthil Kumaran) Date: Thu, 29 Jan 2009 18:27:24 +0530 Subject: [Python-Dev] Examples in Py2 or Py3 In-Reply-To: References: Message-ID: <7c42eba10901290457y4f7ce3d2g79e84ebb58870f1f@mail.gmail.com> > Facundo Batista wrote: > Hi! > > In the Python Argentina mail list there's already people passing > examples and asking help about Python 3. For complete snippets: #!/usr/bin/env python3.0 vs #!/usr/bin/env python2.6 And for blocks of code # this for python 3.0 # this is for python 2.6 I know, it is very rudimentary, but I have followed snippets with written these identifications. -- Senthil From facundobatista at gmail.com Thu Jan 29 14:00:02 2009 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 29 Jan 2009 11:00:02 -0200 Subject: [Python-Dev] Examples in Py2 or Py3 In-Reply-To: <7c42eba10901290457y4f7ce3d2g79e84ebb58870f1f@mail.gmail.com> References: <7c42eba10901290457y4f7ce3d2g79e84ebb58870f1f@mail.gmail.com> Message-ID: 2009/1/29 Senthil Kumaran : > And for blocks of code > > # this for python 3.0 > # this is for python 2.6 Too much work, ;) Seriously, most probably people will forgot to add that after the third example... -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From ben at redfrontdoor.org Thu Jan 29 15:12:02 2009 From: ben at redfrontdoor.org (Ben North) Date: Thu, 29 Jan 2009 14:12:02 +0000 Subject: [Python-Dev] Partial function application 'from the right' Message-ID: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> Hi, I find 'functools.partial' useful, but occasionally I'm unable to use it because it lacks a 'from the right' version. E.g., to create a function which splits a string on commas, you can't say # Won't work when called: split_comma = partial(str.split, sep = ',') and to create a 'log to base 10' function, you can't say # Won't work when called: log_10 = partial(math.log, base = 10.0) because str.split and math.log don't take keyword arguments. PEP-309, which introduces functools.partial, mentions For completeness, another object that appends partial arguments after those supplied in the function call (maybe called rightcurry) has been suggested. 'Completeness' by itself doesn't seem to have been a compelling reason to introduce this feature, but the above cases show that it would be of real use. I've created a patch which adds a 'partial_right' function. The two examples above: >>> import functools, math >>> split_comma = functools.partial_right(str.split, ',') >>> split_comma('a,b,c') ['a', 'b', 'c'] >>> log_10 = functools.partial_right(math.log, 10.0) >>> log_10(100.0) 2.0 and a general illustrative one: >>> def all_args(*args): return args ... >>> functools.partial_right(all_args, 1, 2)(3, 4) (3, 4, 1, 2) I was prompted to look at this by a recent message on python-dev: Xavier Morel , Thu, 22 Jan 2009 14:44:41 +0100: > [...] drop(iterable, n) has to be written islice(iterable, n, None) > (and of course the naming isn't ideal), and you can't really use > functools.partial since the iterator is the first argument (unless > there's a way to partially apply only the tail args without kwargs). Xavier's case becomes: >>> import functools, itertools >>> drop = functools.partial_right(itertools.islice, None, None) >>> list(drop(range(10), 5)) [5, 6, 7, 8, 9] The patch adds a 'from_right' member to partial objects, which can be True for the new from-the-right behaviour, or False for the existing from-the-left behaviour. It's quite small, only c.40 lines, plus a c.70-line change to test_functools.py. I imagine a documention patch would be c.20 lines. Would there be any interest in this? Ben. From aahz at pythoncraft.com Thu Jan 29 15:20:21 2009 From: aahz at pythoncraft.com (Aahz) Date: Thu, 29 Jan 2009 06:20:21 -0800 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> Message-ID: <20090129142021.GA8996@panix.com> On Tue, Jan 27, 2009, Raymond Hettinger wrote: > > It is becoming the norm in 3.x for functions to return iterators, > generators, or views whereever possible. > > I had a thought that pprint() ought to be taught to print iterators: > > pprint(enumerate(seq)) > pprint(map(somefunc, somedata)) > pprint(permutations(elements)) > pprint(mydict.items()) Along the lines of what others have said: pprint() cannot consume an unknown iterator. Therefore, you can pretty up the existing output slightly or special-case certain known iterators. There might also be an API change to pprint() that allowed it to consume iterators. The reason I'm chiming in is that I would welcome a PEP that created a __pprint__ method as an alternative to special-casing. I think that it would be generically useful for user-created objects, plus once you've added this feature other people can easily do some of the grunt work of extending this through the Python core. (Actually, unless someone objects, I don't think a PEP is required, but it would be good for the usual reasons that PEPs are written, to provide a central place documenting the addition.) This can also be done for Python 2.7, too. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From fuzzyman at voidspace.org.uk Thu Jan 29 15:22:47 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 29 Jan 2009 14:22:47 +0000 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <20090129142021.GA8996@panix.com> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> Message-ID: <4981BBB7.50502@voidspace.org.uk> Aahz wrote: > On Tue, Jan 27, 2009, Raymond Hettinger wrote: > >> It is becoming the norm in 3.x for functions to return iterators, >> generators, or views whereever possible. >> >> I had a thought that pprint() ought to be taught to print iterators: >> >> pprint(enumerate(seq)) >> pprint(map(somefunc, somedata)) >> pprint(permutations(elements)) >> pprint(mydict.items()) >> > > Along the lines of what others have said: pprint() cannot consume an > unknown iterator. Therefore, you can pretty up the existing output > slightly or special-case certain known iterators. There might also be an > API change to pprint() that allowed it to consume iterators. > > The reason I'm chiming in is that I would welcome a PEP that created a > __pprint__ method as an alternative to special-casing. I think that it > would be generically useful for user-created objects, plus once you've > added this feature other people can easily do some of the grunt work of > extending this through the Python core. (Actually, unless someone > objects, I don't think a PEP is required, but it would be good for the > usual reasons that PEPs are written, to provide a central place > documenting the addition.) > > This can also be done for Python 2.7, too. > Don't we have a pretty-print API - and isn't it spelled __str__ ? Michael Foord -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From aahz at pythoncraft.com Thu Jan 29 17:06:18 2009 From: aahz at pythoncraft.com (Aahz) Date: Thu, 29 Jan 2009 08:06:18 -0800 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <4981BBB7.50502@voidspace.org.uk> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> <4981BBB7.50502@voidspace.org.uk> Message-ID: <20090129160617.GA21013@panix.com> On Thu, Jan 29, 2009, Michael Foord wrote: > Aahz wrote: >> On Tue, Jan 27, 2009, Raymond Hettinger wrote: >> >>> It is becoming the norm in 3.x for functions to return iterators, >>> generators, or views whereever possible. >>> >>> I had a thought that pprint() ought to be taught to print iterators: >>> >>> pprint(enumerate(seq)) >>> pprint(map(somefunc, somedata)) >>> pprint(permutations(elements)) >>> pprint(mydict.items()) >>> >> >> The reason I'm chiming in is that I would welcome a PEP that created a >> __pprint__ method as an alternative to special-casing. I think that it >> would be generically useful for user-created objects, plus once you've >> added this feature other people can easily do some of the grunt work of >> extending this through the Python core. (Actually, unless someone >> objects, I don't think a PEP is required, but it would be good for the >> usual reasons that PEPs are written, to provide a central place >> documenting the addition.) > > Don't we have a pretty-print API - and isn't it spelled __str__ ? In theory, yes. In practice, we wouldn't be having this discussion if that really worked. But it probably would make sense to see how far using __str__ can take us -- AFAICT enumobject.c doesn't define __str__ (although I may be missing something, I don't know Python at the C level very well). -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From rdmurray at bitdance.com Thu Jan 29 17:17:50 2009 From: rdmurray at bitdance.com (rdmurray at bitdance.com) Date: Thu, 29 Jan 2009 11:17:50 -0500 (EST) Subject: [Python-Dev] Examples in Py2 or Py3 In-Reply-To: References: Message-ID: On Thu, 29 Jan 2009 at 10:50, Facundo Batista wrote: > This introduces the problem that some examples are in Py2 and others > are in Py3. Sometimes this is not explicit, and gets confusing. I'm > trying to avoid this confusion when preparing my own examples. So far, > I use (py3) as a prefix for any example block, like: > > (Py3k) >>>> (some example) > (some result) > > Is there any recommended way to avoid confusion in these cases? (I'm > thinking about changing the prompt in my Python installation, to > something like ">2>>" and ">3>>", to be explicit about it... but I > wanted to know if there's another better way) My suggestion would be to run the examples in the interpreter shell to validate them before posting, and just cut and paste the banner along with the example: Python 2.6.1 (r261:67515, Jan 7 2009, 17:09:13) [GCC 4.3.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> print "hello world" hello world Python 3.0 (r30:67503, Dec 18 2008, 19:09:30) [GCC 4.3.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> print("hello world") hello world A bit noisier, but not much more work than cutting and pasting the example without the banner :) --RDM From phd at phd.pp.ru Thu Jan 29 17:23:17 2009 From: phd at phd.pp.ru (Oleg Broytmann) Date: Thu, 29 Jan 2009 19:23:17 +0300 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <20090129160617.GA21013@panix.com> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> <4981BBB7.50502@voidspace.org.uk> <20090129160617.GA21013@panix.com> Message-ID: <20090129162317.GC27314@phd.pp.ru> On Thu, Jan 29, 2009 at 08:06:18AM -0800, Aahz wrote: > On Thu, Jan 29, 2009, Michael Foord wrote: > > Don't we have a pretty-print API - and isn't it spelled __str__ ? > > In theory, yes. In practice, we wouldn't be having this discussion if > that really worked. But it probably would make sense to see how far > using __str__ can take us -- AFAICT enumobject.c doesn't define __str__ > (although I may be missing something, I don't know Python at the C level > very well). Container objects (tuples/lists/dicts/sets) don't define __str__. Is __pprint__ an attempt to redefine __str__? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From theller at ctypes.org Thu Jan 29 17:27:35 2009 From: theller at ctypes.org (Thomas Heller) Date: Thu, 29 Jan 2009 17:27:35 +0100 Subject: [Python-Dev] Include C++ code in the ctypes test suite? Message-ID: I'm currently working on a patch that adds the __thiscall calling convention to ctypes. This calling convention is used on Windows for calling member functions from C++ classes. The idea is to eventually allow ctypes to wrap C++ classes. To test this functionality it is required to add some C++ source code to the ctypes private test module _ctypes_test.pyd/_ctypes_test.so. Is it appropriate to add C++ source files to the Python repository, or would that create too much trouble on some platforms? -- Thanks, Thomas From benjamin at python.org Thu Jan 29 17:30:21 2009 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 29 Jan 2009 10:30:21 -0600 Subject: [Python-Dev] Include C++ code in the ctypes test suite? In-Reply-To: References: Message-ID: <1afaf6160901290830mb862666od7aa59d5d9941bec@mail.gmail.com> On Thu, Jan 29, 2009 at 10:27 AM, Thomas Heller wrote: > Is it appropriate to add C++ source files to the Python repository, > or would that create too much trouble on some platforms? I don't see a problem with that as long as platforms without C++ compilers aren't affected in the build process. -- Regards, Benjamin From p.f.moore at gmail.com Thu Jan 29 17:39:53 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 29 Jan 2009 16:39:53 +0000 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <20090129162317.GC27314@phd.pp.ru> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> <4981BBB7.50502@voidspace.org.uk> <20090129160617.GA21013@panix.com> <20090129162317.GC27314@phd.pp.ru> Message-ID: <79990c6b0901290839t7080e6b0h81206bd978d85cf0@mail.gmail.com> 2009/1/29 Oleg Broytmann : > On Thu, Jan 29, 2009 at 08:06:18AM -0800, Aahz wrote: >> On Thu, Jan 29, 2009, Michael Foord wrote: >> > Don't we have a pretty-print API - and isn't it spelled __str__ ? >> >> In theory, yes. In practice, we wouldn't be having this discussion if >> that really worked. But it probably would make sense to see how far >> using __str__ can take us -- AFAICT enumobject.c doesn't define __str__ >> (although I may be missing something, I don't know Python at the C level >> very well). > > Container objects (tuples/lists/dicts/sets) don't define __str__. > Is __pprint__ an attempt to redefine __str__? Anyone feel like raising the topic of generic functions again? :-) More practically, the undocumented simplegeneric decorator in pkgutil could be used: Python 2.6.1 (r261:67517, Dec 4 2008, 16:51:00) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> from pkgutil import simplegeneric >>> @simplegeneric ... def f(obj): ... return "Object of type %s" % type(obj) ... >>> def str_f(s): ... return "String: " + s ... >>> f.register(str, str_f) >>> f("Test") 'String: Test' >>> f(1) "Object of type " >>> To me, that seems better than inventing yet another special method. Paul. From lists at cheimes.de Thu Jan 29 17:46:04 2009 From: lists at cheimes.de (Christian Heimes) Date: Thu, 29 Jan 2009 17:46:04 +0100 Subject: [Python-Dev] Include C++ code in the ctypes test suite? In-Reply-To: References: Message-ID: <4981DD4C.5040507@cheimes.de> Thomas Heller schrieb: > To test this functionality it is required to add some C++ source code to the > ctypes private test module _ctypes_test.pyd/_ctypes_test.so. > > Is it appropriate to add C++ source files to the Python repository, > or would that create too much trouble on some platforms? How about creating an additional test library ctypes_test_cpp.cpp? This way we can still run the ctypes tests on a platform without a C++ compiler. Of course you could add some preprocessor magic but that's just another way to ask for trouble. A second test library should make it easier. Christian From solipsis at pitrou.net Thu Jan 29 17:50:32 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 29 Jan 2009 16:50:32 +0000 (UTC) Subject: [Python-Dev] Include C++ code in the ctypes test suite? References: Message-ID: Thomas Heller ctypes.org> writes: > > To test this functionality it is required to add some C++ source code to the > ctypes private test module _ctypes_test.pyd/_ctypes_test.so. Perhaps you should create a separate test module (_ctypes_pp_test?) so that platforms without a properly configured C++ compiler can still run the other tests. (I also suppose configure can detect the presence of a C++ compiler...) Regards Antoine. From barry at python.org Thu Jan 29 17:59:22 2009 From: barry at python.org (Barry Warsaw) Date: Thu, 29 Jan 2009 11:59:22 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <20090129113130.GA2490@amk.local> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <49809C1B.3090805@voidspace.org.uk> <4980CD7A.6000506@v.loewis.de> <87d4e7dj0p.fsf@xemacs.org> <20090129113130.GA2490@amk.local> Message-ID: <7F2AD110-96A2-4367-821E-6DB8E2E3DC86@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 29, 2009, at 6:31 AM, A.M. Kuchling wrote: > If we intend for 3.0 to be a 'beta release', or to weaken the 'no > features in micro releases' rule, then fine; but we have to be *really > clear about it*. Are we? (The 3.0 release page calls it > production-ready.) I think it sets bad precedence to downgrade our confidence in the release. Again, my position is that it's better to stick to the same development processes we've always used, fix the most egregious problems in 3.0.1 with no API changes, but spend most of our energy on a 3.1 release in 6 months. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSYHga3EjvBPtnXfVAQLQxQP+Ipu3J0Ogvj0kW4txTgu8SJ4Hr6q7ll7i uyASnNQdB0WV3My1VsymMb5VlIWJtuvwY4DxYR1fqLHOQY6CloFqmmIkeMpZKt/K qXqNI1OvyLfoqg6QqXI+A4UFnUwlv7bSFHqZUu8wVn4De/kQqVfFUgjxBCoNe0lj 0au4xGdjjYo= =qOne -----END PGP SIGNATURE----- From alexander.belopolsky at gmail.com Thu Jan 29 17:59:39 2009 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 29 Jan 2009 11:59:39 -0500 Subject: [Python-Dev] Include C++ code in the ctypes test suite? In-Reply-To: References: Message-ID: On Thu, Jan 29, 2009 at 11:50 AM, Antoine Pitrou wrote: .. > (I also suppose configure can detect the presence of a C++ compiler...) > This test is already there: $ ./configure ... checking for g++... g++ configure: WARNING: By default, distutils will build C++ extension modules with "g++". If this is not intended, then set CXX on the configure command line. ... From tjreedy at udel.edu Thu Jan 29 18:50:49 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 29 Jan 2009 12:50:49 -0500 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> Message-ID: Ben North wrote: > I find 'functools.partial' useful, but occasionally I'm unable to use it > because it lacks a 'from the right' version. ... > Would there be any interest in this? I think so. Post your patch to the tracker. Even if eventually rejected, it will be there for people to use. From solipsis at pitrou.net Thu Jan 29 18:58:44 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 29 Jan 2009 17:58:44 +0000 (UTC) Subject: [Python-Dev] Partial function application 'from the right' References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> Message-ID: Hello, Ben North redfrontdoor.org> writes: > > I find 'functools.partial' useful, but occasionally I'm unable to use it > because it lacks a 'from the right' version. E.g., to create a function > which splits a string on commas, you can't say > > # Won't work when called: > split_comma = partial(str.split, sep = ',') In py3k, we could also use "..." (the Ellipsis object) to denote places where an argument is missing, so that: split_comma = partial(str.split, ..., ',') would do what you want. Regards Antoine. From guido at python.org Thu Jan 29 19:13:33 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 29 Jan 2009 10:13:33 -0800 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <7F2AD110-96A2-4367-821E-6DB8E2E3DC86@python.org> References: <49809B0C.4020905@egenix.com> <49809C1B.3090805@voidspace.org.uk> <4980CD7A.6000506@v.loewis.de> <87d4e7dj0p.fsf@xemacs.org> <20090129113130.GA2490@amk.local> <7F2AD110-96A2-4367-821E-6DB8E2E3DC86@python.org> Message-ID: > On Jan 29, 2009, at 6:31 AM, A.M. Kuchling wrote: >> If we intend for 3.0 to be a 'beta release', or to weaken the 'no >> features in micro releases' rule, then fine; but we have to be *really >> clear about it*. Are we? (The 3.0 release page calls it >> production-ready.) On Thu, Jan 29, 2009 at 8:59 AM, Barry Warsaw wrote: > I think it sets bad precedence to downgrade our confidence in the release. > Again, my position is that it's better to stick to the same development > processes we've always used, fix the most egregious problems in 3.0.1 with > no API changes, but spend most of our energy on a 3.1 release in 6 months. I'd like to find a middle ground. We can all agree that the users of 3.0 are a small minority compared to the 2.x users. Therefore I think we can bend the rules more than we have done for the recent 2.x releases. Those rules weren't always there (anyone remember the addition of bool, True and False to 2.2.1?). The rules were introduced for the benefit of our most conservative users -- people who introduce Python in an enterprise and don't want to find that they are forced to upgrade in six months. Frankly, I don't really believe the users for whom those rules were created are using 3.0 yet. Instead, I expect there to be two types of users: people in the educational business who don't have a lot of bridges to burn and are eager to use the new features; and developers of serious Python software (e.g. Twisted) who are trying to figure out how to port their code to 3.0. The first group isn't affected by the changes we're considering here (e.g. removing cmp or some obscure functions from the operator module). The latter group *may* be affected, simply because they may have some pre-3.0 code using old features that (by accident) still works under 3.0. On the one hand I understand that those folks want a stable target. On the other hand I think they would prefer to find out sooner rather than later they're using stuff they shouldn't be using any more. It's a delicate balance for sure, and I certainly don't want to open the floodgates here, or rebrand 3.1 as 3.0.1 or anything like that. But I really don't believe that the strictest interpretation of "no new features" will benefit us for 3.0.1. Perhaps we should decide when to go back to a more strict interpretation of the rules based on the uptake of Python 3 compared to Python 2. I don't believe that we risk influencing that uptake by bending the rules; the uptake will depend on the availability of ported 3rd party packages and some performance gains. (I don't know enough about the C reimplementation of io.py to tell whether it could be folded into 3.0 or will have to wait for 3.1.) Finally, to those who claim that 2.6 is a mess because multiprocessing wasn't perfectly stable at introduction: that's never been the standard we've used for totally *new* features. It's always been okay to add slightly immature features at a major release, as long as (a) they don't break anything else, and (b) we can fix things in the next release while maintaining backward compatibility. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Thu Jan 29 19:28:59 2009 From: brett at python.org (Brett Cannon) Date: Thu, 29 Jan 2009 10:28:59 -0800 Subject: [Python-Dev] python breakpoint opcode In-Reply-To: <014f01c981fd$b7bb3c60$2731b520$@com> References: <014f01c981fd$b7bb3c60$2731b520$@com> Message-ID: On Thu, Jan 29, 2009 at 02:38, Dr Andrew Perella wrote: > Hi, > > I was thinking of adding a breakpoint opcode to python to enable less > invasive debugging. > > I came across posts from 1999 by Vladimir Marangozov and Christian Tismer > discussing this issue but the links to the code are all out of date. > > Did anything come of this? There is nothing currently in Python for this, but I was not around for the discussion back then. -Brett From steve at holdenweb.com Thu Jan 29 19:57:43 2009 From: steve at holdenweb.com (Steve Holden) Date: Thu, 29 Jan 2009 13:57:43 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <49809B0C.4020905@egenix.com> <49809C1B.3090805@voidspace.org.uk> <4980CD7A.6000506@v.loewis.de> <87d4e7dj0p.fsf@xemacs.org> <20090129113130.GA2490@amk.local> <7F2AD110-96A2-4367-821E-6DB8E2E3DC86@python.org> Message-ID: <4981FC27.3010208@holdenweb.com> Guido van Rossum wrote: [...] > > Finally, to those who claim that 2.6 is a mess because multiprocessing > wasn't perfectly stable at introduction: that's never been the > standard we've used for totally *new* features. It's always been okay > to add slightly immature features at a major release, as long as (a) > they don't break anything else, and (b) we can fix things in the next > release while maintaining backward compatibility. > There's a large distance between saying its introduction was ill-advised and that 2.6 is a mess. I certainly never intimated such a thing (I said it was "a rushed release"). Did anyone? Of course we can fix it. Of course 2.6 is great. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Thu Jan 29 19:57:43 2009 From: steve at holdenweb.com (Steve Holden) Date: Thu, 29 Jan 2009 13:57:43 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <49809B0C.4020905@egenix.com> <49809C1B.3090805@voidspace.org.uk> <4980CD7A.6000506@v.loewis.de> <87d4e7dj0p.fsf@xemacs.org> <20090129113130.GA2490@amk.local> <7F2AD110-96A2-4367-821E-6DB8E2E3DC86@python.org> Message-ID: <4981FC27.3010208@holdenweb.com> Guido van Rossum wrote: [...] > > Finally, to those who claim that 2.6 is a mess because multiprocessing > wasn't perfectly stable at introduction: that's never been the > standard we've used for totally *new* features. It's always been okay > to add slightly immature features at a major release, as long as (a) > they don't break anything else, and (b) we can fix things in the next > release while maintaining backward compatibility. > There's a large distance between saying its introduction was ill-advised and that 2.6 is a mess. I certainly never intimated such a thing (I said it was "a rushed release"). Did anyone? Of course we can fix it. Of course 2.6 is great. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From guido at python.org Thu Jan 29 20:17:04 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 29 Jan 2009 11:17:04 -0800 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <4981FC27.3010208@holdenweb.com> References: <49809C1B.3090805@voidspace.org.uk> <4980CD7A.6000506@v.loewis.de> <87d4e7dj0p.fsf@xemacs.org> <20090129113130.GA2490@amk.local> <7F2AD110-96A2-4367-821E-6DB8E2E3DC86@python.org> <4981FC27.3010208@holdenweb.com> Message-ID: On Thu, Jan 29, 2009 at 10:57 AM, Steve Holden wrote: > Guido van Rossum wrote: > [...] >> >> Finally, to those who claim that 2.6 is a mess because multiprocessing >> wasn't perfectly stable at introduction: that's never been the >> standard we've used for totally *new* features. It's always been okay >> to add slightly immature features at a major release, as long as (a) >> they don't break anything else, and (b) we can fix things in the next >> release while maintaining backward compatibility. >> > There's a large distance between saying its introduction was ill-advised > and that 2.6 is a mess. I certainly never intimated such a thing (I said > it was "a rushed release"). Did anyone? I don't think that 2.6 as a whole counts as a rushed release, despite the inclusion of multiprocessing. And I don't think it was ill-advised either. > Of course we can fix it. Of course 2.6 is great. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Thu Jan 29 20:26:17 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 29 Jan 2009 20:26:17 +0100 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <7F2AD110-96A2-4367-821E-6DB8E2E3DC86@python.org> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <49809C1B.3090805@voidspace.org.uk> <4980CD7A.6000506@v.loewis.de> <87d4e7dj0p.fsf@xemacs.org> <20090129113130.GA2490@amk.local> <7F2AD110-96A2-4367-821E-6DB8E2E3DC86@python.org> Message-ID: <498202D9.30300@v.loewis.de> > I think it sets bad precedence to downgrade our confidence in the > release. Again, my position is that it's better to stick to the same > development processes we've always used, fix the most egregious problems > in 3.0.1 with no API changes, but spend most of our energy on a 3.1 > release in 6 months. +1 Martin From martin at v.loewis.de Thu Jan 29 20:32:05 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 29 Jan 2009 20:32:05 +0100 Subject: [Python-Dev] Include C++ code in the ctypes test suite? In-Reply-To: References: Message-ID: <49820435.3040501@v.loewis.de> > Is it appropriate to add C++ source files to the Python repository, > or would that create too much trouble on some platforms? I think there will be massive portability problems, which only fade after one or two years, until this actually works everywhere. So failure of this to work shouldn't break the Python build, and, preferably, the build process should suggest the user what might have happened when it failed. Regards, Martin From Scott.Daniels at Acm.Org Thu Jan 29 21:00:15 2009 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Thu, 29 Jan 2009 12:00:15 -0800 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> Message-ID: Antoine Pitrou wrote: > ... > In py3k, we could also use "..." (the Ellipsis object) to denote > places where an argument is missing, so that: > split_comma = partial(str.split, ..., ',') > would do what you want. Thus preventing any use of partial when an argument could be an the Ellipsis instance. --Scott David Daniels Scott.Daniels at Acm.Org From solipsis at pitrou.net Thu Jan 29 21:04:03 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 29 Jan 2009 20:04:03 +0000 (UTC) Subject: [Python-Dev] Partial function application 'from the right' References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> Message-ID: Scott David Daniels Acm.Org> writes: > > Antoine Pitrou wrote: > > ... > > In py3k, we could also use "..." (the Ellipsis object) to denote > > places where an argument is missing, so that: > > split_comma = partial(str.split, ..., ',') > > would do what you want. > > Thus preventing any use of partial when an argument could be an > the Ellipsis instance. Obviously, it is the drawback :) But Ellipsis is hardly used anywhere, and it reads good in this very use case. From Chris.Barker at noaa.gov Thu Jan 29 21:39:31 2009 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Thu, 29 Jan 2009 12:39:31 -0800 Subject: [Python-Dev] Universal newlines, and the gzip module. Message-ID: <49821403.1030603@noaa.gov> Hi all, Over on the matplotlib mailing list, we ran into a problem with trying to use Universal newlines with gzip. In virtually all of my code that reads text files, I use the 'U' flag to open files, it really helps not having to deal with newline issues. Yes, they are fewer now that the Macintosh uses \n, but they can still be a pain. Anyway, we added such support to some matplotlib methods, and found that gzip file reading broken We were passing the flags though into either file() or gzip.open(), and passing 'U' into gzip.open() turns out to be fatal. 1) It would be nice if the gzip module (and the zip lib module) supported Universal newlines -- you could read a compressed text file with "wrong" newlines, and have them handled properly. However, that may be hard to do, so at least: 2) Passing a 'U' flag in to gzip.open shouldn't break it. I took a look at the Python SVN (2.5.4 and 2.6.1) for the gzip lib. I see this: # guarantee the file is opened in binary mode on platforms # that care about that sort of thing if mode and 'b' not in mode: mode += 'b' if fileobj is None: fileobj = self.myfileobj = __builtin__.open(filename, mode or 'rb') this is going to break for 'U' == you'll get 'rUb'. I tested file(filename, 'rUb'), and it looks like it does universal newline translation. So: * Either gzip should be a bit smarter, and remove the 'U' flag (that's what we did in the MPL code), or force 'rb' or 'wb'. * Or: file opening should be a bit smarter -- what does 'rUb' mean? a file can't be both Binary and Universal Text. Should it raise an exception? Somehow I think it would be better to ignore the 'U', but maybe that's only because of the issue I happen to be looking at now. That later seems a better idea -- this issue could certainly come up in other places than the gzip module, but maybe it would break a bunch of code -- who knows? I haven't touched py3 yet, so I have not idea if this issue is different there. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From alexander.belopolsky at gmail.com Thu Jan 29 22:44:09 2009 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 29 Jan 2009 16:44:09 -0500 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> Message-ID: This discussion probably belongs to python-ideas, but since we already have this thread, I'll reply here instead of opening a new thread there. Ellipsis was introduced into python to serve needs of the numeric python community. If you think of numpy multiarrays as functions taking ndim number of arguments, then ellipsis is used to denote any number of missing arguments and : is used to denote a single missing argument. By this analogy, partial(f, ..., *args) is right_partial with '...' standing for any number of missing arguments. I you want to specify exactly one missing argument, you would want to write partial(f, :, *args), which is not a valid syntax even in Py3. If one is willing to use [] instead of () with partial, it is possible to implement partial[f, ..., *args] and partial[f, x, :, z] already in Py2, but I would rather see : allowed in the argument lists or some other short syntax for missing arguments. If such syntax is introduced, the need for partial may even go away with partial(str.split, :, ',') spelled simply as str.split(:, ','). On Thu, Jan 29, 2009 at 3:04 PM, Antoine Pitrou wrote: > Scott David Daniels Acm.Org> writes: >> >> Antoine Pitrou wrote: >> > ... >> > In py3k, we could also use "..." (the Ellipsis object) to denote >> > places where an argument is missing, so that: >> > split_comma = partial(str.split, ..., ',') >> > would do what you want. >> >> Thus preventing any use of partial when an argument could be an >> the Ellipsis instance. > > Obviously, it is the drawback :) But Ellipsis is hardly used anywhere, and it > reads good in this very use case. > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/alexander.belopolsky%40gmail.com > From python at rcn.com Thu Jan 29 22:51:14 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 29 Jan 2009 13:51:14 -0800 Subject: [Python-Dev] Python 3.0.1 References: <49809B0C.4020905@egenix.com> <49809C1B.3090805@voidspace.org.uk> <4980CD7A.6000506@v.loewis.de> <87d4e7dj0p.fsf@xemacs.org><20090129113130.GA2490@amk.local><7F2AD110-96A2-4367-821E-6DB8E2E3DC86@python.org> Message-ID: From: "Guido van Rossum" > On the one hand I understand that those folks want a stable target. On > the other hand I think they would prefer to find out sooner rather > than later they're using stuff they shouldn't be using any more. It's > a delicate balance for sure, and I certainly don't want to open the > floodgates here, or rebrand 3.1 as 3.0.1 or anything like that. But I > really don't believe that the strictest interpretation of "no new > features" will benefit us for 3.0.1. Perhaps we should decide when to > go back to a more strict interpretation of the rules based on the > uptake of Python 3 compared to Python 2. That seems like a smart choice to me. Make the fixups as early as possible, before there has been significant uptake. Am reminded of a cautionary tale from The Art of Unix Programming http://www.faqs.org/docs/artu/ch15s04.html#id2986550 : """ No discussion of make(1) would be complete without an acknowledgement that it includes one of the worst design botches in the history of Unix. The use of tab characters as a required leader for command lines associated with a production means that the interpretation of a makefile can change drastically on the basis of invisible differences in whitespace. "Why the tab in column 1? Yacc was new, Lex was brand new. I hadn't tried either, so I figured this would be a good excuse to learn. After getting myself snarled up with my first stab at Lex, I just did something simple with the pattern newline-tab. It worked, it stayed. And then a few weeks later I had a user population of about a dozen, most of them friends, and I didn't want to screw up my embedded base. The rest, sadly, is history." -- Stuart Feldman """ Raymond From fuzzyman at voidspace.org.uk Thu Jan 29 22:54:28 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 29 Jan 2009 21:54:28 +0000 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <49809B0C.4020905@egenix.com> <49809C1B.3090805@voidspace.org.uk> <4980CD7A.6000506@v.loewis.de> <87d4e7dj0p.fsf@xemacs.org><20090129113130.GA2490@amk.local><7F2AD110-96A2-4367-821E-6DB8E2E3DC86@python.org> Message-ID: <49822594.2000900@voidspace.org.uk> Raymond Hettinger wrote: > From: "Guido van Rossum" >> On the one hand I understand that those folks want a stable target. On >> the other hand I think they would prefer to find out sooner rather >> than later they're using stuff they shouldn't be using any more. It's >> a delicate balance for sure, and I certainly don't want to open the >> floodgates here, or rebrand 3.1 as 3.0.1 or anything like that. But I >> really don't believe that the strictest interpretation of "no new >> features" will benefit us for 3.0.1. Perhaps we should decide when to >> go back to a more strict interpretation of the rules based on the >> uptake of Python 3 compared to Python 2. > > That seems like a smart choice to me. Make the fixups as early as > possible, > before there has been significant uptake. > > Am reminded of a cautionary tale from The Art of Unix Programming > http://www.faqs.org/docs/artu/ch15s04.html#id2986550 : > > """ > > No discussion of make(1) would be complete without an acknowledgement > that it includes one of the worst design botches in the history of > Unix. The use of tab characters as a required leader for command lines > associated with a production means that the interpretation of a > makefile can change drastically on the basis of invisible differences > in whitespace. > > > "Why the tab in column 1? Yacc was new, Lex was brand new. I hadn't > tried either, so I figured this would be a good excuse to learn. After > getting myself snarled up with my first stab at Lex, I just did > something simple with the pattern newline-tab. It worked, it stayed. > And then a few weeks later I had a user population of about a dozen, > most of them friends, and I didn't want to screw up my embedded base. > The rest, sadly, is history." -- Stuart Feldman > > """ > I suspect that the use of significant whitespace is too deeply ingrained in Python for us to change it now - even in Python 3. ;-) Michael > > Raymond > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From ncoghlan at gmail.com Thu Jan 29 22:54:35 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 30 Jan 2009 07:54:35 +1000 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <4981BBB7.50502@voidspace.org.uk> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> <4981BBB7.50502@voidspace.org.uk> Message-ID: <4982259B.4070808@gmail.com> Michael Foord wrote: > Don't we have a pretty-print API - and isn't it spelled __str__ ? For the "reiterable" cases like dictionary views (where the object is not consumed), an appropriate __str__ or __repr__ should be written). Whether that is something as simple as " .items()" for an items view, or something more complicated that more directly shows the content of the view, I'm not sure. For the standard iterators like enumerate and ranged, I would suggest that they be modified to use a repr of the form: "reversed( )" "enumerate( )" "iter( )" "iter( , )" While those obviously won't show how much of the iterable has been consumed, neither do the current representations. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From leif.walsh at gmail.com Thu Jan 29 22:58:25 2009 From: leif.walsh at gmail.com (Leif Walsh) Date: Thu, 29 Jan 2009 16:58:25 -0500 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> Message-ID: On Thu, Jan 29, 2009 at 9:12 AM, Ben North wrote: > I find 'functools.partial' useful, but occasionally I'm unable to use it > because it lacks a 'from the right' version. E.g., to create a function > which splits a string on commas, you can't say First of all, many functions like this are easy to handle yourself. Example: >>> def split_comma(s): >>> return str.split(s, ',') That said, it seems to me that if we're going to add to functools.partial, we should go all the way and allow keyword arguments (or a dict of them, if it's otherwise too hard to implement). Otherwise, in another few {days, weeks, months} we'll see another thread like this clamoring for a keyword-sensitive functools.partial. Come to think of it, I would imagine the next iteration would ask for a way to curry arbitrary positional arguments, and I can't come up with a simple and beautiful way to do that off the top of my head. Maybe this is an argument for keeping functools.partial the way it is and forcing developers to write their own currying functions. -- Cheers, Leif From solipsis at pitrou.net Thu Jan 29 23:04:41 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 29 Jan 2009 22:04:41 +0000 (UTC) Subject: [Python-Dev] Partial function application 'from the right' References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> Message-ID: Alexander Belopolsky gmail.com> writes: > > By this analogy, partial(f, ..., *args) is right_partial with '...' > standing for any number of missing arguments. I you want to specify > exactly one missing argument, you would want to write partial(f, :, > *args), which is not a valid syntax even in Py3. Yes, of course, but... the meaning which numpy attributes to Ellipsis does not have to be the same in other libraries. Otherwise this meaning would have been embedded in the interpreter itself, while it hasn't. The point of using Ellipsis in this case is not to be numpy-friendly, but rather to exploit the fact that it is a very rarely used object, and that it has an alternate spelling which suits very well (visually speaking) the purpose being discussed. Regards Antoine. From aahz at pythoncraft.com Thu Jan 29 23:09:51 2009 From: aahz at pythoncraft.com (Aahz) Date: Thu, 29 Jan 2009 14:09:51 -0800 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <49809B0C.4020905@egenix.com> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> Message-ID: <20090129220951.GA17786@panix.com> On Wed, Jan 28, 2009, M.-A. Lemburg wrote: > > Why don't we just mark 3.0.x as experimental branch and keep updating/ > fixing things that were not sorted out for the 3.0.0 release ?! I > think that's a fair approach, given that the only way to get field > testing for new open-source software is to release early and often. > > A 3.1 release should then be the first stable release of the 3.x > series and mark the start of the usual deprecation mechanisms we have > in the 2.x series. Needless to say, that rushing 3.1 out now would > only cause yet another experimental release... major releases do take > time to stabilize. Speaking as the original author of PEP6 (Bug Fix Releases), this sounds like a reasonable middle ground. I certainly advocate that nobody consider Python 3.0 for production software, and enshrining that into the dev process should work well. At the same time, I think each individual change that doesn't clearly fall into the PEP6 process of being a bugfix needs to be vetted beyond what's permitted for not-yet-released versions. The problem is that the obvious candidate for doing the vetting is the Release Manager, and Barry doesn't like this approach. The vetting does need to be handled by a core committer IMO -- MAL, are you volunteering? Anyone else? Barry, are you actively opposed to marking 3.0.x as experimental, or do you just dislike it? (I.e. are you -1 or -0?) -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From ncoghlan at gmail.com Thu Jan 29 23:12:14 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 30 Jan 2009 08:12:14 +1000 Subject: [Python-Dev] Merging to the 3.0 maintenance branch In-Reply-To: <1afaf6160901290804t3c08134egadfd14493dcb09ae@mail.gmail.com> References: <20090129043706.5614B1E4002@bag.python.org> <1afaf6160901290804t3c08134egadfd14493dcb09ae@mail.gmail.com> Message-ID: <498229BE.4060408@gmail.com> Benjamin Peterson wrote: > On Wed, Jan 28, 2009 at 10:37 PM, brett. cannon > wrote: >> Author: brett.cannon >> Date: Thu Jan 29 05:37:06 2009 >> New Revision: 69093 >> >> Log: >> Backport r69092 by hand since svnmerge keeps saying there is a conflict on '.'. > > Just do "svn resolved ." There are potential problems with doing it that way [1]. The safer option is to do: svn revert . svnmerge merge -M -F Perhaps we should add a "maintmerge" script (along with "maintmerge.bat" batch file) to the root development directory that automates this: #/bin/sh svnmerge merge -r $1 svn revert . svnmerge -M -F $1 (Note that my shell scripting is a little rusty and I haven't actually executed that example...) Then the advice will just be to use svnmerge directly most of the time, and maintmerge when merging a revision that was itself created with svnmerge. Cheers, Nick. [1] How to clobber svnmerge's revision tracking 101: http://mail.python.org/pipermail/python-dev/2008-December/084644.html -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Thu Jan 29 23:19:08 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 30 Jan 2009 08:19:08 +1000 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> Message-ID: <49822B5C.1000405@gmail.com> Leif Walsh wrote: > That said, it seems to me that if we're going to add to > functools.partial, we should go all the way and allow keyword > arguments (or a dict of them, if it's otherwise too hard to > implement). Otherwise, in another few {days, weeks, months} we'll see > another thread like this clamoring for a keyword-sensitive > functools.partial. functools.partial *does* support keyword arguments - it's just that some functions and methods written in C (such as string methods) *don't*, so partial's keyword support doesn't help. A functools.rpartial would go some way towards addressing that. Using the standalone Ellipsis to denote missing arguments would probably start to miss the whole point of functools.partial: the only reason for its existence is that it is *faster than the equivalent Python function*. If partial starts messing about looking for missing arguments and then slotting them in, then it is likely to slow down to the point where you would be better off skipping it and writing a dedicated function that adds the extra arguments. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From solipsis at pitrou.net Thu Jan 29 23:30:27 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 29 Jan 2009 22:30:27 +0000 (UTC) Subject: [Python-Dev] Partial function application 'from the right' References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> <49822B5C.1000405@gmail.com> Message-ID: Nick Coghlan gmail.com> writes: > > If partial starts messing about looking for missing arguments and then > slotting them in, then it is likely to slow down to the point where you > would be better off skipping it and writing a dedicated function that > adds the extra arguments. Looking for missing arguments is very cheap, just raw pointer compares (Ellipsis is a singleton). In comparison, the cost of executing a dedicated Python function would be overwhelming. From steve at pearwood.info Thu Jan 29 23:42:15 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 30 Jan 2009 09:42:15 +1100 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <4981BBB7.50502@voidspace.org.uk> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> <4981BBB7.50502@voidspace.org.uk> Message-ID: <498230C7.2040403@pearwood.info> Michael Foord wrote: > Don't we have a pretty-print API - and isn't it spelled __str__ ? Not really. If it were as simple as calling str(obj), there would be no need for the pprint module. In any case, it seems that the pprint module actually calls repr() on objects other than dicts, tuples and lists. I'm concerned about the number of special methods exploding, but I've also come across times where I needed more than two string representations of an object. Sometimes I solved this by adding a pprint() method, other times I used other names, and it would be nice if there was a standard way of spelling it. So I'm +0.5 on Aahz's suggestion of __pprint__. In my ideal world, __pprint__ should allow (but not require) extra arguments, so that one can do something like the following: pprint(binarytree) # sensible defaults pprint(binarytree, order='preorder') -- Steven From brett at python.org Thu Jan 29 23:45:55 2009 From: brett at python.org (Brett Cannon) Date: Thu, 29 Jan 2009 14:45:55 -0800 Subject: [Python-Dev] Universal newlines, and the gzip module. In-Reply-To: <49821403.1030603@noaa.gov> References: <49821403.1030603@noaa.gov> Message-ID: On Thu, Jan 29, 2009 at 12:39, Christopher Barker wrote: > Hi all, > > Over on the matplotlib mailing list, we ran into a problem with trying to > use Universal newlines with gzip. In virtually all of my code that reads > text files, I use the 'U' flag to open files, it really helps not having to > deal with newline issues. Yes, they are fewer now that the Macintosh uses > \n, but they can still be a pain. > > Anyway, we added such support to some matplotlib methods, and found that > gzip file reading broken We were passing the flags though into either file() > or gzip.open(), and passing 'U' into gzip.open() turns out to be fatal. > > 1) It would be nice if the gzip module (and the zip lib module) supported > Universal newlines -- you could read a compressed text file with "wrong" > newlines, and have them handled properly. However, that may be hard to do, > so at least: > > 2) Passing a 'U' flag in to gzip.open shouldn't break it. > > I took a look at the Python SVN (2.5.4 and 2.6.1) for the gzip lib. I see > this: > > > # guarantee the file is opened in binary mode on platforms > # that care about that sort of thing > if mode and 'b' not in mode: > mode += 'b' > if fileobj is None: > fileobj = self.myfileobj = __builtin__.open(filename, mode or > 'rb') > > this is going to break for 'U' == you'll get 'rUb'. I tested file(filename, > 'rUb'), and it looks like it does universal newline translation. > > So: > > * Either gzip should be a bit smarter, and remove the 'U' flag (that's what > we did in the MPL code), or force 'rb' or 'wb'. > > * Or: file opening should be a bit smarter -- what does 'rUb' mean? a file > can't be both Binary and Universal Text. Should it raise an exception? > Somehow I think it would be better to ignore the 'U', but maybe that's only > because of the issue I happen to be looking at now. > > > That later seems a better idea -- this issue could certainly come up in > other places than the gzip module, but maybe it would break a bunch of code > -- who knows? I think it should be raising an exception as 'rUb' is an invalid value for the argument. -Brett From python at rcn.com Fri Jan 30 00:00:56 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 29 Jan 2009 15:00:56 -0800 Subject: [Python-Dev] Broken Test -- test_distutils Message-ID: In the past couple of days, test_distutils started failing. It looks like a pure python error and may have been introduced by guilherme.polo's checkins: File "c:\py27\lib\distutils\tests\test_sdist.py", line 119, in test_make_distr ibution spawn('tar --help') File "c:\py27\lib\distutils\spawn.py", line 37, in spawn _spawn_nt(cmd, search_path, dry_run=dry_run) File "c:\py27\lib\distutils\spawn.py", line 70, in _spawn_nt cmd = _nt_quote_args(cmd) File "c:\py27\lib\distutils\spawn.py", line 61, in _nt_quote_args args[i] = '"%s"' % args[i] TypeError: 'str' object does not support item assignment 1 test failed: test_distutils Raymond From ziade.tarek at gmail.com Fri Jan 30 00:05:28 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 30 Jan 2009 00:05:28 +0100 Subject: [Python-Dev] Broken Test -- test_distutils In-Reply-To: References: Message-ID: <94bdd2610901291505w1e89bb8dv154499869d982287@mail.gmail.com> On Fri, Jan 30, 2009 at 12:00 AM, Raymond Hettinger wrote: > In the past couple of days, test_distutils started failing. It looks like a > pure python error and may have been introduced by guilherme.polo's checkins: > That's me. I'll fix this problem right now. > > File "c:\py27\lib\distutils\tests\test_sdist.py", line 119, in > test_make_distr > ibution > spawn('tar --help') > File "c:\py27\lib\distutils\spawn.py", line 37, in spawn > _spawn_nt(cmd, search_path, dry_run=dry_run) > File "c:\py27\lib\distutils\spawn.py", line 70, in _spawn_nt > cmd = _nt_quote_args(cmd) > File "c:\py27\lib\distutils\spawn.py", line 61, in _nt_quote_args > args[i] = '"%s"' % args[i] > TypeError: 'str' object does not support item assignment > > 1 test failed: > test_distutils > > > > Raymond > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From ggpolo at gmail.com Fri Jan 30 00:07:45 2009 From: ggpolo at gmail.com (Guilherme Polo) Date: Thu, 29 Jan 2009 21:07:45 -0200 Subject: [Python-Dev] Broken Test -- test_distutils In-Reply-To: References: Message-ID: On Thu, Jan 29, 2009 at 9:00 PM, Raymond Hettinger wrote: > In the past couple of days, test_distutils started failing. It looks like a > pure python error and may have been introduced by guilherme.polo's checkins: > > > File "c:\py27\lib\distutils\tests\test_sdist.py", line 119, in > test_make_distr > ibution > spawn('tar --help') > File "c:\py27\lib\distutils\spawn.py", line 37, in spawn > _spawn_nt(cmd, search_path, dry_run=dry_run) > File "c:\py27\lib\distutils\spawn.py", line 70, in _spawn_nt > cmd = _nt_quote_args(cmd) > File "c:\py27\lib\distutils\spawn.py", line 61, in _nt_quote_args > args[i] = '"%s"' % args[i] > TypeError: 'str' object does not support item assignment > > 1 test failed: > test_distutils > > How did my commits introduced that error ? > > Raymond > -- -- Guilherme H. Polo Goncalves From robert.kern at gmail.com Fri Jan 30 00:13:41 2009 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 29 Jan 2009 17:13:41 -0600 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <20090129142021.GA8996@panix.com> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> Message-ID: On 2009-01-29 08:20, Aahz wrote: > The reason I'm chiming in is that I would welcome a PEP that created a > __pprint__ method as an alternative to special-casing. I think that it > would be generically useful for user-created objects, plus once you've > added this feature other people can easily do some of the grunt work of > extending this through the Python core. (Actually, unless someone > objects, I don't think a PEP is required, but it would be good for the > usual reasons that PEPs are written, to provide a central place > documenting the addition.) I think it's worth looking at Armin Ronacher's pretty.py for a starting point. http://dev.pocoo.org/hg/sandbox/file/tip/pretty I've been using it as my default displayhook under IPython for a few weeks now. It uses a combination of a function registry and a __pretty__ special method to find the right pretty printer. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From python at rcn.com Fri Jan 30 00:08:16 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 29 Jan 2009 15:08:16 -0800 Subject: [Python-Dev] Broken Test -- test_distutils References: <94bdd2610901291505w1e89bb8dv154499869d982287@mail.gmail.com> Message-ID: <888A5B85FC644EF89EAC8A70BBDD9C1D@RaymondLaptop1> [Tarek Ziad?] > That's me. I'll fix this problem right now. Thanks. I appreciate it. Raymond From daniel at stutzbachenterprises.com Fri Jan 30 00:21:11 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Thu, 29 Jan 2009 17:21:11 -0600 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> Message-ID: On Thu, Jan 29, 2009 at 4:04 PM, Antoine Pitrou wrote: > Alexander Belopolsky gmail.com> writes: > > By this analogy, partial(f, ..., *args) is right_partial with '...' > > standing for any number of missing arguments. I you want to specify > > exactly one missing argument, you would want to write partial(f, :, > > *args), which is not a valid syntax even in Py3. > > Yes, of course, but... the meaning which numpy attributes to Ellipsis does > not > have to be the same in other libraries. Otherwise this meaning would have > been > embedded in the interpreter itself, while it hasn't. > The meaning which numpy attributes to Ellipsis is also the meaning that mathematical notation has attached to Ellipsis for a very long time. See: http://en.wikipedia.org/wiki/Ellipsis#In_mathematical_notation -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike.klaas at gmail.com Fri Jan 30 00:24:38 2009 From: mike.klaas at gmail.com (Mike Klaas) Date: Thu, 29 Jan 2009 15:24:38 -0800 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> Message-ID: <5E031BDD-32EC-47CC-8CE4-D4D3888A49F2@gmail.com> On 29-Jan-09, at 3:21 PM, Daniel Stutzbach wrote: > On Thu, Jan 29, 2009 at 4:04 PM, Antoine Pitrou > wrote: > Alexander Belopolsky gmail.com> writes: > > By this analogy, partial(f, ..., *args) is right_partial with '...' > > standing for any number of missing arguments. I you want to specify > > exactly one missing argument, you would want to write partial(f, :, > > *args), which is not a valid syntax even in Py3. > > Yes, of course, but... the meaning which numpy attributes to > Ellipsis does not > have to be the same in other libraries. Otherwise this meaning would > have been > embedded in the interpreter itself, while it hasn't. > > The meaning which numpy attributes to Ellipsis is also the meaning > that mathematical notation has attached to Ellipsis for a very long > time. And yet, python isn't confined to mathematical notation. *, ** are both overloaded for use in argument lists to no-one's peril, AFAICT. -Mike From python at rcn.com Fri Jan 30 00:27:03 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 29 Jan 2009 15:27:03 -0800 Subject: [Python-Dev] Python 3.0.1 References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com><497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> Message-ID: [Aahz] > At the same time, I think each individual > change that doesn't clearly fall into the PEP6 process of being a bugfix > needs to be vetted beyond what's permitted for not-yet-released versions. To get the ball rolling, I have a candidate for discussion. Very late in the 3.0 process (after feature freeze), the bsddb code was ripped out (good riddance). This had the unfortunate side-effect of crippling shelves which now fall back to using dumbdbm. I'm somewhat working on an alternate dbm based on sqlite3: http://code.activestate.com/recipes/576638/ It is a pure python module and probably will not be used directly, but shelves will see an immediate benefit (especially for large shelves) in terms of speed and space. On the one hand, it is an API change or new feature because people can (if they choose) access the dbm directly. OTOH, it is basically a performance fix for shelves whose API won't change at all. The part that is visible and incompatible is that 3.0.1 shelves won't be readable by 3.0.0. > The problem is that the obvious candidate for doing the vetting is the > Release Manager, and Barry doesn't like this approach. The vetting does > need to be handled by a core committer IMO -- MAL, are you volunteering? > Anyone else? It should be someone who is using 3.0 regularly (ideally someone who is working on fixing it). IMO, people who aren't exercising it don't really have a feel for the problems or the cost/benefits of the fixes. > Barry, are you actively opposed to marking 3.0.x as experimental, or do > you just dislike it? (I.e. are you -1 or -0?) My preference is to *not* mark it as experimental. Instead, I prefer doing what it takes to make the 3.0.x series viable. Raymond From collinw at gmail.com Fri Jan 30 00:29:47 2009 From: collinw at gmail.com (Collin Winter) Date: Thu, 29 Jan 2009 15:29:47 -0800 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> Message-ID: <43aa6ff70901291529v3a014d3bo73b63229b8697cf1@mail.gmail.com> On Thu, Jan 29, 2009 at 6:12 AM, Ben North wrote: > Hi, > > I find 'functools.partial' useful, but occasionally I'm unable to use it > because it lacks a 'from the right' version. E.g., to create a function > which splits a string on commas, you can't say > > # Won't work when called: > split_comma = partial(str.split, sep = ',') [snip] > I've created a patch which adds a 'partial_right' function. The two > examples above: > > >>> import functools, math > > >>> split_comma = functools.partial_right(str.split, ',') > >>> split_comma('a,b,c') > ['a', 'b', 'c'] > > >>> log_10 = functools.partial_right(math.log, 10.0) > >>> log_10(100.0) > 2.0 Can you point to real code that this makes more readable? Collin From daniel at stutzbachenterprises.com Fri Jan 30 00:38:29 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Thu, 29 Jan 2009 17:38:29 -0600 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: <5E031BDD-32EC-47CC-8CE4-D4D3888A49F2@gmail.com> References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> <5E031BDD-32EC-47CC-8CE4-D4D3888A49F2@gmail.com> Message-ID: On Thu, Jan 29, 2009 at 5:24 PM, Mike Klaas wrote: > And yet, python isn't confined to mathematical notation. *, ** are both > overloaded for use in argument lists to no-one's peril, AFAICT. > Certainly, but there is no danger of confusion them for multiplication in context, whereas: split_comma = partial(str.split, ..., ',') to me looks like "make ',' the last argument" rather than "make ',' the second argument". -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew-pythondev at puzzling.org Fri Jan 30 00:38:36 2009 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Fri, 30 Jan 2009 10:38:36 +1100 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: <5E031BDD-32EC-47CC-8CE4-D4D3888A49F2@gmail.com> References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> <5E031BDD-32EC-47CC-8CE4-D4D3888A49F2@gmail.com> Message-ID: <20090129233836.GA9550@steerpike.home.puzzling.org> Mike Klaas wrote: > On 29-Jan-09, at 3:21 PM, Daniel Stutzbach wrote: [...] >> The meaning which numpy attributes to Ellipsis is also the meaning >> that mathematical notation has attached to Ellipsis for a very long >> time. > > And yet, python isn't confined to mathematical notation. *, ** are both > overloaded for use in argument lists to no-one's peril, AFAICT. With the crucial difference that * and ** are purely syntax, but Ellipsis is an object. -Andrew. From guido at python.org Fri Jan 30 00:40:41 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 29 Jan 2009 15:40:41 -0800 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> Message-ID: On Thu, Jan 29, 2009 at 3:27 PM, Raymond Hettinger wrote: > To get the ball rolling, I have a candidate for discussion. > > Very late in the 3.0 process (after feature freeze), the bsddb code was > ripped out (good riddance). This had the unfortunate side-effect of > crippling shelves which now fall back to using dumbdbm. > > I'm somewhat working on an alternate dbm based on sqlite3: > http://code.activestate.com/recipes/576638/ > It is a pure python module and probably will not be used directly, but shelves > will see an immediate benefit (especially for large shelves) in terms of speed > and space. > > On the one hand, it is an API change or new feature because people can > (if they choose) access the dbm directly. OTOH, it is basically a > performance fix for shelves whose API won't change at all. The part that is visible > and incompatible is that 3.0.1 shelves won't be readable by 3.0.0. That is too much for 3.0.1. It could affect external file formats which strikes me as a bad idea. Sounds like a good candidate for 3.1, which we should be expecting in 4-6 months I hope. Also you could try find shelve users (are there any?) and recommend they install this as a 3rd party package, with the expectation it'll be built into 3.1. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Fri Jan 30 01:43:37 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 29 Jan 2009 16:43:37 -0800 Subject: [Python-Dev] Python 3.0.1 References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> Message-ID: >> On the one hand, it is an API change or new feature because people can >> (if they choose) access the dbm directly. OTOH, it is basically a >> performance fix for shelves whose API won't change at all. The part that is visible >> and incompatible is that 3.0.1 shelves won't be readable by 3.0.0. > > That is too much for 3.0.1. It could affect external file formats > which strikes me as a bad idea. We should have insisted that bsddb not be taken out until a replacement was put in. The process was broken with the RM insisting on feature freeze early in the game but letting tools like bsddb get ripped-out near the end. IMO, it was foolish to do one without the other. After the second alphas was out, there was resistance to any additions or to revisiting any of the early changes -- that was probably as mistake -- now we're deferring the fix for another 4-6 months and 3.0.x will never have it (at least right out of the box, as shipped). > Also you could try find shelve users (are there > any?) I'm a big fan of shelves and have always used them extensively. Not sure if anyone else cares about them though. > recommend they install this as a 3rd party package, with the > expectation it'll be built into 3.1. Will do. That was my original plan since the day bsddb got ripped out. Raymond From python at rcn.com Fri Jan 30 01:58:59 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 29 Jan 2009 16:58:59 -0800 Subject: [Python-Dev] Python 3.0.1 References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> Message-ID: A couple additional thoughts FWIW: * whichdb() selects from multiple file formats, so 3.0.1 would still be able to read 3.0.0 files. It is the 2.x shelves that won't be readable at all under any scenario. * If you're thinking that shelves have very few users and that 3.0.0 has had few adopters, doesn't that mitigate the effects of making a better format available in 3.0.1? Wouldn't this be the time to do it? * The file format itself is not new or unreadable by 3.0.0. It is just a plain sqlite3 file. Was is new is the ability of shelve's to call sqlite. To me, that is something a little different than changing a pickle protocol or somesuch. Raymond From solipsis at pitrou.net Fri Jan 30 02:04:36 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 30 Jan 2009 01:04:36 +0000 (UTC) Subject: [Python-Dev] Python 3.0.1 References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> Message-ID: Raymond Hettinger rcn.com> writes: > > * If you're thinking that shelves have very few users and that > 3.0.0 has had few adopters, doesn't that mitigate the effects > of making a better format available in 3.0.1? Wouldn't this > be the time to do it? There was already another proposal for an sqlite-based dbm module, you may want to synchronize with it: http://bugs.python.org/issue3783 As I see it, the problem with introducing it in 3.0.1 is that we would be rushing in a new piece of code without much review or polish. Also, there are only two release blockers left for 3.0.1, so we might just finish those and release, then concentrate on 3.1. Regards Antoine. From python at rcn.com Fri Jan 30 02:15:54 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 29 Jan 2009 17:15:54 -0800 Subject: [Python-Dev] pprint(iterator) References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> Message-ID: <6ACB41E9156B42DD87B138113FCBDFD7@RaymondLaptop1> > Along the lines of what others have said: pprint() cannot consume an > unknown iterator. Perhaps so. It's nice to have printing be free of side-effects (other than the actual printing). I've been working with 3.0 daily for several months (on a book project) and mostly think it's great. But sooner or later, we're going to have to address the issue about iterator reprs at the interactive prompt. This is a separate and more general issue than pprint(). My experience so far is that it is that the shift to more things being unviewable at the prompt is bit frustrating and makes the language more opaque. If that has been a source of irritation to me, then it will likely be more acutely felt by people who are starting out and are using the interactive prompt to explore the language. I don't know the right answer here (perhaps an alternate sys.displayhook). Just wanted to provide some early feedback based on my experiences heavily exercising 3.0. Raymond P.S. My other experience with 3.0 is that my most frequent error has changed. It used to be that the number reason for my getting a syntax error was leaving-off a colon. Now, my number one reason is omitting parens in a print() function call. I thought I would get used to it quickly, but it still comes up several times a day. From solipsis at pitrou.net Fri Jan 30 02:19:54 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 30 Jan 2009 01:19:54 +0000 (UTC) Subject: [Python-Dev] pprint(iterator) References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> <6ACB41E9156B42DD87B138113FCBDFD7@RaymondLaptop1> Message-ID: Raymond Hettinger rcn.com> writes: > > P.S. My other experience with 3.0 is that my most frequent error has > changed. It used to be that the number reason for my getting a syntax > error was leaving-off a colon. Now, my number one reason is > omitting parens in a print() function call. I thought I would get used to > it quickly, but it still comes up several times a day. I find myself with the reverse problem. When I code with 2.x, I often put parens around the argument list of a print staement. From stephen at xemacs.org Fri Jan 30 02:34:28 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 30 Jan 2009 10:34:28 +0900 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> Message-ID: <873af1efvf.fsf@xemacs.org> Raymond Hettinger writes: > My preference is to *not* mark it as experimental. Don't take the word "experimental" too seriously. It's clearly an exaggeration given the current state of 3.0.x. What is meant is an explicit announcement that the stability rules chosen in response to the bool-True-False brouhaha will be relaxed for the 3.0.x series *only*. > Instead, I prefer doing what it takes to make the 3.0.x series viable. That's not an "instead", that's two independent choices. The point is that most of the people who are voicing concerns fear precisely that policy. I think that the important question is "can the 3.0.x series be made 'viable' in less than the time frame for 3.1?" If not, I really have to think it's DOA from the point of view of folks who consider 3.0.0 non-viable. I think that's what Barry and Martin are saying. Guido is saying something different. AIUI, he's saying that explicitly introducing controlled instability into 3.0.x of the form "this is what the extremely stable non-buggy inherited-from-3.0 core of 3.1 is going to look like" will be a great service to those who consider 3.0.0 non-viable. The key point is that new features in 3.1 are generally going to be considered less reliable than those inherited from 3.0, and thus a debugged 3.0, even if the implementations have been unstable, provides a way for the very demanding to determine what that set is, and to test how it behaves in their applications. I think it's worth a try, after consultation with some of the major developers who are the ostensible beneficiaries. But if tried, I think it's important to mark 3.0.x as "not yet stable" even if the instability is carefully defined and controlled. From martin at v.loewis.de Fri Jan 30 03:27:03 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 30 Jan 2009 03:27:03 +0100 Subject: [Python-Dev] [Python-checkins] Merging to the 3.0 maintenance branch In-Reply-To: <498229BE.4060408@gmail.com> References: <20090129043706.5614B1E4002@bag.python.org> <1afaf6160901290804t3c08134egadfd14493dcb09ae@mail.gmail.com> <498229BE.4060408@gmail.com> Message-ID: <49826577.4030808@v.loewis.de> > There are potential problems with doing it that way [1]. The safer > option is to do: > > svn revert . > svnmerge merge -M -F I still don't see the potential problem. If you do svnmerge, svn commit, all is fine, right? The problem *only* arises if you do svnmerge, svn up, svn commit - and clearly, you shouldn't do that. If, on commit, you get a conflict, you should revert all your changes, svn up, and start all over with the merge. Regards, Martin From aahz at pythoncraft.com Fri Jan 30 03:33:28 2009 From: aahz at pythoncraft.com (Aahz) Date: Thu, 29 Jan 2009 18:33:28 -0800 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> Message-ID: <20090130023328.GA17511@panix.com> On Fri, Jan 30, 2009, Antoine Pitrou wrote: > Raymond Hettinger rcn.com> writes: >> >> * If you're thinking that shelves have very few users and that >> 3.0.0 has had few adopters, doesn't that mitigate the effects >> of making a better format available in 3.0.1? Wouldn't this >> be the time to do it? > > There was already another proposal for an sqlite-based dbm module, you may > want to synchronize with it: > http://bugs.python.org/issue3783 > > As I see it, the problem with introducing it in 3.0.1 is that we would > be rushing in a new piece of code without much review or polish. Also, > there are only two release blockers left for 3.0.1, so we might just > finish those and release, then concentrate on 3.1. There's absolutely no reason not to have a 3.0.2 before 3.1 comes out. You're probably right that what Raymond wants to is best not done for 3.0.1 -- but once we've agreed in principle that 3.0.x isn't a true production release of Python for PEP6 purposes, we can do "release early, release often". -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From martin at v.loewis.de Fri Jan 30 03:51:44 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 30 Jan 2009 03:51:44 +0100 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <873af1efvf.fsf@xemacs.org> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> <873af1efvf.fsf@xemacs.org> Message-ID: <49826B40.3010708@v.loewis.de> > Don't take the word "experimental" too seriously. It's clearly an > exaggeration given the current state of 3.0.x. What is meant is an > explicit announcement that the stability rules chosen in response to > the bool-True-False brouhaha will be relaxed for the 3.0.x series > *only*. The name for that shouldn't be "experimental", though. I don't think it needs any name at all. It would be sufficient to report, in the release announcement, and some stuff got removed in an incompatible way. This is also different from bool-True-False, which was an addition, not a removal. > I think that the important question is "can the 3.0.x series be made > 'viable' in less than the time frame for 3.1?" If not, I really have > to think it's DOA from the point of view of folks who consider 3.0.0 > non-viable. I think that's what Barry and Martin are saying. DOA == dead on arrival? I don't think Python 3.0 is dead. Instead, I think it is fairly buggy, but those bugs can be fixed. Removal of stuff is *not* a bug fix, of course. The *real* bugs in 3.0 is stuff like "IDLE doesn't work", "bdist_wininst doesn't work", etc. I personally can agree with removal of stuff (despite it not being a bug fix). However, more importantly, I want to support respective authority. If the release manager sets a policy on what is and what is not acceptable for a bug fix release, every committer should implement this policy (or at least not actively break it). With the removals in the code, I do think it is important to release 3.0.1 quickly, like, say, next week. > The key point is that new features in 3.1 are generally going to be > considered less reliable than those inherited from 3.0, and thus a > debugged 3.0, even if the implementations have been unstable, provides > a way for the very demanding to determine what that set is, and to > test how it behaves in their applications. That is fairly abstract. What specific bugs in Python 3.0 are you talking about? Regards, Martin From brett at python.org Fri Jan 30 03:52:00 2009 From: brett at python.org (Brett Cannon) Date: Thu, 29 Jan 2009 18:52:00 -0800 Subject: [Python-Dev] [Python-checkins] Merging to the 3.0 maintenance branch In-Reply-To: <49826577.4030808@v.loewis.de> References: <20090129043706.5614B1E4002@bag.python.org> <1afaf6160901290804t3c08134egadfd14493dcb09ae@mail.gmail.com> <498229BE.4060408@gmail.com> <49826577.4030808@v.loewis.de> Message-ID: On Thu, Jan 29, 2009 at 18:27, "Martin v. L?wis" wrote: >> There are potential problems with doing it that way [1]. The safer >> option is to do: >> >> svn revert . >> svnmerge merge -M -F > > I still don't see the potential problem. If you do svnmerge, svn commit, > all is fine, right? The problem *only* arises if you do svnmerge, > svn up, svn commit - and clearly, you shouldn't do that. If, on commit, > you get a conflict, you should revert all your changes, svn up, and > start all over with the merge. I did do that and I still got conflicts. -Brett From martin at v.loewis.de Fri Jan 30 04:03:54 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Fri, 30 Jan 2009 04:03:54 +0100 Subject: [Python-Dev] [Python-checkins] Merging to the 3.0 maintenance branch In-Reply-To: References: <20090129043706.5614B1E4002@bag.python.org> <1afaf6160901290804t3c08134egadfd14493dcb09ae@mail.gmail.com> <498229BE.4060408@gmail.com> <49826577.4030808@v.loewis.de> Message-ID: <49826E1A.6080809@v.loewis.de> Brett Cannon wrote: > On Thu, Jan 29, 2009 at 18:27, "Martin v. L?wis" wrote: >>> There are potential problems with doing it that way [1]. The safer >>> option is to do: >>> >>> svn revert . >>> svnmerge merge -M -F >> I still don't see the potential problem. If you do svnmerge, svn commit, >> all is fine, right? The problem *only* arises if you do svnmerge, >> svn up, svn commit - and clearly, you shouldn't do that. If, on commit, >> you get a conflict, you should revert all your changes, svn up, and >> start all over with the merge. > > I did do that and I still got conflicts. What is "that"? "svn revert -R" (plus rm for all added files), "svn up", "svnmerge", "svn revert ."? What conflicts? Martin From brett at python.org Fri Jan 30 04:06:13 2009 From: brett at python.org (Brett Cannon) Date: Thu, 29 Jan 2009 19:06:13 -0800 Subject: [Python-Dev] [Python-checkins] Merging to the 3.0 maintenance branch In-Reply-To: <49826E1A.6080809@v.loewis.de> References: <20090129043706.5614B1E4002@bag.python.org> <1afaf6160901290804t3c08134egadfd14493dcb09ae@mail.gmail.com> <498229BE.4060408@gmail.com> <49826577.4030808@v.loewis.de> <49826E1A.6080809@v.loewis.de> Message-ID: On Thu, Jan 29, 2009 at 19:03, "Martin v. L?wis" wrote: > Brett Cannon wrote: >> On Thu, Jan 29, 2009 at 18:27, "Martin v. L?wis" wrote: >>>> There are potential problems with doing it that way [1]. The safer >>>> option is to do: >>>> >>>> svn revert . >>>> svnmerge merge -M -F >>> I still don't see the potential problem. If you do svnmerge, svn commit, >>> all is fine, right? The problem *only* arises if you do svnmerge, >>> svn up, svn commit - and clearly, you shouldn't do that. If, on commit, >>> you get a conflict, you should revert all your changes, svn up, and >>> start all over with the merge. >> >> I did do that and I still got conflicts. > > What is "that"? "svn revert -R" (plus rm for all added files), > "svn up", "svnmerge", "svn revert ."? svn up svnmerge ... conflicts svn revert -R . svn up svnmerge ... same conflicts > > What conflicts? Some metadata on '.'. -Brett From rdmurray at bitdance.com Fri Jan 30 04:08:50 2009 From: rdmurray at bitdance.com (rdmurray at bitdance.com) Date: Thu, 29 Jan 2009 22:08:50 -0500 (EST) Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> Message-ID: On Thu, 29 Jan 2009 at 16:43, Raymond Hettinger wrote: >On Thu, 29 Jan 2009 at 15:40, Guido van Rossum wrote: >> Also you could try find shelve users (are there >> any?) > > I'm a big fan of shelves and have always used them extensively. > Not sure if anyone else cares about them though. I use them. Not in any released products at the moment, though, and I haven't migrated the shelve-using code to 3.0 yet. So I'd be in favor of adding sqlite3 support as soon as practical. --RDM From martin at v.loewis.de Fri Jan 30 04:09:17 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Fri, 30 Jan 2009 04:09:17 +0100 Subject: [Python-Dev] [Python-checkins] Merging to the 3.0 maintenance branch In-Reply-To: References: <20090129043706.5614B1E4002@bag.python.org> <1afaf6160901290804t3c08134egadfd14493dcb09ae@mail.gmail.com> <498229BE.4060408@gmail.com> <49826577.4030808@v.loewis.de> <49826E1A.6080809@v.loewis.de> Message-ID: <49826F5D.7000009@v.loewis.de> > svn up > svnmerge > ... conflicts > svn revert -R . > svn up > svnmerge > ... same conflicts Ah. In the 3.0 branch, always do "svn revert ." after svnmerge. It's ok (Nick says it isn't exactly ok, but I don't understand why) Martin From rrr at ronadam.com Fri Jan 30 04:23:55 2009 From: rrr at ronadam.com (Ron Adam) Date: Thu, 29 Jan 2009 21:23:55 -0600 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <498230C7.2040403@pearwood.info> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> <4981BBB7.50502@voidspace.org.uk> <498230C7.2040403@pearwood.info> Message-ID: <498272CB.5040503@ronadam.com> Steven D'Aprano wrote: > Michael Foord wrote: > >> Don't we have a pretty-print API - and isn't it spelled __str__ ? > > Not really. If it were as simple as calling str(obj), there would be no > need for the pprint module. I agree. And when I want to use pprint, there are usually additional output formatting requirements I need that isn't a "one size fits all" type of problem. In any case, it seems that the pprint module > actually calls repr() on objects other than dicts, tuples and lists. > > I'm concerned about the number of special methods exploding, but I've > also come across times where I needed more than two string > representations of an object. Sometimes I solved this by adding a > pprint() method, other times I used other names, and it would be nice if > there was a standard way of spelling it. So I'm +0.5 on Aahz's > suggestion of __pprint__. I'm -.5 on addint __pprint__ for the above reasons. > In my ideal world, __pprint__ should allow (but not require) extra > arguments, so that one can do something like the following: > > pprint(binarytree) # sensible defaults > pprint(binarytree, order='preorder') It seems to me pprint is one of those functions where output format specifiers and keywords make sense because you are trying to fit the data output of a wide variety of types to a particular output need. It's not reasonably possible for each type to predict what that output need is. Some of the options that sound useful might be: abbreviated form short form long complete detail form tree form column align form right or left margins and alignment options Think of it as how 'dir' is used for examining the contents of a disk drive where different output styles is useful at different times. Looking at it this way, instead of a __pprint__ method, a optional __pprint_style__ attribute could specify a default output style that the pprint function would fall back to. Maybe for iterators, it's not the data produced but rather the current state of use that is more useful? For example for partially consumed iterators it might be useful to express how many items have been taken, and how many are left to take when that info is available. (?) The idea is that pretty printing is usually used to check the status or state of something. Or at least that is how I use it. Ron From rrr at ronadam.com Fri Jan 30 04:23:55 2009 From: rrr at ronadam.com (Ron Adam) Date: Thu, 29 Jan 2009 21:23:55 -0600 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <498230C7.2040403@pearwood.info> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> <4981BBB7.50502@voidspace.org.uk> <498230C7.2040403@pearwood.info> Message-ID: <498272CB.5040503@ronadam.com> Steven D'Aprano wrote: > Michael Foord wrote: > >> Don't we have a pretty-print API - and isn't it spelled __str__ ? > > Not really. If it were as simple as calling str(obj), there would be no > need for the pprint module. I agree. And when I want to use pprint, there are usually additional output formatting requirements I need that isn't a "one size fits all" type of problem. In any case, it seems that the pprint module > actually calls repr() on objects other than dicts, tuples and lists. > > I'm concerned about the number of special methods exploding, but I've > also come across times where I needed more than two string > representations of an object. Sometimes I solved this by adding a > pprint() method, other times I used other names, and it would be nice if > there was a standard way of spelling it. So I'm +0.5 on Aahz's > suggestion of __pprint__. I'm -.5 on addint __pprint__ for the above reasons. > In my ideal world, __pprint__ should allow (but not require) extra > arguments, so that one can do something like the following: > > pprint(binarytree) # sensible defaults > pprint(binarytree, order='preorder') It seems to me pprint is one of those functions where output format specifiers and keywords make sense because you are trying to fit the data output of a wide variety of types to a particular output need. It's not reasonably possible for each type to predict what that output need is. Some of the options that sound useful might be: abbreviated form short form long complete detail form tree form column align form right or left margins and alignment options Think of it as how 'dir' is used for examining the contents of a disk drive where different output styles is useful at different times. Looking at it this way, instead of a __pprint__ method, a optional __pprint_style__ attribute could specify a default output style that the pprint function would fall back to. Maybe for iterators, it's not the data produced but rather the current state of use that is more useful? For example for partially consumed iterators it might be useful to express how many items have been taken, and how many are left to take when that info is available. (?) The idea is that pretty printing is usually used to check the status or state of something. Or at least that is how I use it. Ron From brett at python.org Fri Jan 30 04:59:39 2009 From: brett at python.org (Brett Cannon) Date: Thu, 29 Jan 2009 19:59:39 -0800 Subject: [Python-Dev] 3.0.1/3.1.0 summary Message-ID: This is my attempt to summarize what everyone has been saying so we can get this resolved. >From what I can tell, most people like the idea of doing a 3.0.1 release ASAP (like "in a week or so" fast) with the stuff that should have been removed from 3.0.0 in the first place removed. People also seem to support doing a 3.1 release April/May where new stuff (e.g. io in C, new shelve back-end for sqlite3) is introduced to the rest of the world. This timeline has the benefit of allowing us to do an alpha release at PyCon and puts us at a six month release cycle which does not portray 3.0 or 3.1 as rushed releases. The sticky points I see are: 1. Barry, who is the release manager for 3.0.1, does not like the idea of the cruft that is being proposed removed from 3.0.1. Personally I say we continue to peer pressure him as I think a new major release is not like our typical minor release, but I am not about to force Barry to go against what he thinks is reasonable unless I am willing to step up as release manager (and I am not since I simply don't have the time to learn the process fast enough along with just a lack of time with other Python stuff). 2. Do we label 3.0.x as experimental? I say no since it isn't experimental; we basically had some bugs slip through that happen to be compatibility problems that were overlooked. I for one never viewed 3.0.x as experimental, just not the best we could necessarily do without more input from the community and our own experience with 3.x in general. Let's see if we can get these two points squared away so we can get 3.0.1 in whatever state it is meant to be in out the door quickly. -Brett From stephen at xemacs.org Fri Jan 30 05:08:52 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 30 Jan 2009 13:08:52 +0900 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <49826B40.3010708@v.loewis.de> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> <873af1efvf.fsf@xemacs.org> <49826B40.3010708@v.loewis.de> Message-ID: <87tz7hcu5n.fsf@xemacs.org> "Martin v. L?wis" writes: > > Don't take the word "experimental" too seriously. What is meant > > is an explicit announcement that the stability rules will be > > relaxed for the 3.0.x series *only*. > > The name for that shouldn't be "experimental", though. I don't think > it needs any name at all. That's what I meant. I'm sure that whoever wrote the word "experimental" in the first place regrets it, because it doesn't reflect what they meant. > > I think that the important question is "can the 3.0.x series be made > > 'viable' in less than the time frame for 3.1?" If not, I really have > > to think it's DOA from the point of view of folks who consider 3.0.0 > > non-viable. I think that's what Barry and Martin are saying. > > DOA == dead on arrival? I don't think Python 3.0 is dead. I'm sorry, DOA was poor word choice, especially this context. I meant that people who currently consider 3.0 non-viable are more likely to focus on the branch that will become 3.1 unless a "viable" 3.0.x will arrive *very* quickly. > That is fairly abstract. What specific bugs in Python 3.0 are you > talking about? I'm not talking about specific bugs; I'm perfectly happy with 3.0 for my purposes, and I think it very unlikely that any of the possibly destabilizing changes that have been proposed for 3.0.1 will affect me adversely. Rather, I'm trying to disentangle some of the unfortunate word choices that have been made (and I apologize for making one of my own!), and find common ground so that a policy can be set more quickly. IMO it's likely that there's really no audience for a 3.0.x series that conforms to the rules used for 2.x from 2.2.1 or so on. That is, there are people who really don't care because 3.0 is already a better platform for their application whether there are minor changes or not, and there are people who do care about stability but they're not going to use 3.0.x whether it adheres to the previous rules strictly or not. There are very few who will use 3.0.x if and only if it adheres strictly. From guido at python.org Fri Jan 30 05:11:02 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 29 Jan 2009 20:11:02 -0800 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> Message-ID: On Thu, Jan 29, 2009 at 4:58 PM, Raymond Hettinger wrote: > A couple additional thoughts FWIW: > > * whichdb() selects from multiple file formats, so 3.0.1 would still > be able to read 3.0.0 files. It is the 2.x shelves that won't be > readable at all under any scenario. > > * If you're thinking that shelves have very few users and that > 3.0.0 has had few adopters, doesn't that mitigate the effects > of making a better format available in 3.0.1? Wouldn't this > be the time to do it? > > * The file format itself is not new or unreadable by 3.0.0. > It is just a plain sqlite3 file. Was is new is the ability > of shelve's to call sqlite. To me, that is something a little > different than changing a pickle protocol or somesuch. Sorry, not convinced. This is a change of a different scale than removing stuff that should've been removed. I understand you'd like to see your baby released. But I think it's better to have it tried and tested by others *outside* the core distro first. dbm is not broken in 3.0, just slow. Well so be it, io.py is too and that's a lot more serious. I also note that on some systems at least ndbm and/or gdbm are supported. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Fri Jan 30 05:25:53 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 29 Jan 2009 20:25:53 -0800 Subject: [Python-Dev] Python 3.0.1 References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> Message-ID: [Guido van Rossum] > Sorry, not convinced. No worries. Py3.1 is not far off. Just so I'm clear. Are you thinking that 3.0.x will never have fast shelves, or are you thinking 3.0.2 or 3.0.3 after some external deployment and battle-testing for the module? Raymond From tjreedy at udel.edu Fri Jan 30 05:27:14 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 29 Jan 2009 23:27:14 -0500 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <498272CB.5040503@ronadam.com> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> <4981BBB7.50502@voidspace.org.uk> <498230C7.2040403@pearwood.info> <498272CB.5040503@ronadam.com> Message-ID: Ron Adam wrote: > > > Steven D'Aprano wrote: >> Michael Foord wrote: >> >>> Don't we have a pretty-print API - and isn't it spelled __str__ ? >> >> Not really. If it were as simple as calling str(obj), there would be >> no need for the pprint module. > > I agree. And when I want to use pprint, there are usually additional > output formatting requirements I need that isn't a "one size fits all" > type of problem. Like others, I am wary of over-expanding the list of special methods. Perhap format strings could have a fourth conversion specifier, !p (pretty) in addition to !s, !r, and !a. From tjreedy at udel.edu Fri Jan 30 05:44:16 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 29 Jan 2009 23:44:16 -0500 Subject: [Python-Dev] 3.0.1/3.1.0 summary In-Reply-To: References: Message-ID: Brett Cannon wrote: > This is my attempt to summarize what everyone has been saying so we > can get this resolved. > >>From what I can tell, most people like the idea of doing a 3.0.1 > release ASAP (like "in a week or so" fast) with the stuff that should > have been removed from 3.0.0 in the first place removed. > > People also seem to support doing a 3.1 release April/May where new > stuff (e.g. io in C, new shelve back-end for sqlite3) is introduced to > the rest of the world. This timeline has the benefit of allowing us to > do an alpha release at PyCon and puts us at a six month release cycle > which does not portray 3.0 or 3.1 as rushed releases. > > The sticky points I see are: > > 1. Barry, who is the release manager for 3.0.1, does not like the idea > of the cruft that is being proposed removed from 3.0.1. Personally I > say we continue to peer pressure him as I think a new major release is > not like our typical minor release, but I am not about to force Barry > to go against what he thinks is reasonable unless I am willing to step > up as release manager (and I am not since I simply don't have the time > to learn the process fast enough along with just a lack of time with > other Python stuff). While I prefer cruft removal now, I will, for the same reason, accept and use whatever whatever Barry delivers. > 2. Do we label 3.0.x as experimental? I say no since it isn't > experimental; we basically had some bugs slip through that happen to > be compatibility problems that were overlooked. I for one never viewed > 3.0.x as experimental, just not the best we could necessarily do > without more input from the community and our own experience with 3.x > in general. It is normal for true x.0 releases to be slightly flakey. Experienced users typically wait for x.1 (or SP1) releases for building production systems. I understand that 'normal' is below Python's usual high standards, but it is not a tragedy ;-). > Let's see if we can get these two points squared away so we can get > 3.0.1 in whatever state it is meant to be in out the door quickly. +1 Terry From nnorwitz at gmail.com Fri Jan 30 06:12:48 2009 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 29 Jan 2009 21:12:48 -0800 Subject: [Python-Dev] python breakpoint opcode In-Reply-To: <014f01c981fd$b7bb3c60$2731b520$@com> References: <014f01c981fd$b7bb3c60$2731b520$@com> Message-ID: On Thu, Jan 29, 2009 at 2:38 AM, Dr Andrew Perella wrote: > Hi, > > I was thinking of adding a breakpoint opcode to python to enable less > invasive debugging. > > I came across posts from 1999 by Vladimir Marangozov and Christian Tismer > discussing this issue but the links to the code are all out of date. Can you provide the links? n From martin at v.loewis.de Fri Jan 30 06:53:57 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 30 Jan 2009 06:53:57 +0100 Subject: [Python-Dev] 3.0.1/3.1.0 summary In-Reply-To: References: Message-ID: <498295F5.2050607@v.loewis.de> > 1. Barry, who is the release manager for 3.0.1, does not like the idea > of the cruft that is being proposed removed from 3.0.1. I don't think he actually said that (in fact, I think he said the opposite). It would be good if he clarified, though. Regards, Martin From martin at v.loewis.de Fri Jan 30 06:56:24 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 30 Jan 2009 06:56:24 +0100 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> Message-ID: <49829688.3090200@v.loewis.de> > Just so I'm clear. Are you thinking that 3.0.x will never have > fast shelves As Guido said, shelves are *already* fast in 3.0, if you are using the right operating system. Regards, Martin From eric at trueblade.com Fri Jan 30 07:58:35 2009 From: eric at trueblade.com (Eric Smith) Date: Fri, 30 Jan 2009 01:58:35 -0500 Subject: [Python-Dev] pprint(iterator) In-Reply-To: References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> <4981BBB7.50502@voidspace.org.uk> <498230C7.2040403@pearwood.info> <498272CB.5040503@ronadam.com> Message-ID: <4982A51B.5020801@trueblade.com> Terry Reedy wrote: > Ron Adam wrote: >> >> >> Steven D'Aprano wrote: >>> Michael Foord wrote: >>> >>>> Don't we have a pretty-print API - and isn't it spelled __str__ ? >>> >>> Not really. If it were as simple as calling str(obj), there would be >>> no need for the pprint module. >> >> I agree. And when I want to use pprint, there are usually additional >> output formatting requirements I need that isn't a "one size fits all" >> type of problem. I don't see how you can have a standard interface (like __pprint__), and have additional, per-object formatting parameters. But that's beside the point, I don't like __pprint__ in any event. Too special. > Like others, I am wary of over-expanding the list of special methods. > Perhap format strings could have a fourth conversion specifier, !p > (pretty) in addition to !s, !r, and !a. What would format() do with "!p"? With "!s", it calls str(o), with "!r", it calls repr(o). "!p" could call o.__pprint__(), but that's the special method you're trying to avoid! (I don't recall if I added "!a", and a machine that would know isn't available to me just now.) Eric. From solipsis at pitrou.net Fri Jan 30 11:40:10 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 30 Jan 2009 10:40:10 +0000 (UTC) Subject: [Python-Dev] Python 3.0.1 References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> <20090130023328.GA17511@panix.com> Message-ID: Aahz pythoncraft.com> writes: > > There's absolutely no reason not to have a 3.0.2 before 3.1 comes out. > You're probably right that what Raymond wants to is best not done for > 3.0.1 -- but once we've agreed in principle that 3.0.x isn't a true > production release of Python for PEP6 purposes, we can do "release early, > release often". It's a possibility. To be honest, I didn't envision us releasing a 3.0.2 rather than focusing on 3.1 (which, as others said, can be released in a few months if we keep the amount of changes under control). But then it's only a matter of naming. We can continue the 3.0.x series and incorporate in them whatever was initially planned for 3.1 (including the IO-in-C branch, the dbm.sqlite module, etc.), and release 3.1 only when the whole thing is "good enough". Regards Antoine. From steve at pearwood.info Fri Jan 30 12:04:40 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 30 Jan 2009 22:04:40 +1100 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <4982A51B.5020801@trueblade.com> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> <4981BBB7.50502@voidspace.org.uk> <498230C7.2040403@pearwood.info> <498272CB.5040503@ronadam.com> <4982A51B.5020801@trueblade.com> Message-ID: <4982DEC8.90900@pearwood.info> Eric Smith wrote: > Terry Reedy wrote: >> Ron Adam wrote: >>> >>> >>> Steven D'Aprano wrote: >>>> Michael Foord wrote: >>>> >>>>> Don't we have a pretty-print API - and isn't it spelled __str__ ? >>>> >>>> Not really. If it were as simple as calling str(obj), there would be >>>> no need for the pprint module. >>> >>> I agree. And when I want to use pprint, there are usually additional >>> output formatting requirements I need that isn't a "one size fits >>> all" type of problem. > > I don't see how you can have a standard interface (like __pprint__), and > have additional, per-object formatting parameters. I don't see how you can't. Other standard methods take variable arguments: __init__, __new__, __call__ come to mind. > But that's beside the > point, I don't like __pprint__ in any event. Too special. I'm not sure what you mean by "too special". It's no more special than any other special method. Do you mean the use-case is not common enough? I would find this useful. Whether enough people would find it useful enough to add yet another special method is an open question. -- Steven From ajp at eutechnyx.com Fri Jan 30 11:45:27 2009 From: ajp at eutechnyx.com (Dr Andrew Perella) Date: Fri, 30 Jan 2009 10:45:27 -0000 Subject: [Python-Dev] python breakpoint opcode In-Reply-To: References: <014f01c981fd$b7bb3c60$2731b520$@com> Message-ID: <004e01c982c7$dcf84250$96e8c6f0$@com> Hi Neal, The last post in the thread was: http://mail.python.org/pipermail/python-dev/1999-August/000793.html referencing a download at http://sirac.inrialpes.fr/~marangoz/python/lineno/ Cheers, Andrew This e-mail is confidential and may be privileged. It may be read, copied and used only by the intended recipient. No communication sent by e-mail to or from Eutechnyx is intended to give rise to contractual or other legal liability, apart from liability which cannot be excluded under English law. This email has been scanned for all known viruses by the Email Protection Agency. http://www.epagency.net www.eutechnyx.com Eutechnyx Limited. Registered in England No: 2172322 From mal at egenix.com Fri Jan 30 12:24:00 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 30 Jan 2009 12:24:00 +0100 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> <20090130023328.GA17511@panix.com> Message-ID: <4982E350.9050206@egenix.com> On 2009-01-30 11:40, Antoine Pitrou wrote: > Aahz pythoncraft.com> writes: >> There's absolutely no reason not to have a 3.0.2 before 3.1 comes out. >> You're probably right that what Raymond wants to is best not done for >> 3.0.1 -- but once we've agreed in principle that 3.0.x isn't a true >> production release of Python for PEP6 purposes, we can do "release early, >> release often". > > It's a possibility. To be honest, I didn't envision us releasing a 3.0.2 rather > than focusing on 3.1 (which, as others said, can be released in a few months if > we keep the amount of changes under control). > > But then it's only a matter of naming. We can continue the 3.0.x series and > incorporate in them whatever was initially planned for 3.1 (including the > IO-in-C branch, the dbm.sqlite module, etc.), and release 3.1 only when the > whole thing is "good enough". That would be my preference. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 30 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From steve at holdenweb.com Fri Jan 30 13:03:03 2009 From: steve at holdenweb.com (Steve Holden) Date: Fri, 30 Jan 2009 07:03:03 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> Message-ID: Antoine Pitrou wrote: > Raymond Hettinger rcn.com> writes: >> * If you're thinking that shelves have very few users and that >> 3.0.0 has had few adopters, doesn't that mitigate the effects >> of making a better format available in 3.0.1? Wouldn't this >> be the time to do it? > > There was already another proposal for an sqlite-based dbm module, you may > want to synchronize with it: > http://bugs.python.org/issue3783 > > As I see it, the problem with introducing it in 3.0.1 is that we would be > rushing in a new piece of code without much review or polish. Again > Also, there are > only two release blockers left for 3.0.1, so we might just finish those and > release, then concentrate on 3.1. > Seems to me that every deviation from the policy introduced as a result for the True/False debacle leads to complications and problems. There's no point having a policy instigated for good reasons if we can ignore those reasons on a whim. So to my mind, ignoring the policy *is* effectively declaring 3.0 to be, well, if not a dead parrot then at least a rushed release. Most consistently missing from this picture has been effective communications (in both directions) with the user base. Consequently nobody knows whether specific features are in serious use, and nobody knows whether 3.0 is intended to be a stable base for production software or not. Ignoring users, and acting as though we know what they are doing and what they want, is not going to lead to better acceptance of future releases. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From p.f.moore at gmail.com Fri Jan 30 13:02:24 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 30 Jan 2009 12:02:24 +0000 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <4982DEC8.90900@pearwood.info> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> <4981BBB7.50502@voidspace.org.uk> <498230C7.2040403@pearwood.info> <498272CB.5040503@ronadam.com> <4982A51B.5020801@trueblade.com> <4982DEC8.90900@pearwood.info> Message-ID: <79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com> 2009/1/30 Steven D'Aprano : >> But that's beside the >> >> point, I don't like __pprint__ in any event. Too special. > > I'm not sure what you mean by "too special". It's no more special than any > other special method. Do you mean the use-case is not common enough? I would > find this useful. Whether enough people would find it useful enough to add > yet another special method is an open question. In my view, the issue is that as a special method, *either* it has to be included on all core types (too intrusive for something as non-critical as pprint) *or* pprint has to hard-code the behaviour for core types and still fall back to the special method for non-core types (ugly and a maintenance problem keeping the type tests up to date). Some sort of registry of type-specific implementation functions (whether you call it a generic function or just put together a custom implementation for pprint alone) is more flexible, and less intrusive. It also allows end users to customise the behaviour, even for core types. In all honesty, I think pkgutil.simplegeneric should be documented, exposed, and moved to a library of its own[1]. It's precisely what is needed for this type of situation, which does come up fairly often. I don't think ABCs do what's needed here (although maybe I'm missing something - if so, I'd be interested in knowing what). I'd be willing to look at creating a patch, if the consensus was that this was an appropriate approach and there was a reasonable chance of it being accepted (assuming my code wasn't rubbish :-)) Paul. [1] Note - I have no opinion on the quality of the code, I haven't reviewed it, I am assuming it's OK on the basis that it has been present and in use internally in the pkgutil module for some time now. From ncoghlan at gmail.com Fri Jan 30 13:21:02 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 30 Jan 2009 22:21:02 +1000 Subject: [Python-Dev] [Python-checkins] Merging to the 3.0 maintenance branch In-Reply-To: <49826F5D.7000009@v.loewis.de> References: <20090129043706.5614B1E4002@bag.python.org> <1afaf6160901290804t3c08134egadfd14493dcb09ae@mail.gmail.com> <498229BE.4060408@gmail.com> <49826577.4030808@v.loewis.de> <49826E1A.6080809@v.loewis.de> <49826F5D.7000009@v.loewis.de> Message-ID: <4982F0AE.20308@gmail.com> Martin v. L?wis wrote: >> svn up >> svnmerge >> ... conflicts >> svn revert -R . >> svn up >> svnmerge >> ... same conflicts > > Ah. In the 3.0 branch, always do "svn revert ." after svnmerge. > It's ok (Nick says it isn't exactly ok, but I don't understand why) Doing "svn revert ." before making the commit will lose the metadata changes that svnmerge uses for its bookkeeping (i.e. if this practice is used regularly, the tool will completely lose track of which revisions have already been merged). That won't bother those of us that are only backporting cherry-picked revisions, but is rather inconvenient for anyone checking for revisions that haven't been backported yet, but haven't been explicitly blocked either. Doing "svn resolved ." assumes that you did everything else correctly, and even then I don't see how svnmerge could both backport the py3k changes to the metadata and make its own changes and still get the metadata to a sane state. The consequence of getting this approach wrong is that the merge state of the 3.0 maintenance branch can be clobbered completely (losing track both of which revisions have been backported and which have been blocked). Doing both "svn revert ." and "svnmerge merge -M -F " clears out the conflicted metadata and then correctly updates the metadata for the revisions that have been backported. It will always update the svnmerge metadata correctly, regardless of the relative order of the svnmerge and svn update operations. Given the choice of a method which will always do the right thing, over one which always does the wrong thing and another one which only does the right thing if I did two other things in the right order and will completely trash the bookkeeping if I get it wrong, I prefer the option which is guaranteed to be correct (even if it happens to be a little slower as svnmerge recreates the needed metadata updates). If there's something wrong with my understanding of either svn properties or the operation of svnmerge that means the quicker approaches aren't as broken as I think they are, then I'd be happy to adopt one of them (since they *are* faster than my current approach). But until someone pokes a hole in my logic, I'll stick with the slower-but-always-correct methodology (and continue advocating that approach to everyone else doing updates that affect all four branches). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From phil at riverbankcomputing.com Fri Jan 30 13:21:39 2009 From: phil at riverbankcomputing.com (Phil Thompson) Date: Fri, 30 Jan 2009 12:21:39 +0000 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> Message-ID: <8044d27e31e3f09373f63f87d77bedd5@localhost> On Fri, 30 Jan 2009 07:03:03 -0500, Steve Holden wrote: > Antoine Pitrou wrote: >> Raymond Hettinger rcn.com> writes: >>> * If you're thinking that shelves have very few users and that >>> 3.0.0 has had few adopters, doesn't that mitigate the effects >>> of making a better format available in 3.0.1? Wouldn't this >>> be the time to do it? >> >> There was already another proposal for an sqlite-based dbm module, you >> may >> want to synchronize with it: >> http://bugs.python.org/issue3783 >> >> As I see it, the problem with introducing it in 3.0.1 is that we would be >> rushing in a new piece of code without much review or polish. > > Again > >> Also, there are >> only two release blockers left for 3.0.1, so we might just finish those >> and >> release, then concentrate on 3.1. >> > Seems to me that every deviation from the policy introduced as a result > for the True/False debacle leads to complications and problems. There's > no point having a policy instigated for good reasons if we can ignore > those reasons on a whim. > > So to my mind, ignoring the policy *is* effectively declaring 3.0 to be, > well, if not a dead parrot then at least a rushed release. > > Most consistently missing from this picture has been effective > communications (in both directions) with the user base. Consequently > nobody knows whether specific features are in serious use, and nobody > knows whether 3.0 is intended to be a stable base for production > software or not. Ignoring users, and acting as though we know what they > are doing and what they want, is not going to lead to better acceptance > of future releases. My 2 cents as a user... I wouldn't consider v3.0.n (where n is small) for use in production. v3.1 however implies (to me at least) a level of quality where I would be disappointed if it wasn't production ready. Therefore I would suggest the main purpose of any v3.0.1 release is to make sure that v3.1 is up to scratch. Phil From eric at trueblade.com Fri Jan 30 13:33:19 2009 From: eric at trueblade.com (Eric Smith) Date: Fri, 30 Jan 2009 07:33:19 -0500 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <4982DEC8.90900@pearwood.info> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> <4981BBB7.50502@voidspace.org.uk> <498230C7.2040403@pearwood.info> <498272CB.5040503@ronadam.com> <4982A51B.5020801@trueblade.com> <4982DEC8.90900@pearwood.info> Message-ID: <4982F38F.8030908@trueblade.com> Steven D'Aprano wrote: > Eric Smith wrote: >> Terry Reedy wrote: >>> Ron Adam wrote: >>>> >>>> >>>> Steven D'Aprano wrote: >>>>> Michael Foord wrote: >>>>> >>>>>> Don't we have a pretty-print API - and isn't it spelled __str__ ? >>>>> >>>>> Not really. If it were as simple as calling str(obj), there would >>>>> be no need for the pprint module. >>>> >>>> I agree. And when I want to use pprint, there are usually >>>> additional output formatting requirements I need that isn't a "one >>>> size fits all" type of problem. >> >> I don't see how you can have a standard interface (like __pprint__), >> and have additional, per-object formatting parameters. > > I don't see how you can't. Other standard methods take variable > arguments: __init__, __new__, __call__ come to mind. Those are different, since they're called on known specific objects. Having params to a generic __pprint__ method would be more like having params to __str__ or __repr__. If you know enough about the object to know which parameters to pass to its pretty-print function, then just call a normal method on the object to do the pprint'ing. But, for example, assuming pprint for a list is recursive (as it is for repr), how would you pass the arguments around? > > But that's beside the >> point, I don't like __pprint__ in any event. Too special. > > I'm not sure what you mean by "too special". It's no more special than > any other special method. Do you mean the use-case is not common enough? > I would find this useful. Whether enough people would find it useful > enough to add yet another special method is an open question. Bad choice of words on my part. I meant "too special case" for such machinery. That is, the use case isn't common enough. From ncoghlan at gmail.com Fri Jan 30 13:38:13 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 30 Jan 2009 22:38:13 +1000 Subject: [Python-Dev] [Python-checkins] Merging to the 3.0 maintenance branch In-Reply-To: <49826577.4030808@v.loewis.de> References: <20090129043706.5614B1E4002@bag.python.org> <1afaf6160901290804t3c08134egadfd14493dcb09ae@mail.gmail.com> <498229BE.4060408@gmail.com> <49826577.4030808@v.loewis.de> Message-ID: <4982F4B5.7090001@gmail.com> Martin v. L?wis wrote: >> There are potential problems with doing it that way [1]. The safer >> option is to do: >> >> svn revert . >> svnmerge merge -M -F > > I still don't see the potential problem. If you do svnmerge, svn commit, > all is fine, right? Sort of. svnmerge still gets confused by the fact that the revision being backported already has changes to the svnmerge metadata, so you have to either revert it (which is always wrong), or flag it as resolved (I believe that svnmerge actually does get that case right, but I haven't checked it extensively - since if it does get it right, I don't understand why it leaves the conflict in place instead of automatically marking it as resolved). Regardless, the consequences of forgetting that you did the svn up after the merge instead of before (e.g. if it took some time to get the backported version working, or if something interrupted you between the initial backport/update and the final test and commit step) are fairly hard to clean up, so I prefer the safe approach (despite the extra minute or two it takes for svnmerge to recalculate the metadata changes). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From solipsis at pitrou.net Fri Jan 30 13:38:15 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 30 Jan 2009 12:38:15 +0000 (UTC) Subject: [Python-Dev] [Python-checkins] Merging to the 3.0 maintenance branch References: <20090129043706.5614B1E4002@bag.python.org> <1afaf6160901290804t3c08134egadfd14493dcb09ae@mail.gmail.com> <498229BE.4060408@gmail.com> <49826577.4030808@v.loewis.de> <49826E1A.6080809@v.loewis.de> <49826F5D.7000009@v.loewis.de> <4982F0AE.20308@gmail.com> Message-ID: Nick Coghlan gmail.com> writes: > > Doing "svn resolved ." assumes that you did everything else correctly, > and even then I don't see how svnmerge could both backport the py3k > changes to the metadata and make its own changes and still get the > metadata to a sane state. The metadata are discriminated by source merge URL. That is, the py3k metadata are of the form "/python/trunk: " while the release30-maint metadata are of the form "/python/branches/py3k: ". (*) I guess that's what allows svn to not shoot itself in the foot when merging. I did "svn resolved ." again yesterday and it doesn't seem to have borked anything. (*) (try "svn propget svnmerge-integrated" or "svn propget svnmerge-blocked") Regards Antoine. From ncoghlan at gmail.com Fri Jan 30 14:02:19 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 30 Jan 2009 23:02:19 +1000 Subject: [Python-Dev] [Python-checkins] Merging to the 3.0 maintenance branch In-Reply-To: References: <20090129043706.5614B1E4002@bag.python.org> <1afaf6160901290804t3c08134egadfd14493dcb09ae@mail.gmail.com> <498229BE.4060408@gmail.com> <49826577.4030808@v.loewis.de> <49826E1A.6080809@v.loewis.de> <49826F5D.7000009@v.loewis.de> <4982F0AE.20308@gmail.com> Message-ID: <4982FA5B.7000507@gmail.com> Antoine Pitrou wrote: > Nick Coghlan gmail.com> writes: >> Doing "svn resolved ." assumes that you did everything else correctly, >> and even then I don't see how svnmerge could both backport the py3k >> changes to the metadata and make its own changes and still get the >> metadata to a sane state. > > The metadata are discriminated by source merge URL. That is, the py3k metadata > are of the form "/python/trunk: " while the release30-maint > metadata are of the form "/python/branches/py3k: ". (*) > I guess that's what allows svn to not shoot itself in the foot when merging. Ah, thanks - that's the piece I was missing regarding why the svn resolved trick works (I have used that approach before and checked it as you did - as Martin has pointed out, the only time it definitely goes wrong is if you do an update *after* doing the local merge and the update included other backports). So I'll chalk the fact that svnmerge emits that false alarm up to a deficiency in the tool and only use the "regenerate the metadata" approach when I suspect I may have done the merge+update in the wrong order (since it's a harmless thing to do - it just wastes a couple of minutes relative to the svn resolved approach). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From skip at pobox.com Fri Jan 30 14:04:27 2009 From: skip at pobox.com (skip at pobox.com) Date: Fri, 30 Jan 2009 07:04:27 -0600 Subject: [Python-Dev] Universal newlines, and the gzip module. In-Reply-To: <49821403.1030603@noaa.gov> References: <49821403.1030603@noaa.gov> Message-ID: <18818.64219.137713.112225@montanaro.dyndns.org> Christopher> 1) It would be nice if the gzip module (and the zip lib Christopher> module) supported Universal newlines -- you could read a Christopher> compressed text file with "wrong" newlines, and have Christopher> them handled properly. However, that may be hard to do, Christopher> so at least: Christopher> 2) Passing a 'U' flag in to gzip.open shouldn't break it. I agree with Brett that 'U' is meaningless on the compressed file itself. You want it applied to the contents of the compressed file though, is that right? That makes sense to me. It probably belongs in a separate argument though. Skip From p.f.moore at gmail.com Fri Jan 30 14:28:15 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 30 Jan 2009 13:28:15 +0000 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> Message-ID: <79990c6b0901300528h160c0c81t74e44497fbcf709e@mail.gmail.com> 2009/1/30 Steve Holden : > Most consistently missing from this picture has been effective > communications (in both directions) with the user base. Serious question: does anybody know how to get better communication from the user base? My impression is that it's pretty hard to find out who is actually using 3.0, and get any feedback from them. I suppose a general query on clp might get some feedback, but otherwise, what? I've not seen any significant amount of blog activity on 3.0. As a small contribution, my position is as follows: I use Python mostly for one-off scripts, both at home and at work. I also use Python for a suite of database monitoring tools, as well as using some applications written in Python (Mercurial and MoinMoin, in particular). Ignore the applications, they aren't moving to 3.0 in the short term (based on comments from the application teams). For my own use, the key modules I need are cx_Oracle and pywin32. cx_Oracle was available for 3.0 very quickly (and apparently the port wasn't too hard, which is good feedback!). pywin32 is just now available in preview form. My production box is still using 2.5, and I will probably migrate to 2.6 in due course - but I'll probably leave 3.0 for the foreseeable future (I may rethink if MoinMoin becomes available on 3.0 sooner rather than later). For my desktop PC, I'm using 2.6 but as I do a fair bit of experimenting with modules, I'm taking it slowly (I'd like to see 2.6 binaries for a few more packages, really). I have 3.0 installed, but not as default, so frankly it doesn't get used unless I'm deliberately trying it out. Based on the recent threads, I'm thinking I really should make 3.0 the default just to get a better feel for it. The io-in-C changes would probably help push me to doing so (performance isn't really an issue for me, but I find I'm irrationally swayed by the "3.0 io is slow, but it's getting fixed soon by the io-in-C rewrite" messages I've been seeing - I have no idea if that's a general impression, or just a result of my following python-dev, though). It would make no difference to me, personally, whether *any* of the changes being discussed were released in 3.0.1 or 3.1 (except insofar as I'd like to see them sooner rather than later). So, in summary, for practical purposes I use 2.6. I probably could use 3.0 for a significant proportion of my needs, but the impressions I've been getting make me cautious. I'm using Windows, and although I *can* build a lot of stuff myself, I really don't want to be bothered, so I rely on bdist_wininst installers being available, which is an additional constraint. Paul. From p.f.moore at gmail.com Fri Jan 30 15:51:12 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 30 Jan 2009 14:51:12 +0000 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <49830FB1.3060306@livinglogic.de> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> <4981BBB7.50502@voidspace.org.uk> <498230C7.2040403@pearwood.info> <498272CB.5040503@ronadam.com> <4982A51B.5020801@trueblade.com> <4982DEC8.90900@pearwood.info> <79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com> <49830FB1.3060306@livinglogic.de> Message-ID: <79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com> 2009/1/30 Walter D?rwald : > Paul Moore wrote: > >> [...] >> In all honesty, I think pkgutil.simplegeneric should be documented, >> exposed, and moved to a library of its own[1]. > > http://pypi.python.org/pypi/simplegeneric Thanks, I was aware of that. I assume that the barrier to getting this into the stdlib will be higher than to simply exposing an implementation already available in the stdlib. To be honest, all I would like is for these regular "let's have another special method" discussions to become unnecessary... Paul. From walter at livinglogic.de Fri Jan 30 15:33:21 2009 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Fri, 30 Jan 2009 15:33:21 +0100 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> <4981BBB7.50502@voidspace.org.uk> <498230C7.2040403@pearwood.info> <498272CB.5040503@ronadam.com> <4982A51B.5020801@trueblade.com> <4982DEC8.90900@pearwood.info> <79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com> Message-ID: <49830FB1.3060306@livinglogic.de> Paul Moore wrote: > [...] > In all honesty, I think pkgutil.simplegeneric should be documented, > exposed, and moved to a library of its own[1]. http://pypi.python.org/pypi/simplegeneric > [...] Servus, Walter From barry at python.org Fri Jan 30 16:16:53 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 30 Jan 2009 10:16:53 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <49809B0C.4020905@egenix.com> <49809C1B.3090805@voidspace.org.uk> <4980CD7A.6000506@v.loewis.de> <87d4e7dj0p.fsf@xemacs.org> <20090129113130.GA2490@amk.local> <7F2AD110-96A2-4367-821E-6DB8E2E3DC86@python.org> Message-ID: <16513B54-BBD7-4640-AD40-EE8B8B6FCE78@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 29, 2009, at 1:13 PM, Guido van Rossum wrote: > I'd like to find a middle ground. We can all agree that the users of > 3.0 are a small minority compared to the 2.x users. Therefore I think > we can bend the rules more than we have done for the recent 2.x > releases. Those rules weren't always there (anyone remember the > addition of bool, True and False to 2.2.1?). The rules were introduced > for the benefit of our most conservative users -- people who introduce > Python in an enterprise and don't want to find that they are forced to > upgrade in six months. Removing stuff that should have been removed is fine, and I'm even okay with bending the "should have been" definition. > Frankly, I don't really believe the users for whom those rules were > created are using 3.0 yet. Instead, I expect there to be two types of > users: people in the educational business who don't have a lot of > bridges to burn and are eager to use the new features; and developers > of serious Python software (e.g. Twisted) who are trying to figure out > how to port their code to 3.0. The first group isn't affected by the > changes we're considering here (e.g. removing cmp or some obscure > functions from the operator module). The latter group *may* be > affected, simply because they may have some pre-3.0 code using old > features that (by accident) still works under 3.0. I mostly agree. I'm also concerned about downstream consumers that may be distributing 3.0 and will have a different schedule for doing their upgrades. What I really want to avoid is people having to do stuff like the ugliness to work around the 2.2.1 additions: try: True except NameError: True = 1 False = 0 Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSYMZ5nEjvBPtnXfVAQJZyAP/dAbxc37a3HPfZ6SYH29OxfsyWeist6yk 0jli2WVDiLnc9iYmLky3Bj/B7aijZpq2X2/UOS/F6akOYJhLKfjYckiXzcjBmBIK Ypy3uGrw1wRFxz4ZrJGGzBjxvzSkbYj8ijkGsPqm95FDalq2YOXtrRbOft861dyy 4i2APtZ40AA= =s7U3 -----END PGP SIGNATURE----- From scav at blueyonder.co.uk Fri Jan 30 15:57:04 2009 From: scav at blueyonder.co.uk (scav at blueyonder.co.uk) Date: Fri, 30 Jan 2009 14:57:04 -0000 (GMT) Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: References: Message-ID: <31113.84.19.238.82.1233327424.VFkUQmFaS098Sh0W.squirrel@84.19.238.82> Hi all, > On Thu, Jan 29, 2009 at 6:12 AM, Ben North wrote: >> Hi, >> >> I find 'functools.partial' useful, but occasionally I'm unable to use it >> because it lacks a 'from the right' version. > -1 For me, the main objection to a partial that places its stored positional arguments from the right is that you don't know which positions those arguments will actually occupy until the partial is called. Who *really* thinks that would be a neat feature? There's probably a reason why Haskell doesn't do this... Peter Harris From barry at python.org Fri Jan 30 16:24:15 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 30 Jan 2009 10:24:15 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <20090129220951.GA17786@panix.com> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 29, 2009, at 5:09 PM, Aahz wrote: > The problem is that the obvious candidate for doing the vetting is the > Release Manager, and Barry doesn't like this approach. The vetting > does > need to be handled by a core committer IMO -- MAL, are you > volunteering? > Anyone else? > > Barry, are you actively opposed to marking 3.0.x as experimental, or > do > you just dislike it? (I.e. are you -1 or -0?) I'm opposed to marking 3.0 experimental, so I guess -1 there. It's the first model year of a redesigned nameplate, but it's still got four wheels, a good motor and it turns mostly in the direction you point it. :) No release is ever what everyone wants. There has never been a release where I haven't wanted to add or change something after the fact (see my recent 2.6 unicode grumblings). Perhaps frustratingly, but usually correctly, the community is very resistant to making such feature or API changes after a release is made. That's just life; we deal with it, workaround it and work harder towards the next major release. If that's too burdensome, then maybe it's really the 18 month development cycle that needs to be re-examined. All that aside, I will support whatever community consensus or BDFL pronouncement is made here. Don't be surprised if when you ask me though I'm more conservative than you want. You can always appeal to a higher authority (python-dev or Guido). So don't worry, I'll continue to RM the 3.0 series! Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSYMbn3EjvBPtnXfVAQLsUAP+J3WPGMNgGPSWrawJa8Yp+1RBTIt2vOif rgV+5xyOQqOKnuDntZPAv1R2SqrTCHv8abyLP4pBaoklqtymIDgikiOLJkI2tHij MT+gfPu4Xb7F35HAXE/6vhel124nr8JG15fXBQdEWqiozNZl9GaXEqKZY8tdhgkC 4VDdY6KEwL0= =kvOy -----END PGP SIGNATURE----- From barry at python.org Fri Jan 30 16:28:28 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 30 Jan 2009 10:28:28 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com><497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> Message-ID: <01897A99-0135-4CAC-AD16-A255EC3EDD15@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 29, 2009, at 6:27 PM, Raymond Hettinger wrote: >> The problem is that the obvious candidate for doing the vetting is >> the >> Release Manager, and Barry doesn't like this approach. The vetting >> does >> need to be handled by a core committer IMO -- MAL, are you >> volunteering? >> Anyone else? > > It should be someone who is using 3.0 regularly (ideally someone who > is working on fixing it). IMO, people who aren't exercising it > don't really > have a feel for the problems or the cost/benefits of the fixes. That's not the right way to look at it. I'm using 2.6 heavily these days, does that mean I get to decide what goes in it or not? No. Everyone here, whether they are using 2.6 or not should weigh in, with of course one BDFL to rule them all. Same goes for 3.0. This is a community effort and I feel strongly that we should work toward reaching consensus (that seems to be an American theme these days). Make your case, we'll listen to the pros and cons, decide as a community and then move on. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSYMcnHEjvBPtnXfVAQK+aQQApR5McrCOiYUf6RiNvmrDKmTShMde4iWt Rh9x3wY3EVQskcgdpd+05VSfceVCKJJlqbR1NdMDtnuzM8aD56qQyAxYHhqYyxkh 0adHg1ZmYt/95K0/WE3DM8NoBUPxUFIb4nyeprGBsYola9BUQNc//VSRSIyXf0U6 p3xwN8oQS/c= =KKeq -----END PGP SIGNATURE----- From barry at python.org Fri Jan 30 16:28:54 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 30 Jan 2009 10:28:54 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 29, 2009, at 6:40 PM, Guido van Rossum wrote: > On Thu, Jan 29, 2009 at 3:27 PM, Raymond Hettinger > wrote: >> To get the ball rolling, I have a candidate for discussion. >> >> Very late in the 3.0 process (after feature freeze), the bsddb code >> was >> ripped out (good riddance). This had the unfortunate side-effect of >> crippling shelves which now fall back to using dumbdbm. >> >> I'm somewhat working on an alternate dbm based on sqlite3: >> http://code.activestate.com/recipes/576638/ >> It is a pure python module and probably will not be used directly, >> but shelves >> will see an immediate benefit (especially for large shelves) in >> terms of speed >> and space. >> >> On the one hand, it is an API change or new feature because people >> can >> (if they choose) access the dbm directly. OTOH, it is basically a >> performance fix for shelves whose API won't change at all. The >> part that is visible >> and incompatible is that 3.0.1 shelves won't be readable by 3.0.0. > > That is too much for 3.0.1. It could affect external file formats > which strikes me as a bad idea. > > Sounds like a good candidate for 3.1, which we should be expecting in > 4-6 months I hope. Also you could try find shelve users (are there > any?) and recommend they install this as a 3rd party package, with the > expectation it'll be built into 3.1. I concur. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSYMctnEjvBPtnXfVAQKC3QP/bVCQ6KTI5Kd1H/y2Qp85pkLiC8JAH7ap 8vJ2xPjZde4oe6tz5WRziUparpM5FMA4Cz0fuMg4C7vtt6ZLIG27OKVuXx9i4atG zrtnEfs129Xouq4se6UFiIaIj1KNiNWbZa4cOkSlQFUq37Ww/B25JlrtGnreZB4v 13r8lRzTNOU= =8Fo7 -----END PGP SIGNATURE----- From barry at python.org Fri Jan 30 16:33:09 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 30 Jan 2009 10:33:09 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> Message-ID: <38A60A2A-7B03-4EF3-AB41-C9B462665226@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 29, 2009, at 7:43 PM, Raymond Hettinger wrote: > We should have insisted that bsddb not be taken out until a > replacement > was put in. The process was broken with the RM insisting on feature > freeze early in the game but letting tools like bsddb get ripped-out > near the end. IMO, it was foolish to do one without the other. Very good arguments were made for ripping bsddb out. Guido agreed. A replacement would have delayed 3.0 even more than it originally was, and the replacement would not have been battle tested. It's possible, maybe even likely, that the replacement would have been found inadequate later on and then we'd be saddled with a different mistake. Given that it's easy to make 3rd party packages work, I firmly believe this was the right decision. With a proven, solid, popular replacement available for several months, it will be easy to pull that into the 3.1 release. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSYMdtXEjvBPtnXfVAQK+FAQAlNL26s4ekva/3jpnATfZfXtAkHa+Wqdo f9luB8gkLk3Dk0qXyjm6AisFCMh+Zgu8g+OgrWS3DO6yR+/SlfjVcPbq0kr8nP+L +EXXisuZofeHuxp0JZ3ePoL94ALbv35norx1yHqiKnEMEvUbCfdNWb4sGE2kM5ZE snfeFattlIg= =RQ7t -----END PGP SIGNATURE----- From barry at python.org Fri Jan 30 16:40:42 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 30 Jan 2009 10:40:42 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <873af1efvf.fsf@xemacs.org> References: <1afaf6160901271222i2e2d9525i883367789219f96d@mail.gmail.com> <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> <873af1efvf.fsf@xemacs.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 29, 2009, at 8:34 PM, Stephen J. Turnbull wrote: > I think that the important question is "can the 3.0.x series be made > 'viable' in less than the time frame for 3.1?" If not, I really have > to think it's DOA from the point of view of folks who consider 3.0.0 > non-viable. I think that's what Barry and Martin are saying. Of course, the definition of "viable" is the key thing here. I'm not picking on Raymond, but what is not viable for him will be perfectly viable for other people. We have to be very careful not to view our little group of insiders as the sole universe of Python users (3.0 or otherwise). > Guido is saying something different. AIUI, he's saying that > explicitly > introducing controlled instability into 3.0.x of the form "this is > what the extremely stable non-buggy inherited-from-3.0 core of 3.1 is > going to look like" will be a great service to those who consider > 3.0.0 non-viable. > > The key point is that new features in 3.1 are generally going to be > considered less reliable than those inherited from 3.0, and thus a > debugged 3.0, even if the implementations have been unstable, provides > a way for the very demanding to determine what that set is, and to > test how it behaves in their applications. I'm not sure I agree with that last paragraph. We have a pretty good track record of introducing stable new features in dot-x releases, so there's no reason to believe that the same won't work for 3.x. > I think it's worth a try, after consultation with some of the major > developers who are the ostensible beneficiaries. But if tried, I > think it's important to mark 3.0.x as "not yet stable" even if the > instability is carefully defined and controlled. It all depends on where that instability lies. If 3.0 crashed every time you raised an exception due to some core design flaw, then yeah, we'd have a problem. The fact that a bundled module doesn't do what you want it to does not scream instability to me. The should-have- been-removed features don't either. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSYMfenEjvBPtnXfVAQLIhwP+JVFJWXRoQ5Fz65vrmmGo+8w7ZspjVCWP 9a+yrAh1aGHf0w4vQAirRuBGZNWvl4e5F/Pd4DoWdFVPPKuEhyOiavPAP90ViThy yKHHoEBv6cloUIRXrKendJGzA7L5bDVN0CoQjcPh499mpDxvq7aGgru2lYdD7iT0 KuB21maqMTc= =dWTA -----END PGP SIGNATURE----- From barry at python.org Fri Jan 30 17:02:19 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 30 Jan 2009 11:02:19 -0500 Subject: [Python-Dev] 3.0.1/3.1.0 summary In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 29, 2009, at 10:59 PM, Brett Cannon wrote: > 1. Barry, who is the release manager for 3.0.1, does not like the idea > of the cruft that is being proposed removed from 3.0.1. Personally I > say we continue to peer pressure him as I think a new major release is > not like our typical minor release, but I am not about to force Barry > to go against what he thinks is reasonable unless I am willing to step > up as release manager (and I am not since I simply don't have the time > to learn the process fast enough along with just a lack of time with > other Python stuff). I followed up in a different thread, but just FTR here. I'll continue to RM 3.0. I'll follow the community consensus on specific issues, but if there isn't a clear one and I have to decide, I'll likely take the more conservative path. Appealing to python-dev and Guido is (as always :) allowed. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSYMkjHEjvBPtnXfVAQK/fgP/T4uWwU41k1OEgS6ngXlZvUao3dVh0Hni f+iyeo+cyvWggp6ks1NLoJ+BOH/lpwIybwtuLqUI/FcajctdlOUaTyw2CE2jPjgD SMJID5oj1e/7vpB3Dk26RCIB+trZ6GTg1lC4OjRVn0vrKK/QVRg6dYD2YKcW0Seh fF++3EHxhW0= =TMO+ -----END PGP SIGNATURE----- From barry at python.org Fri Jan 30 17:03:17 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 30 Jan 2009 11:03:17 -0500 Subject: [Python-Dev] 3.0.1/3.1.0 summary In-Reply-To: <498295F5.2050607@v.loewis.de> References: <498295F5.2050607@v.loewis.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 30, 2009, at 12:53 AM, Martin v. L?wis wrote: >> 1. Barry, who is the release manager for 3.0.1, does not like the >> idea >> of the cruft that is being proposed removed from 3.0.1. > > I don't think he actually said that (in fact, I think he said the > opposite). It would be good if he clarified, though. To clarify: cruft that should have been removed 3.0 is fine to remove for 3.0.1, for some definition of "should have been". Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSYMkxXEjvBPtnXfVAQIqtgP+Mra/z5nLY5SU56cw0JjgBwCVY1N3060K TSG90E4R+JpCsXRD7sjf2UfSAzKAGKz6gYja3hnt5awzhnCJMacgN0tvXNaAmuYi b7Qb6N4oV3izDGZPl3x0EO3DGimov2Nq8hCsEZbYnNd3U62MwRlzpW+FJbFJlZHO VR1jiVWX8Ig= =p0VE -----END PGP SIGNATURE----- From nde at comp.leeds.ac.uk Fri Jan 30 17:01:19 2009 From: nde at comp.leeds.ac.uk (Nick Efford) Date: Fri, 30 Jan 2009 16:01:19 +0000 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: Message-ID: <4983244F.3060801@comp.leeds.ac.uk> > Paul Moore wrote: > > Serious question: does anybody know how to get better communication > from the user base? My impression is that it's pretty hard to find out > who is actually using 3.0, and get any feedback from them. I suppose a > general query on clp might get some feedback, but otherwise, what? > I've not seen any significant amount of blog activity on 3.0. I teach programming in a CS dept. at a UK university. We've been teaching Python in one context or another for 5 years now, and are currently in our second year of teaching it as the primary programming language. We have to make decisions on software versions for the coming academic year during the summer months. This means that we've had to be content this year with Python 2.5. We'd love to switch to 3.0 as soon as possible (i.e., Oct 2009), as it is a significantly cleaner language for our purposes. However, we make extensive use of third-party libraries and frameworks such as Pygame, wxPython, etc, to increase the motivation levels of students. The 3.0-readiness of these libraries and frameworks is inevitably going to be a factor in the decision we make this summer. Nick From dickinsm at gmail.com Fri Jan 30 17:50:36 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 30 Jan 2009 16:50:36 +0000 Subject: [Python-Dev] 3.0.1/3.1.0 summary In-Reply-To: References: <498295F5.2050607@v.loewis.de> Message-ID: <5c6f2a5d0901300850q7b261e47k347811f7b183718b@mail.gmail.com> On Fri, Jan 30, 2009 at 4:03 PM, Barry Warsaw wrote: > To clarify: cruft that should have been removed 3.0 is fine to remove for > 3.0.1, for some definition of "should have been". Just to double check, can I take this as a green light to continue with the cmp removal (http://bugs.python.org/issue1717) for 3.0.1? Mark From status at bugs.python.org Fri Jan 30 18:06:48 2009 From: status at bugs.python.org (Python tracker) Date: Fri, 30 Jan 2009 18:06:48 +0100 (CET) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20090130170648.2AB007859E@psf.upfronthosting.co.za> ACTIVITY SUMMARY (01/23/09 - 01/30/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2352 open (+54) / 14582 closed (+20) / 16934 total (+74) Open issues with patches: 788 Average duration of open issues: 697 days. Median duration of open issues: 6 days. Open Issues Breakdown open 2328 (+53) pending 24 ( +1) Issues Created Or Reopened (74) _______________________________ urrlib2/httplib doesn't reset file position between requests 01/23/09 http://bugs.python.org/issue5038 created matejcik Adjust reference-counting note 01/24/09 http://bugs.python.org/issue5039 created tjreedy Bug of CGIXMLRPCRequestHandler 01/24/09 http://bugs.python.org/issue5040 created WayneHuang Memory leak in imp.find_module 01/24/09 CLOSED http://bugs.python.org/issue5041 created ocean-city patch, easy Structure sub-subclass does not initialize with base class posit 01/24/09 http://bugs.python.org/issue5042 created jaraco get_msvcr() returns None rather than [] 01/24/09 http://bugs.python.org/issue5043 created lkcl name not found in generator in eval() 01/24/09 http://bugs.python.org/issue5044 created fjhpy imaplib should remove length of literal strings 01/24/09 http://bugs.python.org/issue5045 created bmoore native win32 and wine mingw+msys build of python2.7 01/24/09 CLOSED http://bugs.python.org/issue5046 created lkcl patch Remove Monterey support from configure.in 01/24/09 http://bugs.python.org/issue5047 created skip.montanaro patch Extending itertools.combinations 01/25/09 CLOSED http://bugs.python.org/issue5048 created konryd ctypes unwilling to allow pickling wide character 01/25/09 http://bugs.python.org/issue5049 created jaraco unicode(C) invokes C.__unicode__ when __unicode__ is defined 01/25/09 CLOSED http://bugs.python.org/issue5050 created livibetter test_update2 in test_os.py invalid due to os.environ.clear() fol 01/25/09 http://bugs.python.org/issue5051 created lkcl Mark distutils to stay compatible with 2.3 01/25/09 http://bugs.python.org/issue5052 reopened tarek http.client.HTTPMessage.getallmatchingheaders() always returns [ 01/25/09 http://bugs.python.org/issue5053 created mwatkins patch CGIHTTPRequestHandler.run_cgi() HTTP_ACCEPT improperly parsed 01/25/09 http://bugs.python.org/issue5054 created mwatkins Distutils-SIG page needs to be updated 01/25/09 CLOSED http://bugs.python.org/issue5055 created akitada PyAPI assumes OS can access shared data in loadable modules (win 01/25/09 CLOSED http://bugs.python.org/issue5056 created lkcl patch Unicode-width dependent optimization leads to non-portable pyc f 01/25/09 http://bugs.python.org/issue5057 created pitrou stop pgen.exe from generating CRLF-ended files and causing mayhe 01/25/09 CLOSED http://bugs.python.org/issue5058 created lkcl Policy.DomainStrict in cookielib example code 01/25/09 http://bugs.python.org/issue5059 created babo gcc profile guided optimization 01/25/09 http://bugs.python.org/issue5060 created rpetrov patch Inadequate documentation of the built-in function open 01/25/09 http://bugs.python.org/issue5061 created MLModel Rlcompleter.Completer does not use __dir__ magic method 01/26/09 http://bugs.python.org/issue5062 created carlj patch python-2.6.spec doesn't build properly 01/26/09 http://bugs.python.org/issue5063 created ptoal patch compiler.parse raises SyntaxErrors without line number informati 01/26/09 http://bugs.python.org/issue5064 created exarkun IDLE improve Subprocess Startup Error message 01/26/09 http://bugs.python.org/issue5065 created stevenjd IDLE documentation for Unix obsolete/incorrect 01/26/09 http://bugs.python.org/issue5066 created stevenjd Error msg from using wrong quotes in JSON is unhelpful 01/26/09 http://bugs.python.org/issue5067 created stevenjd patch tarfile loops forever on broken input 01/26/09 http://bugs.python.org/issue5068 created fijal Use sets instead of list in posixpath._resolve_link 01/26/09 CLOSED http://bugs.python.org/issue5069 created tzot patch Distutils should create install dir if needed 01/26/09 http://bugs.python.org/issue5070 created andybuckley Distutils should not fail if install dir is not in PYTHONPATH 01/26/09 http://bugs.python.org/issue5071 created andybuckley urllib.open sends full URL after GET command instead of local pa 01/26/09 http://bugs.python.org/issue5072 created olemis bsddb/test/test_lock.py sometimes fails due to floating point er 01/26/09 CLOSED http://bugs.python.org/issue5073 created ocean-city patch python3 and ctypes, script causes crash 01/27/09 CLOSED http://bugs.python.org/issue5074 created pooryorick bdist_wininst should not depend on the vc runtime? 01/27/09 CLOSED http://bugs.python.org/issue5075 created mhammond patch bdist_wininst fails on py3k 01/27/09 CLOSED http://bugs.python.org/issue5076 created mhammond patch, patch 2to3 fixer for the removal of operator functions 01/27/09 http://bugs.python.org/issue5077 created alexandre.vassalotti Avoid redundant call to FormatError() 01/27/09 http://bugs.python.org/issue5078 created eckhardt patch time.ctime docs refer to "time tuple" for default 01/27/09 http://bugs.python.org/issue5079 created tlynn PyArg_Parse* should raise TypeError for float parsed with intege 01/27/09 http://bugs.python.org/issue5080 created marketdickinson Unable to print Unicode characters in Python 3 on Windows 01/27/09 CLOSED http://bugs.python.org/issue5081 created giampaolo.rodola Let frameworks to register attributes as builtins 01/27/09 CLOSED http://bugs.python.org/issue5082 created andrea-bs New resource ('gui') for regrtest 01/27/09 CLOSED http://bugs.python.org/issue5083 created gpolo patch unpickling does not intern attribute names 01/27/09 http://bugs.python.org/issue5084 created jakemcguire patch distutils/test_sdist failure on windows 01/27/09 CLOSED http://bugs.python.org/issue5085 created ocean-city patch set_daemon does not exist in Thread 01/28/09 CLOSED http://bugs.python.org/issue5086 created mnewman set_daemon does not exist in Thread 01/28/09 CLOSED http://bugs.python.org/issue5087 created mnewman optparse: inconsistent default value for append actions 01/28/09 http://bugs.python.org/issue5088 created pycurry Error in atexit._run_exitfuncs [...] Exception expected for valu 01/28/09 http://bugs.python.org/issue5089 created marketdickinson import tkinter library Visual C++ Concepts:C Run-Time Error R603 01/28/09 http://bugs.python.org/issue5090 created guxianminer Segfault in PyObject_Malloc(), address out of bounds 01/28/09 http://bugs.python.org/issue5091 created christian.heimes weird memory usage in multiprocessing module 01/29/09 CLOSED http://bugs.python.org/issue5092 created Orlowski 2to3 with a pipe on non-ASCII script 01/29/09 http://bugs.python.org/issue5093 created haypo patch datetime lacks concrete tzinfo impl. for UTC 01/29/09 http://bugs.python.org/issue5094 created brett.cannon msi missing from "bdist --help-formats" 01/29/09 http://bugs.python.org/issue5095 created bethard strange thing after call PyObject_CallMethod 01/29/09 http://bugs.python.org/issue5096 created exe asyncore.dispatcher_with_send undocumented 01/29/09 http://bugs.python.org/issue5097 created exe Environ doesn't escape spaces properly 01/29/09 CLOSED http://bugs.python.org/issue5098 created stuaxo subprocess.POpen.__del__() AttributeError (os module == None!) 01/29/09 http://bugs.python.org/issue5099 created marystern ElementTree.iterparse and Element.tail confusion 01/29/09 http://bugs.python.org/issue5100 created jeroen.dirks test_funcattrs truncated during unittest conversion 01/29/09 http://bugs.python.org/issue5101 created marketdickinson urllib2.py timeouts do not propagate across redirects for 2.6.1 01/29/09 http://bugs.python.org/issue5102 created jacques ssl.SSLSocket timeout not working correctly when remote end is h 01/29/09 http://bugs.python.org/issue5103 created jacques getsockaddrarg() casts port number from int to short without any 01/29/09 http://bugs.python.org/issue5104 created roman.zeyde sqlite3.Row class, handling duplicate column names resulting fro 01/29/09 http://bugs.python.org/issue5105 created sockonafish Update Naming & Binding statement for 3.0 01/30/09 http://bugs.python.org/issue5106 created tjreedy built-in open(..., encoding=vague_default) 01/30/09 http://bugs.python.org/issue5107 created sjmachin Invalid UTF-8 ("%s") length in PyUnicode_FromFormatV() 01/30/09 http://bugs.python.org/issue5108 created haypo patch array.array constructor very slow when passed an array object. 01/30/09 http://bugs.python.org/issue5109 created malcolmp Printing Unicode chars from the interpreter in a non-UTF8 termin 01/30/09 http://bugs.python.org/issue5110 created ezio.melotti patch httplib: wrong Host header when connecting to IPv6 loopback 01/30/09 http://bugs.python.org/issue5111 created gdesmott Issues Now Closed (47) ______________________ [distutils] - error when processing the "--formats=tar" option 30 days http://bugs.python.org/issue1885 tarek patch shutil.destinsrc returns wrong result when source path matches b 357 days http://bugs.python.org/issue2047 pitrou patch, easy msi installs to the incorrect location (C drive) 320 days http://bugs.python.org/issue2271 loewis Ttk support for Tkinter 246 days http://bugs.python.org/issue2983 gpolo Python 2.5.2 Windows Source Distribution missing Visual Studio 2 225 days http://bugs.python.org/issue3105 loewis multiprocessing adds built-in types to the global copyreg.dispat 196 days http://bugs.python.org/issue3350 jnoller Fix gdbinit for Python 3.0 164 days http://bugs.python.org/issue3610 skip.montanaro patch importing from UNC roots doesn't work 152 days http://bugs.python.org/issue3677 ocean-city patch _tkinter._flatten() doesn't check PySequence_Size() error code 136 days http://bugs.python.org/issue3880 benjamin.peterson patch IDLE won't start in custom directory. 130 days http://bugs.python.org/issue3881 loewis patch zipfile and winzip 11 days http://bugs.python.org/issue3997 amaury.forgeotdarc patch, needs review open(0, closefd=False) prints 3 warnings 92 days http://bugs.python.org/issue4233 haypo patch Portability fixes in longobject.c 62 days http://bugs.python.org/issue4393 marketdickinson patch 2.6.1 breaks many applications that embed Python on Windows 52 days http://bugs.python.org/issue4566 mhammond patch, needs review range objects becomes hashable after attribute access 41 days http://bugs.python.org/issue4701 jcea patch python3.0 -u: unbuffered stdout 7 days http://bugs.python.org/issue4705 pitrou patch round(25, 1) should return an integer, not a float 39 days http://bugs.python.org/issue4707 marketdickinson patch [PATCH] zipfile.ZipFile does not extract directories properly 34 days http://bugs.python.org/issue4710 loewis patch, needs review deprecate/delete distutils.mwerkscompiler... 19 days http://bugs.python.org/issue4863 tarek patch Inconsistent usage of next/__next__ in ABC collections; collecti 17 days http://bugs.python.org/issue4920 rhettinger patch Make heapq work with all mutable sequences 14 days http://bugs.python.org/issue4948 rhettinger doctest.testfile should set __name__, can't use namedtuple 6 days http://bugs.python.org/issue5021 rhettinger test_kqueue failure on OS X 3 days http://bugs.python.org/issue5025 marketdickinson patch itertools.fixlen 4 days http://bugs.python.org/issue5034 rhettinger patch Memory leak in imp.find_module 6 days http://bugs.python.org/issue5041 ocean-city patch, easy native win32 and wine mingw+msys build of python2.7 0 days http://bugs.python.org/issue5046 loewis patch Extending itertools.combinations 3 days http://bugs.python.org/issue5048 rhettinger unicode(C) invokes C.__unicode__ when __unicode__ is defined 0 days http://bugs.python.org/issue5050 loewis Distutils-SIG page needs to be updated 0 days http://bugs.python.org/issue5055 loewis PyAPI assumes OS can access shared data in loadable modules (win 4 days http://bugs.python.org/issue5056 eckhardt patch stop pgen.exe from generating CRLF-ended files and causing mayhe 3 days http://bugs.python.org/issue5058 amaury.forgeotdarc Use sets instead of list in posixpath._resolve_link 1 days http://bugs.python.org/issue5069 benjamin.peterson patch bsddb/test/test_lock.py sometimes fails due to floating point er 0 days http://bugs.python.org/issue5073 marketdickinson patch python3 and ctypes, script causes crash 0 days http://bugs.python.org/issue5074 loewis bdist_wininst should not depend on the vc runtime? 2 days http://bugs.python.org/issue5075 mhammond patch bdist_wininst fails on py3k 3 days http://bugs.python.org/issue5076 loewis patch, patch Unable to print Unicode characters in Python 3 on Windows 0 days http://bugs.python.org/issue5081 loewis Let frameworks to register attributes as builtins 0 days http://bugs.python.org/issue5082 loewis New resource ('gui') for regrtest 1 days http://bugs.python.org/issue5083 gpolo patch distutils/test_sdist failure on windows 2 days http://bugs.python.org/issue5085 tarek patch set_daemon does not exist in Thread 0 days http://bugs.python.org/issue5086 benjamin.peterson set_daemon does not exist in Thread 2 days http://bugs.python.org/issue5087 benjamin.peterson weird memory usage in multiprocessing module 2 days http://bugs.python.org/issue5092 LambertDW Environ doesn't escape spaces properly 0 days http://bugs.python.org/issue5098 loewis threading module can deadlock after fork 163 days http://bugs.python.org/issue874900 jcea patch, needs review Improve itertools.starmap 971 days http://bugs.python.org/issue1498370 rhettinger patch cPickle cannot unpickle subnormal floats on some machines 694 days http://bugs.python.org/issue1672332 marketdickinson patch Top Issues Most Discussed (10) ______________________________ 16 Get rid of more references to __cmp__ 106 days open http://bugs.python.org/issue1717 14 Extending itertools.combinations 3 days closed http://bugs.python.org/issue5048 14 round(25, 1) should return an integer, not a float 39 days closed http://bugs.python.org/issue4707 11 python3 closes + home keys 45 days open http://bugs.python.org/issue4676 8 weird memory usage in multiprocessing module 2 days closed http://bugs.python.org/issue5092 8 xml.parsers.expat make a dictionary which keys are broken if bu 8 days open http://bugs.python.org/issue5036 8 python3.0 -u: unbuffered stdout 7 days closed http://bugs.python.org/issue4705 8 Use a named tuple for sys.version_info 83 days open http://bugs.python.org/issue4285 7 bdist_wininst fails on py3k 3 days closed http://bugs.python.org/issue5076 7 Rlcompleter.Completer does not use __dir__ magic method 5 days open http://bugs.python.org/issue5062 From exarkun at divmod.com Fri Jan 30 18:15:54 2009 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Fri, 30 Jan 2009 12:15:54 -0500 Subject: [Python-Dev] Summary of Python tracker Issues In-Reply-To: <20090130170648.2AB007859E@psf.upfronthosting.co.za> Message-ID: <20090130171554.12853.1364077741.divmod.quotient.534@henry.divmod.com> On Fri, 30 Jan 2009 18:06:48 +0100 (CET), Python tracker wrote: > [snip] > >Average duration of open issues: 697 days. >Median duration of open issues: 6 days. It seems there's a bug in the summary tool. I thought it odd a few weeks ago when I noticed the median duration of open issues was one day. I just went back and checked and the week before it was one day it was 2759 days. Perhaps there is some sort of overflow problem when computing this value? Jean-Paul From Scott.Daniels at Acm.Org Fri Jan 30 18:33:15 2009 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Fri, 30 Jan 2009 09:33:15 -0800 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: <31113.84.19.238.82.1233327424.VFkUQmFaS098Sh0W.squirrel@84.19.238.82> References: <31113.84.19.238.82.1233327424.VFkUQmFaS098Sh0W.squirrel@84.19.238.82> Message-ID: scav at blueyonder.co.uk wrote: > Hi all, > >> On Thu, Jan 29, 2009 at 6:12 AM, Ben North wrote: >>> I find 'functools.partial' useful, but occasionally I'm unable to use it >>> because it lacks a 'from the right' version. > -1 > > For me, the main objection to a partial that places > its stored positional arguments from the right is > that you don't know which positions those arguments > will actually occupy until the partial is called. Certainly this interacts in a magical way with keyword args. That definitional problem is the reason there was no curry_right in the original recipe that was the basis of the first partial. If you have: def button(root, position, action=None, text='*', color=None): ... ... blue_button = partial(button, my_root, color=(0,0,1)) Should partial_right(blue_button, 'red') change the color or the text? It is computationally hard to do that (may have to chase chains of **kwarg-passing functions), but even hard to document. So, I'd avoid it. --Scott David Daniels Scott.Daniels at Acm.Org From guido at python.org Fri Jan 30 18:42:19 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 30 Jan 2009 09:42:19 -0800 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> Message-ID: On Thu, Jan 29, 2009 at 8:25 PM, Raymond Hettinger wrote: > > [Guido van Rossum] >> >> Sorry, not convinced. > > No worries. Py3.1 is not far off. > > Just so I'm clear. Are you thinking that 3.0.x will never have > fast shelves, or are you thinking 3.0.2 or 3.0.3 after some > external deployment and battle-testing for the module? I don't know about fast shelves, but I don't think your new module should be added to 3.0.x for any x. Who knows if there even will be a 3.0.2 -- it sounds like it's better to focus on 3.1 after 3.0.1. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From Chris.Barker at noaa.gov Fri Jan 30 18:43:05 2009 From: Chris.Barker at noaa.gov (Christopher Barker) Date: Fri, 30 Jan 2009 09:43:05 -0800 Subject: [Python-Dev] Universal newlines, and the gzip module. In-Reply-To: <18818.64219.137713.112225@montanaro.dyndns.org> References: <49821403.1030603@noaa.gov> <18818.64219.137713.112225@montanaro.dyndns.org> Message-ID: <49833C29.8000106@noaa.gov> skip at pobox.com wrote: > Christopher> 1) It would be nice if the gzip module (and the zip lib > Christopher> module) supported Universal newlines -- you could read a > Christopher> compressed text file with "wrong" newlines, and have > Christopher> them handled properly. However, that may be hard to do, > Christopher> so at least: > > Christopher> 2) Passing a 'U' flag in to gzip.open shouldn't break it. > > I agree with Brett that 'U' is meaningless on the compressed file itself. right -- I think the code that deals with the flags is not smart enough -- it adds the 'b' flag if it isn't already there, but that's all it does. There are only a few flags that make sense for opening a gzip file -- it should only use those, and either ignore others or raise an exception if there are others that don't make sense. > You want it applied to the contents of the compressed file though, is that > right? That would be great. > That makes sense to me. It probably belongs in a separate argument > though. I could go either way on that -- if we simply extracted the 'U' from the passed in mode, we wouldn't have to change the API at all, and it wouldn't break any code that wasn't broken already. As for having 'U' applied to the uncompressed data -- I have no idea how much work that would be -- it depends on how it is currently handling text files (does that work -- i.e \r\n converted to \n on Windows?), and how the Universal newline code is written. In any case, the 'U' flag should NEVER get passed through to the file opening code, and that's easy to fix. I tried to post this to the bug tracker, but my attempt to create an account failed -- do I need to be pre-approved or something? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov From ben at redfrontdoor.org Fri Jan 30 19:35:15 2009 From: ben at redfrontdoor.org (Ben North) Date: Fri, 30 Jan 2009 18:35:15 +0000 Subject: [Python-Dev] Partial function application 'from the right' Message-ID: <5169ff10901301035l3010e678je69a390533179f75@mail.gmail.com> Hi, > [ Potential new "functools.partial_right", e.g., > > split_comma = partial_right(str.split, '.') > ] Thanks for the feedback. Apologies if (as was suggested) this should have gone to python-ideas; I thought as a fairly small extension to existing functionality it would be OK here. I'll try to summarise the responses. There was some very luke-warm support. Terry Reedy suggested it would be worth posting a patch to the tracker, for the record, even if it turns out to be rejected. Nick Coghlan made more clear than I did the main reason a 'partial_right' would be useful: > [...] some functions and methods written in C (such as string methods) > *don't* [support keyword args], so partial's keyword support doesn't > help. > > A functools.rpartial would go some way towards addressing that. On the other hand, Collin Winter asked for more evidence that real benefit (beyond mere 'completeness' of the functools module) would result. I don't really have to hand anything more than the three cases mentioned in my original email (str.split, math.log, itertools.islice), but since the change is so small, I thought the feature worth raising. Leif Walsh pointed out that you could achieve the same effect by defining your own function. This is true, but functools.partial exists because it's sometimes useful to create such functions either more concisely, or anonymously. A 'partial_right' would allow more such functions to be so created. Peter Harris was negative on the idea, pointing out that after g = partial_right(f, 7) you don't know which argument of 'f' the '7' is going to end up as, because it depends on how many are supplied in the eventual call to 'g'. This is true, and would require some care in partial_right's use. Peter also wondered > There's probably a reason why Haskell doesn't do this... I have only written about five lines of Haskell in my life, so take this with a hefty pinch of salt, but: Haskell does have a 'flip' function which reverses the order of a function's arguments, so it looks like you can very easily build a 'partial_right' in Haskell, especially since standard functions are in curried form. There was some discussion (started by Antoine Pitrou) of an idea to generalise 'partial' further, potentially using the Ellipsis object, to allow arbitrarily-placed 'holes' in the argument list. E.g., split_comma = partial(str.split, ..., ',') In some ways I quite like the even-more-completeness of this idea, but think that it might be the wrong side of the complexity/benefit trade-off. Scott David Daniels pointed out that using Ellipsis would have the downside of > [...] preventing any use of partial when an argument could be an the > Ellipsis instance. This could be fixed by making the general form be something with the meaning partial_explicit(f, hole_sentinel, *args, **kwargs) where appearances of the exact object 'hole_sentinel' in 'args' would indicate a hole, to be filled in at the time of the future call. A user wanting to have '...' passed in as a true argument could then do g = partial_explicit(f, None, 3, ..., 4, axis = 2) or hole = object() g = partial_explicit(f, hole, 3, ..., hole, 4, axis = 2) if they wanted a true '...' argument and a hole. (I might have the syntax for this wrong, having not played with Python 3.0, but I hope the idea is clear.) There was some concern expressed (by Daniel Stutzbach, Alexander Belopolsky) that the meaning of '...' would be confusing --- 'one hole' or 'arbitrary many holes'? I think the extra complexity vs extra functionality trade-off is worth considering for 'partial_right', but my personal opinion is that a 'partial_explicit' has that trade-off the wrong way. I'll try to find time to create the patch in the tracker in the next few days, by which time perhaps it'll have become clearer whether the idea is a good one or not. Thanks, Ben. From brett at python.org Fri Jan 30 19:56:46 2009 From: brett at python.org (Brett Cannon) Date: Fri, 30 Jan 2009 10:56:46 -0800 Subject: [Python-Dev] 3.0.1/3.1.0 summary In-Reply-To: References: <498295F5.2050607@v.loewis.de> Message-ID: On Fri, Jan 30, 2009 at 08:03, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Jan 30, 2009, at 12:53 AM, Martin v. L?wis wrote: > >>> 1. Barry, who is the release manager for 3.0.1, does not like the idea >>> of the cruft that is being proposed removed from 3.0.1. >> >> I don't think he actually said that (in fact, I think he said the >> opposite). It would be good if he clarified, though. > > To clarify: cruft that should have been removed 3.0 is fine to remove for > 3.0.1, for some definition of "should have been". Great! Then should we start planning for 3.0.1 in terms of release dates and what to have in the release so we can get this out the door quickly? -Brett From benjamin at python.org Fri Jan 30 21:07:29 2009 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 30 Jan 2009 15:07:29 -0500 Subject: [Python-Dev] 3.0.1/3.1.0 summary In-Reply-To: References: <498295F5.2050607@v.loewis.de> Message-ID: <1afaf6160901301207y1b8e9c3clee9d9b4b071f365b@mail.gmail.com> On Fri, Jan 30, 2009 at 1:56 PM, Brett Cannon wrote: > Great! Then should we start planning for 3.0.1 in terms of release > dates and what to have in the release so we can get this out the door > quickly? I think considering there's only two release blockers we should plan for about a week or two from now. I'm not sure if we want to do a release candidate; we didn't for 2.6.1, but maybe it would be good to see if the community can find any other horrible problems. -- Regards, Benjamin From brett at python.org Fri Jan 30 21:14:02 2009 From: brett at python.org (Brett Cannon) Date: Fri, 30 Jan 2009 12:14:02 -0800 Subject: [Python-Dev] 3.0.1/3.1.0 summary In-Reply-To: <1afaf6160901301207y1b8e9c3clee9d9b4b071f365b@mail.gmail.com> References: <498295F5.2050607@v.loewis.de> <1afaf6160901301207y1b8e9c3clee9d9b4b071f365b@mail.gmail.com> Message-ID: On Fri, Jan 30, 2009 at 12:07, Benjamin Peterson wrote: > On Fri, Jan 30, 2009 at 1:56 PM, Brett Cannon wrote: >> Great! Then should we start planning for 3.0.1 in terms of release >> dates and what to have in the release so we can get this out the door >> quickly? > > I think considering there's only two release blockers we should plan > for about a week or two from now. > > I'm not sure if we want to do a release candidate; we didn't for > 2.6.1, but maybe it would be good to see if the community can find any > other horrible problems. I say it's Barry's call. If he has the time and wants to, then great; they don't hurt. But I know I won't object if we don't have one. -Brett From mike.klaas at gmail.com Fri Jan 30 22:20:31 2009 From: mike.klaas at gmail.com (Mike Klaas) Date: Fri, 30 Jan 2009 13:20:31 -0800 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> <5E031BDD-32EC-47CC-8CE4-D4D3888A49F2@gmail.com> Message-ID: On 29-Jan-09, at 3:38 PM, Daniel Stutzbach wrote: > On Thu, Jan 29, 2009 at 5:24 PM, Mike Klaas > wrote: >> And yet, python isn't confined to mathematical notation. *, ** are >> both overloaded for use in argument lists to no-one's peril, AFAICT. > > Certainly, but there is no danger of confusion them for > multiplication in context, whereas: > > split_comma = partial(str.split, ..., ',') > > to me looks like "make ',' the last argument" rather than "make ',' > the second argument". Yes, I agree. I mistakenly thought that that was the proposal under discussion (that partial(f, ..., 2) == right_curry(f, 2)) -Mike From tjreedy at udel.edu Fri Jan 30 23:27:25 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 30 Jan 2009 17:27:25 -0500 Subject: [Python-Dev] Universal newlines, and the gzip module. In-Reply-To: <49833C29.8000106@noaa.gov> References: <49821403.1030603@noaa.gov> <18818.64219.137713.112225@montanaro.dyndns.org> <49833C29.8000106@noaa.gov> Message-ID: Christopher Barker wrote: > I tried to post this to the bug tracker, but my attempt to create an > account failed -- do I need to be pre-approved or something? No. If you do not get a response from the above, and a retry does not work, you could email webmaster at python.org with details on what you did and how it failed. From tjreedy at udel.edu Sat Jan 31 00:07:28 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 30 Jan 2009 18:07:28 -0500 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <79990c6b0901300528h160c0c81t74e44497fbcf709e@mail.gmail.com> References: <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> <79990c6b0901300528h160c0c81t74e44497fbcf709e@mail.gmail.com> Message-ID: Paul Moore wrote: > Serious question: does anybody know how to get better communication > from the user base? One of the nice things about Python is that the downloads are truly free -- no required 'registration'. On the other hand, there is no option to give feedback either. If PSF/devs wanted to add something to the site, and someone else volunteered to do the implementation, I would volunteer to help with both design and analysis. That said, I think a main determinant of general 3.0 use will be availability of 3rd-party libraries, including Windows binaries. So perhaps we should aim survey efforts at their authors. I have the impression that the C-API porting guide needs improvement for such effort. On the other hand, perhaps they wonder whether ports will be used. In that case, we need more reports like the post of Nick Efford: " > We'd love to switch to 3.0 as soon as possible (i.e., Oct 2009), > as it is a significantly cleaner language for our purposes. > [university CS courses] > However, we make extensive use of third-party libraries and > frameworks such as Pygame, wxPython, etc, to increase the > motivation levels of students. The 3.0-readiness of these > libraries and frameworks is inevitably going to be a factor in > the decision we make this summer. " Terry Jan Reedy From ironfroggy at gmail.com Sat Jan 31 01:38:23 2009 From: ironfroggy at gmail.com (Calvin Spealman) Date: Fri, 30 Jan 2009 19:38:23 -0500 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> <5E031BDD-32EC-47CC-8CE4-D4D3888A49F2@gmail.com> Message-ID: <76fd5acf0901301638x6d4da376w34fe78d327af7c87@mail.gmail.com> I am just replying to the end of this thread to throw in a reminder about my partial.skip patch, which allows the following usage: split_one = partial(str.split, partial.skip, 1) Not looking to say "mine is better", but if the idea is being given merit, I like the skipping arguments method better than just the "right partial", which I think is confusing combined with keyword and optional arguments. And, this patch already exists. Could it be re-evaluated? On Fri, Jan 30, 2009 at 4:20 PM, Mike Klaas wrote: > On 29-Jan-09, at 3:38 PM, Daniel Stutzbach wrote: > >> On Thu, Jan 29, 2009 at 5:24 PM, Mike Klaas wrote: >>> >>> And yet, python isn't confined to mathematical notation. *, ** are both >>> overloaded for use in argument lists to no-one's peril, AFAICT. >> >> Certainly, but there is no danger of confusion them for multiplication in >> context, whereas: >> >> split_comma = partial(str.split, ..., ',') >> >> to me looks like "make ',' the last argument" rather than "make ',' the >> second argument". > > Yes, I agree. I mistakenly thought that that was the proposal under > discussion (that partial(f, ..., 2) == right_curry(f, 2)) > > -Mike > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com > -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy From martin at v.loewis.de Sat Jan 31 01:38:22 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 31 Jan 2009 01:38:22 +0100 Subject: [Python-Dev] [Python-checkins] Merging to the 3.0 maintenance branch In-Reply-To: <4982F0AE.20308@gmail.com> References: <20090129043706.5614B1E4002@bag.python.org> <1afaf6160901290804t3c08134egadfd14493dcb09ae@mail.gmail.com> <498229BE.4060408@gmail.com> <49826577.4030808@v.loewis.de> <49826E1A.6080809@v.loewis.de> <49826F5D.7000009@v.loewis.de> <4982F0AE.20308@gmail.com> Message-ID: <49839D7E.7070902@v.loewis.de> >> Ah. In the 3.0 branch, always do "svn revert ." after svnmerge. >> It's ok (Nick says it isn't exactly ok, but I don't understand why) > > Doing "svn revert ." before making the commit will lose the metadata > changes that svnmerge uses for its bookkeeping (i.e. if this practice is > used regularly, the tool will completely lose track of which revisions > have already been merged). How so? The metadata are getting tracked just fine, no loss whatsoever. > That won't bother those of us that are only > backporting cherry-picked revisions, but is rather inconvenient for > anyone checking for revisions that haven't been backported yet, but > haven't been explicitly blocked either. Take a look at r68901, which I merged using the procedure I described. svn diff -r68900:68901 --depth empty . gives Modified: svnmerge-integrated - /python/branches/py3k:1-67498,67522-67529,67531-67533,67535-67544,67546-67549,67551-67584,67586-67602,67604-67606,67608-67609,67611-67619,67621-67635,67638,67650,67653-67701,67703-67712,67714,67716-67746,67748,67750-67762,67764-67797,67799-67809,67811-67822,67825-67838,67840-67850,67852-67857,67859-67885,67888-67902,67904-67909,67911-67931,67933,67937-67938,67940,67950-67955,67957-67958,67960-67963,67965-67973,67975-67980,67982,67984-68014,68016-68058,68060-68089,68091-68093,68101,68103,68132,68137,68139-68152,68169-68170,68175,68178,68184,68193,68200,68205-68206,68212,68216,68223-68224,68226-68229,68237,68242,68245,68247,68249,68309,68321,68342,68363,68375,68401,68427,68440,68443,68451,68454,68463,68474-68475,68477,68508,68511,68525,68529,68553,68581,68587,68615,68619,68630,68638,68650-68653,68662,68669,68675,68677,68700,68709,68730,68732,68746,68767-68770,68782,68814-68815,68836,68855,68857,68887,68895 + /python/branches/py3k:1-67498,67522-67529,67531-67533,67535-67544,67546-67549,67551-67584,67586-67602,67604-67606,67608-67609,67611-67619,67621-67635,67638,67650,67653-67701,67703-67712,67714,67716-67746,67748,67750-67762,67764-67797,67799-67809,67811-67822,67825-67838,67840-67850,67852-67857,67859-67885,67888-67902,67904-67909,67911-67931,67933,67937-67938,67940,67950-67955,67957-67958,67960-67963,67965-67973,67975-67980,67982,67984-68014,68016-68058,68060-68089,68091-68093,68101,68103,68132,68137,68139-68152,68169-68170,68175,68178,68184,68193,68200,68205-68206,68212,68216,68223-68224,68226-68229,68237,68242,68245,68247,68249,68309,68321,68342,68363,68375,68401,68427,68440,68443,68451,68454,68463,68474-68475,68477,68508,68511,68525,68529,68553,68581,68587,68615,68619,68630,68638,68650-68653,68662,68669,68675,68677,68700,68709,68730,68732,68746,68767-68770,68782,68814-68815,68836,68855,68857,68887,68895,68898 As you can see, 68898 has been added to svnmerge-integrated, and this is indeed the revision that I merged. > Doing "svn resolved ." assumes that you did everything else correctly, > and even then I don't see how svnmerge could both backport the py3k > changes to the metadata and make its own changes and still get the > metadata to a sane state. The *only* interesting metadata in the svnmerge-integrated property are the ones that svnmerge has written, and svnmerge writes them correctly. > The consequence of getting this approach wrong > is that the merge state of the 3.0 maintenance branch can be clobbered > completely (losing track both of which revisions have been backported > and which have been blocked). Not with the procedure I described. > > Doing both "svn revert ." and "svnmerge merge -M -F " clears > out the conflicted metadata and then correctly updates the metadata for > the revisions that have been backported. It will always update the > svnmerge metadata correctly, regardless of the relative order of the > svnmerge and svn update operations. I don't understand why you bring up this "regardless of the relative order"? Who ever proposed a different order? If you do things in the order I suggest, everything will be fine, right? > Given the choice of a method which will always do the right thing, over > one which always does the wrong thing and another one which only does > the right thing if I did two other things in the right order and will > completely trash the bookkeeping if I get it wrong That's open for debate. What *specific* wrong order are you talking about? If you do things in the right order, will it still get the bookkeeping wrong? > If there's something wrong with my understanding of either svn > properties or the operation of svnmerge that means the quicker > approaches aren't as broken as I think they are, then I'd be happy to > adopt one of them (since they *are* faster than my current approach). > But until someone pokes a hole in my logic, I'll stick with the > slower-but-always-correct methodology (and continue advocating that > approach to everyone else doing updates that affect all four branches). See above. You claim that doing things the way I recommend will lose metadata; I believe this claim is false. Regards, Martin From solipsis at pitrou.net Sat Jan 31 01:42:40 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 31 Jan 2009 00:42:40 +0000 (UTC) Subject: [Python-Dev] Partial function application 'from the right' References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> <5E031BDD-32EC-47CC-8CE4-D4D3888A49F2@gmail.com> <76fd5acf0901301638x6d4da376w34fe78d327af7c87@mail.gmail.com> Message-ID: Calvin Spealman gmail.com> writes: > > I am just replying to the end of this thread to throw in a reminder > about my partial.skip patch, which allows the following usage: > > split_one = partial(str.split, partial.skip, 1) > > Not looking to say "mine is better", but if the idea is being given > merit, I like the skipping arguments method better than just the > "right partial", which I think is confusing combined with keyword and > optional arguments. And, this patch already exists. Could it be > re-evaluated? Sorry, where is the patch? If one writes X = partial.skip, it looks quite nice: split_one = partial(str.split, X, 1) Regards Antoine. From martin at v.loewis.de Sat Jan 31 01:43:36 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 31 Jan 2009 01:43:36 +0100 Subject: [Python-Dev] [Python-checkins] Merging to the 3.0 maintenance branch In-Reply-To: <4982F4B5.7090001@gmail.com> References: <20090129043706.5614B1E4002@bag.python.org> <1afaf6160901290804t3c08134egadfd14493dcb09ae@mail.gmail.com> <498229BE.4060408@gmail.com> <49826577.4030808@v.loewis.de> <4982F4B5.7090001@gmail.com> Message-ID: <49839EB8.2060506@v.loewis.de> > (I believe that svnmerge actually does get that case right, but I > haven't checked it extensively - since if it does get it right, I don't > understand why it leaves the conflict in place instead of automatically > marking it as resolved). I think this is a plain bug. It invokes "svn merge", which creates a conflict, then removes the conflicted property (regardless of whether there was a conflict), then writes the property fresh. It doesn't consider the case that there might have been a conflict, just because such conflict didn't occur in their testing. > Regardless, the consequences of forgetting that you did the svn up after > the merge instead of before (e.g. if it took some time to get the > backported version working, or if something interrupted you between the > initial backport/update and the final test and commit step) are fairly > hard to clean up, so I prefer the safe approach (despite the extra > minute or two it takes for svnmerge to recalculate the metadata changes). If I find that it conflicts on commit, I rather restart all over. Regards, Martin From martin at v.loewis.de Sat Jan 31 02:17:45 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 31 Jan 2009 02:17:45 +0100 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <79990c6b0901300528h160c0c81t74e44497fbcf709e@mail.gmail.com> References: <497F6E55.6090608@v.loewis.de> <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> <79990c6b0901300528h160c0c81t74e44497fbcf709e@mail.gmail.com> Message-ID: <4983A6B9.40304@v.loewis.de> > Serious question: does anybody know how to get better communication > from the user base? My impression is that it's pretty hard to find out > who is actually using 3.0, and get any feedback from them. I think the bug tracker is a way in which users communicate with developers. There have been 296 issues since Dec 3rd that got tagged with version 3.0. The absolute majority of these were documentation problems (documentation was incorrect). Then, I would say we have installation problems, and then problems with IDLE. There is also a significant number of 2to3 problems. > I'm using Windows, and although I *can* build a lot of stuff myself, I > really don't want to be bothered, so I rely on bdist_wininst > installers being available, which is an additional constraint. Notice that bdist_wininst doesn't really work in 3.0. So you likely won't see many packages until 3.0.1 is released. Regards, Martin From alexander.belopolsky at gmail.com Sat Jan 31 04:02:21 2009 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Fri, 30 Jan 2009 22:02:21 -0500 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> <5E031BDD-32EC-47CC-8CE4-D4D3888A49F2@gmail.com> <76fd5acf0901301638x6d4da376w34fe78d327af7c87@mail.gmail.com> Message-ID: On Fri, Jan 30, 2009 at 7:42 PM, Antoine Pitrou wrote: .. > If one writes X = partial.skip, it looks quite nice: > > split_one = partial(str.split, X, 1) Or even _ = partial.skip split_one = partial(str.split, _, 1) From aahz at pythoncraft.com Sat Jan 31 05:51:27 2009 From: aahz at pythoncraft.com (Aahz) Date: Fri, 30 Jan 2009 20:51:27 -0800 Subject: [Python-Dev] FINAL REMINDER: OSCON 2009: Call For Participation Message-ID: <20090131045127.GA6423@panix.com> The O'Reilly Open Source Convention has opened up the Call For Participation -- deadline for proposals is Tuesday Feb 3. OSCON will be held July 20-24 in San Jose, California. For more information, see http://conferences.oreilly.com/oscon http://en.oreilly.com/oscon2009/public/cfp/57 -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From ncoghlan at gmail.com Sat Jan 31 06:00:49 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 31 Jan 2009 15:00:49 +1000 Subject: [Python-Dev] [Python-checkins] Merging to the 3.0 maintenance branch In-Reply-To: <49839D7E.7070902@v.loewis.de> References: <20090129043706.5614B1E4002@bag.python.org> <1afaf6160901290804t3c08134egadfd14493dcb09ae@mail.gmail.com> <498229BE.4060408@gmail.com> <49826577.4030808@v.loewis.de> <49826E1A.6080809@v.loewis.de> <49826F5D.7000009@v.loewis.de> <4982F0AE.20308@gmail.com> <49839D7E.7070902@v.loewis.de> Message-ID: <4983DB01.70503@gmail.com> Martin v. L?wis wrote: > See above. You claim that doing things the way I recommend will lose > metadata; I believe this claim is false. I can see how "svn resolved ." gets it right (now that I understand how the conflict is being produced and then fixed automatically by svnmerge, but not actually marked as resolved). I still don't understand how "svn revert ." can avoid losing the metadata changes unless svnmerge is told to modify the properties again after they have been reverted. Or am I misunderstanding SVN, and the revert command doesn't actually revert property changes? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From martin at v.loewis.de Sat Jan 31 08:18:44 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 31 Jan 2009 08:18:44 +0100 Subject: [Python-Dev] [Python-checkins] Merging to the 3.0 maintenance branch In-Reply-To: <4983DB01.70503@gmail.com> References: <20090129043706.5614B1E4002@bag.python.org> <1afaf6160901290804t3c08134egadfd14493dcb09ae@mail.gmail.com> <498229BE.4060408@gmail.com> <49826577.4030808@v.loewis.de> <49826E1A.6080809@v.loewis.de> <49826F5D.7000009@v.loewis.de> <4982F0AE.20308@gmail.com> <49839D7E.7070902@v.loewis.de> <4983DB01.70503@gmail.com> Message-ID: <4983FB54.4050802@v.loewis.de> > I can see how "svn resolved ." gets it right (now that I understand how > the conflict is being produced and then fixed automatically by svnmerge, > but not actually marked as resolved). > > I still don't understand how "svn revert ." can avoid losing the > metadata changes unless svnmerge is told to modify the properties again > after they have been reverted. Or am I misunderstanding SVN, and the > revert command doesn't actually revert property changes? Oops, I meant "svn resolved ." all the time. When I wrote "svn revert .", it was by mistake. Regards, Martin From regebro at gmail.com Sat Jan 31 09:33:19 2009 From: regebro at gmail.com (Lennart Regebro) Date: Sat, 31 Jan 2009 09:33:19 +0100 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <4983A6B9.40304@v.loewis.de> References: <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> <79990c6b0901300528h160c0c81t74e44497fbcf709e@mail.gmail.com> <4983A6B9.40304@v.loewis.de> Message-ID: <319e029f0901310033i404139d3n107f25e87def27b9@mail.gmail.com> Just my 2 eurocents: I think version numbers communicate a couple of things. One thing the communicate is that if you go from x.y.0 to x.y.1 (or from x.y.34 to x.y.35 for that matter) you signify that this is a bug fix release, and that the risk of any of your stuff breaking is close to zero, unless you somehow where relying on what essentially was broken behavior. It's also correct that a .0 anywhere indicates that you should wait, and that a .1 indicated that this should be safer. Of course, you can end up where these two things clash. Where you need to make a major change that breaks something, but you at the same time don't want to flag "Yes, this will be as bugfree as you normally would expect from a .1 release." My opinion is that in that case, the first rule should win out. Don't make potentially incompatible changes in a minor version increase. So it seems to me here that a 3.0.1 bugfix release, and then a 3.1 with the API changes and C IO is at least the type of numbering I would expect. From p.f.moore at gmail.com Sat Jan 31 11:39:44 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 31 Jan 2009 10:39:44 +0000 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <4983A6B9.40304@v.loewis.de> References: <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> <79990c6b0901300528h160c0c81t74e44497fbcf709e@mail.gmail.com> <4983A6B9.40304@v.loewis.de> Message-ID: <79990c6b0901310239y7fe73255o988c4b5dbc7b3ba4@mail.gmail.com> 2009/1/31 "Martin v. L?wis" : > Notice that bdist_wininst doesn't really work in 3.0. So you likely > won't see many packages until 3.0.1 is released. Ah, that might be an issue :-) Can you point me at specifics (bug reports or test cases)? I could see if I can help in fixing things. Paul. From ncoghlan at gmail.com Sat Jan 31 11:45:04 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 31 Jan 2009 20:45:04 +1000 Subject: [Python-Dev] [Python-checkins] Merging to the 3.0 maintenance branch In-Reply-To: <4983FB54.4050802@v.loewis.de> References: <20090129043706.5614B1E4002@bag.python.org> <1afaf6160901290804t3c08134egadfd14493dcb09ae@mail.gmail.com> <498229BE.4060408@gmail.com> <49826577.4030808@v.loewis.de> <49826E1A.6080809@v.loewis.de> <49826F5D.7000009@v.loewis.de> <4982F0AE.20308@gmail.com> <49839D7E.7070902@v.loewis.de> <4983DB01.70503@gmail.com> <4983FB54.4050802@v.loewis.de> Message-ID: <49842BB0.40901@gmail.com> Martin v. L?wis wrote: >> I can see how "svn resolved ." gets it right (now that I understand how >> the conflict is being produced and then fixed automatically by svnmerge, >> but not actually marked as resolved). >> >> I still don't understand how "svn revert ." can avoid losing the >> metadata changes unless svnmerge is told to modify the properties again >> after they have been reverted. Or am I misunderstanding SVN, and the >> revert command doesn't actually revert property changes? > > Oops, I meant "svn resolved ." all the time. When I wrote > "svn revert .", it was by mistake. Ah, in that case we now agree on the right way to do things :) With the explanation as to where the (spurious) conflict is coming from on the initial merge to the maintenance branch, I'm now happy that the only time the revert + regenerate metadata should ever be needed is if someone else checks in a backport between the time when I start a backport and when I go to check it in (which is pretty unlikely in practice). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From martin at v.loewis.de Sat Jan 31 11:58:59 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 31 Jan 2009 11:58:59 +0100 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <79990c6b0901310239y7fe73255o988c4b5dbc7b3ba4@mail.gmail.com> References: <49809B0C.4020905@egenix.com> <20090129220951.GA17786@panix.com> <79990c6b0901300528h160c0c81t74e44497fbcf709e@mail.gmail.com> <4983A6B9.40304@v.loewis.de> <79990c6b0901310239y7fe73255o988c4b5dbc7b3ba4@mail.gmail.com> Message-ID: <49842EF3.4010003@v.loewis.de> > Can you point me at specifics (bug reports or test cases)? I could see > if I can help in fixing things. See r69098. Regards, Martin From barry at python.org Sat Jan 31 14:44:12 2009 From: barry at python.org (Barry Warsaw) Date: Sat, 31 Jan 2009 08:44:12 -0500 Subject: [Python-Dev] 3.0.1/3.1.0 summary In-Reply-To: <5c6f2a5d0901300850q7b261e47k347811f7b183718b@mail.gmail.com> References: <498295F5.2050607@v.loewis.de> <5c6f2a5d0901300850q7b261e47k347811f7b183718b@mail.gmail.com> Message-ID: <95F1D7BB-5729-4127-9167-8B1FAE8EEB66@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 30, 2009, at 11:50 AM, Mark Dickinson wrote: > On Fri, Jan 30, 2009 at 4:03 PM, Barry Warsaw > wrote: >> To clarify: cruft that should have been removed 3.0 is fine to >> remove for >> 3.0.1, for some definition of "should have been". > > Just to double check, can I take this as a green light to continue > with the cmp removal (http://bugs.python.org/issue1717) for 3.0.1? Yep, go ahead. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSYRVrHEjvBPtnXfVAQK9SQQAiJct3mWt/+ZIOkI7DDRoBdz8yFvrmbLX 6AnbW+owvnnlzB9QX5PyDfTaTJa5pLJuoiWYRb7vCzxH1daW9KuFvF9qnaYXUhiO TLkyaO/R40aarB79NkE6J8wyRjYRyMoZgz10/GzxWkQgvTg38ESeKh3b6YRyph0N uo18odqAGEs= =QDP8 -----END PGP SIGNATURE----- From barry at python.org Sat Jan 31 14:46:44 2009 From: barry at python.org (Barry Warsaw) Date: Sat, 31 Jan 2009 08:46:44 -0500 Subject: [Python-Dev] 3.0.1/3.1.0 summary In-Reply-To: References: <498295F5.2050607@v.loewis.de> Message-ID: <592904F8-89F5-4AC7-B3BF-0C51D3B474A6@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 30, 2009, at 1:56 PM, Brett Cannon wrote: > On Fri, Jan 30, 2009 at 08:03, Barry Warsaw wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Jan 30, 2009, at 12:53 AM, Martin v. L?wis wrote: >> >>>> 1. Barry, who is the release manager for 3.0.1, does not like the >>>> idea >>>> of the cruft that is being proposed removed from 3.0.1. >>> >>> I don't think he actually said that (in fact, I think he said the >>> opposite). It would be good if he clarified, though. >> >> To clarify: cruft that should have been removed 3.0 is fine to >> remove for >> 3.0.1, for some definition of "should have been". > > Great! Then should we start planning for 3.0.1 in terms of release > dates and what to have in the release so we can get this out the door > quickly? How about Friday February 13? If that works for everybody, I'll tag the release on my evening of the 12th so that Martin and other east-of- mes will be able to do their thing by my morning of the 13th. I've added this to the Python release calendar. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSYRWRXEjvBPtnXfVAQI/+wQAm95gTGojwZXSU8qfBtNXgD/lALMi1ncK ctEOhueAwnRBCnFg9UyqgX8dcmogWL7M+pikpOjVeH/TUiArXDIlcY+glkVzgMo4 7DizBu5b6SpJq8h1iTvniqsT7SDZeE1S1FhPBIi5cIja78fD2F5Ny5OGV2K377TP GhjZxX8gepw= =OPBI -----END PGP SIGNATURE----- From barry at python.org Sat Jan 31 14:47:23 2009 From: barry at python.org (Barry Warsaw) Date: Sat, 31 Jan 2009 08:47:23 -0500 Subject: [Python-Dev] 3.0.1/3.1.0 summary In-Reply-To: <1afaf6160901301207y1b8e9c3clee9d9b4b071f365b@mail.gmail.com> References: <498295F5.2050607@v.loewis.de> <1afaf6160901301207y1b8e9c3clee9d9b4b071f365b@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 30, 2009, at 3:07 PM, Benjamin Peterson wrote: > On Fri, Jan 30, 2009 at 1:56 PM, Brett Cannon > wrote: >> Great! Then should we start planning for 3.0.1 in terms of release >> dates and what to have in the release so we can get this out the door >> quickly? > > I think considering there's only two release blockers we should plan > for about a week or two from now. > > I'm not sure if we want to do a release candidate; we didn't for > 2.6.1, but maybe it would be good to see if the community can find any > other horrible problems. Let's JFDI. No release candidate. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSYRWa3EjvBPtnXfVAQIVpgQAo1tb/RJ81WFBJHH1GhdhtKagrB5p9MSl U+GfnLx9mEtqBqQ9rnXaQQaPpJjvNmXc10K+8oDdwCJHSX3k66JbK4U4BOBqWgc3 0PTrdIn5/4PqfexT3HWNmH/mZCZXb36HDcE6fxW5CWxuxHbNLypBY7P52XgVJIBW hqMBQVVNxgw= =Zq3w -----END PGP SIGNATURE----- From martin at v.loewis.de Sat Jan 31 16:47:30 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 31 Jan 2009 16:47:30 +0100 Subject: [Python-Dev] Subversion upgraded to 1.5 Message-ID: <49847292.5080401@v.loewis.de> I have now upgraded subversion to 1.5.1 on svn.python.org. Please let me know if you encounter problems. Regards, Martin From ludvig at lericson.se Sat Jan 31 16:50:55 2009 From: ludvig at lericson.se (Ludvig Ericson) Date: Sat, 31 Jan 2009 16:50:55 +0100 Subject: [Python-Dev] Fwd: Partial function application 'from the right' References: <64D950B1-C422-45CE-BBF0-AAA3764BE31B@lericson.se> Message-ID: Begin forwarded message: > From: Ludvig Ericson > Date: January 31, 2009 16:43:50 GMT+01:00 > To: Alexander Belopolsky > Subject: Re: [Python-Dev] Partial function application 'from the > right' > > On Jan 31, 2009, at 04:02, Alexander Belopolsky wrote: > >> On Fri, Jan 30, 2009 at 7:42 PM, Antoine Pitrou >> wrote: >> .. >>> If one writes X = partial.skip, it looks quite nice: >>> >>> split_one = partial(str.split, X, 1) >> >> Or even >> >> _ = partial.skip >> split_one = partial(str.split, _, 1) > > Or even > > ? = partial.skip > split_one = partial(str.split, ?, 1) From p.f.moore at gmail.com Sat Jan 31 17:01:09 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 31 Jan 2009 16:01:09 +0000 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: <49842EF3.4010003@v.loewis.de> References: <79990c6b0901300528h160c0c81t74e44497fbcf709e@mail.gmail.com> <4983A6B9.40304@v.loewis.de> <79990c6b0901310239y7fe73255o988c4b5dbc7b3ba4@mail.gmail.com> <49842EF3.4010003@v.loewis.de> Message-ID: <79990c6b0901310801m33286a0bmee745ff71269f09a@mail.gmail.com> 2009/1/31 "Martin v. L?wis" : >> Can you point me at specifics (bug reports or test cases)? I could see >> if I can help in fixing things. > > See r69098. Thanks. So 3.0.1 and later will be fine - my apologies, I hadn't quite understood what you said. Paul. From digitalxero at gmail.com Sat Jan 31 18:45:12 2009 From: digitalxero at gmail.com (Dj Gilcrease) Date: Sat, 31 Jan 2009 10:45:12 -0700 Subject: [Python-Dev] PEP 374 (DVCS) now in reST In-Reply-To: References: <497BA590.7060406@v.loewis.de> <497BB5E7.4080606@gmail.com> <8E8084F3-0FAF-468E-8820-1ADA3C49380F@python.org> <497CB185.3010601@v.loewis.de> <573EFD3B-2215-4747-B4B0-45C35A9F9F86@gmail.com> Message-ID: a Mercurial "super client" http://blog.red-bean.com/sussman/?p=116 Figured I would link to this for the people doing the HG investigation From g.brandl at gmx.net Sat Jan 31 19:25:45 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 31 Jan 2009 19:25:45 +0100 Subject: [Python-Dev] Python 3.0.1 In-Reply-To: References: <49809B0C.4020905@egenix.com> <49809C1B.3090805@voidspace.org.uk> <4980CD7A.6000506@v.loewis.de> <87d4e7dj0p.fsf@xemacs.org> <20090129113130.GA2490@amk.local> <7F2AD110-96A2-4367-821E-6DB8E2E3DC86@python.org> Message-ID: Guido van Rossum schrieb: > Frankly, I don't really believe the users for whom those rules were > created are using 3.0 yet. Instead, I expect there to be two types of > users: people in the educational business who don't have a lot of > bridges to burn and are eager to use the new features; and developers > of serious Python software (e.g. Twisted) who are trying to figure out > how to port their code to 3.0. The first group isn't affected by the > changes we're considering here (e.g. removing cmp or some obscure > functions from the operator module). The latter group *may* be > affected, simply because they may have some pre-3.0 code using old > features that (by accident) still works under 3.0. > > On the one hand I understand that those folks want a stable target. On > the other hand I think they would prefer to find out sooner rather > than later they're using stuff they shouldn't be using any more. It's > a delicate balance for sure, and I certainly don't want to open the > floodgates here, or rebrand 3.1 as 3.0.1 or anything like that. But I > really don't believe that the strictest interpretation of "no new > features" will benefit us for 3.0.1. Perhaps we should decide when to > go back to a more strict interpretation of the rules based on the > uptake of Python 3 compared to Python 2. +1. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From leif.walsh at gmail.com Sat Jan 31 20:40:12 2009 From: leif.walsh at gmail.com (Leif Walsh) Date: Sat, 31 Jan 2009 14:40:12 -0500 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: <76fd5acf0901301638x6d4da376w34fe78d327af7c87@mail.gmail.com> References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> <5E031BDD-32EC-47CC-8CE4-D4D3888A49F2@gmail.com> <76fd5acf0901301638x6d4da376w34fe78d327af7c87@mail.gmail.com> Message-ID: On Fri, Jan 30, 2009 at 7:38 PM, Calvin Spealman wrote: > I am just replying to the end of this thread to throw in a reminder > about my partial.skip patch, which allows the following usage: > > split_one = partial(str.split, partial.skip, 1) > > Not looking to say "mine is better", but if the idea is being given > merit, I like the skipping arguments method better than just the > "right partial", which I think is confusing combined with keyword and > optional arguments. And, this patch already exists. Could it be > re-evaluated? +1 but I don't know where the patch is. -- Cheers, Leif From martin at v.loewis.de Sat Jan 31 20:43:06 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 31 Jan 2009 20:43:06 +0100 Subject: [Python-Dev] 3.0.1/3.1.0 summary In-Reply-To: <592904F8-89F5-4AC7-B3BF-0C51D3B474A6@python.org> References: <498295F5.2050607@v.loewis.de> <592904F8-89F5-4AC7-B3BF-0C51D3B474A6@python.org> Message-ID: <4984A9CA.1010307@v.loewis.de> > How about Friday February 13? Fine with me (although next Friday (Feb 6) would work slightly better) Martin From dickinsm at gmail.com Sat Jan 31 22:07:37 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 31 Jan 2009 21:07:37 +0000 Subject: [Python-Dev] Removing tp_compare? Message-ID: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com> Here's a question (actually, three questions) for python-dev that came up in the issue 1717 (removing cmp) discussion. Once the cmp removal is complete, the type object's tp_compare slot will no longer be used. The current plan is to rename it to tp_reserved, change its type to (void *), and raise TypeError when initializing any type that attempts to put something nonzero into that slot. But another possibility would be to remove it entirely. So... Questions: (1) Is it desirable to remove tp_compare entirely, instead of just renaming it? (2) If so, for which Python version should that removal take place? 3.0.1? 3.1.0? 4.0? and the all-important bikeshed question: (3) In the meantime, what should the renamed slot be called? tp_reserved? In the issue 1717 discussion, Raymond suggested tp_deprecated_compare. Any thoughts? My own opinion is that it really doesn't matter that much if the slot is left in; it's just a little annoying to have such backwards-compatibility baggage already present in the shiny new 3.0 series. A little like finding a big scratch on your brand-new bright yellow Hummer H3. Or not. N.B. The same questions apply to nb_reserved (which used to be nb_long) in the PyNumberMethods structure. Mark From benjamin at python.org Sat Jan 31 22:18:13 2009 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 31 Jan 2009 15:18:13 -0600 Subject: [Python-Dev] Removing tp_compare? In-Reply-To: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com> References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com> Message-ID: <1afaf6160901311318y74b30a2bt8d9e9804525bb130@mail.gmail.com> On Sat, Jan 31, 2009 at 3:07 PM, Mark Dickinson wrote: > Once the cmp removal is complete, the type object's tp_compare > slot will no longer be used. The current plan is to rename it to > tp_reserved, change its type to (void *), and raise TypeError when > initializing any type that attempts to put something nonzero into > that slot. But another possibility would be to remove it entirely. > So... I think we should keep as tp_reserved in 3.0.1. In 3.1, as I mentioned in the issue, I'd like to reuse it as a slot for __bytes__. Confusion could be avoided still raising a TypeError for a non-null tp_reserved slot unless the type has Py_TPFLAGS_HAVE_BYTES flag set. After a while, we could just make it default. > > N.B. The same questions apply to nb_reserved (which used > to be nb_long) in the PyNumberMethods structure. IMO, it's fine to keep them around, just in case. -- Regards, Benjamin From martin at v.loewis.de Sat Jan 31 22:28:22 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 31 Jan 2009 22:28:22 +0100 Subject: [Python-Dev] Removing tp_compare? In-Reply-To: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com> References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com> Message-ID: <4984C276.70100@v.loewis.de> > (1) Is it desirable to remove tp_compare entirely, instead of > just renaming it? No. > (2) If so, for which Python version should that removal take place? > 3.0.1? 3.1.0? 4.0? If it is removed, it definitely shouldn't be removed in 3.0.1; that would be a binary-incompatible change. > (3) In the meantime, what should the renamed slot be called? > tp_reserved? In the issue 1717 discussion, Raymond suggested > tp_deprecated_compare. tp_reserved sounds fine. In 3.0.1, filling it with a function pointer should give no error, since that would be a binary-incompatible change. > Any thoughts? My own opinion is that it really doesn't matter > that much if the slot is left in; it's just a little annoying to have > such backwards-compatibility baggage already present in > the shiny new 3.0 series. A little like finding a big scratch > on your brand-new bright yellow Hummer H3. Or not. Well, there is also PY_SSIZE_T_CLEAN. I asked before 3.0, and was told that it was too late to remove it. Regards, Martin From greg at krypto.org Sat Jan 31 22:35:23 2009 From: greg at krypto.org (Gregory P. Smith) Date: Sat, 31 Jan 2009 13:35:23 -0800 Subject: [Python-Dev] Subversion upgraded to 1.5 In-Reply-To: <49847292.5080401@v.loewis.de> References: <49847292.5080401@v.loewis.de> Message-ID: <52dc1c820901311335h17ab0ed3h220b0bcbc75f274@mail.gmail.com> I'm seeing the following when trying to svn commit: Transmitting file data ...Read from remote host svn.python.org: Operation timed out svn: Commit failed (details follow): svn: Connection closed unexpectedly ... That was with subversion 1.4.4; copying my changes to a different host with subversion 1.5.1 has the same result. svn update works fine on both hosts in the same sandbox i'm trying to commit from. fwiw, they are both connecting to svn.python.org using IPv6 but that should be irrelevant to svn+ssh as the tcp6 ssh connection works fine. any ideas? -Greg On Sat, Jan 31, 2009 at 7:47 AM, "Martin v. L?wis" wrote: > I have now upgraded subversion to 1.5.1 on svn.python.org. > > Please let me know if you encounter problems. > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/greg%40krypto.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Sat Jan 31 22:37:05 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 31 Jan 2009 21:37:05 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Removing_tp=5Fcompare=3F?= References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com> <4984C276.70100@v.loewis.de> Message-ID: Martin v. L?wis v.loewis.de> writes: > > > Any thoughts? My own opinion is that it really doesn't matter > > that much if the slot is left in; it's just a little annoying to have > > such backwards-compatibility baggage already present in > > the shiny new 3.0 series. A little like finding a big scratch > > on your brand-new bright yellow Hummer H3. Or not. > > Well, there is also PY_SSIZE_T_CLEAN. I asked before 3.0, and was told > that it was too late to remove it. Are all modules PY_SSIZE_T_CLEAN? Last I looked, _ssl.c still used int or long in various places instead of Py_ssize_t. Regards Antoine. From martin at v.loewis.de Sat Jan 31 22:48:44 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 31 Jan 2009 22:48:44 +0100 Subject: [Python-Dev] Removing tp_compare? In-Reply-To: References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com> <4984C276.70100@v.loewis.de> Message-ID: <4984C73C.3000804@v.loewis.de> > Are all modules PY_SSIZE_T_CLEAN? Last I looked, _ssl.c still used int or long > in various places instead of Py_ssize_t. That's probably still the case. Regards, Martin From martin at v.loewis.de Sat Jan 31 23:04:24 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 31 Jan 2009 23:04:24 +0100 Subject: [Python-Dev] Subversion upgraded to 1.5 In-Reply-To: <52dc1c820901311335h17ab0ed3h220b0bcbc75f274@mail.gmail.com> References: <49847292.5080401@v.loewis.de> <52dc1c820901311335h17ab0ed3h220b0bcbc75f274@mail.gmail.com> Message-ID: <4984CAE8.1050509@v.loewis.de> > any ideas? Assuming you reported this right after it happened - sorry, no. I can't find anything relevant in the log files (although a precise time of failure would have helped). Does a plain "ssh pythondev at svn.python.org" still work? What path did you try to check into? Regards, Martin From janssen at parc.com Sat Jan 31 23:06:48 2009 From: janssen at parc.com (Bill Janssen) Date: Sat, 31 Jan 2009 14:06:48 PST Subject: [Python-Dev] =?utf-8?q?Removing_tp=5Fcompare=3F?= In-Reply-To: References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com> <4984C276.70100@v.loewis.de> Message-ID: <57543.1233439608@parc.com> Antoine Pitrou wrote: > Martin v. L?wis v.loewis.de> writes: > > > > > Any thoughts? My own opinion is that it really doesn't matter > > > that much if the slot is left in; it's just a little annoying to have > > > such backwards-compatibility baggage already present in > > > the shiny new 3.0 series. A little like finding a big scratch > > > on your brand-new bright yellow Hummer H3. Or not. > > > > Well, there is also PY_SSIZE_T_CLEAN. I asked before 3.0, and was told > > that it was too late to remove it. > > Are all modules PY_SSIZE_T_CLEAN? Last I looked, _ssl.c still used int or long > in various places instead of Py_ssize_t. _ssl.c does indeed use int or long in various places. I'm not sure how far it can go with Py_ssize_t -- is OpenSSL 64-bit clean? Bill From solipsis at pitrou.net Sat Jan 31 23:09:54 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 31 Jan 2009 22:09:54 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Removing_tp=5Fcompare=3F?= References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com> <4984C276.70100@v.loewis.de> <57543.1233439608@parc.com> Message-ID: Bill Janssen parc.com> writes: > > is OpenSSL 64-bit clean? I'm afraid I'm completely incompetent on this subject...! Regards Antoine. From greg at krypto.org Sat Jan 31 23:49:17 2009 From: greg at krypto.org (Gregory P. Smith) Date: Sat, 31 Jan 2009 14:49:17 -0800 Subject: [Python-Dev] Subversion upgraded to 1.5 In-Reply-To: <4984CAE8.1050509@v.loewis.de> References: <49847292.5080401@v.loewis.de> <52dc1c820901311335h17ab0ed3h220b0bcbc75f274@mail.gmail.com> <4984CAE8.1050509@v.loewis.de> Message-ID: <52dc1c820901311449w6b487195t55f249ca88d163f2@mail.gmail.com> I'm just trying to commit the following to trunk: Sending Lib/test/test_socket.py Sending Misc/NEWS Sending Modules/socketmodule.c Transmitting file data ... I have another svn commit attempt which appesrs to be hanging and destined to timeout running right now. ssh -v pythondev at svn.python.org works fine. -gps On Sat, Jan 31, 2009 at 2:04 PM, "Martin v. L?wis" wrote: > > any ideas? > > Assuming you reported this right after it happened - sorry, no. > I can't find anything relevant in the log files (although a > precise time of failure would have helped). > > Does a plain "ssh pythondev at svn.python.org" still work? > > What path did you try to check into? > > Regards, > Martin > -------------- next part -------------- An HTML attachment was scrubbed... URL:
RetroSearch is an open source project built by @garambo
| Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4