Showing content from https://mail.python.org/pipermail/python-dev/2010-December.txt below:
References: Message-ID: Am 11.12.2010 22:28, schrieb Terry Reedy: > On 12/11/2010 12:19 PM, Georg Brandl wrote: >> Am 11.12.2010 11:36, schrieb Prashant Kumar: >>> I was wondering if we could contribute in porting of python.org >>> website over to python3k. > > I think this is an excellent idea. It will test Python3 and the modules > and external packages used and once successful, advertise that all such > are production-ready Py3 software. If and when there is a test version I > can access (py3.python.org?), I will happily help test by browsing > around and downloading stuff. I don't think a separate test site is useful. Either the build process works with Python 3 or it doesn't -- in the latter case you'll just get an exception while running "make". Georg From steve at pearwood.info Sun Dec 12 02:22:43 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 12 Dec 2010 12:22:43 +1100 Subject: [Python-Dev] futures API In-Reply-To: References: <608275.24121.qm@web27507.mail.ukl.yahoo.com> Message-ID: <4D0423E3.4020805@pearwood.info> Brian Quinlan wrote: > > On Dec 11, 2010, at 6:44 AM, Thomas Nagy wrote: >> I have also observed a minor performance degradation with the executor >> replacement (3 seconds for 5000 work items). The amount of work items >> processed by unit of time does not seem to be a straight line: >> http://www.freehackers.org/~tnagy/runtime_futures_2.png . > > That looks pretty linear to me. Close to, but not quite. The graph seems to be slightly curved, with the amount of work done per second decreasing for large amounts of work. Assuming that this performance degradation is real, and not an artifact of the measurement technique, it seems to be quite small. I'd be happy to describe it as "linear" in the same way we describe dictionary lookups as constant-time, even though technically that's not strictly true. (They're linear in the number of items with a matching hash, and there are probably other complications as well.) As drawn, the curve seems to fall away like a log graph, which might suggest to the casual viewer that this is a good thing. It may be better to reverse the axes, that is to have the independent variable, number of tasks, on the horizontal axis, and the dependent variable, cost (time taken), vertically. This will make it clear that the incremental cost of doing one extra task increases (slightly) as the number of tasks goes up. -- Steven From ncoghlan at gmail.com Sun Dec 12 03:30:37 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 12 Dec 2010 12:30:37 +1000 Subject: [Python-Dev] Format factories (was Re: sWAPcASE Was: transform() and untransform() methods, and the codec registry) In-Reply-To: References: Message-ID: On Sun, Dec 12, 2010 at 5:21 AM, Brett Cannon wrote: > But is this worth it since once you write it you won't be changing it > again, suggesting that taking the time to look up the formatting rules > isn't that much harder and wouldn't burden us w/ writing such > functions and trying to come up with a good API. I suspect what would > be more helpful is either have a rather detailed docstring for > str.format or to at least put the URL to the docs which explains the > mini-language to make it easier to find. Yes, it may be worth it, since it greatly simplifies things like generating a format spec programmatically rather than hardcoding it. You can also run it once at the interactive prompt to figure out the format string you want to hard code, then include the API call as a comment for concise documentation of what your format string does. I helped *define* the new format spec and I still need to reference the docs to create trickier ones. A factory function would be far more convenient than burdening str.format's docstring with a full definition of the format spec syntax. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Sun Dec 12 03:33:34 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 12 Dec 2010 12:33:34 +1000 Subject: [Python-Dev] futures API In-Reply-To: References: <608275.24121.qm@web27507.mail.ukl.yahoo.com> Message-ID: On Sun, Dec 12, 2010 at 6:53 AM, Brian Quinlan wrote: > Is it still unclear why it is there? Maybe you could propose some additional > documentation. Did you get my question the other day as to whether a weakref.WeakKeySet might be a better choice? I believe you would be able to get rid of the periodic sweep for dead references if you did that, and I didn't spot any obvious downsides. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From tnagyemail-mail at yahoo.fr Sun Dec 12 03:32:18 2010 From: tnagyemail-mail at yahoo.fr (Thomas Nagy) Date: Sun, 12 Dec 2010 02:32:18 +0000 (GMT) Subject: [Python-Dev] futures API Message-ID: <229168.97019.qm@web27508.mail.ukl.yahoo.com> --- El s?b, 11/12/10, Brian Quinlan escribi?: > > On Dec 11, 2010, at 6:44 AM, Thomas Nagy wrote: > > > --- El vie, 10/12/10, Brian Quinlan escribi?: > >> On Dec 10, 2010, at 10:51 AM, Thomas Nagy wrote: > >>> --- El vie, 10/12/10, Brian Quinlan > escribi?: > >>>> On Dec 10, 2010, at 5:36 AM, Thomas Nagy > wrote: > >>>>> I have a process running for a long > time, and > >> which > >>>> may use futures of different max_workers > count. I > >> think it > >>>> is not too far-fetched to create a new > futures > >> object each > >>>> time. Yet, the execution becomes slower > after each > >> call, for > >>>> example with http://freehackers.org/~tnagy/futures_test.py: > >>>>> > >>>>> """ > >>>>> import concurrent.futures > >>>>> from queue import Queue > >>>>> import datetime > >>>>> > >>>>> class counter(object): > >>>>>? ? ???def > __init__(self, fut): > >>>>>? ? ? ? > ???self.fut = > >> fut > >>>>> > >>>>>? ? ???def > run(self): > >>>>>? ? ? ? > ???def > >>>> look_busy(num, obj): > >>>>> > >>>>? ???tot = 0 > >>>>> > >>>>? ???for x in > range(num): > >>>>> > >>>>? ???tot += x > >>>>> > >>>>? > ???obj.out_q.put(tot) > >>>>> > >>>>>? ? ? ? > ???start = > >>>> datetime.datetime.utcnow() > >>>>>? ? ? ? > ???self.count = > >> 0 > >>>>>? ? ? ? > ???self.out_q > >> = > >>>> Queue(0) > >>>>>? ? ? ? > ???for x in > >>>> range(1000): > >>>>> > >>>>? ???self.count += 1 > >>>>> > >>>>? > ???self.fut.submit(look_busy, > >> self.count, > >>>> self) > >>>>> > >>>>>? ? ? ? > ???while > >>>> self.count: > >>>>> > >>>>? ???self.count -= 1 > >>>>> > >>>>? ???self.out_q.get() > >>>>> > >>>>>? ? ? ? > ???delta = > >>>> datetime.datetime.utcnow() - start > >>>>> > >>>>? > ???print(delta.total_seconds()) > >>>>> > >>>>> fut = > >>>> > >> > concurrent.futures.ThreadPoolExecutor(max_workers=20) > >>>>> for x in range(100): > >>>>>? ? ???# > comment the following > >> line > >>>>>? ? ???fut = > >>>> > >> > concurrent.futures.ThreadPoolExecutor(max_workers=20) > >>>>>? ? ???c = > counter(fut) > >>>>>? ? > ???c.run() > >>>>> """ > >>>>> > >>>>> The runtime grows after each step: > >>>>> 0.216451 > >>>>> 0.225186 > >>>>> 0.223725 > >>>>> 0.222274 > >>>>> 0.230964 > >>>>> 0.240531 > >>>>> 0.24137 > >>>>> 0.252393 > >>>>> 0.249948 > >>>>> 0.257153 > >>>>> ... > >>>>> > >>>>> Is there a mistake in this piece of > code? > >>>> > >>>> There is no mistake that I can see but I > suspect > >> that the > >>>> circular references that you are building > are > >> causing the > >>>> ThreadPoolExecutor to take a long time to > be > >> collected. Try > >>>> adding: > >>>> > >>>>? ? ? c = counter(fut) > >>>>? ? ? c.run() > >>>> +? ? fut.shutdown() > >>>> > >>>> Even if that fixes your problem, I still > don't > >> fully > >>>> understand this because I would expect the > runtime > >> to fall > >>>> after a while as ThreadPoolExecutors are > >> collected. > >>> > >>> The shutdown call is indeed a good fix :-) > Here is the > >> time response > >>> of the calls to counter() when shutdown is > not > >> called: > >>> http://www.freehackers.org/~tnagy/runtime_futures.png > >> > >> FWIW, I think that you are confusion the term > "future" > >> with > >> "executor". A future represents a single work > item. An > >> executor > >> creates futures and schedules their underlying > work. > > > > Ah yes, sorry. I have also realized that the executor > is not the killer feature I was expecting, it can only > replace a little part of the code I have: controlling the > exceptions and the workflow is the most complicated part. > > > > I have also observed a minor performance degradation > with the executor replacement (3 seconds for 5000 work > items). The amount of work items processed by unit of time > does not seem to be a straight line: http://www.freehackers.org/~tnagy/runtime_futures_2.png > . > > That looks pretty linear to me. Ok. > > Out of curiosity, what is the "_thread_references" > for? > > There is a big comment above it in the code: > > # Workers are created as daemon threads. This is done to > allow the interpreter > # to exit when there are still idle threads in a > ThreadPoolExecutor's thread > # pool (i.e. shutdown() was not called). However, allowing > workers to die with > # the interpreter has two undesirable properties: > #???- The workers would still be running > during interpretor shutdown, > #? ???meaning that they would fail in > unpredictable ways. > #???- The workers could be killed while > evaluating a work item, which could > #? ???be bad if the callable being > evaluated has external side-effects e.g. > #? ???writing to a file. > # > # To work around this problem, an exit handler is installed > which tells the > # workers to exit when their work queues are empty and then > waits until the > # threads finish. > > _thread_references = set() > _shutdown = False > > def _python_exit(): > ? ? global _shutdown > ? ? _shutdown = True > ? ? for thread_reference in _thread_references: > ? ? ? ? thread = thread_reference() > ? ? ? ? if thread is not None: > ? ? ? ? ? ? thread.join() > > Is it still unclear why it is there? Maybe you could > propose some additional documentation. I was thinking that if exceptions have to be caught - and it is likely to be the case in general - then this scheme is redundant. Now I see that the threads are getting their work items from a queue, so it is clear now. Thanks for all the information, Thomas From brian at sweetapp.com Sun Dec 12 03:36:47 2010 From: brian at sweetapp.com (Brian Quinlan) Date: Sat, 11 Dec 2010 18:36:47 -0800 Subject: [Python-Dev] futures API In-Reply-To: References: <608275.24121.qm@web27507.mail.ukl.yahoo.com> Message-ID: <2FA356B1-88E6-43C7-8118-70CAD2BBB315@sweetapp.com> On Dec 11, 2010, at 6:33 PM, Nick Coghlan wrote: > On Sun, Dec 12, 2010 at 6:53 AM, Brian Quinlan > wrote: >> Is it still unclear why it is there? Maybe you could propose some >> additional >> documentation. > > Did you get my question the other day as to whether a > weakref.WeakKeySet might be a better choice? I believe you would be > able to get rid of the periodic sweep for dead references if you did > that, and I didn't spot any obvious downsides. No I didn't, sorry! Could you resent it if it has more details then the paragraph above? Cheers, Brian > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Sun Dec 12 07:05:38 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 12 Dec 2010 16:05:38 +1000 Subject: [Python-Dev] futures API In-Reply-To: <2FA356B1-88E6-43C7-8118-70CAD2BBB315@sweetapp.com> References: <608275.24121.qm@web27507.mail.ukl.yahoo.com> <2FA356B1-88E6-43C7-8118-70CAD2BBB315@sweetapp.com> Message-ID: On Sun, Dec 12, 2010 at 12:36 PM, Brian Quinlan wrote: > > On Dec 11, 2010, at 6:33 PM, Nick Coghlan wrote: > >> On Sun, Dec 12, 2010 at 6:53 AM, Brian Quinlan wrote: >>> >>> Is it still unclear why it is there? Maybe you could propose some >>> additional >>> documentation. >> >> Did you get my question the other day as to whether a >> weakref.WeakKeySet might be a better choice? I believe you would be >> able to get rid of the periodic sweep for dead references if you did >> that, and I didn't spot any obvious downsides. > > No I didn't, sorry! Could you resent it if it has more details then the > paragraph above? There wasn't much more detail, as there actually isn't a great deal to the idea. This was the main comment in the previous email: "It seems to me that using WeakSet would mean you could get rid of _remove_dead_thread_references altogether (the implicit callbacks would handle that part), and then a set() call in _python_exit would give you concrete references to work with for cleanup purposes." Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From lukasz at langa.pl Sun Dec 12 13:01:42 2010 From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=) Date: Sun, 12 Dec 2010 13:01:42 +0100 Subject: [Python-Dev] API for the new sysconfig module In-Reply-To: <037003C3-70F1-4430-8CA4-55C122F40C86@gmail.com> References: <20101210215902.CDB5B1AB60E@kimball.webabinitio.net> <20101211151703.09c8675e@pitrou.net> <037003C3-70F1-4430-8CA4-55C122F40C86@gmail.com> Message-ID: <27F9BCAE-74C1-4863-85A4-1E411E1AC06A@langa.pl> Wiadomo?? napisana przez Raymond Hettinger w dniu 2010-12-11, o godz. 22:18: >> *(I sometimes lose track of which changes were made in both branches >> pre-2.7, which ones were mode post-2.7 release, and which ones went in >> pre-2.7, but were 3.x only regardless) > > Right. I missed that it was already in 2.7. > So, now we're stuck with it, forever. Why not deprecate it for 3.2 (easy since it's probably not yet used anywhere anyway, even in 2.7) and introduce sys.sysconfig. I really like that much better than Java-like accessor functions. -- Best regards, ?ukasz Langa tel. +48 791 080 144 WWW http://lukasz.langa.pl/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Sun Dec 12 14:42:31 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 12 Dec 2010 14:42:31 +0100 Subject: [Python-Dev] API for the new sysconfig module References: <20101210215902.CDB5B1AB60E@kimball.webabinitio.net> <20101211151703.09c8675e@pitrou.net> <037003C3-70F1-4430-8CA4-55C122F40C86@gmail.com> <27F9BCAE-74C1-4863-85A4-1E411E1AC06A@langa.pl> Message-ID: <20101212144231.772efa58@pitrou.net> On Sun, 12 Dec 2010 13:01:42 +0100 ?ukasz Langa wrote: > Wiadomo?? napisana przez Raymond Hettinger w dniu 2010-12-11, o godz. 22:18: > > >> *(I sometimes lose track of which changes were made in both branches > >> pre-2.7, which ones were mode post-2.7 release, and which ones went in > >> pre-2.7, but were 3.x only regardless) > > > > Right. I missed that it was already in 2.7. > > So, now we're stuck with it, forever. > > Why not deprecate it for 3.2 (easy since it's probably not yet used anywhere anyway, even in 2.7) and introduce sys.sysconfig. We already had a lot of churn around these APIs (distutils & friends). I don't think it's a good idea to introduce even more churn. Oh and by the way it's too late to add or remove features from 3.2. > I really like that much better than Java-like accessor functions. Do you actually use sysconfig yourself? It's quite a specialized module, and I don't think API elegance arguments have a great weight here. Regards Antoine. From barry at python.org Sun Dec 12 15:05:06 2010 From: barry at python.org (Barry Warsaw) Date: Sun, 12 Dec 2010 09:05:06 -0500 Subject: [Python-Dev] API for the new sysconfig module In-Reply-To: <20101212144231.772efa58@pitrou.net> References: <20101210215902.CDB5B1AB60E@kimball.webabinitio.net> <20101211151703.09c8675e@pitrou.net> <037003C3-70F1-4430-8CA4-55C122F40C86@gmail.com> <27F9BCAE-74C1-4863-85A4-1E411E1AC06A@langa.pl> <20101212144231.772efa58@pitrou.net> Message-ID: <20101212090506.7c22638e@mission> On Dec 12, 2010, at 02:42 PM, Antoine Pitrou wrote: >On Sun, 12 Dec 2010 13:01:42 +0100 >?ukasz Langa wrote: > >> Wiadomo?? napisana przez Raymond Hettinger w dniu 2010-12-11, o godz. 22:18: >> >> >> *(I sometimes lose track of which changes were made in both branches >> >> pre-2.7, which ones were mode post-2.7 release, and which ones went in >> >> pre-2.7, but were 3.x only regardless) >> > >> > Right. I missed that it was already in 2.7. >> > So, now we're stuck with it, forever. >> >> Why not deprecate it for 3.2 (easy since it's probably not yet used anywhere anyway, even in 2.7) and introduce sys.sysconfig. > >We already had a lot of churn around these APIs (distutils & friends). I >don't think it's a good idea to introduce even more churn. >Oh and by the way it's too late to add or remove features from 3.2. > >> I really like that much better than Java-like accessor functions. > >Do you actually use sysconfig yourself? It's quite a specialized >module, and I don't think API elegance arguments have a great weight >here. I have used them and I do think they're fairly ugly and unwieldy. However, I agree that we should not rush into a different design. Since sysconfig was essentially refactored out of distutils (and now we have two ways to do it!), and we're getting distutils2 for 3.3, I think it would be better to work out a more natural design for sysconfig for 3.3. Then the sysconfig module can go through the natural deprecation cycle. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From ncoghlan at gmail.com Sun Dec 12 15:05:44 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Dec 2010 00:05:44 +1000 Subject: [Python-Dev] API for the new sysconfig module In-Reply-To: <20101212144231.772efa58@pitrou.net> References: <20101210215902.CDB5B1AB60E@kimball.webabinitio.net> <20101211151703.09c8675e@pitrou.net> <037003C3-70F1-4430-8CA4-55C122F40C86@gmail.com> <27F9BCAE-74C1-4863-85A4-1E411E1AC06A@langa.pl> <20101212144231.772efa58@pitrou.net> Message-ID: On Sun, Dec 12, 2010 at 11:42 PM, Antoine Pitrou wrote: >> I really like that much better than Java-like accessor functions. > > Do you actually use sysconfig yourself? It's quite a specialized > module, and I don't think API elegance arguments have a great weight > here. I would also like those advocating replacement of a tried and tested API with "oh, you can just use a single function for it" to rewrite distutils2 and the other tools that currently used distutils.sysconfig based on their "single function" approach before *anything* gets touched. This is not an API that was invented on a whim. It is well-established, with existing use cases that are essential to the wider Python ecosystem, and are more easily managed on some Linux distributions if they don't involve a dependency on distutils. Proposing to throw it away in favour of an ill-defined single function that indiscriminately mixes system data with metadata about that data because some people that don't even use the module find it aesthetically displeasing seems... unwise. Regards, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Sun Dec 12 15:44:22 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Dec 2010 00:44:22 +1000 Subject: [Python-Dev] Using logging in the stdlib and its unit tests In-Reply-To: References: <20101207215053.025ecf52@pitrou.net> <20101208010913.363c6105@pitrou.net> <4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com> <20101210214919.4ac63d27@pitrou.net> Message-ID: On Sun, Dec 12, 2010 at 5:32 AM, Brett Cannon wrote: > On Fri, Dec 10, 2010 at 22:21, Vinay Sajip wrote: >> Nick Coghlan gmail.com> writes: >> >> >>> This could actually make a reasonably good basic for a "task oriented" >>> subsection of the logging documentation. Something like: >>> >> >> Good suggestion, I'll see what I can do. > > Just wanted to +1 on some task-oriented (or at least simple intro) > docs going into the logging module. I think Vinay has made some great improvements to the logging module docs in the last day or two. The latest version out of SVN is available on the site at the usual location: http://docs.python.org/dev/library/logging I am putting some minor notes here for Vinay's benefit (I can put them on the tracker instead, if he would prefer): General It may be worth talking to Georg about how best to split the logging docs up into multiple files. The sidebar menu is getting kinda overwhelmed. 14.7.1.1 Parenthetical comment in first row of second table should start with "(e.g. for" not "(e.g. or" 14.7.1.8 Probably best to say "that's it for the basic tutorial" and then point people towards the advanced tutorial in 14.7.2 before setting them loose on the rest of the docs. The advanced tutorial defines the terminology and gives the necessary structure to help keep the detailed docs in perspective without being overwhelmed by the detail. 14.7.2.1 Something appears to have gone wrong with the first bulleted list. It is missing the "These are the configuration methods:" intro text, as well as a bullet for add/removeHandler The "does not address filters" part should cross-reference the detailed section on filter objects This section should state explicitly whether or not the level setting on a child logger affects which messages it passes to its parent logger 14.7.2.5 This section is out of date, and needs to be caveated to make it clear that it applies only to version prior to Python 3.2 (for 3.2, it can describe the new handler of last resort behaviour) And that's the end of the two tutorials... very nice update :) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From stefan-usenet at bytereef.org Sun Dec 12 15:37:03 2010 From: stefan-usenet at bytereef.org (Stefan Krah) Date: Sun, 12 Dec 2010 15:37:03 +0100 Subject: [Python-Dev] API for the new sysconfig module In-Reply-To: <27F9BCAE-74C1-4863-85A4-1E411E1AC06A@langa.pl> References: <20101210215902.CDB5B1AB60E@kimball.webabinitio.net> <20101211151703.09c8675e@pitrou.net> <037003C3-70F1-4430-8CA4-55C122F40C86@gmail.com> <27F9BCAE-74C1-4863-85A4-1E411E1AC06A@langa.pl> Message-ID: <20101212143703.GA15235@yoda.bytereef.org> Lukasz Langa wrote: > Wiadomo?? napisana przez Raymond Hettinger w dniu 2010-12-11, o godz. 22:18: > > Right. I missed that it was already in 2.7. > So, now we're stuck with it, forever. > > Why not deprecate it for 3.2 (easy since it's probably not yet used anywhere > anyway, even in 2.7) and introduce sys.sysconfig. I really like that much > better than Java-like accessor functions. When I use sysconfig, I just want to get things done as quickly and painlessly as possible. The API suits me just fine (in fact, I find it one of the APIs that are easy to remember). Given that sysconfig will always contain a certain amount of hackery and will always change to accommodate new systems, I'd prefer that it remains a standalone module. Stefan Krah From ziade.tarek at gmail.com Sun Dec 12 16:53:20 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 12 Dec 2010 16:53:20 +0100 Subject: [Python-Dev] API for the new sysconfig module In-Reply-To: <20101212090506.7c22638e@mission> References: <20101210215902.CDB5B1AB60E@kimball.webabinitio.net> <20101211151703.09c8675e@pitrou.net> <037003C3-70F1-4430-8CA4-55C122F40C86@gmail.com> <27F9BCAE-74C1-4863-85A4-1E411E1AC06A@langa.pl> <20101212144231.772efa58@pitrou.net> <20101212090506.7c22638e@mission> Message-ID: On Sun, Dec 12, 2010 at 3:05 PM, Barry Warsaw wrote: > On Dec 12, 2010, at 02:42 PM, Antoine Pitrou wrote: > >>On Sun, 12 Dec 2010 13:01:42 +0100 >>?ukasz Langa wrote: >> >>> Wiadomo?? napisana przez Raymond Hettinger w dniu 2010-12-11, o godz. 22:18: >>> >>> >> *(I sometimes lose track of which changes were made in both branches >>> >> pre-2.7, which ones were mode post-2.7 release, and which ones went in >>> >> pre-2.7, but were 3.x only regardless) >>> > >>> > Right. ?I missed that it was already in 2.7. >>> > So, now we're stuck with it, forever. >>> >>> Why not deprecate it for 3.2 (easy since it's probably not yet used anywhere anyway, even in 2.7) and introduce sys.sysconfig. >> >>We already had a lot of churn around these APIs (distutils & friends). I >>don't think it's a good idea to introduce even more churn. >>Oh and by the way it's too late to add or remove features from 3.2. >> >>> I really like that much better than Java-like accessor functions. >> >>Do you actually use sysconfig yourself? It's quite a specialized >>module, and I don't think API elegance arguments have a great weight >>here. > > I have used them and I do think they're fairly ugly and unwieldy. ?However, I > agree that we should not rush into a different design. ?Since sysconfig was > essentially refactored out of distutils (and now we have two ways to do it!), > and we're getting distutils2 for 3.3, I think it would be better to work out a > more natural design for sysconfig for 3.3. ?Then the sysconfig module can go > through the natural deprecation cycle. I don't think any API refactoring worth the pain here. I don't see the proposed changes make the caller's code that much better. The existing one is good enough in my opinion. Tarek -- Tarek Ziad? | http://ziade.org From vinay_sajip at yahoo.co.uk Sun Dec 12 18:41:46 2010 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Sun, 12 Dec 2010 17:41:46 +0000 (UTC) Subject: [Python-Dev] Using logging in the stdlib and its unit tests References: <20101207215053.025ecf52@pitrou.net> <20101208010913.363c6105@pitrou.net> <4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com> <20101210214919.4ac63d27@pitrou.net> Message-ID: Nick Coghlan gmail.com> writes: Gosh, Nick, that was fast! I'm still making changes, but thanks for spotting and highlighting the typos and omissions. I've just checked in a further update; hopefully it'll get built soon so we can all see the latest changes. Regards, Vinay Sajip From zeljko.grk at gmail.com Sun Dec 12 20:04:49 2010 From: zeljko.grk at gmail.com (Zeljko) Date: Sun, 12 Dec 2010 19:04:49 +0000 (UTC) Subject: [Python-Dev] use case for bytes.format Message-ID: I'm considering to write some pure python pdf lib from from scratch, but found there is no built-in formating for bytes. It's very frustrating to use some inefficient surogate funtion, which is gong to be called thousands times. From p.f.moore at gmail.com Sun Dec 12 23:26:10 2010 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 12 Dec 2010 22:26:10 +0000 Subject: [Python-Dev] Using logging in the stdlib and its unit tests In-Reply-To: <4D029439.7040708@g.nevcal.com> References: <20101207215053.025ecf52@pitrou.net> <20101208010913.363c6105@pitrou.net> <4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com> <20101210214919.4ac63d27@pitrou.net> <4D029439.7040708@g.nevcal.com> Message-ID: On 10 December 2010 20:57, Glenn Linderman wrote: > On 12/10/2010 12:49 PM, Antoine Pitrou wrote: > > And yet, I have helped many people who were baffled by exactly what >> Bill observed: logging.info() didn't do anything. Maybe the default >> should be INFO? > > Funny, because displaying only errors and silencing other messages is > exactly what I expected logging to do by default. > > So we are slowly learning the things that should be on the first couple > pages of the logging docs... > > 1) simple example for one file programs, include an example of specifying > output severity threshold.? I'm with Antoine here on my expectations. > > 2) example for multi-module, showing how a single logging destination causes > logging to happen in all modules, at the same level (if that is the case, > which I hope it is). > > 3) Maybe a small discussion of log formatting should be next?? So the user > realizes he shouldn't do the message formatting himself? > > 4) Then rotating logs for long-running programs. The thing *I* hit very early was wanting to add a command lime option to my script to set the logging level. I'd have liked to be able to add --log=INFO/DEBUG/... but to do that I seem to need to write my own mapping between level names and numbers. A simple example of how to tie command line options to logging config would be a great addition to the documentation. Paul. From tjreedy at udel.edu Sun Dec 12 23:33:19 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 12 Dec 2010 17:33:19 -0500 Subject: [Python-Dev] use case for bytes.format In-Reply-To: References: Message-ID: On 12/12/2010 2:04 PM, Zeljko wrote: This post should have gone to python-list, mirrored as gmane.comp.python.general. Please limit any further response to either of those (or c.l.p) and delete pydev. > I'm considering to write some pure python pdf lib from from scratch, but found > there is no built-in formating for bytes. Bytes do, but you should use text str for general text manipulation. > It's very frustrating to use some inefficient surogate funtion, which is gong to > be called thousands times. This sentence has 3 spelling mistakes, 2 of which would be caught with a mail client like Thunderbird (free) that has spelling correction. Please consider switching. -- Terry Jan Reedy From merwok at netwok.org Sun Dec 12 23:33:22 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Sun, 12 Dec 2010 23:33:22 +0100 Subject: [Python-Dev] Using logging in the stdlib and its unit tests In-Reply-To: References: <20101207215053.025ecf52@pitrou.net> <20101208010913.363c6105@pitrou.net> <4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com> <20101210214919.4ac63d27@pitrou.net> Message-ID: <4D054DB2.8020204@netwok.org> Hi, I suggest to replace ?error? with ?event? in the module doc synopsis. Regards From v+python at g.nevcal.com Mon Dec 13 01:42:49 2010 From: v+python at g.nevcal.com (Glenn Linderman) Date: Sun, 12 Dec 2010 16:42:49 -0800 Subject: [Python-Dev] Using logging in the stdlib and its unit tests In-Reply-To: References: <20101207215053.025ecf52@pitrou.net> <20101208010913.363c6105@pitrou.net> <4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com> <20101210214919.4ac63d27@pitrou.net> <4D029439.7040708@g.nevcal.com> Message-ID: <4D056C09.3090104@g.nevcal.com> On 12/12/2010 2:26 PM, Paul Moore wrote: > The thing*I* hit very early was wanting to add a command lime option > to my script to set the logging level. I'd have liked to be able to > add --log=INFO/DEBUG/... but to do that I seem to need to write my own > mapping between level names and numbers. A simple example of how to > tie command line options to logging config would be a great addition > to the documentation. Oh? import * from logger # change some details to avoid this basicConfig( level= eval( opt.loglevel ) -------------- next part -------------- An HTML attachment was scrubbed... URL: From v+python at g.nevcal.com Mon Dec 13 01:53:05 2010 From: v+python at g.nevcal.com (Glenn Linderman) Date: Sun, 12 Dec 2010 16:53:05 -0800 Subject: [Python-Dev] Using logging in the stdlib and its unit tests In-Reply-To: References: <20101207215053.025ecf52@pitrou.net> <20101208010913.363c6105@pitrou.net> <4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com> <20101210214919.4ac63d27@pitrou.net> Message-ID: <4D056E71.3020704@g.nevcal.com> On 12/12/2010 9:41 AM, Vinay Sajip wrote: > Gosh, Nick, that was fast! I'm still making changes, but thanks for spotting and > highlighting the typos and omissions. I've just checked in a further update; > hopefully it'll get built soon so we can all see the latest changes. I'm not as fast as Nick, but let me add that these changes to the documentation are surely helpful to me. I've read 12% now, of a bigger base, but it was very approachable, and I've come away with being ready to scrap my little logger, I know what I need to do to make my multi-module logging work with the logging module instead, to greater benefit than my little logger, and the only "advanced technique" that I think I need to learn at the moment is formatters, so next chance I get I'll read about those. The mountain doesn't look as steep, now! Thanks for the fast reaction time. -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Mon Dec 13 02:22:47 2010 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 12 Dec 2010 19:22:47 -0600 Subject: [Python-Dev] Using logging in the stdlib and its unit tests In-Reply-To: <4D056C09.3090104@g.nevcal.com> References: <20101207215053.025ecf52@pitrou.net> <20101208010913.363c6105@pitrou.net> <4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com> <20101210214919.4ac63d27@pitrou.net> <4D029439.7040708@g.nevcal.com> <4D056C09.3090104@g.nevcal.com> Message-ID: On 2010-12-12 18:42 , Glenn Linderman wrote: > On 12/12/2010 2:26 PM, Paul Moore wrote: >> The thing*I* hit very early was wanting to add a command lime option >> to my script to set the logging level. I'd have liked to be able to >> add --log=INFO/DEBUG/... but to do that I seem to need to write my own >> mapping between level names and numbers. A simple example of how to >> tie command line options to logging config would be a great addition >> to the documentation. > > Oh? > > import * from logger # change some details to avoid this > basicConfig( level= eval( opt.loglevel ) level = getattr(logging, opt.logLevel) or level = logging._levelNames[opt.logLevel] -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ncoghlan at gmail.com Mon Dec 13 04:30:24 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Dec 2010 13:30:24 +1000 Subject: [Python-Dev] Using logging in the stdlib and its unit tests In-Reply-To: References: <20101207215053.025ecf52@pitrou.net> <20101208010913.363c6105@pitrou.net> <4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com> <20101210214919.4ac63d27@pitrou.net> <4D029439.7040708@g.nevcal.com> <4D056C09.3090104@g.nevcal.com> Message-ID: On Mon, Dec 13, 2010 at 11:22 AM, Robert Kern wrote: > level = getattr(logging, opt.logLevel) While this is the approach I would recommend, it does have a couple of downsides: 1. An upper() call is also needed to allow strings like "info" instead of "INFO": 2. If an integer is available, it would be nice to return it unmodified (or, if we ever get named values in the standard library, convert it to that) 3. The asymmetry with "logging.getLevelName" grates a bit So it would be far more obvious if there was a "logging.getLevel" counterpart to "logging.getLevelName" that looked something like: def getLevel(level): try: return operator.index(level) # Integers and equivalents except TypeError: pass try: key = level.upper() except Exception as ex: raise TypeError("Log level must be an integer or string") from ex return globals()[key] > level = logging._levelNames[opt.logLevel] That doesn't work (_levelNames maps from integers to strings, we want the mapping from strings to integers and it is only the module globals that provides that). Regards, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Mon Dec 13 04:45:50 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Dec 2010 13:45:50 +1000 Subject: [Python-Dev] [Python-checkins] r87202 - python/branches/py3k/Doc/library/logging.rst In-Reply-To: <20101212224535.C828DEE995@mail.python.org> References: <20101212224535.C828DEE995@mail.python.org> Message-ID: On Mon, Dec 13, 2010 at 8:45 AM, vinay.sajip wrote: > +to get the value which you'll pass to :func:`basicConfig` via the *level* > +argument. You may want to error check any user input value, perhaps as in the > +following example:: > + > + ? # assuming loglevel is bound to the string value obtained from the > + ? # command line argument. Convert to upper case to allow the user to > + ? # specify --log=DEBUG or --log=debug > + ? numeric_level = getattr(logging, loglevel.upper(), None) > + ? assert numeric_level is not None, 'Invalid log level: %s' % loglevel > + ? logging.basicConfig(level=numeric_level, ...) Minor nit - using asserts to check user input is generally a bad idea. A more explicit check might be a better example: if not isinstance(numeric_level, int): raise ValueError('Invalid log level: %s' % loglevel) This also covers the case where someone does something weird like pass in "basic_format" as a logging level. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From robert.kern at gmail.com Mon Dec 13 05:13:12 2010 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 12 Dec 2010 22:13:12 -0600 Subject: [Python-Dev] Using logging in the stdlib and its unit tests In-Reply-To: References: <20101207215053.025ecf52@pitrou.net> <20101208010913.363c6105@pitrou.net> <4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com> <20101210214919.4ac63d27@pitrou.net> <4D029439.7040708@g.nevcal.com> <4D056C09.3090104@g.nevcal.com> Message-ID: On 2010-12-12 21:30 , Nick Coghlan wrote: > On Mon, Dec 13, 2010 at 11:22 AM, Robert Kern wrote: >> level = getattr(logging, opt.logLevel) > > While this is the approach I would recommend, it does have a couple of > downsides: > > 1. An upper() call is also needed to allow strings like "info" instead > of "INFO": > 2. If an integer is available, it would be nice to return it > unmodified (or, if we ever get named values in the standard library, > convert it to that) > 3. The asymmetry with "logging.getLevelName" grates a bit > > So it would be far more obvious if there was a "logging.getLevel" > counterpart to "logging.getLevelName" that looked something like: > > def getLevel(level): > try: > return operator.index(level) # Integers and equivalents > except TypeError: > pass > try: > key = level.upper() > except Exception as ex: > raise TypeError("Log level must be an integer or string") from ex > return globals()[key] I don't think that the implementation should use globals(). I wouldn't want logging.getLevel('basic_format') to work. Instead, it should look it up in the known set of levels. >> level = logging._levelNames[opt.logLevel] > > That doesn't work (_levelNames maps from integers to strings, we want > the mapping from strings to integers and it is only the module globals > that provides that). At least in Python 2.6, it maps both ways. But then again, it is an _implementation _detail that should not be relied upon in your programs. I would suggest that there should be two dictionaries as part of the documented API, one mapping numbers to names and one mapping names to numbers. Or functions/methods returning said dictionaries. Having the entire mappings at hand is more useful than having functions that do the translation. They would allow you to auto-generate UIs (e.g. the help text for a --log-level argument or a radio box widget in a GUI). Having separate mappings makes them easier to work with than the 2.6-style _levelNames mapping. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ncoghlan at gmail.com Mon Dec 13 05:27:21 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Dec 2010 14:27:21 +1000 Subject: [Python-Dev] Using logging in the stdlib and its unit tests In-Reply-To: References: <20101207215053.025ecf52@pitrou.net> <20101208010913.363c6105@pitrou.net> <4D01E55B.2050507@g.nevcal.com> <3231.1292004978@parc.com> <20101210214919.4ac63d27@pitrou.net> <4D029439.7040708@g.nevcal.com> <4D056C09.3090104@g.nevcal.com> Message-ID: On Mon, Dec 13, 2010 at 2:13 PM, Robert Kern wrote: >>> level = logging._levelNames[opt.logLevel] >> >> That doesn't work (_levelNames maps from integers to strings, we want >> the mapping from strings to integers and it is only the module globals >> that provides that). > > At least in Python 2.6, it maps both ways. But then again, it is an > _implementation _detail that should not be relied upon in your programs. Ah, you're quite right - I didn't notice that when looking at the contents (the first entries happened to map levels to names) > I > would suggest that there should be two dictionaries as part of the > documented API, one mapping numbers to names and one mapping names to > numbers. Or functions/methods returning said dictionaries. Having the entire > mappings at hand is more useful than having functions that do the > translation. They would allow you to auto-generate UIs (e.g. the help text > for a --log-level argument or a radio box widget in a GUI). Having separate > mappings makes them easier to work with than the 2.6-style _levelNames > mapping. Definitely something worth considering for 3.3. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From rich at noir.com Fri Dec 10 21:20:13 2010 From: rich at noir.com (K. Richard Pixley) Date: Fri, 10 Dec 2010 12:20:13 -0800 Subject: [Python-Dev] logging doc broken Message-ID: <4D028B7D.9030308@noir.com> I'm not sure where to report this but the online doc appears to be mismatched to python-2.6.5 for the logging module. Specifically, for a dir of an instance of a LogRecord, I'm seeing: ['__doc__', '__init__', '__module__', '__str__', 'args', 'created', 'exc_info', 'exc_text', 'filename', 'funcName', 'getMessage', 'levelname', 'levelno', 'lineno', 'module', 'msecs', 'msg', 'name', 'pathname', 'process', 'processName', 'relativeCreated', 'thread', 'threadName'] while the documentation lists a different set of attributes including "lvl". --rich From ncoghlan at gmail.com Mon Dec 13 11:21:18 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Dec 2010 20:21:18 +1000 Subject: [Python-Dev] logging doc broken In-Reply-To: <4D028B7D.9030308@noir.com> References: <4D028B7D.9030308@noir.com> Message-ID: On Sat, Dec 11, 2010 at 6:20 AM, K. Richard Pixley wrote: > I'm not sure where to report this but the online doc appears to be > mismatched to python-2.6.5 for the logging module. > > Specifically, for a dir of an instance of a LogRecord, I'm seeing: > > ['__doc__', '__init__', '__module__', '__str__', 'args', 'created', > 'exc_info', 'exc_text', 'filename', 'funcName', 'getMessage', 'levelname', > 'levelno', 'lineno', 'module', 'msecs', 'msg', 'name', 'pathname', > 'process', 'processName', 'relativeCreated', 'thread', 'threadName'] > > while the documentation lists a different set of attributes including "lvl". As discussed on this list recently, the logging documentation can unfortunately give the impression that the attributes of the log record are the same as the arguments to the log record constructor. This is not the case - the actual list of attributes can be found in the table showing the useful LogRecord attributes that are available to formatters (and filters) when processing records: http://docs.python.org/library/logging#formatter-objects Vinay is currently working on updates to the logging documentation, and this is one of the things that is likely to change (with the table of attributes moved to the LogRecord section and referenced from the sections on Formatter and Filter objects). Regards, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From vinay_sajip at yahoo.co.uk Mon Dec 13 11:31:35 2010 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Mon, 13 Dec 2010 10:31:35 +0000 (UTC) Subject: [Python-Dev] logging doc broken References: <4D028B7D.9030308@noir.com> Message-ID: K. Richard Pixley noir.com> writes: > > I'm not sure where to report this but the online doc appears to be > mismatched to python-2.6.5 for the logging module. > > Specifically, for a dir of an instance of a LogRecord, I'm seeing: > > ['__doc__', '__init__', '__module__', '__str__', 'args', 'created', > 'exc_info', 'exc_text', 'filename', 'funcName', 'getMessage', > 'levelname', 'levelno', 'lineno', 'module', 'msecs', 'msg', 'name', > 'pathname', 'process', 'processName', 'relativeCreated', 'thread', > 'threadName'] > > while the documentation lists a different set of attributes including "lvl". > Please report this on bugs.python.org. Check that you were referring to the 2.6 documentation - I didn't see any references to attributes there (following a quick scan). Ideally, link to the wrong doc section on docs.python.org in your bug report. Thanks, Vinay Sajip From mail at timgolden.me.uk Mon Dec 13 11:35:08 2010 From: mail at timgolden.me.uk (Tim Golden) Date: Mon, 13 Dec 2010 10:35:08 +0000 Subject: [Python-Dev] logging doc broken In-Reply-To: References: <4D028B7D.9030308@noir.com> Message-ID: <4D05F6DC.5040900@timgolden.me.uk> On 13/12/2010 10:31, Vinay Sajip wrote: [...] > Ideally, link to the wrong doc section on docs.python.org in your bug report. Now that's not a piece of advice you see very often :) TJG From ncoghlan at gmail.com Mon Dec 13 11:38:35 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Dec 2010 20:38:35 +1000 Subject: [Python-Dev] logging doc broken In-Reply-To: References: <4D028B7D.9030308@noir.com> Message-ID: On Mon, Dec 13, 2010 at 8:31 PM, Vinay Sajip wrote: > Please report this on bugs.python.org. Check that you were referring to the 2.6 > documentation - I didn't see any references to attributes there (following a > quick scan). Ideally, link to the wrong doc section on docs.python.org in your > bug report. The reference to "lvl" makes me think this is the same mistake I made the other day (i.e. thinking the constructor parameters are also the LogRecord attributes) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From vinay_sajip at yahoo.co.uk Mon Dec 13 11:58:32 2010 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Mon, 13 Dec 2010 10:58:32 +0000 (UTC) Subject: [Python-Dev] logging doc broken References: <4D028B7D.9030308@noir.com> Message-ID: Nick Coghlan gmail.com> writes: > Vinay is currently working on updates to the logging documentation, > and this is one of the things that is likely to change (with the table > of attributes moved to the LogRecord section and referenced from the > sections on Formatter and Filter objects). Yes, and to compound matters, the LogRecord constructor documentation lists the attribute values in the place you'd expect the parameters to appear, using the doc markup for attributes. This I think is what causes the confusion, although the markup is different to that used for parameters it's not clear at first glance. However, these changes are after 2.7 and 3.1: In 2.6 documentation the parameters are named as in the source, but there's no reference in the LogRecord reference documentation about any attributes of the LogRecord. Perhaps it would be best to do the following: - Add constructor params using normal parameter markup (:param:) - Create a separate section "LogRecord Attributes" with a table containing attr name, description, how to use with % formatting and how to use with {}-formatting. There is some overlap here with the parameter documentation for the LogRecord constructor, but I can put in a rider which explains that the param names and attr names are slightly different. (I could kick myself now for not being more scrupulous originally, but I'm not sure I can go back and change the param names in the source code now because in theory, someone might be constructing a LogRecord using LogRecord(**kwargs)). - Link to the latter section from the Formatter, Filter sections. Anyone see problems with this, or have a better suggestion? Regards, Vinay Sajip From vinay_sajip at yahoo.co.uk Mon Dec 13 12:04:26 2010 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Mon, 13 Dec 2010 11:04:26 +0000 (UTC) Subject: [Python-Dev] logging doc broken References: <4D028B7D.9030308@noir.com> <4D05F6DC.5040900@timgolden.me.uk> Message-ID: Tim Golden timgolden.me.uk> writes: > > On 13/12/2010 10:31, Vinay Sajip wrote: > [...] > > Ideally, link to the wrong doc section on docs.python.org in your bug report. > > Now that's not a piece of advice you see very often :) > True, but this area changed after 2.6 was released (after even 2.7, IIRC), so I want to be sure that if I'm going to change the doc sources on release26-maint, I'm doing the right thing. The 2.6 docs seem consistent with the 2.6 code, and while confusing (because the distinction between constructor param names and attr names is not spelled out), I don't see 2.6 docs as broken (or at least, I don't see where the breakage is). Regards, Vinay Sajip From ncoghlan at gmail.com Mon Dec 13 12:17:26 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Dec 2010 21:17:26 +1000 Subject: [Python-Dev] logging doc broken In-Reply-To: References: <4D028B7D.9030308@noir.com> Message-ID: On Mon, Dec 13, 2010 at 8:58 PM, Vinay Sajip wrote: > Nick Coghlan gmail.com> writes: > >> Vinay is currently working on updates to the logging documentation, >> and this is one of the things that is likely to change (with the table >> of attributes moved to the LogRecord section and referenced from the >> sections on Formatter and Filter objects). > > Yes, and to compound matters, the LogRecord constructor documentation lists the > attribute values in the place you'd expect the parameters to appear, using the > doc markup for attributes. This I think is what causes the confusion, although > the markup is different to that used for parameters it's not clear at first > glance. > > However, these changes are after 2.7 and 3.1: In 2.6 documentation the > parameters are named as in the source, but there's no reference in the > LogRecord reference documentation about any attributes of the LogRecord. That's the problem though - if you don't read the intro paragraph closely (which I didn't do), it's easy to jump to the conclusion that those are also the attributes (since there is nothing else in the section). > Perhaps it would be best to do the following: > > - Add constructor params using normal parameter markup (:param:) > - Create a separate section "LogRecord Attributes" with a table containing attr > name, description, how to use with % formatting and how to use with > {}-formatting. There is some overlap here with the parameter documentation for > the LogRecord constructor, but I can put in a rider which explains that the > param names and attr names are slightly different. (I could kick myself now for > not being more scrupulous originally, but I'm not sure I can go back and change > the param names in the source code now because in theory, someone might be > constructing a LogRecord using LogRecord(**kwargs)). > > - Link to the latter section from the Formatter, Filter sections. > > Anyone see problems with this, or have a better suggestion? Yep, that's what I had assumed you were going to do (this did come up in the big thread about the logging docs, but you may have missed it). You're also right about the backwards compatibility problem with changing the parameter names. It's the one downside of keyword arguments - it makes the parameter names part of a function's public API, so they are much harder to fix when you make a mistake. That said, you can't get a one-to-one correspondence in this case anyway, since some of the parameters relate to multiple attributes (e.g. "lvl" becomes both "levelNo" and "levelName"). It is rare that anyone will be constructing a LogRecord manually though, so I don't think the parameter names matter all that much in practice. It is far more common that people will be creating them implicitly through the Logger event notification methods or by deserialising them. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Mon Dec 13 12:21:36 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Dec 2010 21:21:36 +1000 Subject: [Python-Dev] logging doc broken In-Reply-To: References: <4D028B7D.9030308@noir.com> <4D05F6DC.5040900@timgolden.me.uk> Message-ID: On Mon, Dec 13, 2010 at 9:04 PM, Vinay Sajip wrote: > Tim Golden timgolden.me.uk> writes: > >> >> On 13/12/2010 10:31, Vinay Sajip wrote: >> [...] >> > Ideally, link to the wrong doc section on docs.python.org in your bug report. >> >> Now that's not a piece of advice you see very often :) >> > > True, but this area changed after 2.6 was released (after even 2.7, IIRC), so I > want to be sure that if I'm going to change the doc sources on release26-maint, > I'm doing the right thing. It was more a comment on the fact that, at first glance, that sentence looks like an instuction to provide an incorrect link, rather than to provide a link to the section of the docs that is incorrect. Just a quirk of English grammar :) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From zuo at chopin.edu.pl Mon Dec 13 12:44:49 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Mon, 13 Dec 2010 12:44:49 +0100 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation Message-ID: <20101213114449.GA2348@chopin.edu.pl> Dear Python Developers, It is s my first post to python-dev, so let me introduce myself briefly: Jan Kaliszewski, programmer and composer, sometimes also NGO activist. Coming to the matter... The discussion started with remark by Mark Dickinson about such a syntax oddity: > def f(a, b,): ... is fine, but > def f(*, a, b,): ... is a SyntaxError Then some other similar oddities were pointed at (*args/**kwargs-related ones as well as calls like f(*, a=3,) causing SyntaxError too). References: * http://mail.python.org/pipermail/python-dev/2010-July/101636.html * http://bugs.python.org/issue9232 * http://bugs.python.org/issue10682 But yesterday both mentioned issues has been closed as rejected -- with suggestion that it would probably require a PEP to modify Python in this aspect (as there is no clear consensus). So I'd opt for re-opening the discussion -- I suppose that more people could be interested in solving the issue (at least after the end of PEP 3003 moratorium period). I think that seeing that: def f(a, b): ... def f(a, b,): ... def f(a, *, b): ... def f(a, *args, b): ... x(1, 2, 3, 4, z=5) x(1, 2, 3, 4, z=5,) x(1, *(2,3,4), z=5) ...are ok, then -- def f(a, *, b,): ... def f(a, *args, b,): ... x(1, *(2,3,4), z=5,): ... ...should be ok as well, and consequently -- def f(a, *args,): ... def f(a, **kwargs,): ... x(1, *(2,3,4),) x(1, **dict(z=6),) ...should also be ok. Please also note that Py3k's function annotations make one-def-argument- -per-line formattig style the most suitable in some cases, e.g.: def my_func( spam:"Very tasty and nutritious piece of food", ham:"For experts only", *more_spam:"Not less tasty and not less nutritious!", spammish_inquisition:"Nobody expects this!", ) -> "Spam, spam, spam, spam, spam, spam, spam, spam, spam, spam": ... Regards, Jan Kaliszewski From ncoghlan at gmail.com Mon Dec 13 14:25:58 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Dec 2010 23:25:58 +1000 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: <20101213114449.GA2348@chopin.edu.pl> References: <20101213114449.GA2348@chopin.edu.pl> Message-ID: On Mon, Dec 13, 2010 at 9:44 PM, Jan Kaliszewski wrote: > I think that seeing that: > > ? ?def f(a, b): ... > ? ?def f(a, *, b): ... > ? ?def f(a, *args, b): ... > ? ?x(1, 2, 3, 4, z=5) > ? ?x(1, *(2,3,4), z=5) As per the closure of the affected tickets, the likely outcome of such a discussion would be the deprecation and subsequent removal of support for the following two options: def f(a, b,): ... x(1, 2, 3, 4, z=5,): ... Function arguments are not lists. Even when separated onto multiple lines, the closing "):" should remain on the final line with other content. That would be a lot of hassle to get rid of something that people probably aren't doing in the first place, though. Regards, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From mail at timgolden.me.uk Mon Dec 13 14:29:12 2010 From: mail at timgolden.me.uk (Tim Golden) Date: Mon, 13 Dec 2010 13:29:12 +0000 Subject: [Python-Dev] logging doc broken In-Reply-To: References: <4D028B7D.9030308@noir.com> <4D05F6DC.5040900@timgolden.me.uk> Message-ID: <4D061FA8.2000100@timgolden.me.uk> On 13/12/2010 11:21, Nick Coghlan wrote: > On Mon, Dec 13, 2010 at 9:04 PM, Vinay Sajip wrote: >> Tim Golden timgolden.me.uk> writes: >> >>> >>> On 13/12/2010 10:31, Vinay Sajip wrote: >>> [...] >>>> Ideally, link to the wrong doc section on docs.python.org in your bug report. >>> >>> Now that's not a piece of advice you see very often :) >>> >> >> True, but this area changed after 2.6 was released (after even 2.7, IIRC), so I >> want to be sure that if I'm going to change the doc sources on release26-maint, >> I'm doing the right thing. > > It was more a comment on the fact that, at first glance, that sentence > looks like an instuction to provide an incorrect link, rather than to > provide a link to the section of the docs that is incorrect. Just a > quirk of English grammar :) Thanks, Nick. That is what I meant. I wanted to indicate that it was tongue-in-cheek, but I never can remember the sequence of characters which means that... TJG From fuzzyman at voidspace.org.uk Mon Dec 13 14:39:44 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 13 Dec 2010 13:39:44 +0000 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: References: <20101213114449.GA2348@chopin.edu.pl> Message-ID: <4D062220.8040501@voidspace.org.uk> On 13/12/2010 13:25, Nick Coghlan wrote: > On Mon, Dec 13, 2010 at 9:44 PM, Jan Kaliszewski wrote: >> I think that seeing that: >> >> def f(a, b): ... >> def f(a, *, b): ... >> def f(a, *args, b): ... >> x(1, 2, 3, 4, z=5) >> x(1, *(2,3,4), z=5) > As per the closure of the affected tickets, the likely outcome of such > a discussion would be the deprecation and subsequent removal of > support for the following two options: > > def f(a, b,): ... > x(1, 2, 3, 4, z=5,): ... > > Function arguments are not lists. Even when separated onto multiple > lines, the closing "):" should remain on the final line with other > content. Why? For very long signatures I still mildly prefer this: def f(self, first, second, third, fourth, foo=None, bar=None, baz=None, spam=None, eggs=None, ham=None ): Over putting the closing paren: on the last line of the def. Of course not having such long signatures is even more preferable, but *sometimes* they are needed. All the best, Michael Foord > That would be a lot of hassle to get rid of something that people > probably aren't doing in the first place, though. > > Regards, > Nick. > -- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From tseaver at palladion.com Mon Dec 13 14:42:34 2010 From: tseaver at palladion.com (Tres Seaver) Date: Mon, 13 Dec 2010 08:42:34 -0500 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: References: <20101213114449.GA2348@chopin.edu.pl> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 12/13/2010 08:25 AM, Nick Coghlan wrote: > On Mon, Dec 13, 2010 at 9:44 PM, Jan Kaliszewski wrote: >> I think that seeing that: >> >> def f(a, b): ... >> def f(a, *, b): ... >> def f(a, *args, b): ... >> x(1, 2, 3, 4, z=5) >> x(1, *(2,3,4), z=5) > > As per the closure of the affected tickets, the likely outcome of such > a discussion would be the deprecation and subsequent removal of > support for the following two options: > > def f(a, b,): ... > x(1, 2, 3, 4, z=5,): ... > > Function arguments are not lists. Even when separated onto multiple > lines, the closing "):" should remain on the final line with other > content. > > That would be a lot of hassle to get rid of something that people > probably aren't doing in the first place, though. I actually make use of the feature when dealing with APIs which both a) take lots of arguments (more than fit comfortably on two lines at whatever indentation they are called), and b) have optional trailing arguments: I always leave the trailing comma in place in such cases, with the closing paren on the following line, so that adding or removing an argument at the end of the list stays consistent (the diffs are better, too, when I use this pattern). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk0GIsoACgkQ+gerLs4ltQ5HLwCfYFSyVrFtt04h6a39hyK6BD2c t8oAoJdXNS7wIsjF34ZiOQCwQGq9Qs2v =ZWqW -----END PGP SIGNATURE----- From ncoghlan at gmail.com Mon Dec 13 14:54:28 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Dec 2010 23:54:28 +1000 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: References: <20101213114449.GA2348@chopin.edu.pl> Message-ID: On Mon, Dec 13, 2010 at 11:42 PM, Tres Seaver wrote: > I actually make use of the feature when dealing with APIs which both a) > take lots of arguments (more than fit comfortably on two lines at > whatever indentation they are called), and b) have optional trailing > arguments: ?I always leave the trailing comma in place in such cases, > with the closing paren on the following line, so that adding or removing > an argument at the end of the list stays consistent (the diffs are > better, too, when I use this pattern). My personal preferences aren't strong either way, but the issues were closed because committers voiced opinions against making this consistent in the other direction (i.e. always allowing the trailing comma). I don't know that a full PEP is really needed, but the status quo is that some committers said no and others don't really care about the issue all that much, so the current behaviour is going to remain in place unless those in the first group change their mind (or Guido weighs in and says "change it"). Creating a PEP is one way to carry out such persuasion (probably overkill though). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From zuo at chopin.edu.pl Mon Dec 13 15:18:34 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Mon, 13 Dec 2010 15:18:34 +0100 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: References: <20101213114449.GA2348@chopin.edu.pl> Message-ID: <20101213141834.GA5056@chopin.edu.pl> Nick Coghlan dixit (2010-12-13, 23:25): > Function arguments are not lists. Even when separated onto multiple > lines, the closing "):" should remain on the final line with other > content. Not necessarily, IMHO. 1. What about my example with '-> xxx' return-value annotation? (especially when that annotation is a long expression) 2. There are two argument-list-formatting idioms I apply -- depending on which is more suitable in a particular case: a) when argument specs/expressions are not very long and rather if their number is not very big: def function(argument_spec1, argument_spec2, argument_spec3, argument_spec4, argument_spec5, argument_spec6): function_call(expression1, expression2, expression3, expression4, expression5, expression6) b) for long argument lists and/or argument specs/expressions (e.g. when default values or argument annotations are defined as long expressions): def function( long_argument_spec1, long_argument_spec2, long_argument_spec3, long_argument_spec4, long_argument_spec5, long_argument_spec6, ): function_call( long_expression1, long_expression2, long_expression3, long_expression4, long_expression5, long_expression6, ) Note that option 'b' is more convenient for refactorization, diffs etc. Regards, *j From vinay_sajip at yahoo.co.uk Mon Dec 13 16:01:09 2010 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Mon, 13 Dec 2010 15:01:09 +0000 (UTC) Subject: [Python-Dev] logging doc broken References: <4D028B7D.9030308@noir.com> Message-ID: Nick Coghlan gmail.com> writes: > Yep, that's what I had assumed you were going to do (this did come up > in the big thread about the logging docs, but you may have missed it). I hadn't missed it - I'm just spelling out in more detail what I'm going to do. > That said, you can't get a one-to-one correspondence in this case > anyway, since some of the parameters relate to multiple attributes > (e.g. "lvl" becomes both "levelNo" and "levelName"). > > It is rare that anyone will be constructing a LogRecord manually > though, so I don't think the parameter names matter all that much in > practice. It is far more common that people will be creating them > implicitly through the Logger event notification methods or by > deserialising them. I agree, it's a pretty unlikely scenario. Of course, I do make use of the parameter names in the dictConfig approach, so I can't really complain. I should hopefully remember this when I'm writing out a method or function signature for a public API :-) Regards, Vinay From vinay_sajip at yahoo.co.uk Mon Dec 13 16:04:15 2010 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Mon, 13 Dec 2010 15:04:15 +0000 (UTC) Subject: [Python-Dev] logging doc broken References: <4D028B7D.9030308@noir.com> <4D05F6DC.5040900@timgolden.me.uk> <4D061FA8.2000100@timgolden.me.uk> Message-ID: Tim Golden timgolden.me.uk> writes: > > It was more a comment on the fact that, at first glance, that sentence > > looks like an instuction to provide an incorrect link, rather than to > > provide a link to the section of the docs that is incorrect. Just a > > quirk of English grammar :) > > Thanks, Nick. That is what I meant. I wanted to indicate that > it was tongue-in-cheek, but I never can remember the sequence > of characters which means that... > Aaargh! You're right of course, I was too hasty in typing "link to the wrong doc section" rather than the only slightly longer "link to the doc section which is wrong". Tim - when you find those emoticons, be sure to let me know the one for "facepalm" ;-) Regards, Vinay From rdmurray at bitdance.com Mon Dec 13 16:51:19 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 13 Dec 2010 10:51:19 -0500 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: References: <20101213114449.GA2348@chopin.edu.pl> Message-ID: <20101213155119.A52A621CD91@kimball.webabinitio.net> On Mon, 13 Dec 2010 23:25:58 +1000, Nick Coghlan wrote: > On Mon, Dec 13, 2010 at 9:44 PM, Jan Kaliszewski wrote: > > I think that seeing that: > > > > =A0 =A0def f(a, b): ... > > =A0 =A0def f(a, *, b): ... > > =A0 =A0def f(a, *args, b): ... > > =A0 =A0x(1, 2, 3, 4, z=3D5) > > =A0 =A0x(1, *(2,3,4), z=3D5) > > As per the closure of the affected tickets, the likely outcome of such > a discussion would be the deprecation and subsequent removal of > support for the following two options: > > def f(a, b,): ... > x(1, 2, 3, 4, z=3D5,): ... > > Function arguments are not lists. Even when separated onto multiple > lines, the closing "):" should remain on the final line with other > content. > > That would be a lot of hassle to get rid of something that people > probably aren't doing in the first place, though. Counter examples from google code search: http://www.google.com/codesearch/p?hl=en#copo3dCwf5E/django/utils/simplejson/encoder.py&q=^\s*\):&sa=N&cd=5&ct=rc (also appears in json in the stdlib) http://www.google.com/codesearch/p?hl=en#algXCqBNNP0/vendor/python-clientform/ClientForm.py&q=^\ *\):&sa=N&cd=3&ct=rc (class def) http://www.google.com/codesearch/p?hl=en#KT-ZlRkUunU/trunk/code/output/ExprParser.py&q=def\(.*,\s\):&sa=N&cd=2&ct=rc http://www.google.com/codesearch/p?hl=en#XnG7n8Mjf2s/GoogleSearch.py&q=def\(.*,\s\):&sa=N&cd=3&ct=rc http://www.google.com/codesearch/p?hl=en#MokQ50OeeyU/src/python/ndogen/parser/matlab/parser.py&q=def\(.*,\s\):&sa=N&cd=5&ct=rc Not many, granted, but not zero, either, and I'm sure there are lots more out there[*]. I do especially like the fact that there is one in the stdlib :) It seems like the status quo is fine. I wouldn't object to it being made more consistent. I would object to removing the existing cases. -- R. David Murray www.bitdance.com [*] code search's response to various regexes was somewhat surprising; expressions I thought should have been supersets resulted in fewer hits. Nor could I think of a way to search for function invocations ending with a comma. Then again, I usually make lots of mistakes with regexes. From guido at python.org Mon Dec 13 17:54:16 2010 From: guido at python.org (Guido van Rossum) Date: Mon, 13 Dec 2010 08:54:16 -0800 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: References: <20101213114449.GA2348@chopin.edu.pl> Message-ID: On Mon, Dec 13, 2010 at 5:42 AM, Tres Seaver wrote: > I actually make use of the feature when dealing with APIs which both a) > take lots of arguments (more than fit comfortably on two lines at > whatever indentation they are called), and b) have optional trailing > arguments: ?I always leave the trailing comma in place in such cases, > with the closing paren on the following line, so that adding or removing > an argument at the end of the list stays consistent (the diffs are > better, too, when I use this pattern). Same here, and it's a soft style rule at Google that trailing commas are good -- they can help produce shorter diffs. I'm at least +0 on allowing trailing commas in the situation the OP mentioned. -- --Guido van Rossum (python.org/~guido) From vinay_sajip at yahoo.co.uk Mon Dec 13 19:55:12 2010 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Mon, 13 Dec 2010 18:55:12 +0000 (UTC) Subject: [Python-Dev] logging doc broken References: <4D028B7D.9030308@noir.com> Message-ID: Nick Coghlan gmail.com> writes: > Yep, that's what I had assumed you were going to do (this did come up > in the big thread about the logging docs, but you may have missed it). > Ok, I've now checked in this change, and would be grateful for any feedback. Time for a break :-) Thanks a lot for all your patience and valuable feedback, Nick. It's been very helpful, and I hope it will make the logging docs more accessible to those who've been put off in the past. I've emailed Georg about how best to reorganise into: intro + basic tutorial (at current location) advanced tutorial reference cookbook and I'm waiting for his suggestions on how best to implement (in terms of breaking up into different files etc). Thanks and regards, Vinay From alexander.belopolsky at gmail.com Mon Dec 13 20:09:02 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 13 Dec 2010 14:09:02 -0500 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: References: <20101213114449.GA2348@chopin.edu.pl> Message-ID: On Mon, Dec 13, 2010 at 11:54 AM, Guido van Rossum wrote: > I'm at least +0 on > allowing trailing commas in the situation the OP mentioned. > FWIW, I am also about +0.5 on allowing trailing comma. Note that in a similar situation, the C standardization committee has erred on the side of consistency: """ A new feature of C99: a common extension in many implementations allows a trailing comma after the list of enumeration constants. The Committee decided to adopt this feature as an innocuous extension that mirrors the trailing commas allowed in initializers. """ http://www.open-std.org/jtc1/sc22/wg14/www/C99RationaleV5.10.pdf Similarly, I find allowing trailing comma in keyword only arguments lists to be an innocuous extension that mirrors the trailing commas allowed in the positional arguments lists. A possible benefit that I have not seen mentioned is that if developer decides to convert some of the trailing arguments in her function to keyword only, she does not have to worry about removing the trailing comma. From solipsis at pitrou.net Mon Dec 13 20:17:59 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 13 Dec 2010 20:17:59 +0100 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation References: <20101213114449.GA2348@chopin.edu.pl> Message-ID: <20101213201759.422b15de@pitrou.net> On Mon, 13 Dec 2010 14:09:02 -0500 Alexander Belopolsky wrote: > On Mon, Dec 13, 2010 at 11:54 AM, Guido van Rossum wrote: > > I'm at least +0 on > > allowing trailing commas in the situation the OP mentioned. > > > > FWIW, I am also about +0.5 on allowing trailing comma. Note that in a > similar situation, the C standardization committee has erred on the > side of consistency: > > """ > A new feature of C99: a common extension in many implementations > allows a trailing comma after the list of enumeration constants. The > Committee decided to adopt this feature as an innocuous extension that > mirrors the trailing commas allowed in initializers. > """ http://www.open-std.org/jtc1/sc22/wg14/www/C99RationaleV5.10.pdf > > Similarly, I find allowing trailing comma in keyword only arguments > lists to be an innocuous extension that mirrors the trailing commas > allowed in the positional arguments lists. +1 from me as well. Special cases are hard to remember. Regards Antoine. From v+python at g.nevcal.com Mon Dec 13 20:26:44 2010 From: v+python at g.nevcal.com (Glenn Linderman) Date: Mon, 13 Dec 2010 11:26:44 -0800 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: <20101213201759.422b15de@pitrou.net> References: <20101213114449.GA2348@chopin.edu.pl> <20101213201759.422b15de@pitrou.net> Message-ID: <4D067374.8050200@g.nevcal.com> On 12/13/2010 11:17 AM, Antoine Pitrou wrote: > On Mon, 13 Dec 2010 14:09:02 -0500 > Alexander Belopolsky wrote: > >> On Mon, Dec 13, 2010 at 11:54 AM, Guido van Rossum wrote: >>> I'm at least +0 on >>> allowing trailing commas in the situation the OP mentioned. >>> >> FWIW, I am also about +0.5 on allowing trailing comma. Note that in a >> similar situation, the C standardization committee has erred on the >> side of consistency: >> >> """ >> A new feature of C99: a common extension in many implementations >> allows a trailing comma after the list of enumeration constants. The >> Committee decided to adopt this feature as an innocuous extension that >> mirrors the trailing commas allowed in initializers. >> """ http://www.open-std.org/jtc1/sc22/wg14/www/C99RationaleV5.10.pdf >> >> Similarly, I find allowing trailing comma in keyword only arguments >> lists to be an innocuous extension that mirrors the trailing commas >> allowed in the positional arguments lists. > +1 from me as well. Special cases are hard to remember. +1. I tend to put them in, and then take out the ones that Python won't accept. Then, when I add another item to the list, Python tells me to go back and put the one I took out back in again, and take out the one I put in at the new end of the list. Annoying. (for vertically arranged lists, one per line, primarily, with ) on the last line by itself. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Mon Dec 13 20:34:14 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 13 Dec 2010 14:34:14 -0500 Subject: [Python-Dev] logging doc broken In-Reply-To: References: <4D028B7D.9030308@noir.com> <4D05F6DC.5040900@timgolden.me.uk> Message-ID: On 12/13/2010 6:04 AM, Vinay Sajip wrote: > True, but this area changed after 2.6 was released (after even 2.7, IIRC), so I > want to be sure that if I'm going to change the doc sources on release26-maint, 2.6 is in security-fix only mode. I am not sure if that precludes doc fixes, on the chance that there will ever be a security fix, as it does non-security bug fixes, but is it hardly worth the bother compared to improving active releases. -- Terry Jan Reedy From dickinsm at gmail.com Mon Dec 13 20:39:32 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 13 Dec 2010 19:39:32 +0000 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: <20101213155119.A52A621CD91@kimball.webabinitio.net> References: <20101213114449.GA2348@chopin.edu.pl> <20101213155119.A52A621CD91@kimball.webabinitio.net> Message-ID: On Mon, Dec 13, 2010 at 3:51 PM, R. David Murray wrote: > It seems like the status quo is fine. ?I wouldn't object to it being > made more consistent. ?I would object to removing the existing cases. Same here, on all three counts. In one of the projects I'm currently working on, we've settled on a style that does quite a lot of: my_thing = Thing( foo = Foo(arg1, arg2, ...), bar = Bar(arg3, arg4, ...), ... ) and I've found the trailing comma very convenient during refactoring and API experimentation. (There's still good fun to be had arguing about the indentation of that closing parenthesis, though.) Mar From v+python at g.nevcal.com Mon Dec 13 21:08:31 2010 From: v+python at g.nevcal.com (Glenn Linderman) Date: Mon, 13 Dec 2010 12:08:31 -0800 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: References: <20101213114449.GA2348@chopin.edu.pl> <20101213155119.A52A621CD91@kimball.webabinitio.net> Message-ID: <4D067D3F.8030206@g.nevcal.com> On 12/13/2010 11:39 AM, Mark Dickinson wrote: > my_thing = Thing( > foo = Foo(arg1, arg2, ...), > bar = Bar(arg3, arg4, ...), > ... > ) > > and I've found the trailing comma very convenient during refactoring > and API experimentation. (There's still good fun to be had arguing > about the indentation of that closing parenthesis, though.) Clearly it needs to be indented one level, because it is a continuation of the prior line, just like the foo and bar and ... lines are continuations and therefore indented. I'd have argued differently for languages that use {} to delimit blocks. Enjoy! From cs at zip.com.au Mon Dec 13 22:06:50 2010 From: cs at zip.com.au (Cameron Simpson) Date: Tue, 14 Dec 2010 08:06:50 +1100 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: <20101213201759.422b15de@pitrou.net> References: <20101213201759.422b15de@pitrou.net> Message-ID: <20101213210650.GA31778@cskk.homeip.net> On 13Dec2010 20:17, Antoine Pitrou wrote: | On Mon, 13 Dec 2010 14:09:02 -0500 | Alexander Belopolsky wrote: | | > On Mon, Dec 13, 2010 at 11:54 AM, Guido van Rossum wrote: | > > I'm at least +0 on | > > allowing trailing commas in the situation the OP mentioned. | > > | > | > FWIW, I am also about +0.5 on allowing trailing comma. Note that in a | > similar situation, the C standardization committee has erred on the | > side of consistency: | > | > """ | > A new feature of C99: a common extension in many implementations | > allows a trailing comma after the list of enumeration constants. The | > Committee decided to adopt this feature as an innocuous extension that | > mirrors the trailing commas allowed in initializers. | > """ http://www.open-std.org/jtc1/sc22/wg14/www/C99RationaleV5.10.pdf | > | > Similarly, I find allowing trailing comma in keyword only arguments | > lists to be an innocuous extension that mirrors the trailing commas | > allowed in the positional arguments lists. | | +1 from me as well. Special cases are hard to remember. +1 again. Both the special cases and probably an example of diff friendliness: x = f(a, b=3, ##c=4, hacking at dev time d=5) Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ A Master is someone who started before you did. - Gary Zukav From tjreedy at udel.edu Mon Dec 13 22:21:20 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 13 Dec 2010 16:21:20 -0500 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: <20101213201759.422b15de@pitrou.net> References: <20101213114449.GA2348@chopin.edu.pl> <20101213201759.422b15de@pitrou.net> Message-ID: On 12/13/2010 2:17 PM, Antoine Pitrou wrote: > On Mon, 13 Dec 2010 14:09:02 -0500 > Alexander Belopolsky wrote: > >> On Mon, Dec 13, 2010 at 11:54 AM, Guido van Rossum wrote: >>> I'm at least +0 on >>> allowing trailing commas in the situation the OP mentioned. >>> >> >> FWIW, I am also about +0.5 on allowing trailing comma. Note that in a >> similar situation, the C standardization committee has erred on the >> side of consistency: >> >> """ >> A new feature of C99: a common extension in many implementations >> allows a trailing comma after the list of enumeration constants. The >> Committee decided to adopt this feature as an innocuous extension that >> mirrors the trailing commas allowed in initializers. >> """ http://www.open-std.org/jtc1/sc22/wg14/www/C99RationaleV5.10.pdf >> >> Similarly, I find allowing trailing comma in keyword only arguments >> lists to be an innocuous extension that mirrors the trailing commas >> allowed in the positional arguments lists. > > +1 from me as well. Special cases are hard to remember. Same here. A strong +1 for a consistent rule (always or never allowed) with a +1 for always given others use case of one param/arg per line. So I think the issues should be reopened. -- Terry Jan Reedy From raymond.hettinger at gmail.com Mon Dec 13 22:55:17 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Mon, 13 Dec 2010 13:55:17 -0800 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: References: <20101213114449.GA2348@chopin.edu.pl> <20101213201759.422b15de@pitrou.net> Message-ID: <0DE0FA27-5792-4017-A0B0-4B1AF109FA76@gmail.com> On Dec 13, 2010, at 1:21 PM, Terry Reedy wrote: > Same here. A strong +1 for a consistent rule (always or never allowed) with a +1 for always given others use case of one param/arg per line. It seems to me that a trailing comma in an argument list is more likely to be a user error than a deliberate comma-for-the-future. Raymond From guido at python.org Mon Dec 13 23:16:58 2010 From: guido at python.org (Guido van Rossum) Date: Mon, 13 Dec 2010 14:16:58 -0800 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: <0DE0FA27-5792-4017-A0B0-4B1AF109FA76@gmail.com> References: <20101213114449.GA2348@chopin.edu.pl> <20101213201759.422b15de@pitrou.net> <0DE0FA27-5792-4017-A0B0-4B1AF109FA76@gmail.com> Message-ID: On Mon, Dec 13, 2010 at 1:55 PM, Raymond Hettinger wrote: > > On Dec 13, 2010, at 1:21 PM, Terry Reedy wrote: > >> Same here. A strong +1 for a consistent rule (always or never allowed) with a +1 for always given others use case of one param/arg per line. > > > > It seems to me that a trailing comma in an argument list is more likely to be a user error than a deliberate comma-for-the-future. Really? Have you observed this? Even if it was inserted by mistake, it is harmless. Python has a long tradition of allowing redundant trailing commas in comma-separated lists, and it is habit-forming. That's the issue the OP had: he expected it to work in the one context where it doesn't. -- --Guido van Rossum (python.org/~guido) From lukasz at langa.pl Mon Dec 13 23:22:29 2010 From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=) Date: Mon, 13 Dec 2010 23:22:29 +0100 Subject: [Python-Dev] configparser 1.1 - one last bug fix needed Message-ID: <4A6BBCF7-6491-48C3-A02A-84EEA299B1DB@langa.pl> Hi there. There's one last thing that needs to be done with configparser for 3.2. Raymond, Fred, Michael and Georg already expressed their approval on that so unless anybody finds a flaw in the idea expressed below, I'm going to make the change for 3.2b2: - the ConfigParser class will be removed - the SafeConfigParser class will be renamed to ConfigParser - 2to3 will rename SafeConfigParser classes to ConfigParser - 2to3 will warn on the subtle behaviour change when ConfigParser classes are found What's the difference? ---------------------- Both ConfigParser and SafeConfigParser implement interpolation, e.g. option values can contain special tokens similar to those implemented by Python's string formatting: %(example)s. These tokens are replaced during get() operations by values from respective keys (either from the current section or from the DEFAULT section). SafeConfigParser was originally introduced to fix a couple of ConfigParser problems: - when a token didn't match the %(name)s syntax, it was simply treated as a raw value. This caused configuration errors like %var or %(no_closing_s) to be missed. - if someone actually wanted to store arbitrary strings in values, including Python formatting strings, there was no way to escape %(name)s in the configuration. The programmer had to know in advance that some value may hold %(name)s and only get() values from that option using get('section', 'option', raw=True) Then however, that option could not use interpolation anymore. - set() originally allowed to store non-string values in the parser. This was not meant to be a feature and caused trouble when the user tried to save the configuration to a file or get the stored values back using typed get() methods. SafeConfigParser solves these problems by validating interpolation syntax (only %(name)s or %% are allowed, the latter being an escaped percent sign) and raising exceptions on syntax errors, and validating type on set() operations so that no non-string values can be passed to the parser using the API. Why change that? ---------------- When ConfigParser was left alone, it remained the default choice for most end users, including our own distutils and logging libs. This was a very weak choice, and most current ConfigParser users are not aware of the interpolation quirks. I had to close a couple of issues related to people trying to store non-string values internally in the parser. The current situation needlessly complicates the documentation. Explaining all the above quirks to each new user who only wants to parse an INI file is weak at best. Moreover, users trust Python to do the right thing by default and according to their intuition. In this case, going for the default configparser.ConfigParser class without consulting the documentation is clearly a suboptimal choice. One last argument is that SafeConfigParser is an awkward name. It implicates the other parsers are somehow unsafe, or that this specific parser protects users from something. This is generally considered a naming antipattern. When? ----- You might ask whether this can be done for 3.2 (e.g. is that a feature or a bugfix). In Raymond's words, the beta process should be used to flesh out the APIs, test whether they work as expected and fix suboptimal decisions before we hit the release candidate stage. He consideres this essentially a bugfix and I agree. You might ask why do that now and not for 3.3. We believe that 3.2 is the last possible moment of introducing a change like that. The adoption rate is currently still low and application authors porting projects from 2.x expect incompatibilities. When they are non-issues, handled by 2to3, there's nothing to be afraid of. But isn't that... INCOMPATIBLE?! -------------------------------- Yes, it is. Thanks to the low py3k adoption rate now's the only moment where there's marginal risk of introducing silent incompatibility (there are hardly any py3k projects out there). Projects ported from Python 2.x will be informed by 2to3 of the change. We believe that this will fix more bugs than it introduces. Support for bare % signs would be the single case where ConfigParser might have appeared a more natural solution. In those cases we expect that users will rather choose to turn off interpolation whatsoever. Summary ------- If you have any strong, justified arguments against this bugfix, speak up. Otherwise the change will be made on Thursday. -- Interpolating regards, ?ukasz Langa tel. +48 791 080 144 WWW http://lukasz.langa.pl/ From chris at simplistix.co.uk Tue Dec 14 00:41:28 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Mon, 13 Dec 2010 23:41:28 +0000 Subject: [Python-Dev] Using logging in the stdlib and its unit tests In-Reply-To: References: Message-ID: <4D06AF28.3040601@simplistix.co.uk> On 07/12/2010 20:26, Vinay Sajip wrote: > I would suggest that when unit testing, rather than adding StreamHandlers to log > to stderr, that something like TestHandler and Matcher from this post: > > http://plumberjack.blogspot.com/2010/09/unit-testing-and-logging.html For Python 2, my testfixtures package has had some helpful bits in this area for a while now: http://packages.python.org/testfixtures/logging.html I find it important to be able to check my code is logging what I think it should be logging! cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From v+python at g.nevcal.com Tue Dec 14 00:46:46 2010 From: v+python at g.nevcal.com (Glenn Linderman) Date: Mon, 13 Dec 2010 15:46:46 -0800 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: <0DE0FA27-5792-4017-A0B0-4B1AF109FA76@gmail.com> References: <20101213114449.GA2348@chopin.edu.pl> <20101213201759.422b15de@pitrou.net> <0DE0FA27-5792-4017-A0B0-4B1AF109FA76@gmail.com> Message-ID: <4D06B066.5090603@g.nevcal.com> On 12/13/2010 1:55 PM, Raymond Hettinger wrote: > It seems to me that a trailing comma in an argument list is more likely to be a user error than a deliberate comma-for-the-future. It seems to me that a trailing comma, especially in the case of one parameter per line, is a deliberate comma-for-the-future. It's a good reminder that you are dealing with a list of some sort, rather than a statement, when you look at just one parameter on the line. Especially if the ) is on the next line, which I prefer. Yes, a parameter list is not a python list, nor a python tuple, but it is still a generic, comma-separated list, and all such are more conveniently dealt with, in the multi-line case, if trailing commas are permitted. And, of course, the one-entry tuple needs the comma to differentiate it from some other expression, forcing the trailing comma into the syntax... so there can be no consistent rule for all commas that doesn't permit trailing commas. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Dec 14 01:08:32 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 14 Dec 2010 10:08:32 +1000 Subject: [Python-Dev] logging doc broken In-Reply-To: References: <4D028B7D.9030308@noir.com> <4D05F6DC.5040900@timgolden.me.uk> Message-ID: On Tue, Dec 14, 2010 at 5:34 AM, Terry Reedy wrote: > On 12/13/2010 6:04 AM, Vinay Sajip wrote: > > True, but this area changed after 2.6 was released (after even 2.7, IIRC), >> so I >> want to be sure that if I'm going to change the doc sources on >> release26-maint, >> > > 2.6 is in security-fix only mode. I am not sure if that precludes doc > fixes, on the chance that there will ever be a security fix, as it does > non-security bug fixes, but is it hardly worth the bother compared to > improving active releases. > Fixing it in the 2.7 maintenance branch will be sufficient to modify docs.python.org, which is likely the most important place to update. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Dec 14 01:16:07 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 14 Dec 2010 10:16:07 +1000 Subject: [Python-Dev] logging doc broken In-Reply-To: References: <4D028B7D.9030308@noir.com> Message-ID: On Tue, Dec 14, 2010 at 4:55 AM, Vinay Sajip wrote: > Nick Coghlan gmail.com> writes: > > > Yep, that's what I had assumed you were going to do (this did come up > > in the big thread about the logging docs, but you may have missed it). > > > > Ok, I've now checked in this change, and would be grateful for any > feedback. > Time for a break :-) > Hmm, that may not have built correctly, since it isn't showing up in the web version of the dev docs yet. I skimmed the diff on python-checkins though, and it looked good to me. I think the revised docs should make it *much* easier for people to get a handle on how the logging system works (I know my own understanding of it is significantly better than it was a couple of weeks ago). Cheers, Nick. P.S. I'm off visiting family for a couple of weeks, so my email access is going to be sketchy for a while. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Dec 14 01:20:04 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 14 Dec 2010 10:20:04 +1000 Subject: [Python-Dev] configparser 1.1 - one last bug fix needed In-Reply-To: <4A6BBCF7-6491-48C3-A02A-84EEA299B1DB@langa.pl> References: <4A6BBCF7-6491-48C3-A02A-84EEA299B1DB@langa.pl> Message-ID: +1 from me. If anyone complains too much, perhaps we can offer to retain the old ConfigParser as "_BuggyConfigParser"? (that idea is only partially tongue-in-cheek...) Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukasz at langa.pl Tue Dec 14 01:26:44 2010 From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=) Date: Tue, 14 Dec 2010 01:26:44 +0100 Subject: [Python-Dev] configparser 1.1 - one last bug fix needed In-Reply-To: References: <4A6BBCF7-6491-48C3-A02A-84EEA299B1DB@langa.pl> Message-ID: <16704819-23F0-4C2D-8A42-3C3D06FE99EF@langa.pl> Wiadomo?? napisana przez Nick Coghlan w dniu 2010-12-14, o godz. 01:20: > +1 from me. > > If anyone complains too much, perhaps we can offer to retain the old ConfigParser as "_BuggyConfigParser"? (that idea is only partially tongue-in-cheek...) > We may leave the LegacyInterpolation class if someone *really* needs to support that. That way RawConfigParser(interpolation=LegacyInterpolation()) is effectively the old ConfigParser. -- Nondeprecatorily yours, ?ukasz Langa tel. +48 791 080 144 WWW http://lukasz.langa.pl/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Dec 14 01:31:01 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 14 Dec 2010 10:31:01 +1000 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: References: <20101213114449.GA2348@chopin.edu.pl> <20101213155119.A52A621CD91@kimball.webabinitio.net> Message-ID: On Tue, Dec 14, 2010 at 5:39 AM, Mark Dickinson wrote: > On Mon, Dec 13, 2010 at 3:51 PM, R. David Murray > wrote: > > It seems like the status quo is fine. I wouldn't object to it being > > made more consistent. I would object to removing the existing cases. > > Same here, on all three counts. In one of the projects I'm currently > working on, we've settled on a style that does quite a lot of: > > my_thing = Thing( > foo = Foo(arg1, arg2, ...), > bar = Bar(arg3, arg4, ...), > ... > ) > > and I've found the trailing comma very convenient during refactoring > and API experimentation. (There's still good fun to be had arguing > about the indentation of that closing parenthesis, though.) > Another valid use case that occurred to me is building up a string-keyed dictionary: mapping = dict( x=1, y=2, z=3, ) So, on reflection, removing the existing cases where it is supported is certainly unreasonable, which makes the consistency argument that much stronger. For the record, I reopened issue #9232 (noting the lack of consensus), and (as someone suggested on the tracker) changed the resolution on the other one to be as a duplicate of #9232. Cheers, Nick. P.S. As I noted in the logging discussion, my email access is going to be a bit sketchy for the next couple of weeks. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Tue Dec 14 01:34:40 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 14 Dec 2010 01:34:40 +0100 Subject: [Python-Dev] configparser 1.1 - one last bug fix needed References: <4A6BBCF7-6491-48C3-A02A-84EEA299B1DB@langa.pl> Message-ID: <20101214013440.290f2a37@pitrou.net> On Tue, 14 Dec 2010 10:20:04 +1000 Nick Coghlan wrote: > +1 from me. > > If anyone complains too much, perhaps we can offer to retain the old > ConfigParser as "_BuggyConfigParser"? (that idea is only partially > tongue-in-cheek...) Or we can put it in the "buggy" module which receives all buggy code. Regards Antoine. From vinay_sajip at yahoo.co.uk Tue Dec 14 02:22:43 2010 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Tue, 14 Dec 2010 01:22:43 +0000 (UTC) Subject: [Python-Dev] logging doc broken References: <4D028B7D.9030308@noir.com> Message-ID: Nick Coghlan gmail.com> writes: > Hmm, that may not have built correctly, since it isn't showing up in the web > version of the dev docs yet. I skimmed the diff on python-checkins though, and > it looked good to me. I'll keep an eye on the build (built OK on my machine but I'm not sure what the build trigger is for docs.python.org). > I think the revised docs should make it *much* easier for people to get a > handle on how the logging system works (I know my own understanding of it is > significantly better than it was a couple of weeks ago). Glad to hear that. > P.S. I'm off visiting family for a couple of weeks, so my email access is > going to be sketchy for a while. Have a nice time, and thanks for all the help. Regards, Vinay From raymond.hettinger at gmail.com Tue Dec 14 07:21:33 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Mon, 13 Dec 2010 22:21:33 -0800 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: References: <20101213114449.GA2348@chopin.edu.pl> <20101213201759.422b15de@pitrou.net> <0DE0FA27-5792-4017-A0B0-4B1AF109FA76@gmail.com> Message-ID: <6406C2A8-3042-4D88-992D-7389C4C413FC@gmail.com> On Dec 13, 2010, at 2:16 PM, Guido van Rossum wrote: > On Mon, Dec 13, 2010 at 1:55 PM, Raymond Hettinger > wrote: >> >> It seems to me that a trailing comma in an argument list is more likely to be a user error than a deliberate comma-for-the-future. > > Really? Have you observed this? Even if it was inserted by mistake, it > is harmless. I only have one data point, my own mistakes. The SyntaxError has occasionally been helpful to me when working out a function signature or to detect a copy and paste error. In both cases, it meant that there was supposed to be another argument and it had been either forgotten or mispasted. Also, if I were reviewing someone else's code and saw a trailing comma in a function definition, it would seem weird and I would wonder if the author intended a different signature. FWIW, this isn't important to me at all. Was just noting my own experience. Don't put assign much weight to it, I don't have much of a preference either way. > Python has a long tradition of allowing redundant > trailing commas in comma-separated lists, and it is habit-forming. Right. I see that in the wild quite often and use it myself. Raymond From steve at pearwood.info Tue Dec 14 12:24:07 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 14 Dec 2010 22:24:07 +1100 Subject: [Python-Dev] configparser 1.1 - one last bug fix needed In-Reply-To: <20101214013440.290f2a37@pitrou.net> References: <4A6BBCF7-6491-48C3-A02A-84EEA299B1DB@langa.pl> <20101214013440.290f2a37@pitrou.net> Message-ID: <4D0753D7.9060102@pearwood.info> Antoine Pitrou wrote: > On Tue, 14 Dec 2010 10:20:04 +1000 > Nick Coghlan wrote: > >> +1 from me. >> >> If anyone complains too much, perhaps we can offer to retain the old >> ConfigParser as "_BuggyConfigParser"? (that idea is only partially >> tongue-in-cheek...) > > Or we can put it in the "buggy" module which receives all buggy code. The good thing about that idea is that maintenance of buggy.py will be so simple! -- Steven From fdrake at acm.org Tue Dec 14 12:38:51 2010 From: fdrake at acm.org (Fred Drake) Date: Tue, 14 Dec 2010 06:38:51 -0500 Subject: [Python-Dev] configparser 1.1 - one last bug fix needed In-Reply-To: <4D0753D7.9060102@pearwood.info> References: <4A6BBCF7-6491-48C3-A02A-84EEA299B1DB@langa.pl> <20101214013440.290f2a37@pitrou.net> <4D0753D7.9060102@pearwood.info> Message-ID: On Tue, Dec 14, 2010 at 6:24 AM, Steven D'Aprano wrote: > The good thing about that idea is that maintenance of buggy.py will be so > simple! It's self-describing, and needs no further documentation. :-) ? -Fred -- Fred L. Drake, Jr.? ? "A storm broke loose in my mind."? --Albert Einstein From raymond.hettinger at gmail.com Tue Dec 14 18:01:28 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Tue, 14 Dec 2010 09:01:28 -0800 Subject: [Python-Dev] configparser 1.1 - one last bug fix needed In-Reply-To: References: <4A6BBCF7-6491-48C3-A02A-84EEA299B1DB@langa.pl> <20101214013440.290f2a37@pitrou.net> <4D0753D7.9060102@pearwood.info> Message-ID: <7549C51E-0E6F-456F-9469-BBBECB5E8867@gmail.com> On Dec 14, 2010, at 3:38 AM, Fred Drake wrote: > On Tue, Dec 14, 2010 at 6:24 AM, Steven D'Aprano wrote: >> The good thing about that idea is that maintenance of buggy.py will be so >> simple! > > It's self-describing, and needs no further documentation. :-) > And psychologically more effective than deprecation. Anyone writing "import buggy" probably feels like they have got some explaining to do :-) Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: From mail at timgolden.me.uk Tue Dec 14 18:06:20 2010 From: mail at timgolden.me.uk (Tim Golden) Date: Tue, 14 Dec 2010 17:06:20 +0000 Subject: [Python-Dev] configparser 1.1 - one last bug fix needed In-Reply-To: <7549C51E-0E6F-456F-9469-BBBECB5E8867@gmail.com> References: <4A6BBCF7-6491-48C3-A02A-84EEA299B1DB@langa.pl> <20101214013440.290f2a37@pitrou.net> <4D0753D7.9060102@pearwood.info> <7549C51E-0E6F-456F-9469-BBBECB5E8867@gmail.com> Message-ID: <4D07A40C.90507@timgolden.me.uk> On 14/12/2010 17:01, Raymond Hettinger wrote: > > On Dec 14, 2010, at 3:38 AM, Fred Drake wrote: > >> On Tue, Dec 14, 2010 at 6:24 AM, Steven D'Aprano wrote: >>> The good thing about that idea is that maintenance of buggy.py will be so >>> simple! >> >> It's self-describing, and needs no further documentation. :-) >> > > And psychologically more effective than deprecation. > Anyone writing "import buggy" probably feels like they > have got some explaining to do :-) They might be intro retro-robotics: http://www.anf.nildram.co.uk/beebcontrol/buggies/bbc_buggy/index.html (For those not in the UK, this was a very popular schools project kit on the back of the famous BBC micro initiative) TJG From ethan at stoneleaf.us Tue Dec 14 18:30:51 2010 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 14 Dec 2010 09:30:51 -0800 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: References: <20101213114449.GA2348@chopin.edu.pl> <20101213201759.422b15de@pitrou.net> Message-ID: <4D07A9CB.4060409@stoneleaf.us> Terry Reedy wrote: > On 12/13/2010 2:17 PM, Antoine Pitrou wrote: >> On Mon, 13 Dec 2010 14:09:02 -0500 >> Alexander Belopolsky wrote: >> >>> On Mon, Dec 13, 2010 at 11:54 AM, Guido van Rossum >>> wrote: >>>> I'm at least +0 on >>>> allowing trailing commas in the situation the OP mentioned. >>>> >>> >>> FWIW, I am also about +0.5 on allowing trailing comma. Note that in a >>> similar situation, the C standardization committee has erred on the >>> side of consistency: >>> >>> """ >>> A new feature of C99: a common extension in many implementations >>> allows a trailing comma after the list of enumeration constants. The >>> Committee decided to adopt this feature as an innocuous extension that >>> mirrors the trailing commas allowed in initializers. >>> """ http://www.open-std.org/jtc1/sc22/wg14/www/C99RationaleV5.10.pdf >>> >>> Similarly, I find allowing trailing comma in keyword only arguments >>> lists to be an innocuous extension that mirrors the trailing commas >>> allowed in the positional arguments lists. >> >> +1 from me as well. Special cases are hard to remember. > > Same here. A strong +1 for a consistent rule (always or never allowed) > with a +1 for always given others use case of one param/arg per line. +1 on consistency. +1 on allowing the trailing comma. ~Ethan~ From orsenthil at gmail.com Wed Dec 15 09:51:32 2010 From: orsenthil at gmail.com (Senthil Kumaran) Date: Wed, 15 Dec 2010 16:51:32 +0800 Subject: [Python-Dev] Two urllib issues Message-ID: I know it is late to add features in beta release, but still I thought I would ask for a little leeway for these issues, especially as they don't change any API signatures. http://bugs.python.org/issue3243 Has patch, tests and docs http://bugs.python.org/issue1508475 I have patch ready and shall add the tests and docs too. Nothing is dependent on those changes, just that that it would be good to have. Any suggestions on the above? Georg, is it okay if I push this in before 17th? -- Senthil -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Wed Dec 15 13:00:24 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 15 Dec 2010 13:00:24 +0100 Subject: [Python-Dev] r87253 - tracker/instances/python-dev/detectors/autonosy.py References: <20101215005337.D91D4EE9EF@mail.python.org> Message-ID: <20101215130024.31612572@pitrou.net> On Wed, 15 Dec 2010 01:53:37 +0100 (CET) ezio.melotti wrote: > Author: ezio.melotti > Date: Wed Dec 15 01:53:37 2010 > New Revision: 87253 > > Log: > #363: add automatically release managers to the nosy list for release blockers. Initial patch by Georg Brandl. You should probably add deferred blockers too. Antoine. From solipsis at pitrou.net Wed Dec 15 19:39:40 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 15 Dec 2010 19:39:40 +0100 Subject: [Python-Dev] Remove HTTP 0.9 support Message-ID: <20101215193940.31262534@pitrou.net> Hello, I would like to remove HTTP 0.9 support from http.client and http.server. I've opened an issue at http://bugs.python.org/issue10711 for that. Would anyone think it's a bad idea? (HTTP 1.0 was devised in 1996) Regards Antoine. From fdrake at acm.org Wed Dec 15 21:02:24 2010 From: fdrake at acm.org (Fred Drake) Date: Wed, 15 Dec 2010 15:02:24 -0500 Subject: [Python-Dev] Remove HTTP 0.9 support In-Reply-To: <20101215193940.31262534@pitrou.net> References: <20101215193940.31262534@pitrou.net> Message-ID: On Wed, Dec 15, 2010 at 1:39 PM, Antoine Pitrou wrote: > I would like to remove HTTP 0.9 support from http.client and > http.server. +1 ? -Fred -- Fred L. Drake, Jr.? ? "A storm broke loose in my mind."? --Albert Einstein From v+python at g.nevcal.com Wed Dec 15 21:58:51 2010 From: v+python at g.nevcal.com (Glenn Linderman) Date: Wed, 15 Dec 2010 12:58:51 -0800 Subject: [Python-Dev] Remove HTTP 0.9 support In-Reply-To: <20101215193940.31262534@pitrou.net> References: <20101215193940.31262534@pitrou.net> Message-ID: <4D092C0B.8090300@g.nevcal.com> On 12/15/2010 10:39 AM, Antoine Pitrou wrote: > Hello, > > I would like to remove HTTP 0.9 support from http.client and > http.server. I've opened an issue at http://bugs.python.org/issue10711 > for that. Would anyone think it's a bad idea? > > (HTTP 1.0 was devised in 1996) Please address the following comment from the server.py source: # The default request version. This only affects responses up until # the point where the request line is parsed, so it mainly decides what # the client gets back when sending a malformed request line. # Most web servers default to HTTP 0.9, i.e. don't send a status line. default_request_version = "HTTP/0.9" I realize this is a somewhat obscure point, and in general, if your interest in http.client and http.server implies that some of the many outstanding bug reports for that code will get resolved, I have no concern for dropping support for HTTP 0.9 protocol, other than the above. Glenn -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Wed Dec 15 22:25:10 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 15 Dec 2010 22:25:10 +0100 Subject: [Python-Dev] Remove HTTP 0.9 support References: <20101215193940.31262534@pitrou.net> <4D092C0B.8090300@g.nevcal.com> Message-ID: <20101215222510.0b8aa7e7@pitrou.net> On Wed, 15 Dec 2010 12:58:51 -0800 Glenn Linderman wrote: > On 12/15/2010 10:39 AM, Antoine Pitrou wrote: > > Hello, > > > > I would like to remove HTTP 0.9 support from http.client and > > http.server. I've opened an issue at http://bugs.python.org/issue10711 > > for that. Would anyone think it's a bad idea? > > > > (HTTP 1.0 was devised in 1996) > > Please address the following comment from the server.py source: > > # The default request version. This only affects responses up until > # the point where the request line is parsed, so it mainly decides what > # the client gets back when sending a malformed request line. > # Most web servers default to HTTP 0.9, i.e. don't send a status line. > default_request_version = "HTTP/0.9" What do you mean by "address"? The patch changes this to 1.0. And, as the comment says, this only affects what happens when the client sends a syntactically invalid request line, so whether the server does a 0.9-style or 1.0-style response is unimportant. Regards Antoine. From v+python at g.nevcal.com Wed Dec 15 23:20:54 2010 From: v+python at g.nevcal.com (Glenn Linderman) Date: Wed, 15 Dec 2010 14:20:54 -0800 Subject: [Python-Dev] Remove HTTP 0.9 support In-Reply-To: <20101215222510.0b8aa7e7@pitrou.net> References: <20101215193940.31262534@pitrou.net> <4D092C0B.8090300@g.nevcal.com> <20101215222510.0b8aa7e7@pitrou.net> Message-ID: <4D093F46.6050106@g.nevcal.com> On 12/15/2010 1:25 PM, Antoine Pitrou wrote: > On Wed, 15 Dec 2010 12:58:51 -0800 > Glenn Linderman wrote: >> On 12/15/2010 10:39 AM, Antoine Pitrou wrote: >>> Hello, >>> >>> I would like to remove HTTP 0.9 support from http.client and >>> http.server. I've opened an issue at http://bugs.python.org/issue10711 >>> for that. Would anyone think it's a bad idea? >>> >>> (HTTP 1.0 was devised in 1996) >> Please address the following comment from the server.py source: >> >> # The default request version. This only affects responses up until >> # the point where the request line is parsed, so it mainly decides what >> # the client gets back when sending a malformed request line. >> # Most web servers default to HTTP 0.9, i.e. don't send a status line. >> default_request_version = "HTTP/0.9" > What do you mean by "address"? The patch changes this to 1.0. > And, as the comment says, this only affects what happens when the > client sends a syntactically invalid request line, so whether the server > does a 0.9-style or 1.0-style response is unimportant. Just what you did... justify the unimportance of not changing it :) Since now it is different than "most web servers". -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Wed Dec 15 23:29:27 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 15 Dec 2010 23:29:27 +0100 Subject: [Python-Dev] Remove HTTP 0.9 support References: <20101215193940.31262534@pitrou.net> <4D092C0B.8090300@g.nevcal.com> <20101215222510.0b8aa7e7@pitrou.net> <4D093F46.6050106@g.nevcal.com> Message-ID: <20101215232927.475f925c@pitrou.net> On Wed, 15 Dec 2010 14:20:54 -0800 Glenn Linderman wrote: > On 12/15/2010 1:25 PM, Antoine Pitrou wrote: > > On Wed, 15 Dec 2010 12:58:51 -0800 > > Glenn Linderman wrote: > >> On 12/15/2010 10:39 AM, Antoine Pitrou wrote: > >>> Hello, > >>> > >>> I would like to remove HTTP 0.9 support from http.client and > >>> http.server. I've opened an issue at http://bugs.python.org/issue10711 > >>> for that. Would anyone think it's a bad idea? > >>> > >>> (HTTP 1.0 was devised in 1996) > >> Please address the following comment from the server.py source: > >> > >> # The default request version. This only affects responses up until > >> # the point where the request line is parsed, so it mainly decides what > >> # the client gets back when sending a malformed request line. > >> # Most web servers default to HTTP 0.9, i.e. don't send a status line. > >> default_request_version = "HTTP/0.9" > > What do you mean by "address"? The patch changes this to 1.0. > > And, as the comment says, this only affects what happens when the > > client sends a syntactically invalid request line, so whether the server > > does a 0.9-style or 1.0-style response is unimportant. > > Just what you did... justify the unimportance of not changing it :) > Since now it is different than "most web servers". Well, I think the "most web servers" comment itself is outdated. Try e.g. www.mozilla.org or www.google.com or www.msn.com. (but www.python.org or www.apache.org still have the legacy behaviour) Regards Antoine. From fuzzyman at voidspace.org.uk Thu Dec 16 01:35:59 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 16 Dec 2010 00:35:59 +0000 Subject: [Python-Dev] [Python-checkins] r87296 - python/branches/py3k/Doc/library/test.rst In-Reply-To: <20101216002330.E0DBAEE993@mail.python.org> References: <20101216002330.E0DBAEE993@mail.python.org> Message-ID: On 16 December 2010 00:23, eric.araujo wrote: > Author: eric.araujo > Date: Thu Dec 16 01:23:30 2010 > New Revision: 87296 > > Log: > Advertise ?python -m? instead of direct filename. > > > Modified: > python/branches/py3k/Doc/library/test.rst > > Modified: python/branches/py3k/Doc/library/test.rst > > ============================================================================== > --- python/branches/py3k/Doc/library/test.rst (original) > +++ python/branches/py3k/Doc/library/test.rst Thu Dec 16 01:23:30 2010 > @@ -168,14 +168,14 @@ > > Running :mod:`test.regrtest` directly allows what resources are available > for > tests to use to be set. You do this by using the ``-u`` command-line > -option. Run :program:`python regrtest.py -uall` to turn on all > +option. Run :program:`python -m regrtest -uall` to turn on all > Shouldn't this be `python -m test.regrtest`, or even just `python -m test`? Michael > resources; specifying ``all`` as an option for ``-u`` enables all > possible resources. If all but one resource is desired (a more common > case), a > comma-separated list of resources that are not desired may be listed after > -``all``. The command :program:`python regrtest.py -uall,-audio,-largefile` > +``all``. The command :program:`python -m regrtest -uall,-audio,-largefile` > will run :mod:`test.regrtest` with all resources except the ``audio`` and > ``largefile`` resources. For a list of all resources and more command-line > -options, run :program:`python regrtest.py -h`. > +options, run :program:`python -m regrtest -h`. > > Some other ways to execute the regression tests depend on what platform > the > tests are being executed on. On Unix, you can run :program:`make test` at > the > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > -- http://www.voidspace.org.uk -------------- next part -------------- An HTML attachment was scrubbed... URL: From merwok at netwok.org Thu Dec 16 02:07:43 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Thu, 16 Dec 2010 02:07:43 +0100 Subject: [Python-Dev] [Python-checkins] r87296 - python/branches/py3k/Doc/library/test.rst In-Reply-To: References: <20101216002330.E0DBAEE993@mail.python.org> Message-ID: <4D09665F.603@netwok.org> >> -option. Run :program:`python regrtest.py -uall` to turn on all >> +option. Run :program:`python -m regrtest -uall` to turn on all > > Shouldn't this be `python -m test.regrtest`, or even just `python -m test`? It should :) I was about to fix just that but then noticed I could remove more references to regrtest (after all just an implementation detail), so I rewrote the section title and first few lines of the section. My stupid email client forces rewrapping, so I?m attaching the diff as a separate file. I can repost the patch to the tracker if it?s inconvenient to review in email, but I?d like to commit soon since I have a big doc merge to do after this :) Best regards -------------- next part -------------- A non-text attachment was scrubbed... Name: fix-test.rst.diff Type: text/x-diff Size: 2611 bytes Desc: not available URL: From fuzzyman at voidspace.org.uk Thu Dec 16 02:13:42 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 16 Dec 2010 01:13:42 +0000 Subject: [Python-Dev] [Python-checkins] r87296 - python/branches/py3k/Doc/library/test.rst In-Reply-To: <4D09665F.603@netwok.org> References: <20101216002330.E0DBAEE993@mail.python.org> <4D09665F.603@netwok.org> Message-ID: <4D0967C6.4040408@voidspace.org.uk> On 16/12/2010 01:07, ?ric Araujo wrote: >>> -option. Run :program:`python regrtest.py -uall` to turn on all >>> +option. Run :program:`python -m regrtest -uall` to turn on all >> Shouldn't this be `python -m test.regrtest`, or even just `python -m test`? > It should :) I was about to fix just that but then noticed I could > remove more references to regrtest (after all just an implementation > detail), so I rewrote the section title and first few lines of the section. > > My stupid email client forces rewrapping, so I?m attaching the diff as a > separate file. I can repost the patch to the tracker if it?s > inconvenient to review in email, but I?d like to commit soon since I > have a big doc merge to do after this :) The diff looks good to me. All the best, Michael > Best regards -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html From merwok at netwok.org Thu Dec 16 02:36:12 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Thu, 16 Dec 2010 02:36:12 +0100 Subject: [Python-Dev] [Python-checkins] r87299 - in python/branches/py3k: Doc/library/configparser.rst Lib/configparser.py Lib/test/test_cfgparser.py In-Reply-To: <20101216011622.871CDEE9D2@mail.python.org> References: <20101216011622.871CDEE9D2@mail.python.org> Message-ID: <4D096D0C.8020506@netwok.org> > Author: lukasz.langa > New Revision: 87299 > > Log: > Broken ConfigParser removed, SafeConfigParser renamed to ConfigParser. > Life is beatiful once again. IIIUC, this change makes bugs requesting use of SafeConfigParser in distutils and logging obsolete. > @@ -1139,6 +1122,6 @@ > > if __name__ == "__main__": > if "-c" in sys.argv: > - test_coverage('/tmp/cmd.cover') > + test_coverage('/tmp/configparser.cover') > else: > test_main() Consider using the tempfile module. You need to print the filename on stderr, I think. Alternatively, remove this custom functionality entirely and move it to regrtest or unittest. Cheers From merwok at netwok.org Thu Dec 16 02:41:26 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Thu, 16 Dec 2010 02:41:26 +0100 Subject: [Python-Dev] [Python-checkins] r87296 - python/branches/py3k/Doc/library/test.rst In-Reply-To: <4D0967C6.4040408@voidspace.org.uk> References: <20101216002330.E0DBAEE993@mail.python.org> <4D09665F.603@netwok.org> <4D0967C6.4040408@voidspace.org.uk> Message-ID: <4D096E46.20801@netwok.org> > The diff looks good to me. Committed as r87300, thank you sir! From merwok at netwok.org Thu Dec 16 03:41:48 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Thu, 16 Dec 2010 03:41:48 +0100 Subject: [Python-Dev] PEPs and versionadded Message-ID: <4D097C6C.1010109@netwok.org> Hi, I noticed that changes related to PEP 3147 and PEP 3149 in Doc haven?t been accompanied by versionadded/versionchanged directives. Is that on purpose, meaning that everyone should be aware of these PEPs when reading 3.2 docs, or just an oversight? Regards From barry at python.org Thu Dec 16 02:20:56 2010 From: barry at python.org (Barry Warsaw) Date: Wed, 15 Dec 2010 20:20:56 -0500 Subject: [Python-Dev] PEPs and versionadded In-Reply-To: <4D097C6C.1010109@netwok.org> References: <4D097C6C.1010109@netwok.org> Message-ID: <20101215202056.3665b462@mission> On Dec 16, 2010, at 03:41 AM, ?ric Araujo wrote: >I noticed that changes related to PEP 3147 and PEP 3149 in Doc haven?t >been accompanied by versionadded/versionchanged directives. > >Is that on purpose, meaning that everyone should be aware of these PEPs >when reading 3.2 docs, or just an oversight? It's an oversight. The changes should be accompanied by version* directives. I see you already committed a change for compileall.rst - thanks for that! Please feel free to add anything you find missing or submit a bug report and assign it to me. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From merwok at netwok.org Thu Dec 16 07:34:37 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Thu, 16 Dec 2010 07:34:37 +0100 Subject: [Python-Dev] PEPs and versionadded In-Reply-To: <20101215202056.3665b462@mission> References: <4D097C6C.1010109@netwok.org> <20101215202056.3665b462@mission> Message-ID: <4D09B2FD.7080806@netwok.org> > It's an oversight. The changes should be accompanied by version* directives. > I see you already committed a change for compileall.rst - thanks for that! > Please feel free to add anything you find missing or submit a bug report and > assign it to me. The PEP 3149 commits did not touch Doc, apart from sys.rst which got its versionadded added (duh) a short time after the code changes. I don?t know if the introduction of ABI flags ought to be documented more prominently; Doc/using seems still accurate, and Doc/distutils is rather vague about extensions, so I guess we?re good. PEP 3147 modified a handful of files in Doc; I?ve added missing directives in r87312. __cached__ is only described in library/runpy.rst and c-api/import.rst. If there is one canonical place to define what a module is (I couldn?t tell from http://docs.python.org/dev/genindex-M), I think it should be mentioned there. Copy-pasting the paragraph about __cached__ from the PEP would suffice. Both PEPs are well covered in whatsnew. Three cheers for Raymond! Regards From orsenthil at gmail.com Thu Dec 16 09:14:04 2010 From: orsenthil at gmail.com (Senthil Kumaran) Date: Thu, 16 Dec 2010 16:14:04 +0800 Subject: [Python-Dev] Remove HTTP 0.9 support In-Reply-To: <4D093F46.6050106@g.nevcal.com> References: <20101215193940.31262534@pitrou.net> <4D092C0B.8090300@g.nevcal.com> <20101215222510.0b8aa7e7@pitrou.net> <4D093F46.6050106@g.nevcal.com> Message-ID: <20101216081404.GA4603@rubuntu> On Wed, Dec 15, 2010 at 02:20:54PM -0800, Glenn Linderman wrote: > On 12/15/2010 10:39 AM, Antoine Pitrou wrote: > I would like to remove HTTP 0.9 support from http.client and > http.server. I've opened an issue at http://bugs.python.org/issue10711 > for that. Would anyone think it's a bad idea? > > (HTTP 1.0 was devised in 1996) > > Please address the following comment from the server.py source: > > # The default request version. This only affects responses up until > # the point where the request line is parsed, so it mainly decides what > # the client gets back when sending a malformed request line. > # Most web servers default to HTTP 0.9, i.e. don't send a status line. > default_request_version = "HTTP/0.9" > > What do you mean by "address"? The patch changes this to 1.0. > And, as the comment says, this only affects what happens when the > client sends a syntactically invalid request line, so whether the server > does a 0.9-style or 1.0-style response is unimportant. > > > Just what you did... justify the unimportance of not changing it :) Since now > it is different than "most web servers". +1 to removing HTTP 0.9 related code in http.client and http.server. Until today, I hadn't cared to read any details of HTTP 0.9 except that I knew that some code was present in http library to support it. Reading a bit about the HTTP 0.9 'Internet Draft' written 1991 is enough to convince that, if we have to fall back on any old behavior falling back to HTTP 1.0 is perfectly fine. Regarding this comment # Most web servers default to HTTP 0.9, i.e. don't send a status line. Even HTTP 0.9 says that response SHOULD start with status line, but gives a suggestion that clients can "tolerate" bad server server behaviors when they don't send the status line and in that the case response is the body. http://www.w3.org/Protocols/HTTP/Response.html So, It cannot be associated with the behavior "most webservers", back then and even more so now. -- Senthil From orsenthil at gmail.com Thu Dec 16 09:14:56 2010 From: orsenthil at gmail.com (Senthil Kumaran) Date: Thu, 16 Dec 2010 16:14:56 +0800 Subject: [Python-Dev] Remove HTTP 0.9 support In-Reply-To: <20101215232927.475f925c@pitrou.net> References: <20101215193940.31262534@pitrou.net> <4D092C0B.8090300@g.nevcal.com> <20101215222510.0b8aa7e7@pitrou.net> <4D093F46.6050106@g.nevcal.com> <20101215232927.475f925c@pitrou.net> Message-ID: <20101216081456.GB4603@rubuntu> On Wed, Dec 15, 2010 at 11:29:27PM +0100, Antoine Pitrou wrote: > Well, I think the "most web servers" comment itself is outdated. > Try e.g. www.mozilla.org or www.google.com or www.msn.com. > (but www.python.org or www.apache.org still have the legacy behaviour) What legacy behavior did you observe in these? -- Senthil From nd at perlig.de Thu Dec 16 07:42:08 2010 From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=) Date: Thu, 16 Dec 2010 07:42:08 +0100 Subject: [Python-Dev] Remove HTTP 0.9 support In-Reply-To: <20101215193940.31262534@pitrou.net> References: <20101215193940.31262534@pitrou.net> Message-ID: <201012160742.08474@news.perlig.de> * Antoine Pitrou wrote: > Hello, > > I would like to remove HTTP 0.9 support from http.client and > http.server. I've opened an issue at http://bugs.python.org/issue10711 > for that. Would anyone think it's a bad idea? > > (HTTP 1.0 was devised in 1996) HTTP/0.9 support is still recommended (RFC 2616 is from 1999, but still current). I'm wondering, why you would consider touching that at all. Is it broken? Does it stand in the way of anything? If not, why throw away a feature? nd -- Already I've seen people (really!) write web URLs in the form: http:\\some.site.somewhere [...] How soon until greengrocers start writing "apples $1\pound" or something? -- Joona I Palaste in clc From dpritsos at extremepro.gr Thu Dec 16 12:09:41 2010 From: dpritsos at extremepro.gr (Dimitrios Pritsos) Date: Thu, 16 Dec 2010 13:09:41 +0200 Subject: [Python-Dev] Multiprocessing module - Synergeticprocessing (my custom module) - Pickling unPickling issues Message-ID: <4D09F375.40208@extremepro.gr> Hello Core Developers, My name is Dimitrios and I am newbie in python. I am working on a Project (part of my PhD) that is called Synergeticprocessing module. Initially is imitating the multiprocessing built in module but the processes are distributed on a LAN and not Locally. The main issue I have is with Pickle module. And I think I found some kind of BUG in the built in multiprocessing module. (Synergeticprocessing module is located at GitHub: https://github.com/dpritsos/synergeticprocessing) Starting with the "BUG". In case someone uses the multiprocessing.Pool of processes he/she has to face the problem of types.MehtodType Impossible pickling. That is you cannot dispatch an class instance method to the to the Process Pool. However, digging in to the Source Code of the module there are few lines that resolve this issue however are not activated or they are faultily activated so they do not work. This is the 'BUG' _@ ....../multiprocessing/forking.py_ . . . # # Try making some callable types picklable # from pickle import Pickler class ForkingPickler(Pickler): dispatch = Pickler.dispatch.copy() @classmethod def register(cls, type, reduce): def dispatcher(self, obj): rv = reduce(obj) self.save_reduce(obj=obj, *rv) cls.dispatch[type] = dispatcher def _reduce_method(m): if m.im_self is None: return getattr, (m.im_class, m.im_func.func_name) else: return getattr, (m.im_self, m.im_func.func_name) ForkingPickler.register(type(ForkingPickler.save), _reduce_method) def _reduce_method_descriptor(m): return getattr, (m.__objclass__, m.__name__) ForkingPickler.register(type(list.append), _reduce_method_descriptor) ForkingPickler.register(type(int.__add__), _reduce_method_descriptor) #def _reduce_builtin_function_or_method(m): # return getattr, (m.__self__, m.__name__) #ForkingPickler.register(type(list().append), _reduce_builtin_function_or_method) #ForkingPickler.register(type(int().__add__), _reduce_builtin_function_or_method) . . . The RED lines are not doing the job, for some reason they are not managing to register the GREEN function as a global reduce/pickling function even if you call the registration function into you __main__ script. The solution I found is just to do this * import copy_reg import types* def _reduce_method(m): if m.im_self is None: return getattr, (m.im_class, m.im_func.func_name) else: return getattr, (m.im_self, m.im_func.func_name) *copy_reg.pickle(types.MethodType, _reduce_method)* . . . Doing that everything works FINE. But ONLY for local methods i.e. the ones that their class is defined on the __main__ script or other import-ed. In case you want to send something remotely (in an other machine) or to an other __main__ script running separately then you get a message like this: 'module' object has no attribute ' ' The only way to resolve this is firstly to import a script that has defined there and everything works fine. SO the problems it seems to be that the *m.im_class* (look code above) has some attribute __module__ defined as __module__ = '__main__' or something like that. And this is the reason why remote script cannot execute the function. I mean that the _reduce_method() above DOES is pickling the whole CLASS object so there is no reason not to be executed at the remote script. Besides it does as mentioned above in you just import this the user defined class form an other script. I have already spent about 12 weeks working on building my synergeticPool and resolve the issue of Pickling and only 2 days needed for the code of the Pool and the rest of the time was spent for the Pickling issues, and study all the Class related mechanics of python. That was the reason I ve started digging the multipocessessing module and found this say 'BUG', and finally sent this email. Best Regards, Dimitrios -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at voidspace.org.uk Thu Dec 16 13:29:59 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 16 Dec 2010 12:29:59 +0000 Subject: [Python-Dev] Multiprocessing module - Synergeticprocessing (my custom module) - Pickling unPickling issues In-Reply-To: <4D09F375.40208@extremepro.gr> References: <4D09F375.40208@extremepro.gr> Message-ID: <4D0A0647.2050900@voidspace.org.uk> On 16/12/2010 11:09, Dimitrios Pritsos wrote: > > Hello Core Developers, > > My name is Dimitrios and I am newbie in python. I am working on a > Project (part of my PhD) that is called Synergeticprocessing module. > Initially is imitating the multiprocessing built in module but the > processes are distributed on a LAN and not Locally. The main issue I > have is with Pickle module. And I think I found some kind of BUG in > the built in multiprocessing module. > Hello Dimitrios, Please post your bug report to the Python bug tracker. As you think you have a fix for the issue it is much more likely to be applied quickly if you can package it in the form of a test that demonstrates the issue and a patch that fixes it. http://bugs.python.org/ All the best, Michael Foord > (Synergeticprocessing module is located at GitHub: > https://github.com/dpritsos/synergeticprocessing) > > Starting with the "BUG". In case someone uses the multiprocessing.Pool > of processes he/she has to face the problem of types.MehtodType > Impossible pickling. That is you cannot dispatch an class instance > method to the to the Process Pool. However, digging in to the Source > Code of the module there are few lines that resolve this issue however > are not activated or they are faultily activated so they do not work. > This is the 'BUG' > > _@ ....../multiprocessing/forking.py_ > > > . > . > . > > # > # Try making some callable types picklable > # > > from pickle import Pickler > class ForkingPickler(Pickler): > dispatch = Pickler.dispatch.copy() > > @classmethod > def register(cls, type, reduce): > def dispatcher(self, obj): > rv = reduce(obj) > self.save_reduce(obj=obj, *rv) > cls.dispatch[type] = dispatcher > > def _reduce_method(m): > if m.im_self is None: > return getattr, (m.im_class, m.im_func.func_name) > else: > return getattr, (m.im_self, m.im_func.func_name) > ForkingPickler.register(type(ForkingPickler.save), _reduce_method) > > def _reduce_method_descriptor(m): > return getattr, (m.__objclass__, m.__name__) > ForkingPickler.register(type(list.append), _reduce_method_descriptor) > ForkingPickler.register(type(int.__add__), _reduce_method_descriptor) > > #def _reduce_builtin_function_or_method(m): > # return getattr, (m.__self__, m.__name__) > #ForkingPickler.register(type(list().append), > _reduce_builtin_function_or_method) > #ForkingPickler.register(type(int().__add__), > _reduce_builtin_function_or_method) > . > . > . > > The RED lines are not doing the job, for some reason they are not > managing to register the GREEN function as a global reduce/pickling > function even if you call the registration function into you __main__ > script. > > The solution I found is just to do this > * > import copy_reg > import types* > > def _reduce_method(m): > if m.im_self is None: > return getattr, (m.im_class, m.im_func.func_name) > else: > return getattr, (m.im_self, m.im_func.func_name) > > *copy_reg.pickle(types.MethodType, _reduce_method)* > . > . > . > > Doing that everything works FINE. But ONLY for local methods i.e. the > ones that their class is defined on the __main__ script or other > import-ed. > > In case you want to send something remotely (in an other machine) or > to an other __main__ script running separately then you get a message > like this: > > 'module' object has no attribute ' ' > > The only way to resolve this is firstly to import a script that has > defined there and everything works fine. > > SO the problems it seems to be that the *m.im_class* (look code > above) has some attribute __module__ defined as __module__ = > '__main__' or something like that. And this is the reason why remote > script cannot execute the function. I mean that the _reduce_method() > above DOES is pickling the whole CLASS object so there is no reason > not to be executed at the remote script. Besides it does as mentioned > above in you just import this the user defined class form an other > script. > > > I have already spent about 12 weeks working on building my > synergeticPool and resolve the issue of Pickling and only 2 days > needed for the code of the Pool and the rest of the time was spent for > the Pickling issues, and study all the Class related mechanics of > python. That was the reason I ve started digging the multipocessessing > module and found this say 'BUG', and finally sent this email. > > Best Regards, > > > Dimitrios > > > > > > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Thu Dec 16 14:20:37 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 16 Dec 2010 14:20:37 +0100 Subject: [Python-Dev] Remove HTTP 0.9 support In-Reply-To: <20101216081456.GB4603@rubuntu> References: <20101215193940.31262534@pitrou.net> <4D092C0B.8090300@g.nevcal.com> <20101215222510.0b8aa7e7@pitrou.net> <4D093F46.6050106@g.nevcal.com> <20101215232927.475f925c@pitrou.net> <20101216081456.GB4603@rubuntu> Message-ID: <1292505637.3689.0.camel@localhost.localdomain> Le jeudi 16 d?cembre 2010 ? 16:14 +0800, Senthil Kumaran a ?crit : > On Wed, Dec 15, 2010 at 11:29:27PM +0100, Antoine Pitrou wrote: > > Well, I think the "most web servers" comment itself is outdated. > > Try e.g. www.mozilla.org or www.google.com or www.msn.com. > > (but www.python.org or www.apache.org still have the legacy behaviour) > > What legacy behavior did you observe in these? -> Request: xyzzy -> Response: 405 Method Not Allowed Method Not Allowed
The requested method xyzzy is not allowed for the URL /.
Apache/2.3.8 (Unix) mod_ssl/2.3.8 OpenSSL/1.0.0a Server at www.apache.org Port 80 (notice how the response has no headers) From dpritsos at extremepro.gr Thu Dec 16 14:52:19 2010 From: dpritsos at extremepro.gr (Dimitrios Pritsos) Date: Thu, 16 Dec 2010 15:52:19 +0200 Subject: [Python-Dev] Multiprocessing module - Synergeticprocessing (my custom module) - Pickling unPickling issues In-Reply-To: <4D0A0647.2050900@voidspace.org.uk> References: <4D09F375.40208@extremepro.gr> <4D0A0647.2050900@voidspace.org.uk> Message-ID: <4D0A1993.2030309@extremepro.gr> On 12/16/2010 02:29 PM, Michael Foord wrote: > On 16/12/2010 11:09, Dimitrios Pritsos wrote: >> >> Hello Core Developers, >> >> My name is Dimitrios and I am newbie in python. I am working on a >> Project (part of my PhD) that is called Synergeticprocessing module. >> Initially is imitating the multiprocessing built in module but the >> processes are distributed on a LAN and not Locally. The main issue I >> have is with Pickle module. And I think I found some kind of BUG in >> the built in multiprocessing module. >> > > Hello Dimitrios, > > Please post your bug report to the Python bug tracker. As you think > you have a fix for the issue it is much more likely to be applied > quickly if you can package it in the form of a test that demonstrates > the issue and a patch that fixes it. > > http://bugs.python.org/ > > All the best, > > Michael Foord > Hello Michael, OK I will do sent it to the bug tracker. But what about the last issue i.e. that even if the class is transfered-and-pickled-unpickled it raises an exception if the class is defined into the __main__ script and not imported from an other one. (see below) could you please guide me where should I post this for someone to help me fix it! Best Regards, Dimitrios > > >> (Synergeticprocessing module is located at GitHub: >> https://github.com/dpritsos/synergeticprocessing) >> >> Starting with the "BUG". In case someone uses the >> multiprocessing.Pool of processes he/she has to face the problem of >> types.MehtodType Impossible pickling. That is you cannot dispatch an >> class instance method to the to the Process Pool. However, digging in >> to the Source Code of the module there are few lines that resolve >> this issue however are not activated or they are faultily activated >> so they do not work. This is the 'BUG' >> >> _@ ....../multiprocessing/forking.py_ >> >> >> . >> . >> . >> >> # >> # Try making some callable types picklable >> # >> >> from pickle import Pickler >> class ForkingPickler(Pickler): >> dispatch = Pickler.dispatch.copy() >> >> @classmethod >> def register(cls, type, reduce): >> def dispatcher(self, obj): >> rv = reduce(obj) >> self.save_reduce(obj=obj, *rv) >> cls.dispatch[type] = dispatcher >> >> def _reduce_method(m): >> if m.im_self is None: >> return getattr, (m.im_class, m.im_func.func_name) >> else: >> return getattr, (m.im_self, m.im_func.func_name) >> ForkingPickler.register(type(ForkingPickler.save), _reduce_method) >> >> def _reduce_method_descriptor(m): >> return getattr, (m.__objclass__, m.__name__) >> ForkingPickler.register(type(list.append), _reduce_method_descriptor) >> ForkingPickler.register(type(int.__add__), _reduce_method_descriptor) >> >> #def _reduce_builtin_function_or_method(m): >> # return getattr, (m.__self__, m.__name__) >> #ForkingPickler.register(type(list().append), >> _reduce_builtin_function_or_method) >> #ForkingPickler.register(type(int().__add__), >> _reduce_builtin_function_or_method) >> . >> . >> . >> >> The RED lines are not doing the job, for some reason they are not >> managing to register the GREEN function as a global reduce/pickling >> function even if you call the registration function into you __main__ >> script. >> >> The solution I found is just to do this >> * >> import copy_reg >> import types* >> >> def _reduce_method(m): >> if m.im_self is None: >> return getattr, (m.im_class, m.im_func.func_name) >> else: >> return getattr, (m.im_self, m.im_func.func_name) >> >> *copy_reg.pickle(types.MethodType, _reduce_method)* >> . >> . >> . >> >> Doing that everything works FINE. But ONLY for local methods i.e. the >> ones that their class is defined on the __main__ script or other >> import-ed. >> >> In case you want to send something remotely (in an other machine) or >> to an other __main__ script running separately then you get a message >> like this: >> >> 'module' object has no attribute ' ' >> >> The only way to resolve this is firstly to import a script that has >> defined there and everything works fine. >> >> SO the problems it seems to be that the *m.im_class* (look code >> above) has some attribute __module__ defined as __module__ = >> '__main__' or something like that. And this is the reason why remote >> script cannot execute the function. I mean that the _reduce_method() >> above DOES is pickling the whole CLASS object so there is no reason >> not to be executed at the remote script. Besides it does as mentioned >> above in you just import this the user defined class form an other >> script. >> >> >> I have already spent about 12 weeks working on building my >> synergeticPool and resolve the issue of Pickling and only 2 days >> needed for the code of the Pool and the rest of the time was spent >> for the Pickling issues, and study all the Class related mechanics of >> python. That was the reason I ve started digging the >> multipocessessing module and found this say 'BUG', and finally sent >> this email. >> >> Best Regards, >> >> >> Dimitrios >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe:http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > > > -- > > http://www.voidspace.org.uk/ > > May you do good and not evil > May you find forgiveness for yourself and forgive others > May you share freely, never taking more than you give. > -- the sqlite blessinghttp://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Thu Dec 16 15:00:34 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 16 Dec 2010 15:00:34 +0100 Subject: [Python-Dev] Multiprocessing module - Synergeticprocessing (my custom module) - Pickling unPickling issues References: <4D09F375.40208@extremepro.gr> <4D0A0647.2050900@voidspace.org.uk> <4D0A1993.2030309@extremepro.gr> Message-ID: <20101216150034.04711183@pitrou.net> On Thu, 16 Dec 2010 15:52:19 +0200 Dimitrios Pritsos wrote: > > Hello Michael, > > OK I will do sent it to the bug tracker. But what about the last issue > i.e. that even if the class is transfered-and-pickled-unpickled it > raises an exception if the class is defined into the __main__ script and > not imported from an other one. (see below) could you please guide me > where should I post this for someone to help me fix it! The likely explanation is that the __main__ module in the parent process isn't the same as in the child process, so fetching the class from that module doesn't work. If that's really important for you you could open a separate issue. Regards Antoine. From solipsis at pitrou.net Thu Dec 16 15:23:05 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 16 Dec 2010 15:23:05 +0100 Subject: [Python-Dev] Remove HTTP 0.9 support References: <20101215193940.31262534@pitrou.net> <201012160742.08474@news.perlig.de> Message-ID: <20101216152305.13b1fd3a@pitrou.net> On Thu, 16 Dec 2010 07:42:08 +0100 Andr? Malo wrote: > * Antoine Pitrou wrote: > > > Hello, > > > > I would like to remove HTTP 0.9 support from http.client and > > http.server. I've opened an issue at http://bugs.python.org/issue10711 > > for that. Would anyone think it's a bad idea? > > > > (HTTP 1.0 was devised in 1996) > > HTTP/0.9 support is still recommended (RFC 2616 is from 1999, but still > current). > > I'm wondering, why you would consider touching that at all. Is it broken? > Does it stand in the way of anything? If not, why throw away a feature? Well, it complicates maintenance and makes fixing issues such as http://bugs.python.org/issue6791 less likely. Note that the patch still accepts servers and clients which advertise themselves as 0.9 (using "HTTP/0.9" as a version string). It just removes support for the style of "simple response" without headers that 0.9 allowed. Regards Antoine. From exarkun at twistedmatrix.com Thu Dec 16 16:25:56 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Thu, 16 Dec 2010 15:25:56 -0000 Subject: [Python-Dev] Multiprocessing module - Synergeticprocessing (my custom module) - Pickling unPickling issues In-Reply-To: <20101216150034.04711183@pitrou.net> References: <4D09F375.40208@extremepro.gr> <4D0A0647.2050900@voidspace.org.uk> <4D0A1993.2030309@extremepro.gr> <20101216150034.04711183@pitrou.net> Message-ID: <20101216152556.2058.1297404892.divmod.xquotient.262@localhost.localdomain> On 02:00 pm, solipsis at pitrou.net wrote: >On Thu, 16 Dec 2010 15:52:19 +0200 >Dimitrios Pritsos wrote: >> >>Hello Michael, >> >>OK I will do sent it to the bug tracker. But what about the last issue >>i.e. that even if the class is transfered-and-pickled-unpickled it >>raises an exception if the class is defined into the __main__ script >>and >>not imported from an other one. (see below) could you please guide me >>where should I post this for someone to help me fix it! > >The likely explanation is that the __main__ module in the parent >process isn't the same as in the child process, so fetching the class >from that module doesn't work. >If that's really important for you you could open a separate issue. And another option is to use the simple, pleasant fix of not defining any functions or classes in __main__. Define them in a real module and import them for use in __main__. Jean-Paul From nd at perlig.de Thu Dec 16 16:52:43 2010 From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=) Date: Thu, 16 Dec 2010 16:52:43 +0100 Subject: [Python-Dev] Remove HTTP 0.9 support In-Reply-To: <20101216152305.13b1fd3a@pitrou.net> References: <20101215193940.31262534@pitrou.net> <201012160742.08474@news.perlig.de> <20101216152305.13b1fd3a@pitrou.net> Message-ID: <201012161652.43949.nd@perlig.de> On Thursday 16 December 2010 15:23:05 Antoine Pitrou wrote: > On Thu, 16 Dec 2010 07:42:08 +0100 > > Andr? Malo wrote: > > * Antoine Pitrou wrote: > > > Hello, > > > > > > I would like to remove HTTP 0.9 support from http.client and > > > http.server. I've opened an issue at http://bugs.python.org/issue10711 > > > for that. Would anyone think it's a bad idea? > > > > > > (HTTP 1.0 was devised in 1996) > > > > HTTP/0.9 support is still recommended (RFC 2616 is from 1999, but still > > current). > > > > I'm wondering, why you would consider touching that at all. Is it broken? > > Does it stand in the way of anything? If not, why throw away a feature? > > Well, it complicates maintenance and makes fixing issues such as > http://bugs.python.org/issue6791 less likely. I'd vote for removing it from the client code and keeping it in the server. > Note that the patch still accepts servers and clients which advertise > themselves as 0.9 (using "HTTP/0.9" as a version string). HTTP/0.9 doesn't *have* a version string. GET /foo is a HTTP/0.9 request. GET /foo HTTP/0.9 isn't actually (it's a paradoxon, alright ;). It simply isn't a valid HTTP request, which would demand a 505 response. nd From rdmurray at bitdance.com Thu Dec 16 17:04:46 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 16 Dec 2010 11:04:46 -0500 Subject: [Python-Dev] [Python-checkins] r87310 - in python/branches/release27-maint: Doc/library/compileall.rst Lib/compileall.py In-Reply-To: <20101216061502.43686EE992@mail.python.org> References: <20101216061502.43686EE992@mail.python.org> Message-ID: <20101216160446.40E9D1FFBDE@kimball.webabinitio.net> On Thu, 16 Dec 2010 07:15:02 +0100, eric.araujo wrote: > Modified: python/branches/release27-maint/Doc/library/compileall.rst > ============================================================================== > --- python/branches/release27-maint/Doc/library/compileall.rst (original) > +++ python/branches/release27-maint/Doc/library/compileall.rst Thu Dec 16 07:15:02 2010 > @@ -1,4 +1,3 @@ > - > :mod:`compileall` --- Byte-compile Python libraries > =================================================== > > @@ -50,14 +49,14 @@ > > Expand list with its content (file and directory names). I realize you didn't write this line, but note that '-' is accepted as an argument and means "read the list from stdin". > -.. versionadded:: 2.7 > - The ``-i`` option. > +.. versionchanged:: 2.7 > + Added the ``-i`` option. > > > Public functions > ---------------- > > -.. function:: compile_dir(dir[, maxlevels[, ddir[, force[, rx[, quiet]]]]]) > +.. function:: compile_dir(dir[, maxlevels[, ddir[, force[, rx[, quiet]]]]]) > > Recursively descend the directory tree named by *dir*, compiling all :file:`.py` > files along the way. The *maxlevels* parameter is used to limit the depth of > @@ -72,6 +71,23 @@ > If *quiet* is true, nothing is printed to the standard output in normal > operation. > > + > +.. function:: compile_file(fullname[, ddir[, force[, rx[, quiet]]]]) > + > + Compile the file with path *fullname*. If *ddir* is given, it is used as the > + base path from which the filename used in error messages will be generated. > + If *force* is true, modules are re-compiled even if the timestamp is up to > + date. Although this is copied from the other descriptions of *ddir*, it and the other instances (and the description of the -d option) should all really be fixed. As I discovered when writing the tests for the -d option, what ddir is is the path that is "baked in" to the .pyc file. In very old versions of Python that meant it was the path that would show up in tracebacks as the path to the source file. In modern Pythons the ddir path shows up if and only if the .py file does not exist and the .pyc file is being run directly. In 3.2, this means it will never show up normally, since you can't even run the .pyc file without moving it out of __pycache__. Which means 'ddir' is henceforth useful only to those people who want to package sourceless distributions of the python code. (If you want to see this in action check out the -d tests in test_compileall.) So, 'in error messages' really means 'in tracebacks if the .py file doesn't exist'. -- R. David Murray www.bitdance.com From fdrake at acm.org Thu Dec 16 17:20:04 2010 From: fdrake at acm.org (Fred Drake) Date: Thu, 16 Dec 2010 11:20:04 -0500 Subject: [Python-Dev] Remove HTTP 0.9 support In-Reply-To: <201012161652.43949.nd@perlig.de> References: <20101215193940.31262534@pitrou.net> <201012160742.08474@news.perlig.de> <20101216152305.13b1fd3a@pitrou.net> <201012161652.43949.nd@perlig.de> Message-ID: On Thu, Dec 16, 2010 at 10:52 AM, Andr? Malo wrote: > I'd vote for removing it from the client code and keeping it in the server. If it must be maintained anywhere, it should be in the client, according to the basic principle of "accept what you can, generate carefully." Python.org's HTTP/0.9 responses appear to be in response to HTTP/0.9 requests only. A request claiming to be HTTP 1.0, but without a Host: header, gets a redirect to the same page. I'm still in favor of removing HTTP 0.9 support entirely. ? -Fred -- Fred L. Drake, Jr.? ? "A storm broke loose in my mind."? --Albert Einstein From merwok at netwok.org Thu Dec 16 17:41:22 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Thu, 16 Dec 2010 17:41:22 +0100 Subject: [Python-Dev] [Python-checkins] r87310 - in python/branches/release27-maint: Doc/library/compileall.rst Lib/compileall.py In-Reply-To: <20101216160446.40E9D1FFBDE@kimball.webabinitio.net> References: <20101216061502.43686EE992@mail.python.org> <20101216160446.40E9D1FFBDE@kimball.webabinitio.net> Message-ID: <4D0A4132.5020407@netwok.org> Hi, Thanks for double-checking. When I first looked into compileall, I opened http://bugs.python.org/issue10454 where I state that I find the description of those options unclear or even not understandable, so your diagnosis that I just copied text is right. A rewrite to fully cover the module functionality in clear English is needed. Your email was very helpful; can you turn it into a patch and post it to the bug? Thanks in advance. Regards From orsenthil at gmail.com Thu Dec 16 17:52:14 2010 From: orsenthil at gmail.com (Senthil Kumaran) Date: Fri, 17 Dec 2010 00:52:14 +0800 Subject: [Python-Dev] Remove HTTP 0.9 support In-Reply-To: <1292505637.3689.0.camel@localhost.localdomain> References: <20101215193940.31262534@pitrou.net> <4D092C0B.8090300@g.nevcal.com> <20101215222510.0b8aa7e7@pitrou.net> <4D093F46.6050106@g.nevcal.com> <20101215232927.475f925c@pitrou.net> <20101216081456.GB4603@rubuntu> <1292505637.3689.0.camel@localhost.localdomain> Message-ID: <20101216165214.GA7964@rubuntu> On Thu, Dec 16, 2010 at 02:20:37PM +0100, Antoine Pitrou wrote: > > > Try e.g. www.mozilla.org or www.google.com or www.msn.com. > > > (but www.python.org or www.apache.org still have the legacy behaviour) > > > > What legacy behavior did you observe in these? > > -> Request: > xyzzy > > -> Response: > > (notice how the response has no headers) > Well, Error response without headers was observed in www.mozilla.org and www.google.com for Invalid requests. But, I observed something surprising at www.apache.org If you do GET / HTTP/1.0 You do get the valid Response with headers. But if you do GET / You get a valid response Without headers. I was afraid if this behavior was to support HTTP 0.9 style where the the reponse is sent without the headers. Actually, it is turning out to be true: http://ftp.ics.uci.edu/pub/ietf/http/rfc1945.html#Response According to HTTP 1.0, When a request is Simple-Request, it means a VERB URL (without a version) and it generally corresponds to HTTP 0.9 And when a server receives such a Simple-Request, it sends a Simple-Response where it does not send the headers back. I think, the same is exhibited by other Servers as well www.google.com, www.mozilla.org where for Invalid Request without version, you are sending a Simple-Request (HTTP 0.9) style and getting the corresponding response. Given these, any assumption that servers no longer support HTTP/0.9 becomes false. So nuking it will require some thought. -- Senthil -- From foom at fuhm.net Thu Dec 16 17:21:37 2010 From: foom at fuhm.net (James Y Knight) Date: Thu, 16 Dec 2010 11:21:37 -0500 Subject: [Python-Dev] Remove HTTP 0.9 support In-Reply-To: <20101216081404.GA4603@rubuntu> References: <20101215193940.31262534@pitrou.net> <4D092C0B.8090300@g.nevcal.com> <20101215222510.0b8aa7e7@pitrou.net> <4D093F46.6050106@g.nevcal.com> <20101216081404.GA4603@rubuntu> Message-ID: <892B9BB4-C2F4-43FF-9CA4-98A2A8A51EDA@fuhm.net> On Dec 16, 2010, at 3:14 AM, Senthil Kumaran wrote: > Even HTTP 0.9 says that response SHOULD start with status line, but > gives a suggestion that clients can "tolerate" bad server server > behaviors when they don't send the status line and in that the case > response is the body. > > http://www.w3.org/Protocols/HTTP/Response.html > > So, It cannot be associated with the behavior "most webservers", back > then and even more so now. Actually no. That document is describing almost-HTTP 1.0. Here is the actual document you were looking for: http://www.w3.org/Protocols/HTTP/AsImplemented.html HTTP 0.9 had no headers, no status line, nothing but "GET $url " and a stream of data in response. James From orsenthil at gmail.com Thu Dec 16 17:56:13 2010 From: orsenthil at gmail.com (Senthil Kumaran) Date: Fri, 17 Dec 2010 00:56:13 +0800 Subject: [Python-Dev] Remove HTTP 0.9 support In-Reply-To: <892B9BB4-C2F4-43FF-9CA4-98A2A8A51EDA@fuhm.net> References: <20101215193940.31262534@pitrou.net> <4D092C0B.8090300@g.nevcal.com> <20101215222510.0b8aa7e7@pitrou.net> <4D093F46.6050106@g.nevcal.com> <20101216081404.GA4603@rubuntu> <892B9BB4-C2F4-43FF-9CA4-98A2A8A51EDA@fuhm.net> Message-ID: <20101216165613.GB7964@rubuntu> On Thu, Dec 16, 2010 at 11:21:37AM -0500, James Y Knight wrote: > > Even HTTP 0.9 says that response SHOULD start with status line, but > > gives a suggestion that clients can "tolerate" bad server server > > behaviors when they don't send the status line and in that the case > > response is the body. > > > > http://www.w3.org/Protocols/HTTP/Response.html > > > > So, It cannot be associated with the behavior "most webservers", back > > then and even more so now. > > Actually no. That document is describing almost-HTTP 1.0. Yeah. I know it was almost-HTTP 1.0, but the same docs say that if protocol version was not specified, it is assumed to be 0.9. So, I thought it was a good reference point to understand the behavior. > Here is the actual document you were looking for: > http://www.w3.org/Protocols/HTTP/AsImplemented.html > > HTTP 0.9 had no headers, no status line, nothing but "GET $url " and a stream of data in response. Actually, you are right. I seems be be actual defined behavior of HTTP 0.9. As explained in that above doc and also in RFC 1945 Request section. -- Senthil From orsenthil at gmail.com Thu Dec 16 17:58:00 2010 From: orsenthil at gmail.com (Senthil Kumaran) Date: Fri, 17 Dec 2010 00:58:00 +0800 Subject: [Python-Dev] Remove HTTP 0.9 support In-Reply-To: <201012161652.43949.nd@perlig.de> References: <20101215193940.31262534@pitrou.net> <201012160742.08474@news.perlig.de> <20101216152305.13b1fd3a@pitrou.net> <201012161652.43949.nd@perlig.de> Message-ID: <20101216165800.GC7964@rubuntu> On Thu, Dec 16, 2010 at 04:52:43PM +0100, Andr? Malo wrote: > HTTP/0.9 doesn't *have* a version string. > > GET /foo > > is a HTTP/0.9 request. > > GET /foo HTTP/0.9 > > isn't actually (it's a paradoxon, alright ;). It simply isn't a valid HTTP > request, which would demand a 505 response. Yes, this is an important point. Many webservers seem to exhibit the HTTP 0.9 response behaviors when you don't specify the version. -- Senthil From solipsis at pitrou.net Thu Dec 16 18:02:45 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 16 Dec 2010 18:02:45 +0100 Subject: [Python-Dev] Remove HTTP 0.9 support In-Reply-To: <20101216165214.GA7964@rubuntu> References: <20101215193940.31262534@pitrou.net> <4D092C0B.8090300@g.nevcal.com> <20101215222510.0b8aa7e7@pitrou.net> <4D093F46.6050106@g.nevcal.com> <20101215232927.475f925c@pitrou.net> <20101216081456.GB4603@rubuntu> <1292505637.3689.0.camel@localhost.localdomain> <20101216165214.GA7964@rubuntu> Message-ID: <20101216180245.19251acf@pitrou.net> On Fri, 17 Dec 2010 00:52:14 +0800 Senthil Kumaran wrote: > Actually, it is turning out to be true: > > http://ftp.ics.uci.edu/pub/ietf/http/rfc1945.html#Response > > According to HTTP 1.0, When a request is Simple-Request, it means a > VERB URL (without a version) and it generally corresponds to HTTP 0.9 > And when a server receives such a Simple-Request, it sends a > Simple-Response where it does not send the headers back. > > I think, the same is exhibited by other Servers as well > www.google.com, www.mozilla.org where for Invalid Request without > version, you are sending a Simple-Request (HTTP 0.9) style and getting > the corresponding response. Yes, but only error or redirect responses: $ nc www.google.fr 80 GET / HTTP/1.0 302 Found Location: http://www.google.fr/ [etc.] $ nc www.mozilla.org 80 GET / 403 Forbidden Forbidden
You don't have permission to access /error/noindex.html on this server.
That's quite understandable, since most HTTP servers will expect a "host" header to know which site is actually desired. So a HTTP 0.9 client sending Simple-Requests has very little chance of being useful these days. Regards Antoine. From guido at python.org Thu Dec 16 18:34:09 2010 From: guido at python.org (Guido van Rossum) Date: Thu, 16 Dec 2010 09:34:09 -0800 Subject: [Python-Dev] Remove HTTP 0.9 support In-Reply-To: <20101216180245.19251acf@pitrou.net> References: <20101215193940.31262534@pitrou.net> <4D092C0B.8090300@g.nevcal.com> <20101215222510.0b8aa7e7@pitrou.net> <4D093F46.6050106@g.nevcal.com> <20101215232927.475f925c@pitrou.net> <20101216081456.GB4603@rubuntu> <1292505637.3689.0.camel@localhost.localdomain> <20101216165214.GA7964@rubuntu> <20101216180245.19251acf@pitrou.net> Message-ID: All this talk of modern servers that also still support HTTP/0.9 is irrelevant. Unless anybody knows of a server that *only* supports HTTP 0.9 (and that's relevant to users of httplib) let's please kill support in the client. -- --Guido van Rossum (python.org/~guido) From exarkun at twistedmatrix.com Thu Dec 16 19:30:18 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Thu, 16 Dec 2010 18:30:18 -0000 Subject: [Python-Dev] Remove HTTP 0.9 support In-Reply-To: <20101216180245.19251acf@pitrou.net> References: <20101215193940.31262534@pitrou.net> <4D092C0B.8090300@g.nevcal.com> <20101215222510.0b8aa7e7@pitrou.net> <4D093F46.6050106@g.nevcal.com> <20101215232927.475f925c@pitrou.net> <20101216081456.GB4603@rubuntu> <1292505637.3689.0.camel@localhost.localdomain> <20101216165214.GA7964@rubuntu> <20101216180245.19251acf@pitrou.net> Message-ID: <20101216183018.2058.1108731637.divmod.xquotient.281@localhost.localdomain> On 05:02 pm, solipsis at pitrou.net wrote: >On Fri, 17 Dec 2010 00:52:14 +0800 >Senthil Kumaran wrote: >>Actually, it is turning out to be true: >> >>http://ftp.ics.uci.edu/pub/ietf/http/rfc1945.html#Response >> >>According to HTTP 1.0, When a request is Simple-Request, it means a >>VERB URL (without a version) and it generally corresponds to HTTP 0.9 >>And when a server receives such a Simple-Request, it sends a >>Simple-Response where it does not send the headers back. >> >>I think, the same is exhibited by other Servers as well >>www.google.com, www.mozilla.org where for Invalid Request without >>version, you are sending a Simple-Request (HTTP 0.9) style and getting >>the corresponding response. > >Yes, but only error or redirect responses: > >$ nc www.google.fr 80 >GET / >HTTP/1.0 302 Found >Location: http://www.google.fr/ >[etc.] Note that by using `nc` to test this, you're already generating an illegal request (unless you're doing something very special with your keyboard ;). When you hit return, nc will send \n. However, lines are delimited by \r\n in HTTP. I doubt this makes a difference to the point being discussed, but it _could_. I suggest performing your tests with telnet, instead. Jean-Paul From nd at perlig.de Thu Dec 16 19:48:11 2010 From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=) Date: Thu, 16 Dec 2010 19:48:11 +0100 Subject: [Python-Dev] Remove HTTP 0.9 support In-Reply-To: References: <20101215193940.31262534@pitrou.net> <201012161652.43949.nd@perlig.de> Message-ID: <201012161948.11644@news.perlig.de> * Fred Drake wrote: > On Thu, Dec 16, 2010 at 10:52 AM, Andr? Malo wrote: > > I'd vote for removing it from the client code and keeping it in the > > server. > > If it must be maintained anywhere, it should be in the client, > according to the basic principle of "accept what you can, generate > carefully." *scratching head* exactly why I would keep support in the server. nd -- package Hacker::Perl::Another::Just;print qq~@{[reverse split/::/ =>__PACKAGE__]}~; # Andr? Malo # http://www.perlig.de # From fdrake at acm.org Thu Dec 16 19:47:49 2010 From: fdrake at acm.org (Fred Drake) Date: Thu, 16 Dec 2010 13:47:49 -0500 Subject: [Python-Dev] Remove HTTP 0.9 support In-Reply-To: <20101216183018.2058.1108731637.divmod.xquotient.281@localhost.localdomain> References: <20101215193940.31262534@pitrou.net> <4D092C0B.8090300@g.nevcal.com> <20101215222510.0b8aa7e7@pitrou.net> <4D093F46.6050106@g.nevcal.com> <20101215232927.475f925c@pitrou.net> <20101216081456.GB4603@rubuntu> <1292505637.3689.0.camel@localhost.localdomain> <20101216165214.GA7964@rubuntu> <20101216180245.19251acf@pitrou.net> <20101216183018.2058.1108731637.divmod.xquotient.281@localhost.localdomain> Message-ID: On Thu, Dec 16, 2010 at 1:30 PM, wrote: > I doubt this makes a difference to the point being discussed, but it > _could_. ?I suggest performing your tests with telnet, instead. I received similar results using telnet earlier today. ? -Fred -- Fred L. Drake, Jr.? ? "A storm broke loose in my mind."? --Albert Einstein From greg.ewing at canterbury.ac.nz Thu Dec 16 23:15:24 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 17 Dec 2010 11:15:24 +1300 Subject: [Python-Dev] Remove HTTP 0.9 support In-Reply-To: <20101216165214.GA7964@rubuntu> References: <20101215193940.31262534@pitrou.net> <4D092C0B.8090300@g.nevcal.com> <20101215222510.0b8aa7e7@pitrou.net> <4D093F46.6050106@g.nevcal.com> <20101215232927.475f925c@pitrou.net> <20101216081456.GB4603@rubuntu> <1292505637.3689.0.camel@localhost.localdomain> <20101216165214.GA7964@rubuntu> Message-ID: <4D0A8F7C.4080007@canterbury.ac.nz> Senthil Kumaran wrote: > Given these, any assumption that servers no longer support HTTP/0.9 > becomes false. But as long as httplib only sends requests with a version number >= 1.0, it should be able to expect headers in the response, shouldn't it? -- Greg From stefan_ml at behnel.de Fri Dec 17 08:19:07 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 17 Dec 2010 08:19:07 +0100 Subject: [Python-Dev] nonlocal x = value Message-ID: Hi, it seems that Py3 doesn't support setting a "nonlocal" value as part of the "nonlocal" command Python 3.2a4+ (py3k:86480, Nov 16 2010, 16:43:22) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> def x(): ... y = 5 ... def f(): ... nonlocal y = 6 File " ", line 4 nonlocal y = 6 ^ SyntaxError: invalid syntax even though the PEP clearly describes this feature. http://www.python.org/dev/peps/pep-3104/#proposed-solution Either the PEP or the implementation should be updated. Personally, I think the PEP should be changed as I think that the syntax complicates the grammar more than it's worth. Also, the moratorium applies here, given that Py3.1 does not implement this. Comments? Stefan From benjamin at python.org Fri Dec 17 17:08:06 2010 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 17 Dec 2010 10:08:06 -0600 Subject: [Python-Dev] nonlocal x = value In-Reply-To: References: Message-ID: 2010/12/17 Stefan Behnel : > Hi, > > it seems that Py3 doesn't support setting a "nonlocal" value as part of the > "nonlocal" command > > ? ?Python 3.2a4+ (py3k:86480, Nov 16 2010, 16:43:22) > ? ?[GCC 4.4.3] on linux2 > ? ?Type "help", "copyright", "credits" or "license" for more information. > ? ?>>> def x(): > ? ?... ? y = 5 > ? ?... ? def f(): > ? ?... ? ? nonlocal y = 6 > ? ? ?File " ", line 4 > ? ? ? ?nonlocal y = 6 > ? ? ? ? ? ? ? ? ? ^ > ? ?SyntaxError: invalid syntax > > even though the PEP clearly describes this feature. > > http://www.python.org/dev/peps/pep-3104/#proposed-solution > > Either the PEP or the implementation should be updated. Personally, I think > the PEP should be changed as I think that the syntax complicates the grammar > more than it's worth. Also, the moratorium applies here, given that Py3.1 > does not implement this. > > Comments? There's a issue and a patch for this somewhere. I personally don't care; it's not too painful to write two lines. -- Regards, Benjamin From lvh at laurensvh.be Fri Dec 17 17:52:17 2010 From: lvh at laurensvh.be (Laurens Van Houtven) Date: Fri, 17 Dec 2010 17:52:17 +0100 Subject: [Python-Dev] nonlocal x = value In-Reply-To: References: Message-ID: +1 for throwing it out of the PEP. Assignment is a thing, nonlocal/global is a thing, don't mix them up :) (That in addition to the grammar cleanliness argument Stephan already made) cheers lvh. From status at bugs.python.org Fri Dec 17 18:07:04 2010 From: status at bugs.python.org (Python tracker) Date: Fri, 17 Dec 2010 18:07:04 +0100 (CET) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20101217170704.4F0151CCAD@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2010-12-10 - 2010-12-17) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 2537 (-20) closed 19971 (+72) total 22508 (+52) Open issues with patches: 1070 Issues opened (25) ================== #9232: Allow trailing comma in any function argument list. http://bugs.python.org/issue9232 reopened by ncoghlan #10642: Improve the error message of addpackage() (site.py) for defect http://bugs.python.org/issue10642 reopened by r.david.murray #10679: "make altinstall" may clobber OS provided scripts http://bugs.python.org/issue10679 opened by ncoghlan #10680: argparse: titles and add_mutually_exclusive_group don't mix (e http://bugs.python.org/issue10680 opened by Mads.Michelsen #10684: Shutil.move deletes file/folder in windows while renaming http://bugs.python.org/issue10684 opened by harish #10685: trace does nto ignore --ignore-module http://bugs.python.org/issue10685 opened by RusiMody #10686: email.Generator should use unknown-8bit encoded words for head http://bugs.python.org/issue10686 opened by r.david.murray #10690: IDLE Crash when running/saving Module http://bugs.python.org/issue10690 opened by David_Anon #10694: zipfile.py end of central directory detection not robust http://bugs.python.org/issue10694 opened by KevinH #10697: host and port attributes not documented well in function urlli http://bugs.python.org/issue10697 opened by JTMoon79 #10701: Error pickling a dict http://bugs.python.org/issue10701 opened by belopolsky #10702: bytes and bytearray methods are not documented http://bugs.python.org/issue10702 opened by belopolsky #10708: Misc/porting should be folded in to the development FAQ http://bugs.python.org/issue10708 opened by pitrou #10709: Misc/AIX-NOTES needs updating http://bugs.python.org/issue10709 opened by pitrou #10711: Rip out HTTP 0.9 client support http://bugs.python.org/issue10711 opened by pitrou #10712: 2to3 fixer for deprecated unittest method names http://bugs.python.org/issue10712 opened by ezio.melotti #10713: re module doesn't describe string boundaries for \b http://bugs.python.org/issue10713 opened by ralph.corderoy #10715: uninformative error message http://bugs.python.org/issue10715 opened by Phillip.M.Feldman #10716: Modernize pydoc to use CSS http://bugs.python.org/issue10716 opened by rhettinger #10717: Multiprocessing module Pickling unPickling issues http://bugs.python.org/issue10717 opened by dimitrios #10720: test_threadsignals hang on FreeBSD 6.4 http://bugs.python.org/issue10720 opened by pitrou #10721: Remove HTTP 0.9 server support http://bugs.python.org/issue10721 opened by pitrou #10722: IDLE's subprocess didnit make connection ..... Python 2.7 http://bugs.python.org/issue10722 opened by plovet #10723: Undocumented built-in exceptions http://bugs.python.org/issue10723 opened by flashk #10725: Better cache instrumentation http://bugs.python.org/issue10725 opened by rhettinger Most recent 15 issues with no replies (15) ========================================== #10720: test_threadsignals hang on FreeBSD 6.4 http://bugs.python.org/issue10720 #10717: Multiprocessing module Pickling unPickling issues http://bugs.python.org/issue10717 #10713: re module doesn't describe string boundaries for \b http://bugs.python.org/issue10713 #10709: Misc/AIX-NOTES needs updating http://bugs.python.org/issue10709 #10708: Misc/porting should be folded in to the development FAQ http://bugs.python.org/issue10708 #10701: Error pickling a dict http://bugs.python.org/issue10701 #10686: email.Generator should use unknown-8bit encoded words for head http://bugs.python.org/issue10686 #10685: trace does nto ignore --ignore-module http://bugs.python.org/issue10685 #10684: Shutil.move deletes file/folder in windows while renaming http://bugs.python.org/issue10684 #10666: OS X installer variants have confusing readline differences http://bugs.python.org/issue10666 #10664: xml.sax.expatreader should support namespace prefixes http://bugs.python.org/issue10664 #10663: configure shouldn't set a default OPT http://bugs.python.org/issue10663 #10657: os.lstat/os.stat/os.fstat don't set st_dev (st_rdev) on Window http://bugs.python.org/issue10657 #10656: "Out of tree" build fails on AIX 5.3 http://bugs.python.org/issue10656 #10646: ntpath.samefile doesn't work for hard links http://bugs.python.org/issue10646 Most recent 15 issues waiting for review (15) ============================================= #10725: Better cache instrumentation http://bugs.python.org/issue10725 #10723: Undocumented built-in exceptions http://bugs.python.org/issue10723 #10721: Remove HTTP 0.9 server support http://bugs.python.org/issue10721 #10712: 2to3 fixer for deprecated unittest method names http://bugs.python.org/issue10712 #10711: Rip out HTTP 0.9 client support http://bugs.python.org/issue10711 #10694: zipfile.py end of central directory detection not robust http://bugs.python.org/issue10694 #10673: multiprocess.Process join method - timeout indistinguishable f http://bugs.python.org/issue10673 #10671: urllib2 redirect to another host doesn't work http://bugs.python.org/issue10671 #10665: Expand unicodedata module documentation http://bugs.python.org/issue10665 #10657: os.lstat/os.stat/os.fstat don't set st_dev (st_rdev) on Window http://bugs.python.org/issue10657 #10653: test_time test_strptime fails on windows http://bugs.python.org/issue10653 #10652: test___all_ + test_tcl fails (Windows installed binary) http://bugs.python.org/issue10652 #10648: Extend peepholer to reverse loads or stores instead of build/u http://bugs.python.org/issue10648 #10639: reindent.py converts newlines to platform default http://bugs.python.org/issue10639 #10626: Bad interaction between test_logging and test_concurrent_futur http://bugs.python.org/issue10626 Top 10 most discussed issues (10) ================================= #10716: Modernize pydoc to use CSS http://bugs.python.org/issue10716 12 msgs #6791: httplib read status memory usage http://bugs.python.org/issue6791 11 msgs #9234: argparse: aliases for positional arguments (subparsers) http://bugs.python.org/issue9234 11 msgs #10254: unicodedata.normalize('NFC', s) regression http://bugs.python.org/issue10254 11 msgs #10711: Rip out HTTP 0.9 client support http://bugs.python.org/issue10711 11 msgs #9232: Allow trailing comma in any function argument list. http://bugs.python.org/issue9232 8 msgs #9264: trace.py documentation is incomplete http://bugs.python.org/issue9264 8 msgs #9286: email.utils.parseaddr returns garbage for invalid input http://bugs.python.org/issue9286 5 msgs #10515: csv sniffer does not recognize quotes at the end of line http://bugs.python.org/issue10515 5 msgs #10642: Improve the error message of addpackage() (site.py) for defect http://bugs.python.org/issue10642 5 msgs Issues closed (60) ================== #2576: httplib read() very slow due to lack of socket buffer http://bugs.python.org/issue2576 closed by pitrou #4236: Crash when importing builtin module during interpreter shutdow http://bugs.python.org/issue4236 closed by r.david.murray #4402: os.getenv('PATH') return different result between 2.5 and 3. http://bugs.python.org/issue4402 closed by r.david.murray #4661: email.parser: impossible to read messages encoded in a differe http://bugs.python.org/issue4661 closed by r.david.murray #4766: email documentation needs to be precise about strings/bytes http://bugs.python.org/issue4766 closed by r.david.murray #6454: Add "example" keyword argument to optparse constructor http://bugs.python.org/issue6454 closed by rhettinger #6559: add pass_fds paramter to subprocess.Popen() http://bugs.python.org/issue6559 closed by gregory.p.smith #7183: did 2.6.3 regress for some uses of the __doc__ property? http://bugs.python.org/issue7183 closed by r.david.murray #7213: subprocess leaks open file descriptors between Popen instances http://bugs.python.org/issue7213 closed by gregory.p.smith #8127: Add link to PortingPythonToPy3k to What's New documentation http://bugs.python.org/issue8127 closed by r.david.murray #8753: Py_ReprEnter and Py_ReprLeave are undocumented http://bugs.python.org/issue8753 closed by stutzbach #8844: Condition.wait() doesn't raise KeyboardInterrupt http://bugs.python.org/issue8844 closed by pitrou #9048: no OS X buildbots in the stable list http://bugs.python.org/issue9048 closed by r.david.murray #9162: License for multiprocessing files http://bugs.python.org/issue9162 closed by r.david.murray #9721: urlparse.urljoin() cuts off last base character with semicolon http://bugs.python.org/issue9721 closed by orsenthil #10188: tempfile.TemporaryDirectory may throw errors at shutdown http://bugs.python.org/issue10188 closed by ncoghlan #10454: Clarify compileall command-line options http://bugs.python.org/issue10454 closed by r.david.murray #10509: PyTokenizer_FindEncoding can lead to a segfault if bad charact http://bugs.python.org/issue10509 closed by Trundle #10534: difflib.SequenceMatcher: expose junk sets, deprecate undocumen http://bugs.python.org/issue10534 closed by terry.reedy #10559: NameError in tutorial/interpreter http://bugs.python.org/issue10559 closed by r.david.murray #10618: regression in subprocess.call() command quoting http://bugs.python.org/issue10618 closed by r.david.murray #10627: Remove usage of deprecated configparser.ConfigParser class in http://bugs.python.org/issue10627 closed by lukasz.langa #10634: Windows timezone changes not reflected by time.localtime http://bugs.python.org/issue10634 closed by eric.pruitt #10667: collections.Counter object in C http://bugs.python.org/issue10667 closed by rhettinger #10670: Provide search scope limits http://bugs.python.org/issue10670 closed by r.david.murray #10674: Unused dictmaker symbol in 2.* grammar http://bugs.python.org/issue10674 closed by benjamin.peterson #10675: unittest should have an assertChanges context manager http://bugs.python.org/issue10675 closed by r.david.murray #10676: Confusing note in Numeric Types http://bugs.python.org/issue10676 closed by georg.brandl #10677: "make altinstall" includes ABI codes in versioned executable n http://bugs.python.org/issue10677 closed by barry #10678: email.utils.mktime_tz Giving wrong result , by ignoring Timezo http://bugs.python.org/issue10678 closed by r.david.murray #10681: PySlice_GetIndices() signature changed http://bugs.python.org/issue10681 closed by loewis #10682: With '*args' or even bare '*' in def/call argument list, trail http://bugs.python.org/issue10682 closed by loewis #10683: PreLinkEvent error under VC2010 http://bugs.python.org/issue10683 closed by krisvale #10687: Python fails to install with empty ABI flags http://bugs.python.org/issue10687 closed by barry #10688: pydoc removes lib directory http://bugs.python.org/issue10688 closed by r.david.murray #10689: _scproxy extension is NOT build http://bugs.python.org/issue10689 closed by ned.deily #10691: make testall no longer working on Darwin http://bugs.python.org/issue10691 closed by r.david.murray #10692: imap lib server compabilities http://bugs.python.org/issue10692 closed by silversky #10693: error in documentation of distutils.archive_util.make_zipfile http://bugs.python.org/issue10693 closed by eric.araujo #10695: telnetlib.Telnet port number int/str inconsistency http://bugs.python.org/issue10695 closed by r.david.murray #10696: port not split in function urllib.parse.urlsplit http://bugs.python.org/issue10696 closed by r.david.murray #10698: doctest load_tests() typo http://bugs.python.org/issue10698 closed by r.david.murray #10699: fix incorrect help doc with time.tzset http://bugs.python.org/issue10699 closed by r.david.murray #10700: python pickle.dumps AssertionError http://bugs.python.org/issue10700 closed by belopolsky #10703: Regex 0.1.20101210 http://bugs.python.org/issue10703 closed by belopolsky #10704: Regex 0.1.20101210 Python 3.1 install problem Mac OS X 10.6.5 http://bugs.python.org/issue10704 closed by belopolsky #10705: HTTPConnection.set_debuglevel has no information about level r http://bugs.python.org/issue10705 closed by r.david.murray #10706: kill runtests.sh http://bugs.python.org/issue10706 closed by pitrou #10707: compileall is broken http://bugs.python.org/issue10707 closed by r.david.murray #10710: Is Misc/setuid-prog.c still needed? http://bugs.python.org/issue10710 closed by pitrou #10714: httpserver request length http://bugs.python.org/issue10714 closed by pitrou #10718: Idle 2.7.1 from 64-bit installer crashes on OS X when source i http://bugs.python.org/issue10718 closed by ned.deily #10719: compileall no longer warns when cli arguments name non-existen http://bugs.python.org/issue10719 closed by r.david.murray #10724: socket.close close telnet with RST http://bugs.python.org/issue10724 closed by r.david.murray #10726: pydoc: don???t display raw reST in keyword help http://bugs.python.org/issue10726 closed by georg.brandl #1459867: Message.as_string should use "mangle_from_=unixfrom"? http://bugs.python.org/issue1459867 closed by r.david.murray #1078919: email.Header (via add_header) encodes non-ASCII content incorr http://bugs.python.org/issue1078919 closed by r.david.murray #1628205: socket.readline() interface doesn't handle EINTR properly http://bugs.python.org/issue1628205 closed by pitrou #1243654: Faster output if message already has a boundary http://bugs.python.org/issue1243654 closed by r.david.murray #775964: fix test_grp failing when NIS entries present http://bugs.python.org/issue775964 closed by r.david.murray From alexander.belopolsky at gmail.com Fri Dec 17 18:13:47 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Fri, 17 Dec 2010 12:13:47 -0500 Subject: [Python-Dev] nonlocal x = value In-Reply-To: References: Message-ID: On Fri, Dec 17, 2010 at 11:52 AM, Laurens Van Houtven wrote: > +1 for throwing it out of the PEP. Assignment is a thing, > nonlocal/global is a thing, don't mix them up :) (That in addition to > the grammar cleanliness argument Stephan already made) Another +1 for the same reasons. Also, since global does not allow assignment, neither should nonlocal. From benjamin at python.org Fri Dec 17 18:33:50 2010 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 17 Dec 2010 11:33:50 -0600 Subject: [Python-Dev] nonlocal x = value In-Reply-To: References: Message-ID: 2010/12/17 Alexander Belopolsky : > On Fri, Dec 17, 2010 at 11:52 AM, Laurens Van Houtven wrote: >> +1 for throwing it out of the PEP. Assignment is a thing, >> nonlocal/global is a thing, don't mix them up :) (That in addition to >> the grammar cleanliness argument Stephan already made) > > Another +1 for the same reasons. ?Also, since global does not allow > assignment, neither should nonlocal. Note that the PEP stated that global would also be extended. -- Regards, Benjamin From alexander.belopolsky at gmail.com Fri Dec 17 19:00:53 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Fri, 17 Dec 2010 13:00:53 -0500 Subject: [Python-Dev] nonlocal x = value In-Reply-To: References: Message-ID: On Fri, Dec 17, 2010 at 12:33 PM, Benjamin Peterson wrote: .. >> Another +1 for the same reasons. ?Also, since global does not allow >> assignment, neither should nonlocal. > > Note that the PEP stated that global would also be extended. I missed that, so for future reference, the PEP says: """ A shorthand form is also permitted, in which nonlocal is prepended to an assignment or augmented assignment: nonlocal x = 3 The above has exactly the same meaning as nonlocal x; x = 3. (Guido supports a similar form of the global statement.) """ and refers to Guido's post at http://mail.python.org/pipermail/python-3000/2006-November/004166.html In any case, the relevant issue is http://bugs.python.org/issue4199 and it should probably be marked as "after moratorium". Meanwhile an implementation status note can be added to the PEP to avoid this issue being brought up again. From python at mrabarnett.plus.com Sat Dec 18 01:08:47 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 18 Dec 2010 00:08:47 +0000 Subject: [Python-Dev] Locale-specific formatting Message-ID: <4D0BFB8F.2040301@mrabarnett.plus.com> I had a thought about locale-specific formatting. Currently, when we want to do locale-specific formatting we use the locale module like this: >>> locale.format("%d", 12345, grouping=False) '12345' >>> locale.format("%d", 12345, grouping=True) '12,345' This makes it harder to use more than one locale at a time, or one which is different from the default. My thought was that we could specify a locale in the format specification mini-language and the parameter list of str.format, something like this: >>> loc = locale.getlocale() >>> "{0:@1}".format(12345, loc) '12345' >>> "{0:, at 1}".format(12345, loc) '12,345' ... >>> "UK says {value:,.1f at uk} and France says {value:,.1f at france}".format(value=12345, uk=uk_loc, france=france_loc) 'UK says 1,234.5 and France says 1 234,5' Comments? From victor.stinner at haypocalc.com Sat Dec 18 01:55:36 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Sat, 18 Dec 2010 01:55:36 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable Message-ID: <4D0C0688.1080108@haypocalc.com> Hi, In 2008, I proposed a patch to raise a Python exception on SIGSEVG signal. In some cases, it's possible to catch the exception, log it, and continue the execution. But because in some cases, the Python internal state is corrupted, the idea was rejected (see the issue #3999). Someone asked me to display the Python backtrace on SIGSEGV, instead of raising an exception. I implemented this idea in issue #8863. After 9 versions, I think that the patch is ready for inclusion. It catchs SIGSEGV, SIGFPE, SIGBUS and SIGILL signals, and also display the Python backtrace on fatal errors. Because some operating systems have their own fault handler (eg. Ubuntu with apport), Dave Malcolm asked me to add an option disable the Python handler. That's why I added the PYTHONNOFAULTHANDLER environment variable (we all love long variable names!). For an OS vendor, it looks like an environment variable is more practical than a command line option. Eg. Mandriva sets PYTHONDONTWRITEBYTECODE for the whole system. Raymond Hettinger asked me on IRC to write an email about the new environment variable, so here is the question: do you agree to add the new variable? Victor From rdmurray at bitdance.com Sat Dec 18 05:00:28 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 17 Dec 2010 23:00:28 -0500 Subject: [Python-Dev] Locale-specific formatting In-Reply-To: <4D0BFB8F.2040301@mrabarnett.plus.com> References: <4D0BFB8F.2040301@mrabarnett.plus.com> Message-ID: <20101218040028.08CBD237968@kimball.webabinitio.net> On Sat, 18 Dec 2010 00:08:47 +0000, MRAB wrote: > I had a thought about locale-specific formatting. > > Currently, when we want to do locale-specific formatting we use the > locale module like this: > > >>> locale.format("%d", 12345, grouping=False) > '12345' > >>> locale.format("%d", 12345, grouping=True) > '12,345' > > This makes it harder to use more than one locale at a time, or one > which is different from the default. > > My thought was that we could specify a locale in the format > specification mini-language and the parameter list of str.format, > something like this: > > >>> loc = locale.getlocale() > >>> "{0:@1}".format(12345, loc) > '12345' > >>> "{0:, at 1}".format(12345, loc) > '12,345' > ... > >>> "UK says {value:,.1f at uk} and France says > {value:,.1f at france}".format(value=12345, uk=uk_loc, france=france_loc) > 'UK says 1,234.5 and France says 1 234,5' > > Comments? There was at least one long thread on this on python-ideas. Probably worth finding and reading before proceeding with a new discussion... :) I think it was part of the discussion that ultimately led to PEP 378. -- R. David Murray www.bitdance.com From martin at v.loewis.de Sat Dec 18 10:26:34 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 18 Dec 2010 10:26:34 +0100 Subject: [Python-Dev] Locale-specific formatting In-Reply-To: <4D0BFB8F.2040301@mrabarnett.plus.com> References: <4D0BFB8F.2040301@mrabarnett.plus.com> Message-ID: <4D0C7E4A.3010302@v.loewis.de> > Comments? How do you implement that? In particular, how do you retrieve information for different locales in a single program? Regards, Martin From g.brandl at gmx.net Sat Dec 18 12:37:09 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 18 Dec 2010 12:37:09 +0100 Subject: [Python-Dev] r87188 - python/branches/py3k/Doc/library/test.rst In-Reply-To: <20101212182525.C5C6DEE9F3@mail.python.org> References: <20101212182525.C5C6DEE9F3@mail.python.org> Message-ID: I'd rather keep this a note. We don't want to use warnings except for cases where there is a possibility of security implications or crashes. Georg Am 12.12.2010 19:25, schrieb antoine.pitrou: > Author: antoine.pitrou > Date: Sun Dec 12 19:25:25 2010 > New Revision: 87188 > > Log: > Make this a warning and fix indentation > > > > Modified: > python/branches/py3k/Doc/library/test.rst > > Modified: python/branches/py3k/Doc/library/test.rst > ============================================================================== > --- python/branches/py3k/Doc/library/test.rst (original) > +++ python/branches/py3k/Doc/library/test.rst Sun Dec 12 19:25:25 2010 > @@ -5,12 +5,12 @@ > :synopsis: Regression tests package containing the testing suite for Python. > .. sectionauthor:: Brett Cannon > > -.. note:: > - The :mod:`test` package is meant for internal use by Python only. It is > - documented for the benefit of the core developers of Python. Any use of > - this package outside of Python's standard library is discouraged as code > - mentioned here can change or be removed without notice between releases of > - Python. > +.. warning:: > + The :mod:`test` package is meant for internal use by Python only. It is > + documented for the benefit of the core developers of Python. Any use of > + this package outside of Python's standard library is discouraged as code > + mentioned here can change or be removed without notice between releases of > + Python. > > > The :mod:`test` package contains all regression tests for Python as well as the From solipsis at pitrou.net Sat Dec 18 13:07:34 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 18 Dec 2010 13:07:34 +0100 Subject: [Python-Dev] r87188 - python/branches/py3k/Doc/library/test.rst References: <20101212182525.C5C6DEE9F3@mail.python.org> Message-ID: <20101218130734.1f2f471a@pitrou.net> On Sat, 18 Dec 2010 12:37:09 +0100 Georg Brandl wrote: > I'd rather keep this a note. We don't want to use warnings except for cases > where there is a possibility of security implications or crashes. Well, there'll be a crash as soon as someone relies on an API we decide to change or remove :) From g.brandl at gmx.net Sat Dec 18 13:11:09 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 18 Dec 2010 13:11:09 +0100 Subject: [Python-Dev] r87188 - python/branches/py3k/Doc/library/test.rst In-Reply-To: <20101218130734.1f2f471a@pitrou.net> References: <20101212182525.C5C6DEE9F3@mail.python.org> <20101218130734.1f2f471a@pitrou.net> Message-ID: Am 18.12.2010 13:07, schrieb Antoine Pitrou: > On Sat, 18 Dec 2010 12:37:09 +0100 > Georg Brandl wrote: >> I'd rather keep this a note. We don't want to use warnings except for cases >> where there is a possibility of security implications or crashes. > > Well, there'll be a crash as soon as someone relies on an API we decide > to change or remove :) Ah, but I meant a real crash, not an exception. Think marshal or ctypes. Georg From g.brandl at gmx.net Sat Dec 18 13:12:33 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 18 Dec 2010 13:12:33 +0100 Subject: [Python-Dev] nonlocal x = value In-Reply-To: References: Message-ID: Am 17.12.2010 17:52, schrieb Laurens Van Houtven: > +1 for throwing it out of the PEP. Assignment is a thing, > nonlocal/global is a thing, don't mix them up :) (That in addition to > the grammar cleanliness argument Stephan already made) The trouble is what to make of nonlocal x = 3, y Is it two nonlocal declarations or one with a tuple assignment? Georg From g.brandl at gmx.net Sat Dec 18 13:21:45 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 18 Dec 2010 13:21:45 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: <4D0C0688.1080108@haypocalc.com> References: <4D0C0688.1080108@haypocalc.com> Message-ID: Am 18.12.2010 01:55, schrieb Victor Stinner: > Hi, > > In 2008, I proposed a patch to raise a Python exception on SIGSEVG > signal. In some cases, it's possible to catch the exception, log it, and > continue the execution. But because in some cases, the Python internal > state is corrupted, the idea was rejected (see the issue #3999). > > Someone asked me to display the Python backtrace on SIGSEGV, instead of > raising an exception. I implemented this idea in issue #8863. After 9 > versions, I think that the patch is ready for inclusion. It catchs > SIGSEGV, SIGFPE, SIGBUS and SIGILL signals, and also display the Python > backtrace on fatal errors. Because some operating systems have their own > fault handler (eg. Ubuntu with apport), Dave Malcolm asked me to add an > option disable the Python handler. That's why I added the > PYTHONNOFAULTHANDLER environment variable (we all love long variable > names!). For an OS vendor, it looks like an environment variable is more > practical than a command line option. Eg. Mandriva sets > PYTHONDONTWRITEBYTECODE for the whole system. > > Raymond Hettinger asked me on IRC to write an email about the new > environment variable, so here is the question: do you agree to add the > new variable? I very much like having a traceback on (some) segmentation faults, but it's clear there needs to be a way to turn it off. An environment variable seems to be the obvious choice (for the reasons you stated for PYTHONDONTWRITEBYTECODE). Georg From solipsis at pitrou.net Sat Dec 18 13:25:37 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 18 Dec 2010 13:25:37 +0100 Subject: [Python-Dev] r87188 - python/branches/py3k/Doc/library/test.rst References: <20101212182525.C5C6DEE9F3@mail.python.org> <20101218130734.1f2f471a@pitrou.net> Message-ID: <20101218132537.649de53b@pitrou.net> On Sat, 18 Dec 2010 13:11:09 +0100 Georg Brandl wrote: > Am 18.12.2010 13:07, schrieb Antoine Pitrou: > > On Sat, 18 Dec 2010 12:37:09 +0100 > > Georg Brandl wrote: > >> I'd rather keep this a note. We don't want to use warnings except for cases > >> where there is a possibility of security implications or crashes. > > > > Well, there'll be a crash as soon as someone relies on an API we decide > > to change or remove :) > > Ah, but I meant a real crash, not an exception. Think marshal or ctypes. We can add one if necessary ;) Antoine. From foom at fuhm.net Sat Dec 18 14:50:49 2010 From: foom at fuhm.net (James Y Knight) Date: Sat, 18 Dec 2010 08:50:49 -0500 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: <4D0C0688.1080108@haypocalc.com> References: <4D0C0688.1080108@haypocalc.com> Message-ID: On Dec 17, 2010, at 7:55 PM, Victor Stinner wrote: > Hi, > > In 2008, I proposed a patch to raise a Python exception on SIGSEVG signal. In some cases, it's possible to catch the exception, log it, and continue the execution. But because in some cases, the Python internal state is corrupted, the idea was rejected (see the issue #3999). > > Someone asked me to display the Python backtrace on SIGSEGV, instead of raising an exception. I implemented this idea in issue #8863. After 9 versions, I think that the patch is ready for inclusion. It catchs SIGSEGV, SIGFPE, SIGBUS and SIGILL signals, and also display the Python backtrace on fatal errors. Because some operating systems have their own fault handler (eg. Ubuntu with apport), Dave Malcolm asked me to add an option disable the Python handler. I think instead of calling abort() to kill the process, you should: - install the signal handler with SA_NODEFER|SA_RESETHAND (or if sigaction is not present, explicitly reset the action to SIG_DFL and unblock first thing upon entering the handler), and then, - at the end of the handler, kill(getpid(), orig_signal) in order to abort the process. This has two advantages: 1) the process's exit code will actually show the correct signal, 2) it might let the OS fault handlers work properly as well -- I'm not sure. If it does, you may want to experiment with whether having or omitting SA_NODEFER gives a better backtrace (from the OS mechanism) in that case. If #2 actually works, you may not even need the env var, which would be nice. James From victor.stinner at haypocalc.com Sat Dec 18 14:57:12 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Sat, 18 Dec 2010 14:57:12 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: References: <4D0C0688.1080108@haypocalc.com> Message-ID: <4D0CBDB8.6010608@haypocalc.com> Le 18/12/2010 13:21, Georg Brandl a ?crit : > I very much like having a traceback on (some) segmentation faults, Why do you say "some" segmentation faults? > but it's clear there needs to be a way to turn it off. An environment variable > seems to be the obvious choice (for the reasons you stated for > PYTHONDONTWRITEBYTECODE). Amaury proposed a new function sys.setsegfaultenabled(). Such function can be used in a customized site module. I think that I will implement it in my next patch, before commiting the new feature. I suppose that we also need a function to get the status of the fault handler. I just don't like the name. I would prefer sys.setfaulthandlerenabled(), I maybe just sys.setfaulthandler(). As this feature is specific to CPython, should it be protected function? So: * sys._setfaulthandler(bool): enable/disable the fault handler * sys._getfaulthandler()->bool: get the status (enabled/disabled) of the fault handler Victor From phd at phd.pp.ru Sat Dec 18 16:33:31 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Sat, 18 Dec 2010 18:33:31 +0300 Subject: [Python-Dev] Locale-specific formatting In-Reply-To: <4D0BFB8F.2040301@mrabarnett.plus.com> References: <4D0BFB8F.2040301@mrabarnett.plus.com> Message-ID: <20101218153331.GA22017@phd.pp.ru> On Sat, Dec 18, 2010 at 12:08:47AM +0000, MRAB wrote: > This makes it harder to use more than one locale at a time This is quite a known problem, not specific to Python. Locale settings are global for a process, and this is one of the thousands reasons why locale is considered so horrible. ICU is perhaps the only way around the problem. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From g.brandl at gmx.net Sat Dec 18 17:23:12 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 18 Dec 2010 17:23:12 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: <4D0CBDB8.6010608@haypocalc.com> References: <4D0C0688.1080108@haypocalc.com> <4D0CBDB8.6010608@haypocalc.com> Message-ID: Am 18.12.2010 14:57, schrieb Victor Stinner: > Le 18/12/2010 13:21, Georg Brandl a ?crit : >> I very much like having a traceback on (some) segmentation faults, > Why do you say "some" segmentation faults? Well, without a closer I assume that for some crashes it's just not possible anymore for the Python interpreter to even print out the traceback? Georg From g.brandl at gmx.net Sat Dec 18 17:50:38 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 18 Dec 2010 17:50:38 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: <4D0CBDB8.6010608@haypocalc.com> References: <4D0C0688.1080108@haypocalc.com> <4D0CBDB8.6010608@haypocalc.com> Message-ID: Am 18.12.2010 14:57, schrieb Victor Stinner: > Le 18/12/2010 13:21, Georg Brandl a ?crit : >> I very much like having a traceback on (some) segmentation faults, > Why do you say "some" segmentation faults? >> but it's clear there needs to be a way to turn it off. An environment variable >> seems to be the obvious choice (for the reasons you stated for >> PYTHONDONTWRITEBYTECODE). > Amaury proposed a new function sys.setsegfaultenabled(). Such function > can be used in a customized site module. I think that I will implement > it in my next patch, before commiting the new feature. I suppose that we > also need a function to get the status of the fault handler. > > I just don't like the name. I would prefer sys.setfaulthandlerenabled(), > I maybe just sys.setfaulthandler(). As this feature is specific to > CPython, should it be protected function? So: > * sys._setfaulthandler(bool): enable/disable the fault handler > * sys._getfaulthandler()->bool: get the status (enabled/disabled) of > the fault handler In any case, this is coming pretty late; beta 2 is scheduled for this weekend, and even if this is something that only kicks in when all hope is lost anyway, it is a new feature. I should like to hear approval from a few more devs before I will let this go into 3.2. Georg From solipsis at pitrou.net Sat Dec 18 18:50:03 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 18 Dec 2010 18:50:03 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable References: <4D0C0688.1080108@haypocalc.com> <4D0CBDB8.6010608@haypocalc.com> Message-ID: <20101218185003.3298ba57@pitrou.net> On Sat, 18 Dec 2010 17:50:38 +0100 Georg Brandl wrote: > Am 18.12.2010 14:57, schrieb Victor Stinner: > > Le 18/12/2010 13:21, Georg Brandl a ?crit : > >> I very much like having a traceback on (some) segmentation faults, > > Why do you say "some" segmentation faults? > >> but it's clear there needs to be a way to turn it off. An environment variable > >> seems to be the obvious choice (for the reasons you stated for > >> PYTHONDONTWRITEBYTECODE). > > Amaury proposed a new function sys.setsegfaultenabled(). Such function > > can be used in a customized site module. I think that I will implement > > it in my next patch, before commiting the new feature. I suppose that we > > also need a function to get the status of the fault handler. > > > > I just don't like the name. I would prefer sys.setfaulthandlerenabled(), > > I maybe just sys.setfaulthandler(). As this feature is specific to > > CPython, should it be protected function? So: > > * sys._setfaulthandler(bool): enable/disable the fault handler > > * sys._getfaulthandler()->bool: get the status (enabled/disabled) of > > the fault handler > > In any case, this is coming pretty late; beta 2 is scheduled for this > weekend, and even if this is something that only kicks in when all hope > is lost anyway, it is a new feature. I should like to hear approval > from a few more devs before I will let this go into 3.2. I'm +1 on Victor's patch, although I don't think the sys function is useful at all. The env var is enough IMO. (I would even be for backporting it to the bugfix branches if there's no downside) Regards Antoine. From python at mrabarnett.plus.com Sat Dec 18 19:26:43 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 18 Dec 2010 18:26:43 +0000 Subject: [Python-Dev] Locale-specific formatting In-Reply-To: <4D0C7E4A.3010302@v.loewis.de> References: <4D0BFB8F.2040301@mrabarnett.plus.com> <4D0C7E4A.3010302@v.loewis.de> Message-ID: <4D0CFCE3.2050906@mrabarnett.plus.com> On 18/12/2010 09:26, "Martin v. L?wis" wrote: >> Comments? > > How do you implement that? In particular, how do you retrieve > information for different locales in a single program? > The locale module would be able to return a named locale dict: >>> loc = locale.getnamedlocale('en_UK') or: >>> loc = locale.namedlocales['en_UK'] The dict would be like what locale.localeconv() returns. The str.format method would then be able to use that dict. From alexander.belopolsky at gmail.com Sat Dec 18 19:55:00 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sat, 18 Dec 2010 13:55:00 -0500 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: References: <4D0C0688.1080108@haypocalc.com> <4D0CBDB8.6010608@haypocalc.com> Message-ID: On Sat, Dec 18, 2010 at 11:50 AM, Georg Brandl wrote: .. > In any case, this is coming pretty late; beta 2 is scheduled for this > weekend, and even if this is something that only kicks in when all hope > is lost anyway, it is a new feature. ?I should like to hear approval > from a few more devs before I will let this go into 3.2. > I am -1 on the feature as written. I would be -0 if it did not install signal handlers by default and even better was implemented in a separate module, not in core. This feature is similar to the one that is implemented by R and with R, it was always a problem when R was embedded as a library. It has always been a welcome feature of Python that its core did not mess with the application global state. Before this is approved, I would like to see a discussion of the consequences for embedded Python and particularly for the case when there are multiple interpreters in the same process. I think it would be best to start with this feature as an library module or even as a 3rd party add-on and see how useful it will be found in the wild. From martin at v.loewis.de Sat Dec 18 20:36:50 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 18 Dec 2010 20:36:50 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: References: <4D0C0688.1080108@haypocalc.com> <4D0CBDB8.6010608@haypocalc.com> Message-ID: <4D0D0D52.5020204@v.loewis.de> Am 18.12.2010 19:55, schrieb Alexander Belopolsky: > On Sat, Dec 18, 2010 at 11:50 AM, Georg Brandl wrote: > .. >> In any case, this is coming pretty late; beta 2 is scheduled for this >> weekend, and even if this is something that only kicks in when all hope >> is lost anyway, it is a new feature. I should like to hear approval >> from a few more devs before I will let this go into 3.2. >> > > I am -1 on the feature as written. I would be -0 if it did not > install signal handlers by default and even better was implemented in > a separate module, not in core. This is also what I think. Installing a signal handler is a fairly drastic action, and I don't think the code has been sufficiently reviewed yet. So I'd rather see this deferred until after 3.2, to allow for additional reviews (e.g. if some of the Twisted guys would approve it - that would be convincing). I also share Alexander's concern that Python just shouldn't mess with signal handlers at all, ideally. So some trade-off has to be found to address that concern (e.g. by making the signal handlers only active for the executable interpreter, but not in the embedded case). Regards, Martin From solipsis at pitrou.net Sat Dec 18 21:18:36 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 18 Dec 2010 21:18:36 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable References: <4D0C0688.1080108@haypocalc.com> <4D0CBDB8.6010608@haypocalc.com> <4D0D0D52.5020204@v.loewis.de> Message-ID: <20101218211836.0c2b0792@pitrou.net> On Sat, 18 Dec 2010 20:36:50 +0100 "Martin v. L?wis" wrote: > Am 18.12.2010 19:55, schrieb Alexander Belopolsky: > > On Sat, Dec 18, 2010 at 11:50 AM, Georg Brandl wrote: > > .. > >> In any case, this is coming pretty late; beta 2 is scheduled for this > >> weekend, and even if this is something that only kicks in when all hope > >> is lost anyway, it is a new feature. I should like to hear approval > >> from a few more devs before I will let this go into 3.2. > >> > > > > I am -1 on the feature as written. I would be -0 if it did not > > install signal handlers by default and even better was implemented in > > a separate module, not in core. > > This is also what I think. Installing a signal handler is a fairly > drastic action, and I don't think the code has been sufficiently > reviewed yet. How much more review should it receive? > I also share Alexander's concern that Python just shouldn't mess with > signal handlers at all, ideally. So some trade-off has to be found to > address that concern (e.g. by making the signal handlers only active > for the executable interpreter, but not in the embedded case). Well, Python already does (and also apparently messes with other things such as FPU state), so that's a separate issue altogether. Regards Antoine. From solipsis at pitrou.net Sat Dec 18 21:48:30 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 18 Dec 2010 21:48:30 +0100 Subject: [Python-Dev] r87389 - in python/branches/py3k: Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS References: <20101218200004.99110EE985@mail.python.org> Message-ID: <20101218214830.1aa8be32@pitrou.net> On Sat, 18 Dec 2010 21:00:04 +0100 (CET) ezio.melotti wrote: > Author: ezio.melotti > Date: Sat Dec 18 21:00:04 2010 > New Revision: 87389 > > Log: > #10573: use actual/expected consistently in unittest methods. IMHO, this should be reverted. The API currently doesn't treat these arguments differently, so they should really be labeled "first" and "second". Otherwise, the user will wrongly assume that the signature is asymmetric and that they should be careful about which order they pass the arguments in. Regards Antoine. From g.brandl at gmx.net Sat Dec 18 22:04:02 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 18 Dec 2010 22:04:02 +0100 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: <4D067D3F.8030206@g.nevcal.com> References: <20101213114449.GA2348@chopin.edu.pl> <20101213155119.A52A621CD91@kimball.webabinitio.net> <4D067D3F.8030206@g.nevcal.com> Message-ID: Am 13.12.2010 21:08, schrieb Glenn Linderman: > On 12/13/2010 11:39 AM, Mark Dickinson wrote: >> my_thing = Thing( >> foo = Foo(arg1, arg2, ...), >> bar = Bar(arg3, arg4, ...), >> ... >> ) >> >> and I've found the trailing comma very convenient during refactoring >> and API experimentation. (There's still good fun to be had arguing >> about the indentation of that closing parenthesis, though.) > > > Clearly it needs to be indented one level, because it is a continuation > of the prior line, just like the foo and bar and ... lines are > continuations and therefore indented. Clearly Emacs is superior to Vim because (insert some random fact here). Clearly the only thing that is clear about coding style details (even if we all more or less agree on PEP 8) is that it is a matter of personal taste. Georg From v+python at g.nevcal.com Sat Dec 18 23:41:09 2010 From: v+python at g.nevcal.com (Glenn Linderman) Date: Sat, 18 Dec 2010 14:41:09 -0800 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists -- continuation In-Reply-To: References: <20101213114449.GA2348@chopin.edu.pl> <20101213155119.A52A621CD91@kimball.webabinitio.net> <4D067D3F.8030206@g.nevcal.com> Message-ID: <4D0D3885.80903@g.nevcal.com> On 12/18/2010 1:04 PM, Georg Brandl wrote: > Am 13.12.2010 21:08, schrieb Glenn Linderman: >> On 12/13/2010 11:39 AM, Mark Dickinson wrote: >>> my_thing = Thing( >>> foo = Foo(arg1, arg2, ...), >>> bar = Bar(arg3, arg4, ...), >>> ... >>> ) >>> >>> and I've found the trailing comma very convenient during refactoring >>> and API experimentation. (There's still good fun to be had arguing >>> about the indentation of that closing parenthesis, though.) >> >> Clearly it needs to be indented one level, because it is a continuation >> of the prior line, just like the foo and bar and ... lines are >> continuations and therefore indented. > Clearly Emacs is superior to Vim because (insert some random fact here). Sure you have that right. > Clearly the only thing that is clear about coding style details (even if > we all more or less agree on PEP 8) is that it is a matter of personal > taste. And this too. But apparently you missed the fact that Mark wanted some good fun arguing about the indentation of the closing parenthesis... and didn't quote my "Enjoy!" that implied that that was all I was giving him. But then, you are release manager, which would make it very difficult, but hopefully you can still have a Merry Christmas! (or whatever end-of-year holiday suits your fancy) -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Sat Dec 18 23:46:23 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 18 Dec 2010 17:46:23 -0500 Subject: [Python-Dev] r87389 - in python/branches/py3k: Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS In-Reply-To: <20101218214830.1aa8be32@pitrou.net> References: <20101218200004.99110EE985@mail.python.org> <20101218214830.1aa8be32@pitrou.net> Message-ID: On 12/18/2010 3:48 PM, Antoine Pitrou wrote: > On Sat, 18 Dec 2010 21:00:04 +0100 (CET) > ezio.melotti wrote: >> Author: ezio.melotti >> Date: Sat Dec 18 21:00:04 2010 >> New Revision: 87389 >> >> Log: >> #10573: use actual/expected consistently in unittest methods. Change was requested by M. Foord and R. Hettinger (and G.Brandl for b2). > IMHO, this should be reverted. The API currently doesn't treat these > arguments differently, so they should really be labeled "first" and > "second". Otherwise, the user will wrongly assume that the signature is > asymmetric and that they should be careful about which order they pass > the arguments in. The error report on assert failure *is* often asymmetrical ;=). From Michael's post: "This is particularly relevant for the methods that produce 'diffed' output on failure - as the order determines whether mismatched items are missing from the expected or additional to the expected." This change struck me as a nice bit of polishing. -- Terry Jan Reedy From tjreedy at udel.edu Sun Dec 19 00:21:24 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 18 Dec 2010 18:21:24 -0500 Subject: [Python-Dev] Locale-specific formatting In-Reply-To: <20101218153331.GA22017@phd.pp.ru> References: <4D0BFB8F.2040301@mrabarnett.plus.com> <20101218153331.GA22017@phd.pp.ru> Message-ID: On 12/18/2010 10:33 AM, Oleg Broytman wrote: > This is quite a known problem, not specific to Python. Locale > settings are global for a process, and this is one of the thousands > reasons why locale is considered so horrible. > ICU is perhaps the only way around the problem. This is about the third mention of 'ICU' in the last few weeks, so I looked it up: International Components for Unicode http://site.icu-project.org/ Several libraries (C/C++,Java), including prebuilt binaries for Windows (and some others). There is already a Python .cpp wrapper (but no Windows binaries, which limits usefulness) http://pyicu.osafoundation.org/ http://pypi.python.org/pypi/PyICU/1.0.1 -- Terry Jan Reedy From grygoriy.fuchedzhy at gmail.com Sat Dec 18 22:37:40 2010 From: grygoriy.fuchedzhy at gmail.com (Grygoriy Fuchedzhy) Date: Sat, 18 Dec 2010 23:37:40 +0200 Subject: [Python-Dev] [feature request] add .svgz to mimetypes.suffix_map Message-ID: Hi. I've created bug on bugtracker, and then I was told there that I should post this on this mailing list for discussion. Here is link to bug: http://bugs.python.org/issue10730 Please add '.svgz': '.svg.gz' map to mimetypes.suffix_map -- Grygoriy Fuchedzhy -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Sun Dec 19 01:31:37 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 19 Dec 2010 01:31:37 +0100 Subject: [Python-Dev] Locale-specific formatting In-Reply-To: <4D0CFCE3.2050906@mrabarnett.plus.com> References: <4D0BFB8F.2040301@mrabarnett.plus.com> <4D0C7E4A.3010302@v.loewis.de> <4D0CFCE3.2050906@mrabarnett.plus.com> Message-ID: <4D0D5269.1060304@v.loewis.de> Am 18.12.2010 19:26, schrieb MRAB: > On 18/12/2010 09:26, "Martin v. L?wis" wrote: >>> Comments? >> >> How do you implement that? In particular, how do you retrieve >> information for different locales in a single program? >> > The locale module would be able to return a named locale dict: > >>>> loc = locale.getnamedlocale('en_UK') Ok, so, in turn: how do you implement *that* (what C API specifically do you use to implement that getnamedlocale function)? Regards, Martin From python at mrabarnett.plus.com Sun Dec 19 02:11:36 2010 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 19 Dec 2010 01:11:36 +0000 Subject: [Python-Dev] Locale-specific formatting In-Reply-To: <4D0D5269.1060304@v.loewis.de> References: <4D0BFB8F.2040301@mrabarnett.plus.com> <4D0C7E4A.3010302@v.loewis.de> <4D0CFCE3.2050906@mrabarnett.plus.com> <4D0D5269.1060304@v.loewis.de> Message-ID: <4D0D5BC8.7080804@mrabarnett.plus.com> On 19/12/2010 00:31, "Martin v. L?wis" wrote: > Am 18.12.2010 19:26, schrieb MRAB: >> On 18/12/2010 09:26, "Martin v. L?wis" wrote: >>>> Comments? >>> >>> How do you implement that? In particular, how do you retrieve >>> information for different locales in a single program? >>> >> The locale module would be able to return a named locale dict: >> >>>>> loc = locale.getnamedlocale('en_UK') > > Ok, so, in turn: how do you implement *that* (what C API > specifically do you use to implement that getnamedlocale function)? > I suppose there could be some sort of locale database. A downloadable, up-to-date copy of the database could be maintained on the Python website. From tjreedy at udel.edu Sun Dec 19 03:24:32 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 18 Dec 2010 21:24:32 -0500 Subject: [Python-Dev] [feature request] add .svgz to mimetypes.suffix_map In-Reply-To: References: Message-ID: On 12/18/2010 4:37 PM, Grygoriy Fuchedzhy wrote: > Hi. > I've created bug on bugtracker, and then I was told there that I should > post this on this mailing list for discussion. > Here is link to bug: http://bugs.python.org/issue10730 > > Please add'.svgz':'.svg.gz' map to mimetypes.suffix_map This issue is actually a request to add .svg to the types map and the the abbreviation .svgz for .svg.gx to the suffix_map. I believe Scalable Vector Graphics are well on the way to becoming *the* standard vector graphics format for the web, especially with upcoming IE9 support. https://secure.wikimedia.org/wikipedia/en/wiki/Svg It (they) are already supported by the other major browsers. In addition, "Google announced on 31 August 2010 that it had begun to index SVG content on the web, whether it is in standalone files or embedded in HTML." So it might be sensible to add these now rather than two years from now. But if it is too late for 3.2, then I expect addition for 3.3 will be obvious. -- Terry Jan Reedy From guido at python.org Sun Dec 19 05:23:49 2010 From: guido at python.org (Guido van Rossum) Date: Sat, 18 Dec 2010 20:23:49 -0800 Subject: [Python-Dev] r87389 - in python/branches/py3k: Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS In-Reply-To: References: <20101218200004.99110EE985@mail.python.org> <20101218214830.1aa8be32@pitrou.net> Message-ID: I may be unique, but I fear there is no great answer. On the one hand I almost always code it as e.g. assertEqual(actual, expected), which matches my preference for e.g. "if x == 5:" rather than "if 5 == x:". On the other hand in those assert* functions that show a nice diff of two lists, when reading such a diff my expectation is that "old, new" corresponds to "expected, actual". Which then freaks me out until I realize that I coded it as "actual, expected"... And yet "expected, actual" still looks weird to me. :-( On Sat, Dec 18, 2010 at 2:46 PM, Terry Reedy wrote: > On 12/18/2010 3:48 PM, Antoine Pitrou wrote: >> >> On Sat, 18 Dec 2010 21:00:04 +0100 (CET) >> ezio.melotti ?wrote: >>> >>> Author: ezio.melotti >>> Date: Sat Dec 18 21:00:04 2010 >>> New Revision: 87389 >>> >>> Log: >>> #10573: use actual/expected consistently in unittest methods. > > Change was requested by M. Foord and R. Hettinger (and G.Brandl for b2). > >> IMHO, this should be reverted. The API currently doesn't treat these >> arguments differently, so they should really be labeled "first" and >> "second". Otherwise, the user will wrongly assume that the signature is >> asymmetric and that they should be careful about which order they pass >> the arguments in. > > The error report on assert failure *is* often asymmetrical ;=). > From Michael's post: > "This is particularly relevant for the methods that produce 'diffed' output > on failure - as the order determines whether mismatched items are missing > from the expected or additional to the expected." > > This change struck me as a nice bit of polishing. > > -- > Terry Jan Reedy > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (python.org/~guido) From rdmurray at bitdance.com Sun Dec 19 07:52:37 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Sun, 19 Dec 2010 01:52:37 -0500 Subject: [Python-Dev] r87389 - in python/branches/py3k: Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS In-Reply-To: References: <20101218200004.99110EE985@mail.python.org> <20101218214830.1aa8be32@pitrou.net> Message-ID: <20101219065237.9FF242379ED@kimball.webabinitio.net> On Sat, 18 Dec 2010 20:23:49 -0800, Guido van Rossum wrote: > I may be unique, but I fear there is no great answer. On the one hand > I almost always code it as e.g. assertEqual(actual, expected), which > matches my preference for e.g. "if x =3D=3D 5:" rather than "if 5 =3D=3D x:= > ". > On the other hand in those assert* functions that show a nice diff of > two lists, when reading such a diff my expectation is that "old, new" > corresponds to "expected, actual". Which then freaks me out until I > realize that I coded it as "actual, expected"... And yet "expected, > actual" still looks weird to me. :-( You aren't unique, I feel the same way. But it seems to me that the most important thing is to be consistent, so that I don't freak out for long. -- R. David Murray www.bitdance.com From phd at phd.pp.ru Sun Dec 19 11:44:00 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Sun, 19 Dec 2010 13:44:00 +0300 Subject: [Python-Dev] Locale-specific formatting In-Reply-To: References: <4D0BFB8F.2040301@mrabarnett.plus.com> <20101218153331.GA22017@phd.pp.ru> Message-ID: <20101219104400.GA21740@phd.pp.ru> On Sat, Dec 18, 2010 at 06:21:24PM -0500, Terry Reedy wrote: > On 12/18/2010 10:33 AM, Oleg Broytman wrote: > >> This is quite a known problem, not specific to Python. Locale >> settings are global for a process, and this is one of the thousands >> reasons why locale is considered so horrible. >> ICU is perhaps the only way around the problem. > > This is about the third mention of 'ICU' in the last few weeks, so I > looked it up: International Components for Unicode > http://site.icu-project.org/ > Several libraries (C/C++,Java), including prebuilt binaries for Windows > (and some others). > There is already a Python .cpp wrapper (but no Windows binaries, which > limits usefulness) > http://pyicu.osafoundation.org/ > http://pypi.python.org/pypi/PyICU/1.0.1 A month ago there was a long thread that mentioned ICU many times: http://mail.python.org/pipermail/python-dev/2010-November/106068.html Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From solipsis at pitrou.net Sun Dec 19 14:13:55 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 19 Dec 2010 14:13:55 +0100 Subject: [Python-Dev] r87389 - in python/branches/py3k: Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS References: <20101218200004.99110EE985@mail.python.org> <20101218214830.1aa8be32@pitrou.net> Message-ID: <20101219141355.5ac7cd2f@pitrou.net> On Sat, 18 Dec 2010 20:23:49 -0800 Guido van Rossum wrote: > I may be unique, but I fear there is no great answer. On the one hand > I almost always code it as e.g. assertEqual(actual, expected), which > matches my preference for e.g. "if x == 5:" rather than "if 5 == x:". > On the other hand in those assert* functions that show a nice diff of > two lists, when reading such a diff my expectation is that "old, new" > corresponds to "expected, actual". Which then freaks me out until I > realize that I coded it as "actual, expected"... And yet "expected, > actual" still looks weird to me. :-( This could be nicely resolved by renaming the arguments "a" and "b", and having the diff display "a, b". It's quite natural (both the diff ordering and the arguments ordering), and they are consistent with each other. Regards Antoine. From solipsis at pitrou.net Sun Dec 19 14:23:02 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 19 Dec 2010 14:23:02 +0100 Subject: [Python-Dev] r87399 - in python/branches/py3k: Doc/library/http.client.rst Doc/library/urllib.request.rst Lib/http/client.py Lib/test/test_httplib.py Lib/test/test_urllib2.py Lib/urllib/request.py Misc/NEWS References: <20101219104952.EABC8EE9A9@mail.python.org> Message-ID: <20101219142302.399ef3b4@pitrou.net> Hello Senthil, > + raise TypeError("data should be byte-like object\ Should be "bytes-like". > + request.add_unredirected_header( > + 'Content-length', '%d' % len(mv) * mv.itemsize) There is an operator precedence problem here: >>> "%d" % 4 * 5 '44444' > + """ > + file_obj = io.StringIO() > + file_obj.write("Something\nSomething\nSomething\n") > + Why is this whole thing commented out? If it wasn't, you would have seen the test failing. Thanks Antoine. From guido at python.org Sun Dec 19 19:41:34 2010 From: guido at python.org (Guido van Rossum) Date: Sun, 19 Dec 2010 10:41:34 -0800 Subject: [Python-Dev] r87389 - in python/branches/py3k: Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS In-Reply-To: <20101219141355.5ac7cd2f@pitrou.net> References: <20101218200004.99110EE985@mail.python.org> <20101218214830.1aa8be32@pitrou.net> <20101219141355.5ac7cd2f@pitrou.net> Message-ID: On Sun, Dec 19, 2010 at 5:13 AM, Antoine Pitrou wrote: > On Sat, 18 Dec 2010 20:23:49 -0800 > Guido van Rossum wrote: >> I may be unique, but I fear there is no great answer. On the one hand >> I almost always code it as e.g. assertEqual(actual, expected), which >> matches my preference for e.g. "if x == 5:" rather than "if 5 == x:". >> On the other hand in those assert* functions that show a nice diff of >> two lists, when reading such a diff my expectation is that "old, new" >> corresponds to "expected, actual". Which then freaks me out until I >> realize that I coded it as "actual, expected"... And yet "expected, >> actual" still looks weird to me. :-( > > This could be nicely resolved by renaming the arguments "a" and "b", > and having the diff display "a, b". It's quite natural (both the diff > ordering and the arguments ordering), and they are consistent with each > other. So 'a' stands for 'after' and 'b' for 'before', right? :-) -- --Guido van Rossum (python.org/~guido) From solipsis at pitrou.net Sun Dec 19 19:49:49 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 19 Dec 2010 19:49:49 +0100 Subject: [Python-Dev] r87389 - in python/branches/py3k: Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS In-Reply-To: References: <20101218200004.99110EE985@mail.python.org> <20101218214830.1aa8be32@pitrou.net> <20101219141355.5ac7cd2f@pitrou.net> Message-ID: <1292784589.3679.1.camel@localhost.localdomain> Le dimanche 19 d?cembre 2010 ? 10:41 -0800, Guido van Rossum a ?crit : > On Sun, Dec 19, 2010 at 5:13 AM, Antoine Pitrou wrote: > > On Sat, 18 Dec 2010 20:23:49 -0800 > > Guido van Rossum wrote: > >> I may be unique, but I fear there is no great answer. On the one hand > >> I almost always code it as e.g. assertEqual(actual, expected), which > >> matches my preference for e.g. "if x == 5:" rather than "if 5 == x:". > >> On the other hand in those assert* functions that show a nice diff of > >> two lists, when reading such a diff my expectation is that "old, new" > >> corresponds to "expected, actual". Which then freaks me out until I > >> realize that I coded it as "actual, expected"... And yet "expected, > >> actual" still looks weird to me. :-( > > > > This could be nicely resolved by renaming the arguments "a" and "b", > > and having the diff display "a, b". It's quite natural (both the diff > > ordering and the arguments ordering), and they are consistent with each > > other. > > So 'a' stands for 'after' and 'b' for 'before', right? :-) Ouch. I guess I don't natively think in English. From martin at v.loewis.de Sun Dec 19 19:49:30 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 19 Dec 2010 19:49:30 +0100 Subject: [Python-Dev] Locale-specific formatting In-Reply-To: <4D0D5BC8.7080804@mrabarnett.plus.com> References: <4D0BFB8F.2040301@mrabarnett.plus.com> <4D0C7E4A.3010302@v.loewis.de> <4D0CFCE3.2050906@mrabarnett.plus.com> <4D0D5269.1060304@v.loewis.de> <4D0D5BC8.7080804@mrabarnett.plus.com> Message-ID: <4D0E53BA.40101@v.loewis.de> > I suppose there could be some sort of locale database. A downloadable, > up-to-date copy of the database could be maintained on the Python > website. I think you are quite underestimating the implementation effort. So -0 on your original proposal until such a thing actually exists. Regards, Martin From martin at v.loewis.de Sun Dec 19 19:53:43 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 19 Dec 2010 19:53:43 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: <20101218211836.0c2b0792@pitrou.net> References: <4D0C0688.1080108@haypocalc.com> <4D0CBDB8.6010608@haypocalc.com> <4D0D0D52.5020204@v.loewis.de> <20101218211836.0c2b0792@pitrou.net> Message-ID: <4D0E54B7.2070604@v.loewis.de> >> This is also what I think. Installing a signal handler is a fairly >> drastic action, and I don't think the code has been sufficiently >> reviewed yet. > > How much more review should it receive? There should be at least one reviewer with an established track record of being interested/knowledgable in the POSIX APIs and cross-platform aspects of it. As I said, any of the Twisted guys would qualify. > Well, Python already does (and also apparently messes with other things > such as FPU state), so that's a separate issue altogether. Not at all. Some signal handlers are more likely to interfere with the the rest of the application than others. Regards, Martin From solipsis at pitrou.net Sun Dec 19 20:20:22 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 19 Dec 2010 20:20:22 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: <4D0E54B7.2070604@v.loewis.de> References: <4D0C0688.1080108@haypocalc.com> <4D0CBDB8.6010608@haypocalc.com> <4D0D0D52.5020204@v.loewis.de> <20101218211836.0c2b0792@pitrou.net> <4D0E54B7.2070604@v.loewis.de> Message-ID: <1292786422.3679.24.camel@localhost.localdomain> Le dimanche 19 d?cembre 2010 ? 19:53 +0100, "Martin v. L?wis" a ?crit : > >> This is also what I think. Installing a signal handler is a fairly > >> drastic action, and I don't think the code has been sufficiently > >> reviewed yet. > > > > How much more review should it receive? > > There should be at least one reviewer with an established track record > of being interested/knowledgable in the POSIX APIs and cross-platform > aspects of it. > As I said, any of the Twisted guys would qualify. James Knight already posted in this thread. Is this sufficient to you? I don't think it's reasonable to expect "the Twisted guys" (whoever they are...) to come and review our code, while they struggle with their own bug queue. It would certainly be beneficial for the stdlib to get as much review attention as you are asking for this patch, but it is also totally impractical to expect such a level of attention from a community of volunteers. If we aren't knowledgeable enough to maintain signal handling code, then we should simply remove the signal module from the stdlib (with an appropriate deprecation period, that is) and let interested people maintain it as a third-party module. That would also close some lingering issues in the tracker, and free up some of our time. > > Well, Python already does (and also apparently messes with other things > > such as FPU state), so that's a separate issue altogether. > > Not at all. Some signal handlers are more likely to interfere with the > the rest of the application than others. Can you clarify why you think those signal handlers fall into that description? After all, SIGINT, SIGILL and friends only get triggered in catastrophic conditions. Thanks Antoine. From raymond.hettinger at gmail.com Sun Dec 19 20:55:55 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Sun, 19 Dec 2010 11:55:55 -0800 Subject: [Python-Dev] r87389 - in python/branches/py3k: Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS In-Reply-To: References: <20101218200004.99110EE985@mail.python.org> <20101218214830.1aa8be32@pitrou.net> <20101219141355.5ac7cd2f@pitrou.net> Message-ID: <31185718-F276-46CE-82E2-6C6B6EC920E8@gmail.com> On Dec 19, 2010, at 10:41 AM, Guido van Rossum wrote: > On Sun, Dec 19, 2010 at 5:13 AM, Antoine Pitrou wrote: >> On Sat, 18 Dec 2010 20:23:49 -0800 >> Guido van Rossum wrote: >>> I may be unique, but I fear there is no great answer. On the one hand >>> I almost always code it as e.g. assertEqual(actual, expected), which >>> matches my preference for e.g. "if x == 5:" rather than "if 5 == x:". >>> On the other hand in those assert* functions that show a nice diff of >>> two lists, when reading such a diff my expectation is that "old, new" >>> corresponds to "expected, actual". Which then freaks me out until I >>> realize that I coded it as "actual, expected"... And yet "expected, >>> actual" still looks weird to me. :-( >> >> This could be nicely resolved by renaming the arguments "a" and "b", >> and having the diff display "a, b". It's quite natural (both the diff >> ordering and the arguments ordering), and they are consistent with each >> other. > > So 'a' stands for 'after' and 'b' for 'before', right? :-) If you go down the a / b path instead of actual/expected, the diffs are straight-forward but some of the other output styles needed to be changed also (replace the messages for "unexpected" and "missing" elements to "things in a but not in b" and "things in b but not in a". Raymond From alexander.belopolsky at gmail.com Sun Dec 19 21:38:41 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 19 Dec 2010 15:38:41 -0500 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: <1292786422.3679.24.camel@localhost.localdomain> References: <4D0C0688.1080108@haypocalc.com> <4D0CBDB8.6010608@haypocalc.com> <4D0D0D52.5020204@v.loewis.de> <20101218211836.0c2b0792@pitrou.net> <4D0E54B7.2070604@v.loewis.de> <1292786422.3679.24.camel@localhost.localdomain> Message-ID: On Sun, Dec 19, 2010 at 2:20 PM, Antoine Pitrou wrote: .. >> There should be at least one reviewer with an established track record >> of being interested/knowledgable in the POSIX APIs and cross-platform >> aspects of it. >> As I said, any of the Twisted guys would qualify. > > James Knight already posted in this thread. Is this sufficient to you? > I cannot speak for Martin of course, but I think we should first see if James Knight's recommendations get incorporated into the patch and whether that will be sufficient for James Knight to speak in favor of applying the patch. From victor.stinner at haypocalc.com Sun Dec 19 22:59:01 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Sun, 19 Dec 2010 22:59:01 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: <4D0E54B7.2070604@v.loewis.de> References: <4D0C0688.1080108@haypocalc.com> <4D0CBDB8.6010608@haypocalc.com> <4D0D0D52.5020204@v.loewis.de> <20101218211836.0c2b0792@pitrou.net> <4D0E54B7.2070604@v.loewis.de> Message-ID: <1292795941.5914.23.camel@marge> Le dimanche 19 d?cembre 2010 ? 19:53 +0100, "Martin v. L?wis" a ?crit : > There should be at least one reviewer with an established track record > of being interested/knowledgable in the POSIX APIs and cross-platform > aspects of it. Functions used by the fault handler: - write() - PyUnicode_Check() - PyFrame_GetLineNumber() - DebugBreak() (Windows, in debug mode, only) - abort() - (macro) PyUnicode_GET_SIZE() and PyUnicode_AS_UNICODE() - PyUnicode_Check(), PyFrame_Check() - PyFrame_GetLineNumber() - _Py_atomic_load_relaxed() - _PyThreadState_GetFrame() I suppose that all *Py*() functions are POSIX compliants. DebugBreak() is specific to Windows. Only write() and abort() have to be POSIX compliant. According to Linux manual pages: - abort() conforms to SVr4, POSIX.1-2001, 4.3BSD, C89, C99. - write() conforms to SVr4, 4.3BSD, POSIX.1-2001 -- Functions used by the other parts of the patch: - signal() or sigaction()+sigemptyset() - (optional) sigaltstack() - PyMem_Malloc(), PyMem_Free() - Py_GETENV() - (optional) getpid(), kill() (used by the tests) signal() conforms to C89, C99, POSIX.1-2001. So I don't see any function incompatible with POSIX. (Don't read old versions of my patch, read the latest version) -- About the cross-platform question: it looks like SIGFPE fault is not catchable on Windows. I only tried Linux and Windows, but there are a unit test for each signal and we have 60+ buildbots with various platforms. Can't we wait for the buildbot results? > > Well, Python already does (and also apparently messes with other things > > such as FPU state), so that's a separate issue altogether. > > Not at all. Some signal handlers are more likely to interfere with the > the rest of the application than others. Python installs its own SIGINT handler. I think that a SIGINT handler is more intrusive than a SIGSEGV or SIGILL handler. Victor From victor.stinner at haypocalc.com Sun Dec 19 23:10:43 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Sun, 19 Dec 2010 23:10:43 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: References: <4D0C0688.1080108@haypocalc.com> <4D0CBDB8.6010608@haypocalc.com> Message-ID: <1292796643.5914.34.camel@marge> Le samedi 18 d?cembre 2010 ? 17:23 +0100, Georg Brandl a ?crit : > Well, without a closer I assume that for some crashes it's just not > possible anymore for the Python interpreter to even print out the > traceback? The worst case occurs if the frame objects are corrupted, eg. if the filename of a frame is an invalid object (eg. pointer to NULL or to a corrupted unicode object). If the file descriptor 2 is not stderr (eg. if there is no file descriptor 2), the backtrace cannot be printed. Except of these two issues, I think that the fault handler is always able to display the backtrace, even on stack overflow or invalid memory write. -- Older versions of my patch were less reliable: - allocate memory on the heap => only use the stack (only few bytes) - call "complex" Python functions (like UTF-8 codec with backslashreplace error handler) => use a very simple ASCII +backslashreplace implementation and avoid complex functions - use not signal-safe functions like printf => use write() and simple functions to format numbers - may go into an unlimited loop if there is a loop in the frame linked list => limit the loop to MAX_DEPTH (100) iterations Victor From victor.stinner at haypocalc.com Sun Dec 19 23:32:03 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Sun, 19 Dec 2010 23:32:03 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: References: <4D0C0688.1080108@haypocalc.com> <4D0CBDB8.6010608@haypocalc.com> Message-ID: <1292797923.5914.55.camel@marge> Le samedi 18 d?cembre 2010 ? 13:55 -0500, Alexander Belopolsky a ?crit : > I am -1 on the feature as written. I would be -0 if it did not > install signal handlers by default and even better was implemented in > a separate module, not in core. I think that the feature is useless if it is disabled by default. How do you know that you application will crash? The idea is to give informations to the user when an application crashs: the user can use the backtrace or send it to the developer. Segmentation faults are usually not (easilly) reproductible :-( So even if you enable the fault handler, you may not be able to replay the crash. Or even worse, the fault may not occurs at all when you enable the fault handler... (Heisenbugs!) > This feature is similar to the one that is implemented by R and with > R, it was always a problem when R was embedded as a library. Which kind of problem? A conflict with an existing fault handler? > It has always been a welcome feature of Python that its core did > not mess with the application global state. What do mess the application global state? > Before this is approved, I would like to see a discussion of the > consequences for embedded Python I suppose that if a fault occurs outside Python, the fault handler displays an irrevelent or empty Python backtrace. The fault handler is installed by Py_InitializeEx(), and uninstalled by Py_Finalize(). If the program has its own fault handler or doesn't want Python fault handler, it can set PYTHONNOFAULTHANDLER environment variable. Should we add a C function to the Python API to configure (disable) the fault handler? I only know *two* applications catching faults: Mplayer and aMule. I don't think that they embed Python. If they do, I think that these *two* applications can do something special. Since there are only two applications having their own fault handler, I don't think that we should disable the fault handler by default. (System wide fault handler, like Apport on Ubuntu, can be configured system wide by setting the PYTHONNOFAULTHANDLER environment variable) > and particularly for the case when there are multiple interpreters > in the same process. What? Is it a myth or does Python really support multiple interpreters in the same process? How is it possible? Who uses this? _Py_DumpBacktrace() function (of my patch) uses _Py_atomic_load_relaxed(&_PyThreadState_Current) to get the current thread state. I picked up this instruction from PyThreadState_Get(). If it doesn't work, PyThreadState_Get() should be fixed too. Victor From scott+python-dev at scottdial.com Sun Dec 19 23:08:51 2010 From: scott+python-dev at scottdial.com (Scott Dial) Date: Sun, 19 Dec 2010 17:08:51 -0500 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: References: <4D0C0688.1080108@haypocalc.com> Message-ID: <4D0E8273.1040606@scottdial.com> On 12/18/2010 8:50 AM, James Y Knight wrote: > I think instead of calling abort() to kill the process, you should: > - install the signal handler with SA_NODEFER|SA_RESETHAND (or if > sigaction is not present, explicitly reset the action to SIG_DFL and > unblock first thing upon entering the handler), and then, > - at the end of the handler, kill(getpid(), orig_signal) in order to > abort the process. I concur with this being the correct way to right such a handler. > This has two advantages: 1) the process's exit code will actually > show the correct signal, It's more than an advantage: it's the only correct way to handle a termination signal. > 2) it might let the OS fault handlers work properly > as well -- I'm not sure. If it does, you may want to experiment with > whether having or omitting SA_NODEFER gives a better backtrace (from the > OS mechanism) in that case. Even if that doesn't work, things like the grsecurity patches to linux use these signals to detect exploits and log them and do throttling. Calling abort() effectively converts all of these faults into SIGABRT terminations that are considered (more?) innocent terminations. -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From victor.stinner at haypocalc.com Sun Dec 19 23:37:58 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Sun, 19 Dec 2010 23:37:58 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: <1292786422.3679.24.camel@localhost.localdomain> References: <4D0C0688.1080108@haypocalc.com> <4D0CBDB8.6010608@haypocalc.com> <4D0D0D52.5020204@v.loewis.de> <20101218211836.0c2b0792@pitrou.net> <4D0E54B7.2070604@v.loewis.de> <1292786422.3679.24.camel@localhost.localdomain> Message-ID: <1292798278.5914.56.camel@marge> Le dimanche 19 d?cembre 2010 ? 20:20 +0100, Antoine Pitrou a ?crit : > Can you clarify why you think those signal handlers fall into that > description? After all, SIGINT, SIGILL and friends only get triggered in > catastrophic conditions. SIGSEGV, not SIGINT Victor From solipsis at pitrou.net Sun Dec 19 23:51:03 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 19 Dec 2010 23:51:03 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable References: <4D0C0688.1080108@haypocalc.com> <4D0CBDB8.6010608@haypocalc.com> <4D0D0D52.5020204@v.loewis.de> <20101218211836.0c2b0792@pitrou.net> <4D0E54B7.2070604@v.loewis.de> <1292786422.3679.24.camel@localhost.localdomain> <1292798278.5914.56.camel@marge> Message-ID: <20101219235103.227aaad8@pitrou.net> On Sun, 19 Dec 2010 23:37:58 +0100 Victor Stinner wrote: > Le dimanche 19 d?cembre 2010 ? 20:20 +0100, Antoine Pitrou a ?crit : > > Can you clarify why you think those signal handlers fall into that > > description? After all, SIGINT, SIGILL and friends only get triggered in > > catastrophic conditions. > > SIGSEGV, not SIGINT Oops, sorry for the mistake. I hope this didn't stir any confusion beyond my own ;) cheers Antoine. From victor.stinner at haypocalc.com Mon Dec 20 00:02:25 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Mon, 20 Dec 2010 00:02:25 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: References: <4D0C0688.1080108@haypocalc.com> Message-ID: <1292799745.5914.76.camel@marge> Le samedi 18 d?cembre 2010 ? 08:50 -0500, James Y Knight a ?crit : > I think instead of calling abort() to kill the process, you should: > - install the signal handler with SA_NODEFER|SA_RESETHAND (or if sigaction is not present, explicitly reset the action to SIG_DFL and unblock first thing upon entering the handler), and then, > - at the end of the handler, kill(getpid(), orig_signal) in order to abort the process. I don't think that kill() is available on Windows. We may call directly the previous handler in the new handler, but it is more difficult. It looks like the prototype of a signal handler depends on its options. Or is it possible to always pass (signum, siginfo, ucontext) arguments? I don't know if siginfo and ucontect are portable or not... Another possible approach is to restore the previous handler in the fault handler, and don't call abort(). The fault should occur again, and so the previous signal handler will be called. I choosed to not call the previous handler to keep the patch simple. If you consider that we must call the previous handler, I can work on a new patch doing that. > This has two advantages: 1) the process's exit code will actually show the correct signal Oh, I didn't noticed that the exit code is different. > 2) it might let the OS fault handlers work properly as well -- I'm not sure. If it does, you may want to experiment with whether having or omitting SA_NODEFER gives a better backtrace (from the OS mechanism) in that case. For the OS fault handler: we should try, I am not sure that calling another signal handler (my fault handler) keeps the process state unchanged (especially if sigaltstk() is not available). I don't understand the SA_NODEFER option. What does it change for this fault handler? > If #2 actually works, you may not even need the env var, which would be nice. I prefer to keep it optional because we don't know yet how it interacts exactly with existing debug tools, OS tools, embedded Python interpreters, etc. Victor From tjreedy at udel.edu Mon Dec 20 00:54:55 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 19 Dec 2010 18:54:55 -0500 Subject: [Python-Dev] r87389 - in python/branches/py3k: Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS In-Reply-To: References: <20101218200004.99110EE985@mail.python.org> <20101218214830.1aa8be32@pitrou.net> <20101219141355.5ac7cd2f@pitrou.net> Message-ID: On 12/19/2010 1:41 PM, Guido van Rossum wrote: > On Sun, Dec 19, 2010 at 5:13 AM, Antoine Pitrou wrote: >> This could be nicely resolved by renaming the arguments "a" and "b", >> and having the diff display "a, b". It's quite natural (both the diff >> ordering and the arguments ordering), and they are consistent with each >> other. > > So 'a' stands for 'after' and 'b' for 'before', right? :-) difflib uses 'a' and 'b' for before and after (orig,new in svn terms, with edits/diffs from a to b) respectively. Not really great. The docs then have to explain what 'a' and 'b' are and the implications for interpreting the output. -- Terry Jan Reedy From merwok at netwok.org Mon Dec 20 01:09:19 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Mon, 20 Dec 2010 01:09:19 +0100 Subject: [Python-Dev] Locale-specific formatting In-Reply-To: <20101218153331.GA22017@phd.pp.ru> References: <4D0BFB8F.2040301@mrabarnett.plus.com> <20101218153331.GA22017@phd.pp.ru> Message-ID: <4D0E9EAF.6010000@netwok.org> Le 18/12/2010 16:33, Oleg Broytman a ?crit : > This is quite a known problem, not specific to Python. Locale > settings are global for a process, and this is one of the thousands > reasons why locale is considered so horrible. > ICU is perhaps the only way around the problem. Babel rocks: http://babel.edgewall.org/ Cheers From victor.stinner at haypocalc.com Mon Dec 20 01:19:35 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Mon, 20 Dec 2010 01:19:35 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: <1292799745.5914.76.camel@marge> References: <4D0C0688.1080108@haypocalc.com> <1292799745.5914.76.camel@marge> Message-ID: <1292804375.5914.84.camel@marge> Le lundi 20 d?cembre 2010 ? 00:02 +0100, Victor Stinner a ?crit : > Another possible approach is to restore the previous handler in the > fault handler, and don't call abort(). The fault should occur again, and > so the previous signal handler will be called. I implemented this simple approach in the version 10 of my patch: the previous signal handler is called. http://bugs.python.org/issue8863#msg124373 On my Debian Sid, the libc fatal error message is written as expected (eg. "Segmentation fault") and the exit code is different for each fault (eg. 132 for SIGILL). So we keep the existing behaviour, the new Python fault handler just write more information if a fault occurs. I suppose that it is compatible with the grsecurity thing and Apport, but I didn't tried. Can someone test it? I prefer to restore the previous signal handler than restoring the *default* signal handler. > I choosed to not call the previous handler to keep the patch simple. It's cool: restoring the signal approach keeps the code simple, and it looks like it works :-) Victor From solipsis at pitrou.net Mon Dec 20 01:48:40 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 20 Dec 2010 01:48:40 +0100 Subject: [Python-Dev] r87389 - in python/branches/py3k: Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS References: <20101218200004.99110EE985@mail.python.org> <20101218214830.1aa8be32@pitrou.net> <20101219141355.5ac7cd2f@pitrou.net> Message-ID: <20101220014840.6b734274@pitrou.net> On Sun, 19 Dec 2010 18:54:55 -0500 Terry Reedy wrote: > On 12/19/2010 1:41 PM, Guido van Rossum wrote: > > On Sun, Dec 19, 2010 at 5:13 AM, Antoine Pitrou wrote: > > >> This could be nicely resolved by renaming the arguments "a" and "b", > >> and having the diff display "a, b". It's quite natural (both the diff > >> ordering and the arguments ordering), and they are consistent with each > >> other. > > > > So 'a' stands for 'after' and 'b' for 'before', right? :-) > > difflib uses 'a' and 'b' for before and after (orig,new in svn terms, > with edits/diffs from a to b) respectively. Not really great. For a non-native English speaker, 'a' and 'b' don't evoke 'after' and 'before' but simply the first two letters of the latin alphabet, and their ordering is therefore obvious with respect to function arguments. By the way, hg uses a/b as well, and so does git apparently, so Python's difflib is not exotic in that regard: $ hg diff diff -r 56867877575b README --- a/README Fri Dec 17 21:43:27 2010 +0100 +++ b/README Mon Dec 20 01:42:57 2010 +0100 @@ -1,3 +1,4 @@ +some change This is Python version 3.2 beta 1 ================================= Regards Antoine. From martin at v.loewis.de Mon Dec 20 02:05:30 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 20 Dec 2010 02:05:30 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: <1292795941.5914.23.camel@marge> References: <4D0C0688.1080108@haypocalc.com> <4D0CBDB8.6010608@haypocalc.com> <4D0D0D52.5020204@v.loewis.de> <20101218211836.0c2b0792@pitrou.net> <4D0E54B7.2070604@v.loewis.de> <1292795941.5914.23.camel@marge> Message-ID: <4D0EABDA.3060702@v.loewis.de> > Functions used by the fault handler: > - write() > - PyUnicode_Check() > - PyFrame_GetLineNumber() > - DebugBreak() (Windows, in debug mode, only) > - abort() > - (macro) PyUnicode_GET_SIZE() and PyUnicode_AS_UNICODE() > - PyUnicode_Check(), PyFrame_Check() > - PyFrame_GetLineNumber() > - _Py_atomic_load_relaxed() > - _PyThreadState_GetFrame() > > I suppose that all *Py*() functions are POSIX compliants. The problem is that merely being POSIX compliant would not be enough to allow calling a function in a signal handler. Instead, the function *also* needs to be async-signal safe. However, > - abort() conforms to SVr4, POSIX.1-2001, 4.3BSD, C89, C99. > - write() conforms to SVr4, 4.3BSD, POSIX.1-2001 These two functions are indeed async-signal safe. So assuming that none of the functions above indirectly calls a non-async-signal-safe function, this part is fine. Looking at your function list, my other concern is that you are calling Python API without holding the GIL, IIUC. In particular, you are accessing _PyThreadState_Current, which may not point to the current thread if the current thread has released the GIL. Regards, Martin From martin at v.loewis.de Mon Dec 20 02:11:34 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 20 Dec 2010 02:11:34 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: <1292797923.5914.55.camel@marge> References: <4D0C0688.1080108@haypocalc.com> <4D0CBDB8.6010608@haypocalc.com> <1292797923.5914.55.camel@marge> Message-ID: <4D0EAD46.209@v.loewis.de> > What? Is it a myth or does Python really support multiple interpreters > in the same process? How is it possible? Who uses this? [Not sure if you are joking] There is certainly some support for multiple interpreters in Python; the most prominent user of this feature is mod_python. Regards, Martin From fuzzyman at voidspace.org.uk Mon Dec 20 01:57:04 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 20 Dec 2010 00:57:04 +0000 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: <1292799745.5914.76.camel@marge> References: <4D0C0688.1080108@haypocalc.com> <1292799745.5914.76.camel@marge> Message-ID: <4D0EA9E0.4080707@voidspace.org.uk> On 19/12/2010 23:02, Victor Stinner wrote: > Le samedi 18 d?cembre 2010 ? 08:50 -0500, James Y Knight a ?crit : >> I think instead of calling abort() to kill the process, you should: >> - install the signal handler with SA_NODEFER|SA_RESETHAND (or if sigaction is not present, explicitly reset the action to SIG_DFL and unblock first thing upon entering the handler), and then, >> - at the end of the handler, kill(getpid(), orig_signal) in order to abort the process. > I don't think that kill() is available on Windows. Thanks to Brian Curtin there is now an os.kill() on Windows, but it doesn't handle the full range of signals. Michael -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html From victor.stinner at haypocalc.com Mon Dec 20 05:53:49 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Mon, 20 Dec 2010 05:53:49 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: <4D0EABDA.3060702@v.loewis.de> References: <4D0C0688.1080108@haypocalc.com> <4D0CBDB8.6010608@haypocalc.com> <4D0D0D52.5020204@v.loewis.de> <20101218211836.0c2b0792@pitrou.net> <4D0E54B7.2070604@v.loewis.de> <1292795941.5914.23.camel@marge> <4D0EABDA.3060702@v.loewis.de> Message-ID: <1292820829.4763.16.camel@marge> Le lundi 20 d?cembre 2010 ? 02:05 +0100, "Martin v. L?wis" a ?crit : > The problem is that merely being POSIX compliant would not be enough > to allow calling a function in a signal handler. Instead, the function > *also* needs to be async-signal safe. Yes, this issue was fixed in an older version of my patch. Starting at version 9, the signal handler only call signal-safe functions. > Looking at your function list, my other concern is that you are calling > Python API without holding the GIL, IIUC. In particular, you are > accessing _PyThreadState_Current, which may not point to the current > thread if the current thread has released the GIL. Ah? Where does _PyThreadState_Current point to if the GIL is not hold when the fault handler is called? My patch changes also Py_FatalError() to display the Python backtrace. It looks that _PyThreadState_Current can be NULL if the GIL is released. In this case, _Py_DumpBacktrace() just do nothing. There is also a gil_last_holder variable: can it be used to get the thread state (especially if the thread state was deleted)? I don't think that it will possible the acquire the GIL in Py_FatalError() or in the fault handler. Victor From martin at v.loewis.de Mon Dec 20 08:22:40 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 20 Dec 2010 08:22:40 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: <1292820829.4763.16.camel@marge> References: <4D0C0688.1080108@haypocalc.com> <4D0CBDB8.6010608@haypocalc.com> <4D0D0D52.5020204@v.loewis.de> <20101218211836.0c2b0792@pitrou.net> <4D0E54B7.2070604@v.loewis.de> <1292795941.5914.23.camel@marge> <4D0EABDA.3060702@v.loewis.de> <1292820829.4763.16.camel@marge> Message-ID: <4D0F0440.5070806@v.loewis.de> >> Looking at your function list, my other concern is that you are calling >> Python API without holding the GIL, IIUC. In particular, you are >> accessing _PyThreadState_Current, which may not point to the current >> thread if the current thread has released the GIL. > > Ah? Where does _PyThreadState_Current point to if the GIL is not hold > when the fault handler is called? The GIL is likely held by a different thread, then. _PyThreadState_Current will point to the state of this other thread. > It looks that _PyThreadState_Current can be NULL if the GIL is released. > In this case, _Py_DumpBacktrace() just do nothing. There is also a > gil_last_holder variable: can it be used to get the thread state > (especially if the thread state was deleted)? Of this thread? I don't think so. gil_last_holder might also refer to a different thread. > I don't think that it will possible the acquire the GIL in > Py_FatalError() or in the fault handler. I agree. Regards, Martin From phd at phd.pp.ru Mon Dec 20 10:23:02 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Mon, 20 Dec 2010 12:23:02 +0300 Subject: [Python-Dev] Locale-specific formatting In-Reply-To: <4D0E9EAF.6010000@netwok.org> References: <4D0BFB8F.2040301@mrabarnett.plus.com> <20101218153331.GA22017@phd.pp.ru> <4D0E9EAF.6010000@netwok.org> Message-ID: <20101220092302.GA5973@phd.pp.ru> On Mon, Dec 20, 2010 at 01:09:19AM +0100, ??ric Araujo wrote: > Le 18/12/2010 16:33, Oleg Broytman a ??crit : > > This is quite a known problem, not specific to Python. Locale > > settings are global for a process, and this is one of the thousands > > reasons why locale is considered so horrible. > > ICU is perhaps the only way around the problem. > > Babel rocks: http://babel.edgewall.org/ Unicode CLDR! Never heard of it before, thank you for pointing this out. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From steve at pearwood.info Mon Dec 20 10:55:10 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 20 Dec 2010 20:55:10 +1100 Subject: [Python-Dev] r87389 - in python/branches/py3k: Doc/library/unittest.rst Lib/unittest/case.py Misc/NEWS In-Reply-To: <20101220014840.6b734274@pitrou.net> References: <20101218200004.99110EE985@mail.python.org> <20101218214830.1aa8be32@pitrou.net> <20101219141355.5ac7cd2f@pitrou.net> <20101220014840.6b734274@pitrou.net> Message-ID: <4D0F27FE.4020706@pearwood.info> Antoine Pitrou wrote: > For a non-native English speaker, 'a' and 'b' don't evoke 'after' and > 'before' but simply the first two letters of the latin alphabet, and > their ordering is therefore obvious with respect to function arguments. It's not just non-native English speakers either. I too think of a, b as being first, second rather than after, before. -- Steven From stefan at bytereef.org Mon Dec 20 12:08:35 2010 From: stefan at bytereef.org (Stefan Krah) Date: Mon, 20 Dec 2010 12:08:35 +0100 Subject: [Python-Dev] Issue #8863 adds a new PYTHONNOFAULTHANDLER environment variable In-Reply-To: References: <4D0C0688.1080108@haypocalc.com> <4D0CBDB8.6010608@haypocalc.com> Message-ID: <20101220110835.GA13825@yoda.bytereef.org> Alexander Belopolsky wrote: > On Sat, Dec 18, 2010 at 11:50 AM, Georg Brandl wrote: > .. > > In any case, this is coming pretty late; beta 2 is scheduled for this > > weekend, and even if this is something that only kicks in when all hope > > is lost anyway, it is a new feature. ?I should like to hear approval > > from a few more devs before I will let this go into 3.2. > > > > I am -1 on the feature as written. I would be -0 if it did not > install signal handlers by default and even better was implemented in > a separate module, not in core. I would not want this to be the default either. I think the output is not particularly informative: $ ./python Lib/test/crashers/gc_inspection.py (
RetroSearch is an open source project built by @garambo
| Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4