A RetroSearch Logo

Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Search Query:

Showing content from https://mail.python.org/pipermail/python-dev/2011-September/113543.html below:

[Python-Dev] python -m tokenize in 3.x ?

[Python-Dev] python -m tokenize in 3.x ?Guido van Rossum guido at python.org
Thu Sep 8 05:48:19 CEST 2011
My guess that there was no specific intent -- most likely it occurred
to nobody that the main() functionality was actually useful. I'd say
it's fine to put it back, and then document it (so it won't be removed
again :-).

--Guido

On Wed, Sep 7, 2011 at 7:06 PM, Meador Inge <meadori at gmail.com> wrote:
> Hi All,
>
> I have been investing some 'tokenize' bugs recently.  As a part of
> that investigation I was trying to use '-m tokenize', which works
> great in 2.x:
>
> [meadori at motherbrain cpython]$ python2.7 -m tokenize test.py
> 1,0-1,5:        NAME    'print'
> 1,6-1,21:       STRING  '"Hello, World!"'
> 1,21-1,22:      NEWLINE '\n'
> 2,0-2,0:        ENDMARKER       ''
>
> In 3.x, however, the functionality has been removed and replaced with
> some hard-wired test code:
>
> [meadori at motherbrain cpython]$ python3 -m tokenize test.py
> TokenInfo(type=57 (ENCODING), string='utf-8', start=(0, 0), end=(0, 0), line='')
> TokenInfo(type=1 (NAME), string='def', start=(1, 0), end=(1, 3),
> line='def parseline(self, line):')
> TokenInfo(type=1 (NAME), string='parseline', start=(1, 4), end=(1,
> 13), line='def parseline(self, line):')
> TokenInfo(type=53 (OP), string='(', start=(1, 13), end=(1, 14),
> line='def parseline(self, line):')
> ...
>
> Why is this?  I found the commit where the functionality was removed
> [1], but no explanation.  Any objection to adding this feature back?
>
> [1] http://hg.python.org/cpython/rev/51e24512e305/
>
> --
> # Meador
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
More information about the Python-Dev mailing list

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4