On Mon, 27 Sep 2010 23:45:45 -0400 Steve Holden <steve at holdenweb.com> wrote: > On 9/27/2010 11:27 PM, Benjamin Peterson wrote: > > 2010/9/27 Meador Inge <meadori at gmail.com>: > >> which, as seen in the trace, is because the 'detect_encoding' function in > >> 'Lib/tokenize.py' searches for 'BOM_UTF8' (a 'bytes' object) in the string > >> to tokenize 'first' (a 'str' object). It seems to me that strings should > >> still be able to be tokenized, but maybe I am missing something. > >> Is the implementation of 'detect_encoding' correct in how it attempts to > >> determine an encoding or should I open an issue for this? > > > > Tokenize only works on bytes. You can open a feature request if you desire. > > > Working only on bytes does seem rather perverse. I agree, the morality of bytes objects could have been better :)
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4