Showing content from http://mail.python.org/pipermail/python-dev/attachments/20160208/c7e674dc/attachment.html below:
<div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Mon, 8 Feb 2016 at 06:33 Victor Stinner <<a href="mailto:victor.stinner@gmail.com">victor.stinner@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
Since 3.3, functions of the os module started to emit<br>
DeprecationWarning when called with bytes filenames.<br>
<br>
The rationale is quite simple: Windows native type for filenames is<br>
Unicode, and the Windows has a weird behaviour when you use bytes. For<br>
example, os.listdir(b'.') gives you paths which cannot be used with<br>
open() on filenames which are not encodable the ANSI code page.<br>
Unencodable characters are replaced with "?". The following issue was<br>
opened to document this weird behaviour (but the doc was never<br>
completed):<br>
<br>
"Document that bytes OS API can returns unusable results on Windows"<br>
<a href="http://bugs.python.org/issue16700" rel="noreferrer" target="_blank">http://bugs.python.org/issue16700</a><br>
<br>
<br>
When the new os.scandir() API was designed, I asked to *not* support<br>
bytes filenames since they are "broken by design".<br>
<a href="https://www.python.org/dev/peps/pep-0471/" rel="noreferrer" target="_blank">https://www.python.org/dev/peps/pep-0471/</a><br>
<br>
Recently, an user complained that os.walk() doesn't work with bytes on<br>
Windows anymore:<br>
<br>
"Regression: os.walk now using os.scandir() breaks bytes filenames on windows"<br>
<a href="http://bugs.python.org/issue25911" rel="noreferrer" target="_blank">http://bugs.python.org/issue25911</a><br>
<br>
<br>
Serhiy Storchaka just pushed a change to reintroduce support bytes<br>
support on Windows in os.walk(), but I would prefer to do the<br>
*opposite*: drop supports for bytes filenames on Windows.<br>
<br>
Are we brave enough to force users to use the "right" type for filenames?<br>
<br>
--<br>
<br>
On Python 2, it wasn't possible to use Unicode for filenames, many<br>
functions fail badly with Unicode, especially when you mix bytes and<br>
Unicode.<br>
<br>
On Python 3, Unicode is the "natural" types, most Python functions<br>
prefer Unicode, and the PEP 383 (surrogateescape) allows to safetely<br>
use Unicode on UNIX even with undecodable filenames (invalid bytes are<br>
stored as Unicode surrogate characters).<br></blockquote><div><br></div><div>If Unicode string don't work in Python 2 then what is Python 2/3 to do as a cross-platform solution if we completely remove bytes support in Python 3? Wouldn't that mean there is no common type between Python 2 & 3 that one can use which will work with the os module except native strings (which are difficult to get right)? <br></div></div></div>
RetroSearch is an open source project built by @garambo
| Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4