Showing content from http://mail.python.org/pipermail/python-dev/attachments/20161104/7fdda1cc/attachment-0001.html below:
<div dir="ltr"><div class="gmail_default" style="color:rgb(0,0,0)"><div class="gmail_default" style="font-size:12.8px"><div class="gmail_default" style="font-size:12.8px">Answers inline...</div><div class="gmail_default" style="font-size:12.8px"><br></div><div class="gmail_extra" style="color:rgb(34,34,34);font-size:12.8px"><div class="gmail_quote"><span class="gmail-im">On Fri, Nov 4, 2016 at 5:56 AM, Eric V. Smith <span dir="ltr"><<a href="mailto:eric@trueblade.com" target="_blank">eric@trueblade.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="gmail-m_-947583576949057914gmail-">On 11/3/2016 3:06 PM, Fabio Zadrozny wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi Python-Dev,<br><br>I'm trying to get my head around on what's accepted in f-strings --<br><a href="https://www.python.org/dev/peps/pep-0498/" rel="noreferrer" target="_blank">https://www.python.org/dev/pep<wbr>s/pep-0498/</a> seems very light on the<br>details on what it does accept as an expression and how things should<br>actually be parsed (and the current implementation still doesn't seem to<br>be in a state for a final release, so, I thought asking on python-dev<br>would be a reasonable option).<br></blockquote><br></span>In what way do you think the implementation isn't ready for a final release?</blockquote><div><br></div></span><div><div class="gmail_default" style="color:rgb(0,0,0)">Well, the cases listed in the docsâ (<a href="https://hg.python.org/cpython/file/default/Doc/reference/lexical_analysis.rst" target="_blank">https://hg.python.org/<wbr>cpython/file/default/Doc/<wbr>reference/lexical_analysis.rst</a><wbr>) don't work in the latest release (with SyntaxErrors) -- and the bug I created related to it: <a href="http://bugs.python.org/issue28597" target="_blank">http://bugs.python.org/<wbr>issue28597</a> was promptly closed as duplicate -- so, I assumed (maybe wrongly?) that the parsing still needs work.</div></div><span class="gmail-im"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="gmail-m_-947583576949057914gmail-"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">I was thinking there'd be some grammar for it (something as<br><a href="https://docs.python.org/3.6/reference/grammar.html" rel="noreferrer" target="_blank">https://docs.python.org/3.6/re<wbr>ference/grammar.html</a>), but all I could<br>find related to this is a quote saying that f-strings should be<br>something as:<br><br>f ' <text> { <expression> <optional !s, !r, or !a> <optional : format<br>specifier> } <text><br><br>So, given that, is it safe to assume that <expression> would be equal to<br>the "test" node from the official grammar?<br></blockquote><br></span>No. There are really three phases here:<br><br>1. The f-string is tokenized as a regular STRING token, like all other strings (f-, b-, u-, r-, etc).<br>2. The parser sees that it's an f-string, and breaks it into expression and text parts.<br>3. For each expression found, the expression is compiled with PyParser_ASTFromString(..., Py_eval_input, ...).<br><br>Step 2 is the part that limits what types of expressions are allowed. While scanning for the end of an expression, it stops at the first '!', ':', or '}' that isn't inside of a string and isn't nested inside of parens, braces, and brackets.<br></blockquote><div><br></div></span><div><div class="gmail_default" style="color:rgb(0,0,0)">âIt'd be nice if at least this description could be added to the PEP (as all other language implementations and IDEs will have to work the same way and will probably reference it) -- a grammar example, even if not used would be helpful (personally, I think hand-crafted parsers are always worse in the long run compared to having a proper grammar with a parser, although I understand that if you're not really used to it, it may be more work to set it up).</div></div><div class="gmail_default" style="color:rgb(0,0,0)"><br></div><div class="gmail_default" style="color:rgb(0,0,0)">Also, I find it a bit troubling that PyParser_ASTFromString is used there and not just the node which would be related to an expression, although I understand it's probably an easier approach, although in the end you probably have to filter it and end up just accepting what's beneath the "test" from the grammar, no? (i.e.: that's what a lambda body accepts).</div><span class="gmail-im"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">The nesting-tracking is why these work:<br>>>> f'{(lambda x:3)}'<br>'<function <lambda> at 0x000000000296E560>'<br>>>> f'{(lambda x:3)!s:.20}'<br>'<function <lambda> a'<br><br>But this doesn't:<br>>>> f'{lambda x:3}'<br> File "<fstring>", line 1<br>  (lambda x)<br>       ^<br>SyntaxError: unexpected EOF while parsing<br><br>Also, backslashes are not allowed anywhere inside of the expression. This was a late change right before beta 1 (I think), and differs from the PEP and docs. I have an open item to fix them.<span class="gmail-m_-947583576949057914gmail-"><br><br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">I initially thought it would obviously be, but the PEP says that using a<br>lamda inside the expression would conflict because of the colon (which<br>wouldn't happen if a proper grammar was actually used for this parsing<br>as there'd be no conflict as the lamda would properly consume the<br>colon), so, I guess some pre-parser steps takes place to separate the<br>expression to then be parsed, so, I'm interested on knowing how exactly<br>that should work when the implementation is finished -- lots of plus<br>points if there's actually a grammar to back it up :)<br></blockquote><br></span>I've considered using the grammar and tokenizer to implement f-string parsing, but I doubt it will ever happen. It's a lot of work, and everything that produced or consumed tokens would have to be aware of it. As it stands, if you don't need to look inside of f-strings, you can just treat them as regular STRING tokens.<br></blockquote><div><br></div></span><div><div class="gmail_default" style="color:rgb(0,0,0)">âWell, I think all language implementations / IDEs (or at least those which want to give syntax errors) will *have* to look inside f-strings.</div></div><div class="gmail_default" style="color:rgb(0,0,0)"><br></div><div class="gmail_default" style="color:rgb(0,0,0)">Also, you could still have a separate grammar saying how to look inside f-strings (this would make the lives of other implementors easier) even if it was a post-processing step as you're doing now.</div><span class="gmail-im"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>I hope that helps.<br><br>Eric.<br></blockquote><div><br></div></span><div><div class="gmail_default" style="color:rgb(0,0,0);display:inline">âIt does, thank you very much.</div></div><div><br></div><div><div class="gmail_default" style="color:rgb(0,0,0)">âBest Regards,</div><div class="gmail_default" style="color:rgb(0,0,0)"><br></div><div class="gmail_default" style="color:rgb(0,0,0)">Fabioâ</div></div></div></div></div></div></div>
RetroSearch is an open source project built by @garambo
| Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4