Showing content from http://mail.python.org/pipermail/python-dev/attachments/20180105/bbc7446e/attachment.html below:
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">Some inline responses to Paul (I snipped everything I agree with).<br></div><div class="gmail_quote"><br></div><div class="gmail_quote">On Wed, Jan 3, 2018 at 3:34 AM, Paul Moore <span dir="ltr"><<a href="mailto:p.f.moore@gmail.com" target="_blank">p.f.moore@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On 28 December 2017 at 06:08, Yury Selivanov <<a href="mailto:yselivanov.ml@gmail.com">yselivanov.ml@gmail.com</a>> wrote:<br>
> This is a second version of PEP 567.<br>
[...]</span><br><div><div class="h5"><br>
> The notion of "current value" deserves special consideration:<br>
> different asynchronous tasks that exist and execute concurrently<br>
> may have different values for the same key. This idea is well-known<br>
> from thread-local storage but in this case the locality of the value is<br>
> not necessarily bound to a thread. Instead, there is the notion of the<br>
> "current ``Context``" which is stored in thread-local storage, and<br>
> is accessed via ``contextvars.copy_context()`` function.<br>
<br>
</div></div>Accessed by copying it? That seems weird to me. I'd expect either that you'd be<br>
able to access the current Context directly, *or* that you'd say that the<br>
current Context is not directly accessible by the user, but that a copy can be<br>
obtained using copy_context. But given that the Context is immutable, why the<br>
need to copy it?<br></blockquote><div><br></div><div>Because it's not immutable. (I think by now people following this thread understand that.) The claims or implications in the PEP that Context is immutable are wrong (and contradict the recommended use of run() for asyncio, for example).<br></div><div>Â </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Also, the references to threads in the above are confusing. It says that this<br>
is a well-known concept in terms of thread-local storage, but this case is<br>
different. It then goes on to say that the current Context is stored in thread<br>
local storage, which gives me the impression that the new idea *is* related to<br>
thread local storage...<br></blockquote><div><br></div><div>The PEP's language does seem confused. This is because it doesn't come out and define the concept of "task". The correspondence is roughly that thread-locals are to threads what Contexts and ContextVars are to tasks. (This still requires quite a bit of squinting due to the API differences but it's better than what the PEP says.)<br></div><div>Â </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I think that the fact that a Context is held in thread-local storage is an<br>
implementation detail. Assuming I'm right, don't bother mentioning it - simply<br>
say that there's a notion of a current Context and leave it at that.<span class=""><br></span></blockquote><div><br></div><div>No, actually it's important that each thread has its own current context. It is even possible to pass Contexts between threads, e.g. if you have a ThreadExecutor e, you can call e.submit(ctx.run, some_function, args...).</div><div><br></div><div>However, run() is not thread-safe! @Yury: There's an example that does almost exactly this in the PEP, but I think it could result in a race condition if you called run() concurrently on that same context in a different thread -- whichever run() finishes last will overwrite the other's state. I think we should just document this and recommend always using a fresh copy in such scenarios. Hm, does this mean it would be good to have an explicit Context.copy() method? Or should we show how to create a copy of an arbitrary Context using ctx.run(copy_context)?<br></div><div>Â </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">
> Manipulation of the current ``Context`` is the responsibility of the<br>
> task framework, e.g. asyncio.<br>
><br>
> A ``Context`` is conceptually a read-only mapping, implemented using<br>
> an immutable dictionary. The ``ContextVar.get()`` method does a<br>
> lookup in the current ``Context`` with ``self`` as a key, raising a<br>
> ``LookupError``Â or returning a default value specified in<br>
> the constructor.<br>
><br>
> The ``ContextVar.set(value)`` method clones the current ``Context``,<br>
> assigns the ``value`` to it with ``self`` as a key, and sets the<br>
> new ``Context`` as the new current ``Context``.<br>
><br>
<br>
</span>On first reading, this confused me because I didn't spot that you're saying a<br>
*Context* is read-only, but a *ContextVar* has get and set methods.<br>
<br>
Maybe reword this to say that a Context is a read-only mapping from ContextVars<br>
to values. A ContextVar has a get method that looks up its value in the current<br>
Context, and a set method that replaces the current Context with a new one that<br>
associates the specified value with this ContextVar.<br>
<br>
(The current version feels confusing to me because it goes into too much detail<br>
on how the implementation does this, rather than sticking to the high-level<br>
specification)<span class=""><br></span></blockquote><div><br></div><div>We went over this passage in another subthread. IMO what it says about ContextVar.set() is incorrect.<br></div><div>Â </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">
> Specification<br>
> =============<br>
><br>
> A new standard library module ``contextvars`` is added with the<br>
> following APIs:<br>
><br>
> 1. ``copy_context() -> Context`` function is used to get a copy of<br>
>Â Â the current ``Context`` object for the current OS thread.<br>
><br>
> 2. ``ContextVar`` class to declare and access context variables.<br>
><br>
> 3. ``Context`` class encapsulates context state. Every OS thread<br>
>Â Â stores a reference to its current ``Context`` instance.<br>
>Â Â It is not possible to control that reference manually.<br>
>Â Â Instead, the ``Context.run(callable, *args, **kwargs)`` method is<br>
>Â Â used to run Python code in another context.<br>
<br>
</span>Context.run() came a bit out of nowhere here. Maybe the part from "It<br>
is not possible..." should be in the introduction above? Something<br>
like the following, covering this and copy_context:<br>
<br>
  The current Context cannot be accessed directly by user code. If the<br>
  framework wants to run some code in a different Context, the<br>
  Context.run(callable, *args, **kwargs) method is used to do that. To<br>
  construct a new context for this purpose, the current context can be copied<br>
  via the copy_context function, and manipulated prior to the call to run().<span class=""><br></span></blockquote><div><br></div><div>I agree that run() comes out of nowhere but I'd suggest a simpler fix -- just say "Instead, Context.run() must be used, see below."<br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">
><br>
> contextvars.ContextVar<br>
> ----------------------<br>
><br>
> The ``ContextVar`` class has the following constructor signature:<br>
> ``ContextVar(name, *, default=_NO_DEFAULT)``. The ``name`` parameter<br>
> is used only for introspection and debug purposes, and is exposed<br>
> as a read-only ``ContextVar.name`` attribute. The ``default``<br>
> parameter is optional. Example::<br>
><br>
>Â Â Â # Declare a context variable 'var' with the default value 42.<br>
>Â Â Â var = ContextVar('var', default=42)<br>
><br>
> (The ``_NO_DEFAULT`` is an internal sentinel object used to<br>
> detect if the default value was provided.)<br>
<br>
</span>My first thought was that default was the context variable's initial value. But<br>
if that's what it is, why not call it that? If the default has another effect<br>
as well as being the initial value, maybe clarify here what that is?<span class=""><br></span></blockquote><div><br></div><div>IMO it's more and different than the "initial value". The ContextVar never gets set directly to the default -- you can verify this by checking "var in ctx" for a variable that has a default but isn't set -- it's not present. It really is used as the "default default" by ContextVar.get(). That's not an implementation detail.<br></div><div>Â </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">
> ``ContextVar.get()`` returns a value for context variable from the<br>
> current ``Context``::<br>
><br>
>Â Â Â # Get the value of `var`.<br>
>Â Â Â var.get()<br>
><br>
> ``ContextVar.set(value) -> Token`` is used to set a new value for<br>
> the context variable in the current ``Context``::<br>
><br>
>Â Â Â # Set the variable 'var' to 1 in the current context.<br>
>Â Â Â var.set(1)<br>
><br>
> ``ContextVar.reset(token)`` is used to reset the variable in the<br>
> current context to the value it had before the ``set()`` operation<br>
> that created the ``token``::<br>
><br>
>Â Â Â assert var.get(None) is None<br>
<br>
</span>get doesn't take an argument. Typo?<span class=""><br></span></blockquote><div><br></div><div>Actually it does, the argument specifies a default (to override the "default default" set in the constructor). However this hasn't been mentioned yet at this point (the description of ContextVar.get() earlier doesn't mention it, only the implementation below). It would be good to update the earlier description of ContextVar.get() to mention the optional default (and how it interacts with the "default default").<br></div><div class="h5"><br>
> asyncio<br>
> -------<br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">[...]<br>
><br>
> C API<br>
> -----<br>
><br>
[...]<br>
<br>
I haven't commented on these as they aren't my area of expertise.<span class=""><br></span></blockquote><div><br></div><div>(Too bad, since there's an important clue about the mutability of Context hidden in this section! :-)<br></div><div>Â </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">
> Implementation<br>
> ==============<br>
><br>
> This section explains high-level implementation details in<br>
> pseudo-code. Some optimizations are omitted to keep this section<br>
> short and clear.<br>
<br>
</span>Again, I'm ignoring this as I don't really have an interest in how the facility<br>
is implemented.<br></blockquote><div><br></div><div>(Again, too bad, since despite the section heading this acts as a pseudo-code specification that is much more exact than the "specification" section above.)<br></div><div>Â <span class=""></span><br><span class=""></span></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">
> Implementation Notes<br>
> ====================<br>
><br>
> * The internal immutable dictionary for ``Context`` is implemented<br>
>  using Hash Array Mapped Tries (HAMT). They allow for O(log N)<br>
>Â Â ``set`` operation, and for O(1) ``copy_context()`` function, where<br>
>  *N* is the number of items in the dictionary. For a detailed<br>
>Â Â analysis of HAMT performance please refer to :pep:`550` [1]_.<br>
<br>
</span>Would it be worth exposing this data structure elsewhere, in case<br>
other uses for it exist?<span class=""><br></span></blockquote><div><br></div><div>I've asked Yury this several times, but he's shy about exposing it. Maybe it's better to wait until 3.8 so the implementation and its API can stabilize a bit before the API is constrained by backwards compatibility.<br></div><div>Â </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">
> * ``ContextVar.get()`` has an internal cache for the most recent<br>
>  value, which allows to bypass a hash lookup. This is similar<br>
>Â Â to the optimization the ``decimal`` module implements to<br>
>Â Â retrieve its context from ``PyThreadState_GetDict()``.<br>
>Â Â See :pep:`550` which explains the implementation of the cache<br>
>Â Â in a great detail.<br>
><br>
<br>
</span>Should the cache (or at least the performance guarantees it implies) be part of<br>
the spec? Do we care if other implementations fail to implement a cache?<br></blockquote></div></div><div class="gmail_extra"><br></div><div class="gmail_extra">IMO it's a quality-of-implementation issue, but the speed of the CPython implementation plays an important role in acceptance of the PEP (since we don't want to slow down e.g. asyncio task creation).<br></div><div class="gmail_extra"><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">--Guido van Rossum (<a href="http://python.org/~guido" target="_blank">python.org/~guido</a>)</div>
</div></div>
RetroSearch is an open source project built by @garambo
| Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4