Some functions and types of the C-API are designated unstable, meaning that they will not change in patch (bugfix/security) releases, but may change between minor releases (e.g. between 3.11 and 3.12) without deprecation warnings.
Any C API with a leading underscore is designated internal, meaning that it may change or disappear without any notice.
Motivation & Rationale Unstable C API tierThe Python C-API is currently divided into three stability tiers:
Tools requiring access to CPython internals (e.g. advanced debuggers and JIT compilers) are often built for minor series releases of CPython, and assume that the C-API internals used do not change in patch releases. To support these tools, we need a tier between the Public and Private C-API, with guarantees on stability throughout the minor-series release: the proposed Unstable tier.
Some functions, like PyCode_New(), are documented as unstable (“Calling [it] directly can bind you to a precise Python version”), and also often change in practice. The unstable tier should make their status obvious even to people who don’t read the docs carefully enough, making them hard to use accidentally.
Currently, CPython developers don’t agree on the exact meaning of a leading underscore in API names. It is used to mean two different things:
The unclear meaning makes the underscore less useful than it could be. If it only marked private API, CPython developers could change underscored functions, or remove unused ones, without researching how they’re documented or used outside CPython.
With the introduction of a dedicated unstable tier, we can clarify the meaning of the leading underscore. It should mark private API only.
Not breaking code unnecessarilyThis PEP specifies that API in the unstable tier should have a special name prefix. This means functions (macros, etc.) will need to be renamed. After a rename, the old name should continue to be available until an incompatible change is made (i.e. until call sites need to be updated anyway). In other words, just changing the tier of a function shouldn’t break users’ code.
SpecificationThe C API is divided by stability expectations into three “sections” (internal, public, and limited). We’ll now call these stability tiers, or tiers for short.
An Unstable tier will be added.
APIs (functions, types, etc.) in this tier will named with the PyUnstable_ prefix, with no leading underscore.
They will be declared in headers used for public API (Include/*.h, rather than in a subdirectory like Include/unstable/).
Several rules for dealing with the unstable tier will be introduced:
PyUnstable_* name.
The old name should be deprecated (e.g. with Py_DEPRECATED), but continue to be available until an incompatible change is made to the API. Per Python’s backwards compatibility policy (PEP 387), this deprecation needs to last at least two releases (without an SC exceptions). But it can also last indefinitely – for example, if PEP 590’s “provisional” _PyObject_Vectorcall was added today, it would be initially named PyUnstable_Object_Vectorcall and there would be no plan to remove this name.
In the following cases, an incompatible change (and thus removing the deprecated name) is allowed without an SC exception, as if the function was already part of the Unstable tier:
For examples, see the initial unstable API specified in this PEP.
PyUnstable_* name.
If the old name is documented, or widely used externally, it should continue to be available until an incompatible change is made (and call sites need to be updated). It should start raising deprecation warnings (e.g. using Py_DEPRECATED).
PyUnstable_* prefix.
The old name should remain available until the API is deprecated or removed.
These rules will be documented in the devguide, and user documentation will be updated accordingly.
Reference docs for C API named PyUnstable_* will automatically show notes with links to the unstable tier documentation.
C API named with a leading underscore, as well as API only available with Py_BUILD_CORE, will be considered internal. This means:
This might happen long after this PEP is accepted. Consequently, for a few years core devs should do some research before changing underscored API, especially if it doesn’t need Py_BUILD_CORE.
Users of the C API are encouraged to search their codebase for _Py and _PY identifier prefixes, and treat any hits as issues to be eventually fixed – either by switching to an existing alternative, or by opening a CPython issue to request exposing public API for their use case, and eventually switching to that.
The following API will be moved to the Unstable tier in the initial implementation as proof of the concept.
Code object constructors:
PyUnstable_Code_New() (renamed from PyCode_New)PyUnstable_Code_NewWithPosOnlyArgs() (renamed from PyCode_NewWithPosOnlyArgs)Code extra information (PEP 523):
PyUnstable_Eval_RequestCodeExtraIndex() (renamed from _PyEval_RequestCodeExtraIndex)PyUnstable_Code_GetExtra() (renamed from _PyCode_GetExtra)PyUnstable_Code_SetExtra() (renamed from _PyCode_SetExtra)More are expected in Python 3.12, without the need for another PEP.
Backwards CompatibilityThe C API backwards compatibility expectations will be made clearer.
All renamed API will be available under old names for as long as feasible.
How to Teach ThisThe changes affect advanced C programmers, who should consult the updated reference documentation, devguide and/or What’s New document.
Reference Implementationhttps://github.com/python/cpython/compare/main…encukou:unstable-tier
Rejected Ideas No special prefixIn the initial version of this PEP, unstable API didn’t have the PyUnstable prefix. Instead, defining Py_USING_UNSTABLE_API made the API available in a given source file, signifying acknowledgement that the file as a whole will potentially need to be revisited for each Python release.
However, it was decided that unstable-ness needs to be exposed in the individual names.
Underscore prefixIt would be possible to mark both private and unstable API with leading underscores. However, that would dilute the meaning of _Py prefix. Reserving the prefix for internal API only makes it trivial to search for.
It might be good to add a similar tier in the Python (not C) API, e.g. for types.CodeType. However, the mechanism for that would need to be different. This is outside the scope of the PEP.
This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4