This PEP has been rejected in favour of PEP 631 due to its popularity, consistency with the existing usage of PEP 508 strings, and compatibility with existing packaging tool suites.
AbstractThis PEP specifies how to write a project’s dependencies in a pyproject.toml file for packaging-related tools to consume using the fields defined in PEP 621, as an alternative to the PEP 508-based approach defined in PEP 631.
There are multiple benefits to using TOML tables and other data-types to represent requirements rather than PEP 508 strings:
Most of this is taken from discussions in the PEP 621 dependencies topic. This has elements from Pipfile, Poetry, Dart’s dependencies and Rust’s Cargo. A comparison document shows advantages and disadvantages between this format and PEP 508-style specifiers.
In the specification of multiple requirements with the same distribution name (where environment markers choose the appropriate dependency), the chosen solution is similar to Poetry’s, where an array of requirements is allowed.
The direct-reference keys closely align with and utilise PEP 610 and PEP 440 as to reduce differences in the packaging ecosystem and rely on previous work in specification.
SpecificationAs in PEP 621, if metadata is improperly specified then tools MUST raise an error. The metadata MUST conform to the TOML specification.
To reduce confusion with this document being a specification for specifying dependencies, the word “requirement” is used to mean a PEP 508 dependency specification.
The following tables are added to the project table specified in PEP 621.
dependencies
Format: table
The keys inside this table are the names of the required distribution. The values can have one of the following types:
version in the requirement table, except allowing the empty string "" to place no restriction on the version.[] as a value.The keys of the requirement table are as follows (all are optional):
version (string): a PEP 440 version specifier, which is a comma-delimited list of version specifier clauses. The string MUST be non-empty.extras (array of strings): a list of PEP 508 extras declarations for the distribution. The list MUST be non-empty.markers (string): a PEP 508 environment marker expression. The string MUST be non-empty.url (string): the URL of the artifact to install and satisfy the requirement. Note that file:// is the prefix used for packages to be retrieved from the local filesystem.git, hg, bzr or svn (string): the URL of a VCS repository (as specified in PEP 440) to clone, whose tree will be installed to satisfy the requirement. Further VCS keys will be added via amendments to PEP 610, however tools MAY opt to support other VCS’s using their command-line command prior to the acceptance of the amendment.revision (string): the identifier for a specific revision of the specified VCS repository to check-out before installation. Users MUST only provide this when one of git, hg, bzr, svn, or another VCS key is used to identify the distribution to install. Revision identifiers are suggested in PEP 610.At most one of the following keys can be specified simultaneously, as they logically conflict with each other in the requirement: version, url, git, hg, bzr, svn, and any other VCS key.
An empty requirement table {} places no restriction on the requirement, in addition to the empty string "".
Any keys provided which are not specified in this document MUST cause an error in parsing.
optional-dependencies
Format: table
The keys inside this table are the names of an extra’s required distribution. The values can have one of the following types:
These requirement tables have the same specification as above, with the addition of the following required key:
for-extra (string): the name of the PEP 508 extra that this requirement is required for.Tools will need to convert this format to PEP 508 requirement strings. Below is an example implementation of that conversion (assuming validation is already performed):
def convert_requirement_to_pep508(name, requirement):
if isinstance(requirement, str):
requirement = {"version": requirement}
pep508 = name
if "extras" in requirement:
pep508 += " [" + ", ".join(requirement["extras"]) + "]"
if "version" in requirement:
pep508 += " " + requirement["version"]
if "url" in requirement:
pep508 += " @ " + requirement["url"]
for vcs in ("git", "hg", "bzr", "svn"):
if vcs in requirement:
pep508 += " @ " + vcs + "+" + requirement[vcs]
if "revision" in requirement:
pep508 += "@" + requirement["revision"]
extra = None
if "for-extra" in requirement:
extra = requirement["for-extra"]
if "markers" in requirement:
markers = requirement["markers"]
if extra:
markers = "extra = '" + extra + "' and (" + markers + ")"
pep508 += "; " + markers
return pep508, extra
def convert_requirements_to_pep508(dependencies):
pep508s = []
extras = set()
for name, req in dependencies.items():
if isinstance(req, list):
for sub_req in req:
pep508, extra = convert_requirement_to_pep508(name, sub_req)
pep508s.append(pep508)
if extra:
extras.add(extra)
else:
pep508, extra = convert_requirement_to_pep508(name, req)
pep508s.append(pep508)
if extra:
extras.add(extra)
return pep508s, extras
def convert_project_requirements_to_pep508(project):
reqs, _ = convert_requirements_to_pep508(project.get("dependencies", {}))
optional_reqs, extras = convert_requirements_to_pep508(
project.get("optional-dependencies", {})
)
reqs += optional_reqs
return reqs, extras
JSON schema
For initial validation, a JSON-schema can be used. Not only does this help tools have a consistent validation, but it allows code editors to highlight validation errors as users are building the dependencies list.
{
"$id": "spam",
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Project metadata",
"type": "object",
"definitions": {
"requirementTable": {
"title": "Full project dependency specification",
"type": "object",
"properties": {
"extras": {
"title": "Dependency extras",
"type": "array",
"items": {
"title": "Dependency extra",
"type": "string"
}
},
"markers": {
"title": "Dependency environment markers",
"type": "string"
}
},
"propertyNames": {
"enum": [
"extras",
"markers",
"version",
"url",
"git",
"hg",
"bzr",
"svn",
"for-extra"
]
},
"oneOf": [
{
"title": "Version requirement",
"properties": {
"version": {
"title": "Version",
"type": "string"
}
}
},
{
"title": "URL requirement",
"properties": {
"url": {
"title": "URL",
"type": "string",
"format": "uri"
}
},
"required": [
"url"
]
},
{
"title": "VCS requirement",
"properties": {
"revision": {
"title": "VCS repository revision",
"type": "string"
}
},
"oneOf": [
{
"title": "Git repository",
"properties": {
"git": {
"title": "Git URL",
"type": "string",
"format": "uri"
}
},
"required": [
"git"
]
},
{
"title": "Mercurial repository",
"properties": {
"hg": {
"title": "Mercurial URL",
"type": "string",
"format": "uri"
}
},
"required": [
"hg"
]
},
{
"title": "Bazaar repository",
"properties": {
"bzr": {
"title": "Bazaar URL",
"type": "string",
"format": "uri"
}
},
"required": [
"bzr"
]
},
{
"title": "Subversion repository",
"properties": {
"svn": {
"title": "Subversion URL",
"type": "string",
"format": "uri"
}
},
"required": [
"svn"
]
}
]
}
]
},
"requirementVersion": {
"title": "Version project dependency specification",
"type": "string"
},
"requirement": {
"title": "Project dependency specification",
"oneOf": [
{
"$ref": "#/definitions/requirementVersion"
},
{
"$ref": "#/definitions/requirementTable"
},
{
"title": "Multiple specifications",
"type": "array",
"items": {
"$ref": "#/definitions/requirementTable"
},
"minLength": 1
}
]
},
"optionalRequirementTable": {
"title": "Project optional dependency specification table",
"allOf": [
{
"$ref": "#/definitions/requirementTable"
},
{
"properties": {
"for-extra": {
"title": "Dependency's extra",
"type": "string"
}
},
"required": [
"for-extra"
]
}
]
},
"optionalRequirement": {
"title": "Project optional dependency specification",
"oneOf": [
{
"$ref": "#/definitions/optionalRequirementTable"
},
{
"title": "Multiple specifications",
"type": "array",
"items": {
"$ref": "#/definitions/optionalRequirementTable"
},
"minLength": 1
}
]
}
},
"properties": {
"dependencies": {
"title": "Project dependencies",
"type": "object",
"additionalProperties": {
"$ref": "#/definitions/requirement"
}
},
"optional-dependencies": {
"title": "Project dependencies",
"type": "object",
"additionalProperties": {
"$ref": "#/definitions/optionalRequirement"
}
}
}
}
Examples
Full artificial example:
[project.dependencies]
flask = { }
django = { }
requests = { version = ">= 2.8.1, == 2.8.*", extras = ["security", "tests"], markers = "python_version < '2.7'" }
pip = { url = "https://github.com/pypa/pip/archive/1.3.1.zip" }
sphinx = { git = "ssh://git@github.com/sphinx-doc/sphinx.git" }
numpy = "~=1.18"
pytest = [
{ version = "<6", markers = "python_version < '3.5'" },
{ version = ">=6", markers = "python_version >= '3.5'" },
]
[project.optional-dependencies]
pytest-timout = { for-extra = "dev" }
pytest-mock = [
{ version = "<6", markers = "python_version < '3.5'", for-extra = "dev" },
{ version = ">=6", markers = "python_version >= '3.5'", for-extra = "dev" },
]
In homage to PEP 631, the following is an equivalent dependencies specification for docker-compose:
[project.dependencies]
cached-property = ">= 1.2.0, < 2"
distro = ">= 1.2.0, < 2"
docker = { extras = ["ssh"], version = ">= 4.2.2, < 5" }
docopt = ">= 0.6.1, < 1"
jsonschema = ">= 2.5.1, < 4"
PyYAML = ">= 3.10, < 6"
python-dotenv = ">= 0.13.0, < 1"
requests = ">= 2.20.0, < 3"
texttable = ">= 0.9.0, < 2"
websocket-client = ">= 0.32.0, < 1"
# Conditional
"backports.shutil_get_terminal_size" = { version = "== 1.0.0", markers = "python_version < '3.3'" }
"backports.ssl_match_hostname" = { version = ">= 3.5, < 4", markers = "python_version < '3.5'" }
colorama = { version = ">= 0.4, < 1", markers = "sys_platform == 'win32'" }
enum34 = { version = ">= 1.0.4, < 2", markers = "python_version < '3.4'" }
ipaddress = { version = ">= 1.0.16, < 2", markers = "python_version < '3.3'" }
subprocess32 = { version = ">= 3.5.4, < 4", markers = "python_version < '3.2'" }
[project.optional-dependencies]
PySocks = { version = ">= 1.5.6, != 1.5.7, < 2", for-extra = "socks" }
ddt = { version = ">= 1.2.2, < 2", for-extra = "tests" }
pytest = { version = "< 6", for-extra = "tests" }
mock = { version = ">= 1.0.1, < 4", markers = "python_version < '3.4'", for-extra = "tests" }
Compatibility Examples
The authors of this PEP recognise that various tools need to both read from and write to this format for dependency specification. This section aims to provide direct comparison with and examples for translating to/from the currently used standard, PEP 508.
Note
For simplicity and clarity, various ways in which TOML allows you to specify each specification is not represented. These examples use the standard inline representation.
For example, while following are considered equivalent in TOML, we choose the second form for the examples in this section.
aiohttp.version = "== 3.6.2"
aiohttp = { version = "== 3.6.2" }
Version Constrained Dependencies
No Version Constraint
Simple Version Constraint
aiohttp >= 3.6.2, < 4.0.0
aiohttp = { version = ">= 3.6.2, < 4.0.0" }
Note
This can, for conciseness, be also represented as a string.
aiohttp = ">= 3.6.2, < 4.0.0"Direct Reference Dependencies
URL Dependency
aiohttp @ https://files.pythonhosted.org/packages/97/d1/1cc7a1f84097d7abdc6c09ee8d2260366f081f8e82da36ebb22a25cdda9f/aiohttp-3.6.2-cp35-cp35m-macosx_10_13_x86_64.whl
aiohttp = { url = "https://files.pythonhosted.org/packages/97/d1/1cc7a1f84097d7abdc6c09ee8d2260366f081f8e82da36ebb22a25cdda9f/aiohttp-3.6.2-cp35-cp35m-macosx_10_13_x86_64.whl" }
VCS Dependency
aiohttp @ git+ssh://git@github.com/aio-libs/aiohttp.git@master
aiohttp = { git = "ssh://git@github.com/aio-libs/aiohttp.git", revision = "master" }
Environment Markers
aiohttp >= 3.6.1; python_version >= '3.8'
aiohttp = { version = ">= 3.6.1", markers = "python_version >= '3.8'" }
A slightly extended example of the above, where a particular version of aiohttp is required based on the interpreter version.
aiohttp >= 3.6.1; python_version >= '3.8' aiohttp >= 3.0.0, < 3.6.1; python_version < '3.8'
aiohttp = [
{ version = ">= 3.6.1", markers = "python_version >= '3.8'" },
{ version = ">= 3.0.0, < 3.6.1", markers = "python_version < '3.8'" }
]
Complex Examples
Version Constraint
aiohttp [speedups] >= 3.6.2; python_version >= '3.8' and extra == 'http'
aiohttp = { version = ">= 3.6.2", extras = ["speedups"], markers = "python_version >= '3.8'", for-extra = "http" }
Direct Reference (VCS)
aiohttp [speedups] @ git+ssh://git@github.com/aio-libs/aiohttp.git@master ; python_version >= '3.8' and extra == 'http'
aiohttp = { git = "ssh://git@github.com/aio-libs/aiohttp.git", revision = "master", extras = ["speedups"], markers = "python_version >= '3.8'", for-extra = "http" }
Rejected Ideas Switch to an array for dependencies
Use an array instead of a table in order to have each element only be a table (with a name key) and no arrays of requirement tables. This was very verbose and restrictive in the TOML format, and having multiple requirements for a given distribution isn’t very common.
direct table in requirement
Include the direct-reference keys in a direct table, have the VCS specified as the value of a vcs key. This was more explicit and easier to include in a JSON-schema validation, but was decided to be too verbose and not as readable.
Include hash in direct-reference requirements. This was only for package lock-files, and didn’t really have a place in the project’s metadata.
Environment marker keysMake each PEP 508 environment marker as a key (or child-table key) in the requirement. This arguably increases readability and ease of parsing. The markers key would still be allowed for more advanced specification, with which the key-specified environment markers are and’d with the result of. This was deferred as more design needs to be undertaken.
This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4