This PEP describes the second version of a built-package format for Python called “wheel”. Wheel provides a Python-specific, relocatable package format that allows people to install software more quickly and predictably than re-building from source each time.
A wheel is a ZIP-format archive with a specially formatted file name and the .whl extension. It contains a single distribution nearly as it would be installed according to PEP 376 with a particular installation scheme. Simple wheels can be unpacked onto sys.path and used directly but wheels are usually installed with a specialized installer.
This version of the wheel specification adds support for installing distributions into many different directories, and adds a way to find those files after they have been installed.
PEP DeferralThis PEP is not currently being actively pursued, with Python packaging improvements currently focusing on the package build process rather than expanding the binary archive format to cover additional use cases.
Some specific elements to be addressed when work on this PEP is resumed in the future:
Wheel 1.0 is best at installing files into site-packages and a few other locations specified by distutils, but users would like to install files from single distribution into many directories – perhaps separate locations for docs, data, and code. Unfortunately not everyone agrees on where these install locations should be relative to the root directory. This version of the format adds many more categories, each of which can be installed to a different destination based on policy. Since it might also be important to locate the installed files at runtime, this version of the format also adds a way to record the installed paths in a way that can be read by the installed software.
Wheel installation notionally consists of two phases:
distribution-1.0.dist-info/WHEEL.distribution-1.0.dist-info/ and (if there is data) distribution-1.0.data/.distribution-1.0.data/ onto its destination path. Each subdirectory of distribution-1.0.data/ is a key into a dict of destination directories, such as distribution-1.0.data/(purelib|platlib|headers|scripts|data).#!python to point to the correct interpreter. (Note: Python scripts are usually handled by package metadata, and not included verbatim in wheel.)distribution-1.0.dist.info/RECORD with the installed paths.distribution-1.0.data directory.In practice, installers will usually extract files directly from the archive to their destinations without writing a temporary distribution-1.0.data/ directory.
#!python.
{distribution}-{version}.data/scripts/. If the first line of a file in scripts/ starts with exactly b'#!python', rewrite to point to the correct interpreter. Unix installers may need to add the +x bit to these files if the archive was created on Windows.
The b'#!pythonw' convention is allowed. b'#!pythonw' indicates a GUI script instead of a console script.
module:callable string in package metadata, and are not included verbatim in the wheel archive’s scripts directory. This kind of script gives the installer an opportunity to generate platform specific wrappers.
.dist-info at the end of the archive.
.dist-info files physically at the end of the archive. This enables some potentially interesting ZIP tricks including the ability to amend the metadata without rewriting the entire archive.
The wheel filename is {distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl.
For example, distribution-1.0-1-py27-none-any.whl is the first build of a package called ‘distribution’, and is compatible with Python 2.7 (any Python 2.7 implementation), with no ABI (pure Python), on any CPU architecture.
The last three components of the filename before the extension are called “compatibility tags.” The compatibility tags express the package’s basic interpreter requirements and are detailed in PEP 425.
Escaping and UnicodeEach component of the filename is escaped by replacing runs of non-alphanumeric characters with an underscore _:
re.sub("[^\w\d.]+", "_", distribution, re.UNICODE)
The archive filename is Unicode. The packaging tools may only support ASCII package names, but Unicode filenames are supported in this specification.
The filenames inside the archive are encoded as UTF-8. Although some ZIP clients in common use do not properly display UTF-8 filenames, the encoding is supported by both the ZIP specification and Python’s zipfile.
The contents of a wheel file, where {distribution} is replaced with the name of the package, e.g. beaglevote and {version} is replaced with its version, e.g. 1.0.0, consist of:
/, the root of the archive, contains all files to be installed in purelib or platlib as specified in WHEEL. purelib and platlib are usually both site-packages.{distribution}-{version}.dist-info/ contains metadata.{distribution}-{version}.data/ contains one subdirectory for each non-empty install scheme key not already covered, where the subdirectory name is an index into a dictionary of install paths (e.g. data, scripts, include, purelib, platlib).scripts and begin with exactly b'#!python' in order to enjoy script wrapper generation and #!python rewriting at install time. They may have any or no extension.{distribution}-{version}.dist-info/METADATA is Metadata version 1.1 or greater format metadata.{distribution}-{version}.dist-info/WHEEL is metadata about the archive itself in the same basic key: value format:
Wheel-Version: 1.9 Generator: bdist_wheel 1.9 Root-Is-Purelib: true Tag: py2-none-any Tag: py3-none-any Build: 1 Install-Paths-To: wheel/_paths.py Install-Paths-To: wheel/_paths.json
Wheel-Version is the version number of the Wheel specification.Generator is the name and optionally the version of the software that produced the archive.Root-Is-Purelib is true if the top level directory of the archive should be installed into purelib; otherwise the root should be installed into platlib.Tag is the wheel’s expanded compatibility tags; in the example the filename would contain py2.py3-none-any.Build is the build number and is omitted if there is no build number.Install-Paths-To is a location relative to the archive that will be overwritten with the install-time paths of each category in the install scheme. See the install paths section. May appear 0 or more times.Any file that is not normally installed inside site-packages goes into the .data directory, named as the .dist-info directory but with the .data/ extension:
distribution-1.0.dist-info/ distribution-1.0.data/
The .data directory contains subdirectories with the scripts, headers, documentation and so forth from the distribution. During installation the contents of these subdirectories are moved onto their destination paths.
If a subdirectory is not found in the install scheme, the installer should emit a warning, and it should be installed at distribution-1.0.data/... as if the package was unpacked by a standard unzip tool.
In addition to the distutils install paths, wheel now includes the listed categories based on GNU autotools. This expanded scheme should help installers to implement system policy, but installers may root each category at any location.
A UNIX install scheme might map the categories to their installation paths like this:
{
'bindir': '$eprefix/bin',
'sbindir': '$eprefix/sbin',
'libexecdir': '$eprefix/libexec',
'sysconfdir': '$prefix/etc',
'sharedstatedir': '$prefix/com',
'localstatedir': '$prefix/var',
'libdir': '$eprefix/lib',
'static_libdir': r'$prefix/lib',
'includedir': '$prefix/include',
'datarootdir': '$prefix/share',
'datadir': '$datarootdir',
'mandir': '$datarootdir/man',
'infodir': '$datarootdir/info',
'localedir': '$datarootdir/locale',
'docdir': '$datarootdir/doc/$dist_name',
'htmldir': '$docdir',
'dvidir': '$docdir',
'psdir': '$docdir',
'pdfdir': '$docdir',
'pkgdatadir': '$datadir/$dist_name'
}
If a package needs to find its files at runtime, it can request they be written to a specified file or files by the installer and included in those same files inside the archive itself, relative to their location within the archive (so a wheel is still installed correctly if unpacked with a standard unzip tool, or perhaps not unpacked at all).
If the WHEEL metadata contains these fields:
Install-Paths-To: wheel/_paths.py Install-Paths-To: wheel/_paths.json
Then the wheel installer, when it is about to unpack wheel/_paths.py from the archive, replaces it with the actual paths used at install time. The paths may be absolute or relative to the generated file.
If the filename ends with .py then a Python script is written. The script MUST be executed to get the paths, but it will probably look like this:
data='../wheel-0.26.0.dev1.data/data' headers='../wheel-0.26.0.dev1.data/headers' platlib='../wheel-0.26.0.dev1.data/platlib' purelib='../wheel-0.26.0.dev1.data/purelib' scripts='../wheel-0.26.0.dev1.data/scripts' # ...
If the filename ends with .json then a JSON document is written:
{ "data": "../wheel-0.26.0.dev1.data/data", ... }
Only the categories actually used by a particular wheel must be written to this file.
These files are designed to be written to a location that can be found by the installed package without introducing any dependency on a packaging library.
Signed wheel filesWheel files include an extended RECORD that enables digital signatures. PEP 376’s RECORD is altered to include a secure hash digestname=urlsafe_b64encode_nopad(digest) (urlsafe base64 encoding with no trailing = characters) as the second column instead of an md5sum. All possible entries are hashed, including any generated files such as .pyc files, but not RECORD which cannot contain its own hash. For example:
file.py,sha256=AVTFPZpEKzuHr7OvQZmhaU3LvwKz06AJw8mT\_pNh2yI,3144 distribution-1.0.dist-info/RECORD,,
The signature file(s) RECORD.jws and RECORD.p7s are not mentioned in RECORD at all since they can only be added after RECORD is generated. Every other file in the archive must have a correct hash in RECORD or the installation will fail.
If JSON web signatures are used, one or more JSON Web Signature JSON Serialization (JWS-JS) signatures is stored in a file RECORD.jws adjacent to RECORD. JWS is used to sign RECORD by including the SHA-256 hash of RECORD as the signature’s JSON payload:
{ "hash": "sha256=ADD-r2urObZHcxBW3Cr-vDCu5RJwT4CaRTHiFmbcIYY" }
(The hash value is the same format used in RECORD.)
If RECORD.p7s is used, it must contain a detached S/MIME format signature of RECORD.
A wheel installer is not required to understand digital signatures but MUST verify the hashes in RECORD against the extracted file contents. When the installer checks file hashes against RECORD, a separate signature checker only needs to establish that RECORD matches the signature.
See
Why does wheel include attached signatures?This specification does not have an opinion on how you should organize your code. The .data directory is just a place for any files that are not normally installed inside
site-packagesor on the PYTHONPATH. In other words, you may continue to usepkgutil.get_data(package, resource)even though those files will usually not be distributed in wheel’s.datadirectory.
Why does wheel allow JWS signatures?Attached signatures are more convenient than detached signatures because they travel with the archive. Since only the individual files are signed, the archive can be recompressed without invalidating the signature or individual files can be verified without having to download the whole archive.
Why does wheel also allow S/MIME signatures?The JOSE specifications of which JWS is a part are designed to be easy to implement, a feature that is also one of wheel’s primary design goals. JWS yields a useful, concise pure-Python implementation.
What’s the deal with “purelib” vs. “platlib”?S/MIME signatures are allowed for users who need or want to use existing public key infrastructure with wheel.
Signed packages are only a basic building block in a secure package update system. Wheel only provides the building block.
Is it possible to import Python code directly from a wheel file?Wheel preserves the “purelib” vs. “platlib” distinction, which is significant on some platforms. For example, Fedora installs pure Python packages to ‘/usr/lib/pythonX.Y/site-packages’ and platform dependent packages to ‘/usr/lib64/pythonX.Y/site-packages’.
A wheel with “Root-Is-Purelib: false” with all its files in
{name}-{version}.data/purelibis equivalent to a wheel with “Root-Is-Purelib: true” with those same files in the root, and it is legal to have files in both the “purelib” and “platlib” categories.In practice a wheel should have only one of “purelib” or “platlib” depending on whether it is pure Python or not and those files should be at the root with the appropriate setting given for “Root-is-purelib”.
AppendixTechnically, due to the combination of supporting installation via simple extraction and using an archive format that is compatible with
zipimport, a subset of wheel files
dosupport being placed directly on
sys.path. However, while this behaviour is a natural consequence of the format design, actually relying on it is generally discouraged.
Firstly, wheel is designed primarily as a distribution format, so skipping the installation step also means deliberately avoiding any reliance on features that assume full installation (such as being able to use standard tools like
pipandvirtualenvto capture and manage dependencies in a way that can be properly tracked for auditing and security update purposes, or integrating fully with the standard build machinery for C extensions by publishing header files in the appropriate place).Secondly, while some Python software is written to support running directly from a zip archive, it is still common for code to be written assuming it has been fully installed. When that assumption is broken by trying to run the software from a zip archive, the failures can often be obscure and hard to diagnose (especially when they occur in third party libraries). The two most common sources of problems with this are the fact that importing C extensions from a zip archive is not supported by CPython (since doing so is not supported directly by the dynamic loading machinery on any platform) and that when running from a zip archive the
__file__attribute no longer refers to an ordinary filesystem path, but to a combination path that includes both the location of the zip archive on the filesystem and the relative path to the module inside the archive. Even when software correctly uses the abstract resource APIs internally, interfacing with external components may still require the availability of an actual on-disk file.Like metaclasses, monkeypatching and metapath importers, if you’re not already sure you need to take advantage of this feature, you almost certainly don’t need it. If you do decide to use it anyway, be aware that many projects will require a failure to be reproduced with a fully installed package before accepting it as a genuine bug.
Example urlsafe-base64-nopad implementation:
# urlsafe-base64-nopad for Python 3
import base64
def urlsafe_b64encode_nopad(data):
return base64.urlsafe_b64encode(data).rstrip(b'=')
def urlsafe_b64decode_nopad(data):
pad = b'=' * (4 - (len(data) & 3))
return base64.urlsafe_b64decode(data + pad)
Copyright
This document has been placed into the public domain.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4