Type expressions provide a standardized way to specify types in the Python type system. When a type expression is evaluated at runtime, the resulting type form object encodes the information supplied in the type expression. This enables a variety of use cases including runtime type checking, introspection, and metaprogramming.
Such use cases have proliferated, but there is currently no way to accurately annotate functions that accept type form objects. Developers are forced to use an overly-wide type like object, which makes some use cases impossible and generally reduces type safety. This PEP addresses this limitation by introducing a new special form typing.TypeForm.
This PEP makes no changes to the Python grammar. Correct usage of TypeForm is intended to be enforced only by type checkers, not by the Python runtime.
A function that operates on type form objects must understand how type expression details are encoded in these objects. For example, int | str, "int | str", list[int], and MyTypeAlias are all valid type expressions, and they evaluate to instances of types.UnionType, builtins.str, types.GenericAlias, and typing.TypeAliasType, respectively.
There is currently no way to indicate to a type checker that a function accepts type form objects and knows how to work with them. TypeForm addresses this limitation. For example, here is a function that checks whether a value is assignable to a specified type and returns None if it is not:
def trycast[T](typx: TypeForm[T], value: object) -> T | None: ...
The use of TypeForm and the type variable T describes a relationship between the type form passed to parameter typx and the function’s return type.
TypeForm can also be used with TypeIs to define custom type narrowing behaviors:
def isassignable[T](value: object, typx: TypeForm[T]) -> TypeIs[T]: ...
request_json: object = ...
if isassignable(request_json, MyTypedDict):
assert_type(request_json, MyTypedDict) # Type of variable is narrowed
The isassignable function implements something like an enhanced isinstance check. This is useful for validating whether a value decoded from JSON conforms to a particular structure of nested TypedDicts, lists, unions, Literals, or any other type form that can be described with a type expression. This kind of check was alluded to in PEP 589 but could not be implemented without TypeForm.
type[C]?
One might think that type[C] would suffice for these use cases. However, only class objects (instances of the builtins.type class) are assignable to type[C]. Many type form objects do not meet this requirement:
def trycast[T](typx: type[T], value: object) -> T | None: ... trycast(str, 'hi') # OK trycast(Literal['hi'], 'hi') # Type violation trycast(str | None, 'hi') # Type violation trycast(MyProtocolClass, obj) # Type violationTypeForm use cases
A survey of Python libraries reveals several categories of functions that would benefit from TypeForm:
def is_assignable[T](value: object, typx: TypeForm[T]) -> TypeIs[T]def is_match[T](value: object, typx: TypeForm[T]) -> TypeGuard[T]def convert[T](value: object, typx: TypeForm[T]) -> T
class Converter[T]:
def __init__(self, typx: TypeForm[T]) -> None: ...
def convert(self, value: object) -> T: ...
class Field[T]:
value_type: TypeForm[T]
The survey also identified some introspection functions that accept runtime type forms as input. Today, these functions are annotated with object:
def get_annotation_info(typx: object) -> objectThese functions accept values evaluated from arbitrary annotation expressions, not just type expressions, so they cannot be altered to use TypeForm.
When a type expression is evaluated at runtime, the resulting value is a type form object. This value encodes the information supplied in the type expression, and it represents the type described by that type expression.
TypeForm is a special form that, when used in a type expression, describes a set of type form objects. It accepts a single type argument, which must be a valid type expression. TypeForm[T] describes the set of all type form objects that represent the type T or types that are assignable to T. For example, TypeForm[str | None] describes the set of all type form objects that represent a type assignable to str | None:
ok1: TypeForm[str | None] = str | None # OK ok2: TypeForm[str | None] = str # OK ok3: TypeForm[str | None] = None # OK ok4: TypeForm[str | None] = Literal[None] # OK ok5: TypeForm[str | None] = Optional[str] # OK ok6: TypeForm[str | None] = "str | None" # OK ok7: TypeForm[str | None] = Any # OK err1: TypeForm[str | None] = str | int # Error err2: TypeForm[str | None] = list[str | None] # Error
By this same definition, TypeForm[object] describes a type form object that represents the type object or any type that is assignable to object. Since all types in the Python type system are assignable to object, TypeForm[object] describes the set of all type form objects evaluated from all valid type expressions.
TypeForm[Any] describes a TypeForm type whose type argument is not statically known but is a valid type form object. It is thus assignable both to and from any other TypeForm type (because Any is assignable both to and from any type).
The type expression TypeForm, with no type argument provided, is equivalent to TypeForm[Any].
TypeForm Evaluation
When a static type checker encounters a valid type expression, the evaluated type of this expression should be assignable to TypeForm[T] if the type it describes is assignable to T.
For example, if a static type checker encounters the expression str | None, it may normally evaluate its type as UnionType because it produces a runtime value that is an instance of types.UnionType. However, because this expression is a valid type expression, it is also assignable to the type TypeForm[str | None]:
v1_actual: UnionType = str | None # OK v1_type_form: TypeForm[str | None] = str | None # OK v2_actual: type = list[int] # OK v2_type_form: TypeForm = list[int] # OK
The Annotated special form is allowed in type expressions, so it can also appear in an expression that is assignable to TypeForm. Consistent with the typing spec’s rules for Annotated, a static type checker may choose to ignore any Annotated metadata that it does not understand:
v3: TypeForm[int | str] = Annotated[int | str, "metadata"] # OK v4: TypeForm[Annotated[int | str, "metadata"]] = int | str # OK
A string literal expression containing a valid type expression should likewise be assignable to TypeForm:
v5: TypeForm[set[str]] = "set[str]" # OKValid Type Expressions
The typing spec defines syntactic rules for type expressions in the form of a formal grammar. Semantic rules are specified as comments along with the grammar definition. Contextual requirements are detailed throughout the typing spec in sections that discuss concepts that appear within type expressions. For example, the special form Self can be used in a type expression only within a class, and a type variable can be used within a type expression only when it is associated with a valid scope.
A valid type expression is an expression that follows all of the syntactic, semantic, and contextual rules for a type expression.
Expressions that are not valid type expressions should not evaluate to a TypeForm type:
bad1: TypeForm = tuple() # Error: Call expression not allowed in type expression bad2: TypeForm = (1, 2) # Error: Tuple expression not allowed in type expression bad3: TypeForm = 1 # Non-class object not allowed in type expression bad4: TypeForm = Self # Error: Self not allowed outside of a class bad5: TypeForm = Literal[var] # Error: Variable not allowed in type expression bad6: TypeForm = Literal[f""] # Error: f-strings not allowed in type expression bad7: TypeForm = ClassVar[int] # Error: ClassVar not allowed in type expression bad8: TypeForm = Required[int] # Error: Required not allowed in type expression bad9: TypeForm = Final[int] # Error: Final not allowed in type expression bad10: TypeForm = Unpack[Ts] # Error: Unpack not allowed in this context bad11: TypeForm = Optional # Error: Invalid use of Optional special form bad12: TypeForm = T # Error if T is an out-of-scope TypeVar bad13: TypeForm = "int + str" # Error: invalid quoted type expressionExplicit
TypeForm Evaluation
TypeForm also acts as a function that can be called with a single argument. Type checkers should validate that this argument is a valid type expression:
x1 = TypeForm(str | None)
reveal_type(v1) # Revealed type is "TypeForm[str | None]"
x2 = TypeForm("list[int]")
revealed_type(v2) # Revealed type is "TypeForm[list[int]]"
x3 = TypeForm('type(1)') # Error: invalid type expression
At runtime the TypeForm(...) callable simply returns the value passed to it.
This explicit syntax serves two purposes. First, it documents the developer’s intent to use the value as a type form object. Second, static type checkers validate that all rules for type expressions are followed:
x4 = type(1) # No error, evaluates to "type[int]" x5 = TypeForm(type(1)) # Error: call not allowed in type expressionAssignability
TypeForm has a single type parameter, which is covariant. That means TypeForm[B] is assignable to TypeForm[A] if B is assignable to A:
def get_type_form() -> TypeForm[int]: ... t1: TypeForm[int | str] = get_type_form() # OK t2: TypeForm[str] = get_type_form() # Error
type[T] is a subtype of TypeForm[T], which means that type[B] is assignable to TypeForm[A] if B is assignable to A:
def get_type() -> type[int]: ... t3: TypeForm[int | str] = get_type() # OK t4: TypeForm[str] = get_type() # Error
TypeForm is a subtype of object and is assumed to have all of the attributes and methods of object.
This PEP clarifies static type checker behaviors when evaluating type expressions in “value expression” contexts (that is, contexts where type expressions are not mandated by the typing spec). In the absence of a TypeForm type annotation, existing type evaluation behaviors persist, so no backward compatibility issues are anticipated. For example, if a static type checker previously evaluated the type of expression str | None as UnionType, it will continue to do so unless this expression is assigned to a variable or parameter whose type is annotated as TypeForm.
Type expressions are used in annotations to describe which values are accepted by a function parameter, returned by a function, or stored in a variable:
parameter type return type
| |
v v
def plus(n1: int, n2: int) -> int:
sum: int = n1 + n2
^
|
variable type
return sum
Type expressions evaluate to valid type form objects at runtime and can be assigned to variables and manipulated like any other data in a program:
a variable a type expression
| |
v v
int_type_form: TypeForm = int | None
^
|
the type of a type form object
TypeForm[] is how you spell the type of a type form object, which is a runtime representation of a type.
TypeForm is similar to type, but type is compatible only with class objects like int, str, list, or MyClass. TypeForm accommodates any type form that can be expressed using a valid type expression, including those with brackets (list[int]), union operators (int | None), and special forms (Any, LiteralString, Never, etc.).
Most programmers will not define their own functions that accept a TypeForm parameter or return a TypeForm value. It is more common to pass a type form object to a library function that knows how to decode and use such objects.
For example, the isassignable function in the trycast library can be used like Python’s built-in isinstance function to check whether a value matches the shape of a particular type. isassignable accepts any type form object as input.
from trycast import isassignable
if isassignable(some_object, MyTypedDict): # OK: MyTypedDict is a TypeForm[]
...
if isinstance(some_object, MyTypedDict): # ERROR: MyTypedDict is not a type[]
...
If you want to write your own runtime type checker or a function that manipulates type form objects as values at runtime, this section provides examples of how such a function can use TypeForm.
Functions like typing.get_origin and typing.get_args can be used to extract components of some type form objects.
import typing
from typing import TypeForm, cast
def strip_annotated_metadata[T](typx: TypeForm[T]) -> TypeForm[T]:
if typing.get_origin(typx) is typing.Annotated:
typx = cast(TypeForm[T], typing.get_args(typx)[0])
return typx
isinstance and is can also be used to distinguish between different kinds of type form objects:
import types
import typing
from typing import TypeForm, cast
def split_union(typx: TypeForm) -> tuple[TypeForm, ...]:
if isinstance(typx, types.UnionType): # X | Y
return cast(tuple[TypeForm, ...], typing.get_args(typx))
if typing.get_origin(typx) is typing.Union: # Union[X, Y]
return cast(tuple[TypeForm, ...], typing.get_args(typx))
if typx in (typing.Never, typing.NoReturn,):
return ()
return (typx,)
Combining with a type variable
TypeForm can be parameterized by a type variable that is used elsewhere within the same function definition:
def as_instance[T](typx: TypeForm[T]) -> T | None:
return typx() if isinstance(typx, type) else None
Combining with type
Both TypeForm and type can be parameterized by the same type variable within the same function definition:
def as_type[T](typx: TypeForm[T]) -> type[T] | None:
return typx if isinstance(typx, type) else None
Combining with TypeIs and TypeGuard
A type variable can also be used by a TypeIs or TypeGuard return type:
def isassignable[T](value: object, typx: TypeForm[T]) -> TypeIs[T]: ...
count: int | str = ...
if isassignable(count, int):
assert_type(count, int)
else:
assert_type(count, str)
Challenges When Accepting All TypeForms
A function that takes an arbitrary TypeForm as input must support a variety of possible type form objects. Such functions are not easy to write.
'list[str]') must be parsed (to something like list[str]).eval(), which is difficult to use in a safe way.IntTree = list[int | 'IntTree'] are difficult to resolve.QuerySet[User]) can introduce non-standard behaviors that require runtime support.Pyright (version 1.1.379) provides a reference implementation for TypeForm.
Mypy contributors also plan to implement support for TypeForm.
A reference implementation of the runtime component is provided in the typing_extensions module.
Alternate names were considered for TypeForm. TypeObject and TypeType were deemed too generic. TypeExpression and TypeExpr were also considered, but these were considered confusing because these objects are not themselves “expressions” but rather the result of evaluating a type expression.
type[C] to support all type expressions
type was designed to describe class objects, subclasses of the type class. A value with the type type is assumed to be instantiable through a constructor call. Widening the meaning of type to represent arbitrary type form objects would present backward compatibility problems and would eliminate a way to describe the set of values limited to subclasses of type.
Certain special forms act as type qualifiers and can be used in some but not all annotation contexts:
For example. the type qualifier Final can be used as a variable type but not as a parameter type or a return type:
some_const: Final[str] = ... # OK def foo(not_reassignable: Final[object]): ... # Error: Final not allowed here def nonsense() -> Final[object]: ... # Error: Final not allowed here
With the exception of Annotated, type qualifiers are not allowed in type expressions. TypeForm is limited to type expressions because its assignability rules are based on the assignability rules for types. It is nonsensical to ask whether Final[int] is assignable to int because the former is not a valid type expression.
Functions that wish to operate on objects that are evaluated from annotation expressions can continue to accept such inputs as object parameters.
It was asserted that some functions may wish to pattern match on the interior of type expressions in their signatures.
One use case is to allow a function to explicitly enumerate all the specific kinds of type expressions it supports as input. Consider the following possible pattern matching syntax:
@overload def checkcast(typx: TypeForm[AT=Annotated[T, *A]], value: str) -> T: ... @overload def checkcast(typx: TypeForm[UT=Union[*Ts]], value: str) -> Union[*Ts]: ... @overload def checkcast(typx: type[C], value: str) -> C: ... # ... (more)
All functions observed in the wild that conceptually accept type form objects generally try to support all kinds of type expressions, so it doesn’t seem valuable to enumerate a particular subset.
Additionally, the above syntax isn’t precise enough to fully describe the input constraints for a typical function in the wild. For example, many functions do not support type expressions with quoted subexpressions like list['Movie'].
A second use case for pattern matching is to explicitly match an Annotated form to extract the interior type argument and strip away any metadata:
def checkcast(
typx: TypeForm[T] | TypeForm[AT=Annotated[T, *A]],
value: object
) -> T:
However, Annotated[T, metadata] is already treated equivalent to T by static type checkers. There’s no additional value in being explicit about this behavior. The example above could more simply be written as the equivalent:
def checkcast(typx: TypeForm[T], value: object) -> T:Acknowledgements
typing_extensions implementation of the TypeExpr special form.type to be assigned to TypeForm or not.TypeForm special form in a real-world runtime type checker (beartype).This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4