PEP 673 – Self Type

Author:: Pradeep Kumar Srinivasan <gohanpra at gmail.com>, James Hilton-Balfe <gobot1234yt at gmail.com>
Sponsor:: Jelle Zijlstra <jelle.zijlstra at gmail.com>
Discussions-To:: Typing-SIG list
Status:: Final
Type:: Standards Track
Topic:: Typing
Created:: 10-Nov-2021
Python-Version:: 3.11
Post-History:: 17-Nov-2021
Resolution:: Python-Dev thread

Table of Contents

Abstract
Motivation
- Usage statistics
Specification
Valid Locations for Self
Runtime behavior
Rejected Alternatives
- Allow the Type Checker to Infer the Return Type
Reference Implementations
Resources
Copyright

Important

This PEP is a historical document: see Self and typing.Self for up-to-date specs and documentation. Canonical typing specs are maintained at the typing specs site; runtime typing behaviour is described in the CPython documentation.

×ばつ

See the typing specification update process for how to propose changes to the typing spec.

Abstract

This PEP introduces a simple and intuitive way to annotate methods that return an instance of their class. This behaves the same as the TypeVar-based approach specified in PEP 484 but is more concise and easier to follow.

Motivation

A common use case is to write a method that returns an instance of the same class, usually by returning self.

classShape:
 defset_scale(self, scale: float):
 self.scale = scale
 return self
Shape().set_scale(0.5) # => should be Shape

One way to denote the return type is to specify it as the current class, say, Shape. Using the method makes the type checker infer the type Shape, as expected.

classShape:
 defset_scale(self, scale: float) -> Shape:
 self.scale = scale
 return self
Shape().set_scale(0.5) # => Shape

However, when we call set_scale on a subclass of Shape, the type checker still infers the return type to be Shape. This is problematic in situations such as the one shown below, where the type checker will return an error because we are trying to use attributes or methods not present on the base class.

classCircle(Shape):
 defset_radius(self, r: float) -> Circle:
 self.radius = r
 return self
Circle().set_scale(0.5) # *Shape*, not Circle
Circle().set_scale(0.5).set_radius(2.7)
# => Error: Shape has no attribute set_radius

The present workaround for such instances is to define a TypeVar with the base class as the bound and use it as the annotation for the self parameter and the return type:

fromtypingimport TypeVar
TShape = TypeVar("TShape", bound="Shape")
classShape:
 defset_scale(self: TShape, scale: float) -> TShape:
 self.scale = scale
 return self
classCircle(Shape):
 defset_radius(self, radius: float) -> Circle:
 self.radius = radius
 return self
Circle().set_scale(0.5).set_radius(2.7) # => Circle

Unfortunately, this is verbose and unintuitive. Because self is usually not explicitly annotated, the above solution doesn’t immediately come to mind, and even if it does, it is very easy to go wrong by forgetting either the bound on the TypeVar(bound="Shape") or the annotation for self.

This difficulty means that users often give up and either use fallback types like Any or just omit the type annotation completely, both of which make the code less safe.

We propose a more intuitive and succinct way of expressing the above intention. We introduce a special form Self that stands for a type variable bound to the encapsulating class. For situations such as the one above, the user simply has to annotate the return type as Self:

fromtypingimport Self
classShape:
 defset_scale(self, scale: float) -> Self:
 self.scale = scale
 return self
classCircle(Shape):
 defset_radius(self, radius: float) -> Self:
 self.radius = radius
 return self

By annotating the return type as Self, we no longer have to declare a TypeVar with an explicit bound on the base class. The return type Self mirrors the fact that the function returns self and is easier to understand.

As in the above example, the type checker will correctly infer the type of Circle().set_scale(0.5) to be Circle, as expected.

Usage statistics

We analyzed popular open-source projects and found that patterns like the above were used about 40% as often as popular types like dict or Callable. For example, in typeshed alone, such "Self" types are used 523 times, compared to 1286 uses of dict and 1314 uses of Callable as of October 2021. This suggests that a Self type will be used quite often and users will benefit a lot from the simpler approach above.

Users of Python types have also frequently requested this feature, both on the proposal doc and on GitHub.

Specification

Use in Method Signatures

Self used in the signature of a method is treated as if it were a TypeVar bound to the class.

fromtypingimport Self
classShape:
 defset_scale(self, scale: float) -> Self:
 self.scale = scale
 return self

is treated equivalently to:

fromtypingimport TypeVar
SelfShape = TypeVar("SelfShape", bound="Shape")
classShape:
 defset_scale(self: SelfShape, scale: float) -> SelfShape:
 self.scale = scale
 return self

This works the same for a subclass too:

classCircle(Shape):
 defset_radius(self, radius: float) -> Self:
 self.radius = radius
 return self

which is treated equivalently to:

SelfCircle = TypeVar("SelfCircle", bound="Circle")
classCircle(Shape):
 defset_radius(self: SelfCircle, radius: float) -> SelfCircle:
 self.radius = radius
 return self

One implementation strategy is to simply desugar the former to the latter in a preprocessing step. If a method uses Self in its signature, the type of self within a method will be Self. In other cases, the type of self will remain the enclosing class.

Use in Classmethod Signatures

The Self type annotation is also useful for classmethods that return an instance of the class that they operate on. For example, from_config in the following snippet builds a Shape object from a given config.

classShape:
 def__init__(self, scale: float) -> None: ...
 @classmethod
 deffrom_config(cls, config: dict[str, float]) -> Shape:
 return cls(config["scale"])

However, this means that Circle.from_config(...) is inferred to return a value of type Shape, when in fact it should be Circle:

classCircle(Shape):
 defcircumference(self) -> float: ...
shape = Shape.from_config({"scale": 7.0})
# => Shape
circle = Circle.from_config({"scale": 7.0})
# => *Shape*, not Circle
circle.circumference()
# Error: `Shape` has no attribute `circumference`

The current workaround for this is unintuitive and error-prone:

Self = TypeVar("Self", bound="Shape")
classShape:
 @classmethod
 deffrom_config(
 cls: type[Self], config: dict[str, float]
 ) -> Self:
 return cls(config["scale"])

We propose using Self directly:

fromtypingimport Self
classShape:
 @classmethod
 deffrom_config(cls, config: dict[str, float]) -> Self:
 return cls(config["scale"])

This avoids the complicated cls: type[Self] annotation and the TypeVar declaration with a bound. Once again, the latter code behaves equivalently to the former code.

Use in Parameter Types

Another use for Self is to annotate parameters that expect instances of the current class:

Self = TypeVar("Self", bound="Shape")
classShape:
 defdifference(self: Self, other: Self) -> float: ...
 defapply(self: Self, f: Callable[[Self], None]) -> None: ...

We propose using Self directly to achieve the same behavior:

fromtypingimport Self
classShape:
 defdifference(self, other: Self) -> float: ...
 defapply(self, f: Callable[[Self], None]) -> None: ...

Note that specifying self: Self is harmless, so some users may find it more readable to write the above as:

classShape:
 defdifference(self: Self, other: Self) -> float: ...

Use in Attribute Annotations

Another use for Self is to annotate attributes. One example is where we have a LinkedList whose elements must be subclasses of the current class.

fromdataclassesimport dataclass
fromtypingimport Generic, TypeVar
T = TypeVar("T")
@dataclass
classLinkedList(Generic[T]):
 value: T
 next: LinkedList[T] | None = None
# OK
LinkedList[int](value=1, next=LinkedList[int](value=2))
# Not OK
LinkedList[int](value=1, next=LinkedList[str](value="hello"))

However, annotating the next attribute as LinkedList[T] allows invalid constructions with subclasses:

@dataclass
classOrdinalLinkedList(LinkedList[int]):
 defordinal_value(self) -> str:
 return as_ordinal(self.value)
# Should not be OK because LinkedList[int] is not a subclass of
# OrdinalLinkedList, # but the type checker allows it.
xs = OrdinalLinkedList(value=1, next=LinkedList[int](value=2))
if xs.next:
 print(xs.next.ordinal_value()) # Runtime Error.

We propose expressing this constraint using next: Self | None:

fromtypingimport Self
@dataclass
classLinkedList(Generic[T]):
 value: T
 next: Self | None = None
@dataclass
classOrdinalLinkedList(LinkedList[int]):
 defordinal_value(self) -> str:
 return as_ordinal(self.value)
xs = OrdinalLinkedList(value=1, next=LinkedList[int](value=2))
# Type error: Expected OrdinalLinkedList, got LinkedList[int].
if xs.next is not None:
 xs.next = OrdinalLinkedList(value=3, next=None) # OK
 xs.next = LinkedList[int](value=3, next=None) # Not OK

The code above is semantically equivalent to treating each attribute containing a Self type as a property that returns that type:

fromdataclassesimport dataclass
fromtypingimport Any, Generic, TypeVar
T = TypeVar("T")
Self = TypeVar("Self", bound="LinkedList")
classLinkedList(Generic[T]):
 value: T
 @property
 defnext(self: Self) -> Self | None:
 return self._next
 @next.setter
 defnext(self: Self, next: Self | None) -> None:
 self._next = next
classOrdinalLinkedList(LinkedList[int]):
 defordinal_value(self) -> str:
 return str(self.value)

Use in Generic Classes

Self can also be used in generic class methods:

classContainer(Generic[T]):
 value: T
 defset_value(self, value: T) -> Self: ...

This is equivalent to writing:

Self = TypeVar("Self", bound="Container[Any]")
classContainer(Generic[T]):
 value: T
 defset_value(self: Self, value: T) -> Self: ...

The behavior is to preserve the type argument of the object on which the method was called. When called on an object with concrete type Container[int], Self is bound to Container[int]. When called with an object of generic type Container[T], Self is bound to Container[T]:

defobject_with_concrete_type() -> None:
 int_container: Container[int]
 str_container: Container[str]
 reveal_type(int_container.set_value(42)) # => Container[int]
 reveal_type(str_container.set_value("hello")) # => Container[str]
defobject_with_generic_type(
 container: Container[T], value: T,
) -> Container[T]:
 return container.set_value(value) # => Container[T]

The PEP doesn’t specify the exact type of self.value within the method set_value. Some type checkers may choose to implement Self types using class-local type variables with Self = TypeVar("Self", bound=Container[T]), which will infer a precise type T. However, given that class-local type variables are not a standardized type system feature, it is also acceptable to infer Any for self.value. We leave this up to the type checker.

Note that we reject using Self with type arguments, such as Self[int]. This is because it creates ambiguity about the type of the self parameter and introduces unnecessary complexity:

classContainer(Generic[T]):
 deffoo(
 self, other: Self[int], other2: Self,
 ) -> Self[str]: # Rejected
 ...

In such cases, we recommend using an explicit type for self:

classContainer(Generic[T]):
 deffoo(
 self: Container[T],
 other: Container[int],
 other2: Container[T]
 ) -> Container[str]: ...

Use in Protocols

Self is valid within Protocols, similar to its use in classes:

fromtypingimport Protocol, Self
classShapeProtocol(Protocol):
 scale: float
 defset_scale(self, scale: float) -> Self:
 self.scale = scale
 return self

is treated equivalently to:

fromtypingimport TypeVar
SelfShape = TypeVar("SelfShape", bound="ShapeProtocol")
classShapeProtocol(Protocol):
 scale: float
 defset_scale(self: SelfShape, scale: float) -> SelfShape:
 self.scale = scale
 return self

See PEP 544 for details on the behavior of TypeVars bound to protocols.

Checking a class for compatibility with a protocol: If a protocol uses Self in methods or attribute annotations, then a class Foo is considered compatible with the protocol if its corresponding methods and attribute annotations use either Self or Foo or any of Foo’s subclasses. See the examples below:

fromtypingimport Protocol
classShapeProtocol(Protocol):
 defset_scale(self, scale: float) -> Self: ...
classReturnSelf:
 scale: float = 1.0
 defset_scale(self, scale: float) -> Self:
 self.scale = scale
 return self
classReturnConcreteShape:
 scale: float = 1.0
 defset_scale(self, scale: float) -> ReturnConcreteShape:
 self.scale = scale
 return self
classBadReturnType:
 scale: float = 1.0
 defset_scale(self, scale: float) -> int:
 self.scale = scale
 return 42
classReturnDifferentClass:
 scale: float = 1.0
 defset_scale(self, scale: float) -> ReturnConcreteShape:
 return ReturnConcreteShape(...)
defaccepts_shape(shape: ShapeProtocol) -> None:
 y = shape.set_scale(0.5)
 reveal_type(y)
defmain() -> None:
 return_self_shape: ReturnSelf
 return_concrete_shape: ReturnConcreteShape
 bad_return_type: BadReturnType
 return_different_class: ReturnDifferentClass
 accepts_shape(return_self_shape) # OK
 accepts_shape(return_concrete_shape) # OK
 accepts_shape(bad_return_type) # Not OK
 # Not OK because it returns a non-subclass.
 accepts_shape(return_different_class)

Valid Locations for `Self`

A Self annotation is only valid in class contexts, and will always refer to the encapsulating class. In contexts involving nested classes, Self will always refer to the innermost class.

The following uses of Self are accepted:

classReturnsSelf:
 deffoo(self) -> Self: ... # Accepted
 @classmethod
 defbar(cls) -> Self: # Accepted
 return cls()
 def__new__(cls, value: int) -> Self: ... # Accepted
 defexplicitly_use_self(self: Self) -> Self: ... # Accepted
 # Accepted (Self can be nested within other types)
 defreturns_list(self) -> list[Self]: ...
 # Accepted (Self can be nested within other types)
 @classmethod
 defreturn_cls(cls) -> type[Self]:
 return cls
classChild(ReturnsSelf):
 # Accepted (we can override a method that uses Self annotations)
 deffoo(self) -> Self: ...
classTakesSelf:
 deffoo(self, other: Self) -> bool: ... # Accepted
classRecursive:
 # Accepted (treated as an @property returning ``Self | None``)
 next: Self | None
classCallableAttribute:
 deffoo(self) -> int: ...
 # Accepted (treated as an @property returning the Callable type)
 bar: Callable[[Self], int] = foo
classHasNestedFunction:
 x: int = 42
 deffoo(self) -> None:
 # Accepted (Self is bound to HasNestedFunction).
 defnested(z: int, inner_self: Self) -> Self:
 print(z)
 print(inner_self.x)
 return inner_self
 nested(42, self) # OK
classOuter:
 classInner:
 deffoo(self) -> Self: ... # Accepted (Self is bound to Inner)

The following uses of Self are rejected.

deffoo(bar: Self) -> Self: ... # Rejected (not within a class)
bar: Self # Rejected (not within a class)
classFoo:
 # Rejected (Self is treated as unknown).
 defhas_existing_self_annotation(self: T) -> Self: ...
classFoo:
 defreturn_concrete_type(self) -> Self:
 return Foo() # Rejected (see FooChild below for rationale)
classFooChild(Foo):
 child_value: int = 42
 defchild_method(self) -> None:
 # At runtime, this would be Foo, not FooChild.
 y = self.return_concrete_type()
 y.child_value
 # Runtime error: Foo has no attribute child_value
classBar(Generic[T]):
 defbar(self) -> T: ...
classBaz(Bar[Self]): ... # Rejected

We reject type aliases containing Self. Supporting Self outside class definitions can require a lot of special-handling in type checkers. Given that it also goes against the rest of the PEP to use Self outside a class definition, we believe the added convenience of aliases is not worth it:

TupleSelf = Tuple[Self, Self] # Rejected
classAlias:
 defreturn_tuple(self) -> TupleSelf: # Rejected
 return (self, self)

Note that we reject Self in staticmethods. Self does not add much value since there is no self or cls to return. The only possible use cases would be to return a parameter itself or some element from a container passed in as a parameter. These don’t seem worth the additional complexity.

classBase:
 @staticmethod
 defmake() -> Self: # Rejected
 ...
 @staticmethod
 defreturn_parameter(foo: Self) -> Self: # Rejected
 ...

Likewise, we reject Self in metaclasses. Self in this PEP consistently refers to the same type (that of self). But in metaclasses, it would have to refer to different types in different method signatures. For example, in __mul__, Self in the return type would refer to the implementing class Foo, not the enclosing class MyMetaclass. But, in __new__, Self in the return type would refer to the enclosing class MyMetaclass. To avoid confusion, we reject this edge case.

classMyMetaclass(type):
 def__new__(cls, *args: Any) -> Self: # Rejected
 return super().__new__(cls, *args)
 def__mul__(cls, count: int) -> list[Self]: # Rejected
 return [cls()] * count
classFoo(metaclass=MyMetaclass): ...

Runtime behavior

Because Self is not subscriptable, we propose an implementation similar to typing.NoReturn.

@_SpecialForm
defSelf(self, params):
"""Used to spell the type of "self" in classes.
 Example::
 from typing import Self
 class ReturnsSelf:
 def parse(self, data: bytes) -> Self:
 ...
 return self
 """
 raise TypeError(f"{self} is not subscriptable")

Rejected Alternatives

Allow the Type Checker to Infer the Return Type

One proposal is to leave the Self type implicit and let the type checker infer from the body of the method that the return type must be the same as the type of the self parameter:

classShape:
 defset_scale(self, scale: float):
 self.scale = scale
 return self # Type checker infers that we are returning self

We reject this because Explicit Is Better Than Implicit. Beyond that, the above approach will fail for type stubs, which don’t have method bodies to analyze.

Reference Implementations

Mypy: Proof of concept implementation in Mypy.

Pyright: v1.1.184

Runtime implementation of Self: PR.

Resources

Similar discussions on a Self type in Python started in Mypy around 2016: Mypy issue #1212 - SelfType or another way to spell "type of self". However, the approach ultimately taken there was the bounded TypeVar approach shown in our "before" examples. Other issues that discuss this include Mypy issue #2354 - Self types in generic classes.

Pradeep made a concrete proposal at the PyCon Typing Summit 2021:: recorded talk, slides.

James brought up the proposal independently on typing-sig: Typing-sig thread.

Other languages have similar ways to express the type of the enclosing class:

TypeScript has the this type (TypeScript docs)
Rust has the Self type (Rust docs)

Thanks to the following people for their feedback on the PEP:

Jia Chen, Rebecca Chen, Sergei Lebedev, Kaylynn Morgan, Tuomas Suutari, Eric Traut, Alex Waygood, Shannon Zhu, and Никита Соболев

Copyright

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.

Source: https://github.com/python/peps/blob/main/peps/pep-0673.rst

Last modified: 2024年06月11日 22:12:09 GMT