homepage

typing._GenericAlias represents two different types: user defined (like List[int]) and special (like List). They have different behavior, and common methods contain special cases.
The proposed PR rewrites the implementation in more object-oriented paradigm: different classes for different behavior. _GenericAlias is split on three classes: user defined (it may be replaced with GenericAlias in future), special, and the base class for common methods. Its subclasses are also split on classes for special types Tuple and Callable and for parametrized Callable[] and Annotated[]. The work is not finished yet and the code is still complex.

New changeset c1c7d8ead9eb214a6149a43e31a3213c52448877 by Serhiy Storchaka in branch 'master':
bpo-40397: Refactor typing._GenericAlias (GH-19719)
https://github.com/python/cpython/commit/c1c7d8ead9eb214a6149a43e31a3213c52448877

The next PR removes __args__ and __parameters__ from _SpecialGenericAlias.
* No existing test is failed. It is an evidence that these attributes are not intended, but a side effect of mixing two different kinds in one class.
* get_args() ignores __args__ and returns () for _SpecialGenericAlias (after resolving issue40398).
* Nested _SpecialGenericAlias is not considered a generic class containing type variables:
>>> from typing import *
>>> T = TypeVar('T')
>>> Dict[int, List[T]][str]
typing.Dict[int, typing.List[str]]
>>> Dict[int, List][str]
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/serhiy/py/cpython-release/Lib/typing.py", line 244, in inner
 return func(*args, **kwds)
 File "/home/serhiy/py/cpython-release/Lib/typing.py", line 695, in __getitem__
 _check_generic(self, params)
 File "/home/serhiy/py/cpython-release/Lib/typing.py", line 194, in _check_generic
 raise TypeError(f"{cls} is not a generic class")
TypeError: typing.Dict[int, typing.List] is not a generic class
It also fixes the following error:
>>> from typing import *
>>> T = TypeVar('T')
>>> Dict[T, List][str]
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/serhiy/py/cpython-release/Lib/typing.py", line 241, in inner
 return cached(*args, **kwds)
 File "/home/serhiy/py/cpython-release/Lib/typing.py", line 703, in __getitem__
 subargs = tuple(subst[x] for x in arg.__parameters__)
 File "/home/serhiy/py/cpython-release/Lib/typing.py", line 703, in <genexpr>
 subargs = tuple(subst[x] for x in arg.__parameters__)
KeyError: ~T
and allows to simplify __eq__ and __hash__ for _SpecialGenericAlias. Currently it has a weird behavior in corner case:
>>> from typing import *
>>> from typing import T
>>> List == List[T]
True

The most recent change here caused a regression. The following file:
```
from typing import Generic, TypeVar, Union
class CannotTransform(Exception): pass
T = TypeVar("T")
E = TypeVar("E", bound=Exception)
class Ok(Generic[T]): pass
class Err(Generic[E]): pass
Result = Union[Ok[T], Err[E]]
TResult = Result[T, CannotTransform]
TMatchResult = TResult[int]
```
Now fails with:
```
Traceback (most recent call last):
 File "/Users/jzijlstra-mpbt/py/cpython/Lib/../../black/black.py", line 22, in <module>
 TMatchResult = TResult[int]
 File "/Users/jzijlstra-mpbt/py/cpython/Lib/typing.py", line 244, in inner
 return func(*args, **kwds)
 File "/Users/jzijlstra-mpbt/py/cpython/Lib/typing.py", line 704, in __getitem__
 arg = arg[subargs]
 File "/Users/jzijlstra-mpbt/py/cpython/Lib/typing.py", line 244, in inner
 return func(*args, **kwds)
 File "/Users/jzijlstra-mpbt/py/cpython/Lib/typing.py", line 695, in __getitem__
 _check_generic(self, params)
 File "/Users/jzijlstra-mpbt/py/cpython/Lib/typing.py", line 194, in _check_generic
 raise TypeError(f"{cls} is not a generic class")
TypeError: __main__.Err[__main__.CannotTransform] is not a generic class
```
Before commit c1c7d8ead9eb214a6149a43e31a3213c52448877 it was fine. This was found when we added 3.9-dev to CI for Black (https://github.com/psf/black/pull/1393).

@Serhiy can you look at this? A possible fix might be:
diff --git a/Lib/typing.py b/Lib/typing.py
index 681ab6d21e..adcef1e82b 100644
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -701,7 +701,8 @@ class _GenericAlias(_BaseGenericAlias, _root=True):
 arg = subst[arg]
 elif isinstance(arg, (_BaseGenericAlias, GenericAlias)):
 subargs = tuple(subst[x] for x in arg.__parameters__)
- arg = arg[subargs]
+ if subargs:
+ arg = arg[subargs]
 new_args.append(arg)
 return self.copy_with(tuple(new_args))
 
The immediate cause of the problem seems to be that we call arg[subargs] with subargs being (). So arg is a _GenericAlias but its __parameters__ is (). I'm not sure what path was taken previously. :-(

I think I have a shorter repro, not involving unions:
from typing import *
T = TypeVar("T")
S = TypeVar("S")
U = TypeVar("U")
class A(Generic[T]): ...
class B(Generic[T]): ...
class C(Generic[T, S]): ...
print(C[A[U], B[int]][str])
Fails in the same place, works in 3.8 (I didn't check just before the offending commit), and the same fix works.

New changeset fcb285609a2e55f2dc63dcfbb32e4e2fddf71546 by Serhiy Storchaka in branch 'master':
bpo-40397: Remove __args__ and __parameters__ from _SpecialGenericAlias (GH-19984)
https://github.com/python/cpython/commit/fcb285609a2e55f2dc63dcfbb32e4e2fddf71546

Thank you Jelle for your report. There is even simpler example:
Dict[Tuple[T], List[int]][str]

Even simpler: Dict[T, List[int]][str].
Dict[T, list[int]][str] also fails. But dict[T, list[int]][str] works as expected (and there are tests).

New changeset 0122d48681b1df27015cf396559fb283ba364d6d by Serhiy Storchaka in branch 'master':
bpo-40397: Fix subscription of nested generic alias without parameters. (GH-20021)
https://github.com/python/cpython/commit/0122d48681b1df27015cf396559fb283ba364d6d

Thanks Serhyi! I can confirm that the issue I posted is fixed.

Looks like there's nothing left to do here.