-
Notifications
You must be signed in to change notification settings - Fork 288
-
Hi All,
I eventually managed to release a py.typed version of testfixtures, but to do so I had to give up on a bunch of overloads which I thought helped document intended methods of use:
simplistix/testfixtures@70283c2
I've tried to get my head around this a few times over the last few months:
- How do I type an overloaded function's keyword arguments that may not be specified?
- mypy complaining about calls to a datetime factory #1961
I even ended up logging a mypy bug:
...and as well as several attempts myself, LLMs have also been struggling, even once the underlying typing was sorted out:
At a high level, I'm essentially looking to revert simplistix/testfixtures@70283c2 but with functioning type annotations, essentially providing these three calling patterns for the factory:
@overload def mock_datetime( tzinfo: TZInfo | None = None, delta: float | None = None, delta_type: str = 'seconds', date_type: type[date] = date, strict: bool = False ) -> type[MockDateTime]: ... @overload def mock_datetime( year: int, month: int, day: int, hour: int = ..., minute: int = ..., second: int = ..., microsecond: int = ..., tzinfo: TZInfo | None = None, delta: float | None = None, delta_type: str = 'seconds', date_type: type[date] = date, strict: bool = False ) -> type[MockDateTime]: ... @overload def mock_datetime( default: datetime, tzinfo: TZInfo | None = None, delta: float | None = None, delta_type: str = 'seconds', date_type: type[date] = date, strict: bool = False ) -> type[MockDateTime]: ... @overload def mock_datetime( default: None, # explicit None positional tzinfo: TZInfo | None = None, delta: float | None = None, delta_type: str = 'seconds', date_type: type[date] = date, strict: bool = False ) -> type[MockDateTime]: ... def mock_datetime( *args, tzinfo: TZInfo | None = None,
...and then these overload patterns for the methods for manipulating the queue of upcoming mock datetimes:
class MockDateTime(MockedCurrent, datetime): @overload @classmethod def add( cls, year: int, month: int, day: int, hour: int = ..., minute: int = ..., second: int = ..., microsecond: int = ..., tzinfo: TZInfo = ..., ) -> None: ... @overload @classmethod def add( cls, instance: datetime, ) -> None: ... @classmethod def add(cls, *args, **kw): """ @@ -170,29 +147,6 @@ def add(cls, *args, **kw): """ return super().add(*args, **kw) @overload @classmethod def set( cls, year: int, month: int, day: int, hour: int = ..., minute: int = ..., second: int = ..., microsecond: int = ..., tzinfo: TZInfo = ..., ) -> None: ... @overload @classmethod def set( cls, instance: datetime, ) -> None: ... @classmethod def set(cls, *args, **kw): """ @@ -205,28 +159,6 @@ def set(cls, *args, **kw): """ return super().set(*args, **kw) @overload @classmethod def tick( cls, days: float = ..., seconds: float = ..., microseconds: float = ..., milliseconds: float = ..., minutes: float = ..., hours: float = ..., weeks: float = ..., ) -> None: ... @overload @classmethod def tick( cls, delta: timedelta, # can become positional-only when Python 3.8 minimum ) -> None: ... @classmethod def tick(cls, *args, **kw) -> None: """
...and similar across mock_datetime, mock_time and mock_date and the helper types they construct.
Is Python typing just "not there yet" or am I missing something? Happy to be shown the way, ecstatic if someone feels inspired to throw up a PR 😅
Beta Was this translation helpful? Give feedback.
All reactions
Replies: 2 comments 14 replies
-
Can you describe the problem you're seeing in more detail? You mentioned in the other thread that the ellipses make mypy unhappy. When I paste your code into a new document, add the missing import statements, and run mypy on it, I don't see any errors.
Normally, you'd need to supply real default argument values rather than using an ellipsis, but type checkers generally allow you to use an ellipsis as a stand-in for the actual value when you're writing overloads or stubs. It's still a good idea to provide the actual default argument values because it allows language servers to show users these values when they call the function. For example, here's what I see when I start typing a call to MockDateTime.set when using the pyright language server.
It would be better if you supplied the actual default values. I presume they are zero by default? If so, replace the ... with 0.
Beta Was this translation helpful? Give feedback.
All reactions
-
@hauntsaninja - thanks, your branch is happy with the minimal examples above, however, is this another related bug?
class Foo: pass @overload def foo(x: int) -> None: ... @overload def foo(*, x: Foo) -> None: ... @overload def foo(a: str, /) -> None: ... def foo(*args: int | str, **kw: int | Foo) -> None: pass
...gives:
$ mypy demo.py
demo.py:40: error: Overloaded function implementation does not accept all possible arguments of signature 1 [misc]
The key bit appears to be the | Foo on kw, so while the above is "more complete" (and relates to the TZInfo type from my real world problem), I can still reproduce the error with the following, more minimal:
class Foo: pass @overload def foo(x: int) -> None: ... @overload def foo(a: str, /) -> None: ... def foo(*args: int | str, **kw: int | Foo) -> None: pass
$ mypy demo.py
demo.py:37: error: Overloaded function implementation does not accept all possible arguments of signature 1 [misc]
Interestingly, the | str does appear important to trigger the bug, as:
class Foo: pass @overload def foo(x: int) -> None: ... @overload def foo(*, x: Foo) -> None: ... def foo(*args: int, **kw: int | Foo) -> None: pass
$ mypy demo.py
Success: no issues found in 1 source file
Beta Was this translation helpful? Give feedback.
All reactions
-
Yeah, that corresponds to the comment in that diff (got merged faster than I expected). Let's try python/mypy#19619
Beta Was this translation helpful? Give feedback.
All reactions
-
Okay, great, thanks, and now on to the next one(s):
class MockedCurrent: @classmethod def add(cls, *args: int | datetime, **kw: int | TZInfo | None) -> None: return None class MockDateTime(MockedCurrent): @overload @classmethod def add( cls, year: int, month: int, day: int, hour: int = 0, minute: int = 0, second: int = 0, microsecond: int = 0, *, tzinfo: TZInfo | None = None, fold: int = 0 ) -> None: ... @overload @classmethod def add( cls, instance: datetime, / ) -> None: ... @classmethod def add(cls, *args: int | datetime, **kw: int | TZInfo | None) -> None: return cls.add(*args, **kw)
On mypy master, I get:
demo.py:14: error: Signature of "add" incompatible with supertype "MockedCurrent" [override]
demo.py:14: note: Superclass:
demo.py:14: note: @classmethod
demo.py:14: note: def add(cls, *args: int | datetime, **kw: int | tzinfo | None) -> None
demo.py:14: note: Subclass:
demo.py:14: note: @overload
demo.py:14: note: @classmethod
demo.py:14: note: def add(cls, year: int, month: int, day: int, hour: int = ..., minute: int = ..., second: int = ..., microsecond: int = ..., *, tzinfo: tzinfo | None = ..., fold: int = ...) -> None
demo.py:14: note: @overload
demo.py:14: note: @classmethod
demo.py:14: note: def add(cls, datetime, /) -> None
demo.py:42: error: Argument 1 to "add" of "MockDateTime" has incompatible type "*tuple[int | datetime, ...]"; expected "int" [arg-type]
demo.py:42: error: Argument 2 to "add" of "MockDateTime" has incompatible type "**dict[str, int | tzinfo | None]"; expected "int" [arg-type]
demo.py:42: error: Argument 2 to "add" of "MockDateTime" has incompatible type "**dict[str, int | tzinfo | None]"; expected "tzinfo | None" [arg-type]
Why is the superclass considered incompatible? It can be called in exactly the same way as the subclass.
However, more confusing are the subsequent errors, what are those trying to communicate? Are they further bugs?
Beta Was this translation helpful? Give feedback.
All reactions
-
Here's a minimal reproducer for the the confusing stuff above:
class Base: @overload def add(cls, x: int) -> None: ... @overload def add(cls, y: str, /) -> None: ... def add(cls, *args: int | str, **kw: int) -> None: return None class Sample(Base): @overload def add(cls, x: int) -> None: ... @overload def add(cls, y: str, /) -> None: ... def add(cls, *args: int | str, **kw: int) -> None: return cls.add(*args, **kw)
demo.py:26: error: Argument 1 to "add" of "Sample" has incompatible type "*tuple[int | str, ...]"; expected "int" [arg-type]
...although if this is a bug, I suspect there's a parallel one for **kw as shown in the previous comment...
Beta Was this translation helpful? Give feedback.
All reactions
-
I think both errors are legit:
Incompatible with supertype
This boils down to:
class A:
def foo(self, *args: int, **kw: str) -> None: ...
class B(A):
def foo(self, year: int, *, tzinfo: str) -> None: ...
which is unsound because you could do:
def calls_foo(a: A):
a.foo(1, 2, 3, 4, 5, oops="A.foo handles these args, Liskov substitution principle implies B.foo should as well")
calls_foo(B()) # boom
pyright doesn't complain about the datetime example in your second last comment though, so maybe myself and mypy are missing something
Errors when calling superclass from your subclass
This is because only the overload types matter to the caller. The caller needs to prove to the type checker that the way it is calling the function matches the overloads.
pyright complains about this as well
Beta Was this translation helpful? Give feedback.