I'm wondering how should a StrictTypeGuard[str] behave if it's defined input type is Any, and it's given an argument int in a usecase:

def tg(x: Any) -> StrictTypeGuard[str]:
 return isinstance(x, str)
def foo(y: int):
 if tg(y): # do we raise here because we narrow y type to empty set?
 print("got string")
 else:
 print("got int")

Effectively narrowed type is empty set ("it can't return true for this input type"). Does this mean rising a type error or just ignoring a branch in if/else?

In your example above, the type of y is narrowed to Never in the positive case and int in the negative case.

In the latest published version of pyright (1.1.210), I've removed support for the two-argument form of TypeGuard (based on lukwarm-to-negative feedback it garnered) and I've implemented StrictTypeGuard. If you're interested in this topic, please give it a try and provide feedback.

I just hit the need for StrictTypeGuard which brought me here, and the experimental support in pyright has solved my problem, thanks, so it would be great if it was standardised. Reading about TypeAssert, however, I've realised that would be great too. I currently do TypeAssert like this

def to_str(x: object) -> str:
 if isinstance(x, str):
 return x
 raise TypeError("expected str")
def process_part_of_json(input: object):
 input = to_str(input)
 use(input)

which different from the way I'd naturally write untyped Python like

def check_is_str(x):
 if not isinstance(x, str):
 raise TypeError("expected str")
def process_part_of_json(input: object):
 check_is_str(input)
 use(input)

As I've adopted regular TypeGuards, I've also realized that I have a need for what you call StrictTypeGuard more often than not. It would be a great addition.

After playing around with type guards, I've noticed that the 3rd point is often insufficient for many cases and not what I would expect, especially when trying to write generic type guards. The problem is that types are allowed to expand beyond their original types, at least on mypy.

def isiterable(x: Any) -> TypeGuard[Iterable]:
 return isinstance(x, Iterable)
x: list[int]
if isiterable(x):
 reveal_type(x) # Got Iterable[Any], expected list[int].
if isinstance(x, Iterable):
 reveal_type(x) # Got expected list[int].

My expectation is that type guards should at least work like isinstance in the positive case.

The & operator (which translates to the magic method set.__and__) requires an AbstractSet operand. It doesn't work with an arbitrary Collection, at least not according to the builtins.pyi stub.

class set(MutableSet[_T], Generic[_T]):
 ...
 def __and__(self, __s: AbstractSet[object]) -> set[_T]: ...

So I don't think it's enough to narrow item to Collection. You need to verify that it's an instance of AbstractSet. Something like this...

def is_present(collection: set[Any], item: Collection[Any]) -> bool:
 if isinstance(item, AbstractSet):
 return bool(collection & item)
 return item in collection

@gandhis1 @JelleZijlstra

[we can discuss this through the PEP process]

@erictraut created this discussion to get feedback on the proposed feature, so I'm giving feedback here. It's a little strange that people are responding to my feedback by saying there should be a PEP where, presumably, I can go repeat the same feedback.

The is_int/is_positive_int situation is a known problem. Wouldn't it make more sense to try to sort that out here instead of submitting a PEP with a known problem and figuring it out then?

@gandhis1

I really just see this as an extension of a feature that was already approved into the language.

Others (not me) have emphasized in this discussion that StrictTypeGuard is a separate thing from TypeGuard, not an extension, so that's something you can discuss directly with them. Personally I feel the relationship between StrictTypeGuard and TypeGuard is part of why this will be confusing to non-experts, and in particular (going back to @erictraut's comment here) that makes this proposed Python change more confusing than what's in TypeScript, which only has the StrictTypeGuard approach.

@JelleZijlstra @erictraut

[provisional support in Pyright]

I really don't want to get too sidetracked or upset anyone, but you all seem to be saying or implying that provisional support in a major type checker is a necessary but not sufficient condition to typing PEP acceptance. This gives the type checker devs, maybe in some cases a particular person, a lot of influence. Maybe that's as it should be, I couldn't say, but the fact remains.

@harahu I'm sorry, I'm not really sure what you mean by "validated" in your comment. However, I'm saying the difficulty is that novice developers can get unintuitive results by importing StrictTypeGuard code they didn't write. If you look again at the nested logic in func2 in my earlier comment, you can imagine that a developer could import is_int and is_positive_int from some library, naively use them and then get weird typing results. To figure this out they'd have to somehow know these is_* functions are the culprit, go look up their source, and then figure out what StrictTypeGuard is doing. (I wrote something similar in another earlier comment here.)

For your second point about pandas, also in the func2 comment I wrote:

So, on one hand we have a new feature that is useful to big projects like NumPy with lots of Python experts, while on the other hand it feels like opening the door to allow bad code that can trip up even experienced users for years to come.

I'm not sure what I think about it. Ideally we'd also want to hear from all the people down the road who will be confused by this new stuff, or who will have to debug bad library code that uses it, not just the experts in this discussion.

So it's not that this super advanced feature is only useful to something like pandas, it's that, practically speaking, only projects with Python typing experts will be able to use StrictTypeGuard effectively. All the other projects will not use it or will be confused by it when they find it in library code. In other words it advantages experts and is neutral to or disadvantages everyone else.

@jeremyn By validation I'm referring to the practice of having type checkers make sure that the definition of a function is in line with its annotation. Just like a type checker wouldn't be ok with the return type annotation of my meaning_of_life function example above, it shouldn't be ok with your is_positive_int either.

My point is that for your is_positive_int example to be a valid point, it must somehow be realistic. And, I think it is only realistic if it is "easy" to create such invalid StrictTypeGuards. The moment the type checker validates them, and complains if it doesn't agree that some function is "safe" as a StrictTypeGuard, the probability of seeing is_positive_int (and its brethren) annotated as StrictTypeGuard goes down enormously. After all, it would imply either getting a cast or # type: ignore past code review.

This, in my mind, makes the problem you're pointing out not very different from one we already have. Functions you haven't written yourself can have too narrow return annotations due to casting. Should you use them and encounter a type error, you have to go through the process of discovering that a certain external function is the culprit, and you have to work out how it doesn't do what it says on the tin. I don't think this process is made significantly harder (or more likely to occur) by the introduction of a validated StrictTypeGuard.

only projects with Python typing experts will be able to use StrictTypeGuard effectively. All the other projects will not use it or will be confused by it when they find it in library code.

I'm not a typing expert. I'm also not confused by this concept, and I want to make use of it. What gives?

@harahu Your meaning_of_life function is easy to check because it's returning a str when it should get an int. However as I understand it a *TypeGuard function just expects a bool and that's what it's getting in every case we've discussed, so that basic check won't work. It sounds like you want the checker to logically reason about whether the function is coming up with the boolean in a sensible way, which would require near-human intelligence, especially if you still permit stuff like is to find None, and ==/</etc to narrow down Literals. If you have a simple proposal for implementing that validation, I would be interested in hearing it.

Also anyone who has found their way to this forum and can meaningfully discuss these things is a typing expert compared to the vast majority. Also note that you only discovered a need for StrictTypeGuard after using TypeGuard a lot, TypeGuard itself being described as an "advanced feature" by @erictraut.

Sorry for forgetting to come back to you @jeremyn. I just realized given that I was reminded of this discussion by and issue touching upon a usecase for this feature: rustedpy/result#69

It sounds like you want the checker to logically reason about whether the function is coming up with the boolean in a sensible way, which would require near-human intelligence [...]

It wouldn't have to be very intelligent, no. I wouldn't mind having to use casts or ignore statements in cases where I am absolutely confident in the guard, but the type checker is not. The point is more that this is a psychological boundary that people feel bad about crossing unless they know what they're doing. It will mean that you end up using a StrictTypeGuard only when it is easy to statically verify that it is indeed correct, or when it is easy for you to make the case that the type checker is wrong in not trusting your code. And the latter won't be the case if you don't really understand the concept.

As someone who introduced type checking into our CI process at my current technology group of about ~50 developers, and have had to deal with the fallout of helping a number of people who have never used types before understand how they work from the ground up, I am strongly in favor of any solution that (1) matches user's natural intuition and (2) reduces cognitive burden, especially when it comes to concepts with a lot of edge cases like narrowing.

To that extent, I have a strong preference for option 2. The example I gave above, and am repeating here, is the annotation of pd.isna in pandas. Basic intuition would lead one to believe that if that returns True, the value will be narrowed to the null-equivalent members of the Union, and the actual object in the negative case.

In my opinion enhancements to type narrowing are one of the highest ROI enhancements one can make to the type system. Almost all of these kinds of enhancements have the effect of improving a user's ability to represent known facts at runtime statically in code. Even today there are still gaps in where one lacks the necessary syntax to be able to represent something through types, or alternatively must jump through excessive hoops like assert or cast to get things that we know to be true, to be understood to be true by type checkers. Any time we can eliminate one of those, it is a massive step forward.

What exactly do you mean by TypeAssert?

Recently I needed to assert that values are not None inside a generator expression and narrow the type accordingly. I came up with the following solution:

from typing import TypeVar
ValueT = TypeVar('ValueT')
def assert_not_none(value: ValueT | None) -> ValueT:
 """Raise an exception if the value is None.

 Args:
 value: The value to check.

 Raises:
 ValueError: If the value is None.

 Returns:
 The value, guaranteed to not be None.
 """
 if value is None:
 raise ValueError("Value cannot be None.")
 return value

Note: I am still not sure if the signature assert_not_none(value: ValueT | None) -> ValueT will work for any type in place of ValueT.

Usage:

def pass_int(value: int) -> int:
 return value
sequence: list[int | None] = [1, 2, 3, 4, None, 5]
values = (pass_int(assert_not_none(value)) for value in sequence)

I would certainly welcome a more general version of this asserting type guard in the standard library.

I have also found an interesting discussion in this issue: Request: an AssertingTypeGuard type for TypeGuard-like semantics #930 There you will see other needs for such an asserting type guard.

Typescript seems to behave the way we're suggesting TypeGuard should behave:

function is_positive_int(value: number | string): value is number { // Equivalent of TypeGuard[int]
 return typeof value === 'number' && value > 0;
}
function output(val: number | string) {
 if (is_positive_int(val)) {
 console.log(val + 1);
 } else {
 console.log(val - 1); // Error here because the assumption is that val is a string
 }
 
}

So, there's a precedent for assuming the negative type even when it might not make sense.

sorry for unsolicited comment, but I wanted to chime in regarding the Guido's example. In my mind the reason why it is really contrived is that if one is trying to distinguish positive ints as a type (hence using static typing), they should declare it as such. Thus the real-world example should have looked like this:

PosInt = NewType("PosInt", int)
def is_positive_int(a: PosInt|str) -> TypeGuard[PosInt]:
 return isinstance(a, int) and a >= 0
def test2(x: str | PosInt):
 if is_positive_int(x):
 reveal_type(x)
 else:
 reveal_type(x) # Pyright (at least the reference implementation) is saying x is 'str' here

This would also disallow (from the static type checker perspective) passing regular int from the upstream code.

This is just to support the notion that the else branch behavior should narrow the type without an introduction of another StrictTypeGuard. After all, the reasoning to not enforce strict narrowing in the original PEP says:

However, there are many ways a determined or uninformed developer can subvert type safety – most commonly by using cast or Any. If a Python developer takes the time to learn about and implement user-defined type guards within their code, it is safe to assume that they are interested in type safety and will not write their type guard functions in a way that will undermine type safety or produce nonsensical results.

Why not apply the same principle here? If someone shoots themselves in a foot by creating typeguadrs that are "narrowing" to implicit derived types (like the positive int), then it is just should be considered a "known" choice.

Why not apply the same principle here? If someone shoots themselves in a foot by creating typeguadrs that are "narrowing" to implicit derived types (like the positive int), then it is just should be considered a "known" choice.

I guess my argument is it wasn't a "known" choice before. So now people have to know that this is the new behavior. If the new behavior is counter-intuitive, it's harder for people to guess what's going to happen.

I'm having a hard time finding any examples where type guards are used with unions that wouldn't work with the new concept though.

Oh and @mishamsk, thanks for the feedback. I'm going to use your example in the PEP as to how people should handle that case that Guido had:

PosInt = NewType('PosInt', int)
def is_positive_int(val: PosInt | int | str) -> TypeGuard[PosInt]:
 return isinstance(val, int) and val > 0
def func(val: int | str):
 if is_positive_int(val):
 # Type checker assumes PosInt here
 else:
 # Type checker assumes str | int here

@rchiodo happy to help. Hope this PEP will get through!

@ikamensh, this idea was proposed earlier. I implemented it in prototype form, and the feedback was quite negative. The problem is that it doesn't address all of the needs. It provides support for narrowing in the negative case, but it doesn't support narrowing of union types as explained in point number 3 in this post.

Thanks for the hint. Not using information from more narrow type in actual calls is indeed very significant.

Thinking about it a bit more, I guess there could be StrictTypeGuard[A,B] - it would narrow types in not mutually exclusive way on both True and False. This would not have the limitation of not using narrower type in actual calls.

As I said, that was previously proposed and prototyped, and it received negative feedback for reasons I discuss above. The PEP can discuss it in the "Rejected Ideas" section, but I don't think there's a need to discuss it any further.

Note that all major type checkers now support TypeIs (https://peps.python.org/pep-0742/), which does allow narrowing in the negative case.

Thoughts about updating the typeshed hints for iscoroutinefunction?

@overload
def iscoroutinefunction(obj: Callable[_P, Coroutine[Any, Any, _T]]) -> TypeIs[Callable[_P, Coroutine[Any, Any, _T]]]: ...
@overload
def iscoroutinefunction(obj: Callable[_P, Awaitable[_T]]) -> TypeIs[Callable[_P, Coroutine[Any, Any, _T]]]: ...
@overload
def iscoroutinefunction(obj: object) -> TypeIs[Callable[..., Coroutine[Any, Any, Any]]]: ...

I believe that would be unsafe, because there may be callables that returns a coroutine but that iscoroutinefunction() does not recognize.

That is a relatively recent PEP, but why wasn't this mentioned before? I think PEP-742 simply settles this whole discussion! Does anybody disagree?

And now that the feature exists, turning any existing TypeGuard into a TypeIs should be a new issue being tracked in the projects releasing each TypeGuard