homepage

Hi, I'm working on a class which implements the __contains__ method but the way I would like it to work is by generating an object that will be evaluated later.
It'll return a custom object instead of True/False
class C:
 def __contains__(self, x):
 return "I will evaluate this thing later... Don't bother now"
but when I do:
>>> 1 in C()
True
It seems to evaluate the answer with bool!
Reading the docs (http://docs.python.org/py3k/reference/expressions.html#membership-test-details) It says:
"`x in y` is true if and only if `y.__contains__(x)` is true."
It looks like the docs doesn't match the code and the code is trying to mimic the behavior of lists/tuples where "x in y" is the same as
any(x is e or x == e for e in y)
and always yield True or False.
There is a reason why it is that way?
Thanks!

"an object is true" is a short way of saying "bool(obj) is True". So the docs match the behavior.
Returning the actual object instead of True/False from the "in" operator is a feature request.

@Georg Brandl
Oh sorry, now I see... true != True
But still, why is that the default behavior? Shouldn't it use whatever the method returns?

Well, usually what you want *is* a boolean indicating whether the element is in the collection or not.
Being able to overload "in", mostly for metaprogramming purposes, is a request that probably nobody thought of when implementing "in".

I think the idea has some merit. I think it should be well vetted on python-ideas, though. One thing that will certianly weigh against it is that implementation would not be trivial.

I see that every other comparison operator (<, >, <=, >=, ==, !=) except for `is` work the way I expect and is able to return anything.
e.g.
>>> numpy.arange(5) < 3
array([ True, True, True, False, False], dtype=bool)
I didn't checked the code (and probably I'm talking nonsense), but seems like the `in` operator has an extra call to `PyObject_IsTrue` that maybe could be dropped?
Of course it can break code relying on `x in y` being True/False but it would only happen on customized classes.
Another option that won't break code is to add a different method to handle these cases. Something like "__contains_non_bool__", but that'd be a big api change.

2011年12月28日 João Bernardo <report@bugs.python.org>:
>
> João Bernardo <jbvsmo@gmail.com> added the comment:
>
> I see that every other comparison operator (<, >, <=, >=, ==, !=) except for `is` work the way I expect and is able to return anything.
>
> e.g.
>
>>>> numpy.arange(5) < 3
> array([ True, True, True, False, False], dtype=bool)
>
> I didn't checked the code (and probably I'm talking nonsense), but seems like the `in` operator has an extra call to `PyObject_IsTrue` that maybe could be dropped?
I'm not sure what you're referring to, but I doubt that would do the job.
>
> Of course it can break code relying on `x in y` being True/False but it would only happen on customized classes.
>
> Another option that won't break code is to add a different method to handle these cases. Something like "__contains_non_bool__", but that'd be a big api change.
And completely hideous.

Using my poor grep abilities I found that on Objects/typeobject.c
(I replaced some declarations/error checking from the code with ...)
static int
slot_sq_contains(PyObject *self, PyObject *value) {
 ...
 func = lookup_maybe(self, "__contains__", &contains_str);
 if (func != NULL) {
 ...
 res = PyObject_Call(func, args, NULL);
 ...
 if (res != NULL) {
 result = PyObject_IsTrue(res);
 Py_DECREF(res);
 }
 }
 else if (! PyErr_Occurred()) {
 /* Possible results: -1 and 1 */
 result = (int)_PySequence_IterSearch(self, value,
 PY_ITERSEARCH_CONTAINS);
 }
}
 
I don't know if I'm in the right place, but the function returns `int` and evaluates the result to 1 or 0 if __contains__ is found.
I also don't know what SQSLOT means, but unlike the other operators (which are defined as TPSLOT), `slot_sq_contains` is a function returning "int" while `slot_tp_richcompare` returns "PyObject *".
Why is that defined that way?

It's defined that way because it's a slot returning a bool, so it doesn't need to return anything except for 0 or 1.
Changing this to return a PyObject would mean that every extension module (i.e. module written in C) that defines a custom __contains__ would need to be adapted. That is the non-trivial implementation that Benjamin was talking about.

-1 on this proposal. It has everyone paying a price for a questionable feature that would benefit very few.

For what it's worth I proposed this on -ideas a while ago, the sticking points were what does `not in` do (no one had an answer anyone was happy with for this), and do we need a way to override it from the other perspective (e.g. if I want to do `SpecialObj() in [1, 2, 3]`, is there a way to do that?).

The problem with `not in` is because it must evaluate the result. It's not just another operator like "==" and "!=".
Looks like we're suffering from premature optimization and now it would break a lot of code to make it good.
For my application, I created a different method to generate the object (Not as good as I wanted, but there's no option right now):
`MyClass().has(1)` instead of `1 in MyClass()`
So, if no one comes up with a better idea, this issue should be closed.
Thanks