[Python-Dev] NotImplemented reaching top-level

Wed Dec 28 23:14:30 CET 2005

Hi Marc,
On Wed, Dec 28, 2005 at 09:56:43PM +0100, M.-A. Lemburg wrote:
> >>>>> d += 1.2
> >>>>> d
> >> NotImplemented
>> The PEP documenting the coercion logic has complete tables
> for what should happen:

Well, '+=' does not invoke coercion at all, with new-style classes like
Decimal.
> Looking at the code in abstract.c the above problem appears
> to be related to the special cases applied to += and *=
> in case both operands cannot deal with the type combination.
>> In such a case, a check is done whether the operation could
> be interpreted as sequence operation (concat or repeat) and
> then delegated to the appropriate handlers.

Indeed. The bug was caused by this delegation, which (prior to my
patch) would also return a Py_NotImplemented that would leak through
abstract.c. My patch is to remove this unnecessary delegation by not
defining sq_concat/sq_repeat for user-defined classes, and restoring the
original expectation that the sq_concat/sq_repeat slots should not
return Py_NotImplemented. How does this relate to coercion?
> But then again, looking in typeobject.c, the following code
> could be the cause for leaking a NotImplemented singleton
> reference:
>> #define SLOT1BINFULL(FUNCNAME, TESTFUNC, SLOTNAME, OPSTR, ROPSTR) \
> static PyObject * \
> FUNCNAME(PyObject *self, PyObject *other) \
> { \
> 	static PyObject *cache_str, *rcache_str; \
> 	int do_other = self->ob_type != other->ob_type && \
> 	 other->ob_type->tp_as_number != NULL && \
> 	 other->ob_type->tp_as_number->SLOTNAME == TESTFUNC; \
> 	if (self->ob_type->tp_as_number != NULL && \
> 	 self->ob_type->tp_as_number->SLOTNAME == TESTFUNC) { \
> 		PyObject *r; \
> 		if (do_other && \
> 		 PyType_IsSubtype(other->ob_type, self->ob_type) && \
> 		 method_is_overloaded(self, other, ROPSTR)) { \
> 			r = call_maybe( \
> 				other, ROPSTR, &rcache_str, "(O)", self); \
> 			if (r != Py_NotImplemented) \
> 				return r; \
> 			Py_DECREF(r); \
> 			do_other = 0; \
> 		} \
> 		r = call_maybe( \
> 			self, OPSTR, &cache_str, "(O)", other); \
> 		if (r != Py_NotImplemented || \
> 		 other->ob_type == self->ob_type) \
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> If both types are of the same type, then a NotImplemented returng
> value would be returned.

Indeed, however:
>> 			return r; \
> 		Py_DECREF(r); \
> 	} \
> 	if (do_other) { \
> 		return call_maybe( \
> 			other, ROPSTR, &rcache_str, "(O)", self); \
> 	} \
> 	Py_INCREF(Py_NotImplemented); \
> 	return Py_NotImplemented; \
> }

This last statement also returns Py_NotImplemented. So it's expected of
this function to be able to return Py_NotImplemented, isn't it? The
type slots like nb_add can return Py_NotImplemented; the code that
converts it to a TypeError is in the caller, which is abstract.c.
A bientot,
Armin