What kind of objects `yield from` can be used with?

Question

Initially (PEP 380), yield from syntax was introduced to be used for delegating to a "subgenerator." Later it was used with now deprecated generator-based coroutines.

I cannot find out what kind of objects yield from can be applied to in general. My first conjecture was that it only requires __iter__ method on the object to return an iterator. Indeed, the following works with Python 3.8:

class C:
 def __init__(self, n):
 self.n = n
 def __iter__(self):
 return iter(range(self.n))
def g(n):
 yield from C(n)
print(tuple(g(3)))

However, it also works with some awaitables, like asyncio.sleep(1), which do not have __iter__ method.

What is the general rule? What determines if an object can be given as an argument to yield from form?

• felipe: You are in the right path, but perhaps yield from asyncio.sleep(1) is confusing you. yield from is Python's 3.4 syntax equivalent of Python 3.5 await. Check out the documentation for Python 3.4 asyncio: 18.5.9.3. Concurrency and multithreading.
• Alexey: @Felipe, do you mean that yield from has two completely unrelated meanings? Does it have just two of them, or more? In the code example that i gave, you cannot replace yield from with await, so they are not always equivalent.
• Alexey: @Felipe, i do not see where in the documentation you pointed to my question is answered.
• felipe: The first paragraph on the link documentation above. "An event loop runs in a thread and executes all callbacks and tasks in the same thread. While a task is running in the event loop, no other task is running in the same thread. But when the task uses yield from, the task is suspended and the event loop executes the next task."
• felipe: Asynchronous programming is a different concept all together in Python (different from generators). In today's world, you use the keyword await to utilize asynchronous functions, but back before 3.4, you would use yield from as opposed to await (which is why you might see yield from sprinkled around in code that does not seem to have generators).

Answer 1

You can check how CPython evaluates that statement. From this follows it needs to be either a coroutine or an iterable:

case TARGET(GET_YIELD_FROM_ITER): {
 /* before: [obj]; after [getiter(obj)] */
 PyObject *iterable = TOP();
 PyObject *iter;
 if (PyCoro_CheckExact(iterable)) {
 /* `iterable` is a coroutine */
 if (!(co->co_flags & (CO_COROUTINE | CO_ITERABLE_COROUTINE))) {
 /* and it is used in a 'yield from' expression of a
 regular generator. */
 Py_DECREF(iterable);
 SET_TOP(NULL);
 _PyErr_SetString(tstate, PyExc_TypeError,
 "cannot 'yield from' a coroutine object "
 "in a non-coroutine generator");
 goto error;
 }
 }
 else if (!PyGen_CheckExact(iterable)) {
 /* `iterable` is not a generator. */
 iter = PyObject_GetIter(iterable);
 Py_DECREF(iterable);
 SET_TOP(iter);
 if (iter == NULL)
 goto error;
 }
 PREDICT(LOAD_CONST);
 DISPATCH();
}

• Alexey: So, there are really two unrelated meanings... Do you know if it was always so, or if historically all coroutines, including asyncio.sleep, had __iter__ method?
• a_guest: @Alexey Back then there was no real difference between coroutines and generators. As the 3.4 docs read: "A coroutine is a generator that follows certain conventions.". No separate coroutine type existed. PEP 492 which describes the change to async and await syntax is also informative with respect to the old behavior. PEP 3156 also provides information on that topic (e.g. search for yield from in these docs, it's a long read).
• Alexey: I wanted to see how concurrency was implemented with generators, but apparently it is too late, as the implementation changed. I would have to downgrade to Python 3.3, i suppose.