homepage

http://stackoverflow.com/questions/7306522/combining-itertools-and-multiprocessing/7307078 suggests (and the idea itself seems reasonable) that it would sometimes be useful for multiprocessing to operate correctly (ie lazily) with lazy input (iterables). for example, if the input is large, or perhaps generated by some other process "on demand".
obviously this complicates matters, given the asynchronous nature of a worker pool, and would mean re-allocating the results list as required. but in principle i suspect it would be possible and might be a useful extension.

Since it's a feature request, I would suggest to look whether it can apply to concurrent.futures instead.

If you want lazy operation then you should use imap(f, it[, chunksize]) rather than using map_async(f, it).
This will return an iterator rather than a list. Also, the iterator's next() method has a timeout argument. (chunksize is the number of items which get sent to a worker in each batch, with 1 being the default.)

Unless you have a reason why imap() does not solve the problem I will eventually close the issue as rejected.

hi - i'm the original author (may be using a different account). as far as i remember, i raised this because it seemed relevant given the link i gave. if you've looked at the issue and think your approach would work, or that this should be closed, or whatever, that's fine by me. i'm not going to check myself - i can't remember anything about this now (nearly a year later) and it's not my place to worry about your code (no offence - just trying to clarify that i have no skin in this game).

I'll close then.