Iterating over two iterables

Question

I've written a little function that iterates over two iterables. The first one returns an object which is used to convert an object of the second iterable. I've got a working implementation, but would like to get some feedback.

A simple use case would look like this:

list(serialize([String(), String()], [1, 2])) # should work
list(serialize([String(), String()], range(3))) # should fail, too many values
list(serialize([String(), String()], range(1))) # should fail, too few values

But sometimes the number of values is not known but the type is the same. Therefore infinite iterators should be allowed as types argument:

list(serialize(it.repeat(String()), range(3))) # should work

Of course finite iterators are allowed as well:

list(serialize(it.repeat(String(), times=3), range(3))) # should work
list(serialize(it.repeat(String(), times=4), range(5))) # should fail
# should fail, but probably not that easy to do.
list(serialize(it.repeat(String(), times=4), range(3)))

My working implementation (except for the last case) looks like this:

class String(object):
 def dump(self, value):
 return str(value)
def serialize(types, values):
 class Stop(object):
 """Since None can be valid value we use it as fillvalue."""
 if isinstance(types, collections.Sized):
 # Since None is a valid value, use a different fillvalue.
 tv_pairs = it.izip_longest(types, values, fillvalue=Stop)
 else:
 tv_pairs = it.izip(it.chain(types, (Stop,)), values)
 for type, value in tv_pairs:
 if type is Stop:
 raise ValueError('Too many values.')
 if value is Stop:
 raise ValueError('Too few values.')
 yield type.dump(value)

• user26977: you could be checking to see if the lengths of the two lists match instead of messing around with izip?
• P3trus: @omouse This won't work if a generator is used.

Answer 1

There is no (general) way of telling the difference between an infinite and a finite iterator.

types might be a generator that stops yielding after a million items have been yielded. The only fool-proof way of telling would be to consume the generator up to the point where it stops yielding items. Of course, even if we've consumed a billion items and it hasn't stopped, we can't be sure it wouldn't stop at a later point.

This is why I recommend you change the specification for your function. The new version will yield items until values have been up. If there are too few types, it raises ValueError.

def serialize(types, values):
 types_iter = iter(types)
 for value in values:
 try:
 type = next(types_iter)
 except StopIteration:
 raise ValueError("Too few types.")
 yield type.dump(value)

If you really want to, you can check with len() that the lengths of types and values are the same, but as you have said, this will only work for some iterables.

def serialize(types, values):
 try:
 len_types, len_values = len(types), len(values)
 except TypeError:
 pass
 else:
 if len_values > len_types:
 raise ValueError("Too many values.")
 if len_values < len_types:
 raise ValueError('Too few values.')
 for type, value in itertools.izip(types, values):
 yield type.dump(value)
 return
 types_iter = iter(types)
 for value in values:
 try:
 type = next(types_iter)
 except StopIteration:
 raise ValueError("Too many values.")
 yield type.dump(value)