Infinitely iterable list class in Python

Question

itertools.cycle is cool, but it has an internal "storage" that uses up memory. And I wanted to practice "tampering" with built-in methods of classes, which is a new field for me. I tried to implement a class which mostly behaves like a list, but when we try to iterate over it (with for loop, for example), it would jump back to the beginning when it reaches the end, therefore cycling indefinitely. Here's what I've come up with:

class InfiniteListEmptyError(Exception):
 pass
class InfiniteList(list):
 """
 Pretty much a regular list, but when iterated over, it does not stop in the end. 
 Instead, it iterates indefinitely.
 """
 def __init__(self, arg):
 super(InfiniteList, self).__init__(arg)
 def __getattr__(self, name):
 if name == "it":
 # get iterator
 result = object.__getattribute__(self, name)
 else:
 try:
 result = super(InfiniteList, self).__getattribute__(name)
 except AttributeError:
 try:
 result = self.it.__getattribute__(name)
 except AttributeError:
 # Initialize iterator cuz it's not initialized
 self.__iter__()
 result = self.it.__getattribute__(name)
 return result
 def __iter__(self):
 it = super(InfiniteList, self).__iter__()
 self.it = it
 return self
 def __next__(self):
 try:
 result = next(self.it)
 except StopIteration:
 self.__iter__()
 try:
 result = next(self.it)
 except StopIteration:
 raise InfiniteListEmptyError("Could not iterate. List is empty!")
 return result
# TESTS
a = InfiniteList(tuple()) #empty list
print(a)
print(a.__length_hint__()) #testing iterator attributes
print(a.__eq__([1,3,5])) # false
# should raise exception, since the list is empty
try:
 for i in a:
 print(i)
except InfiniteListEmptyError:
 print("List is empty!")
a.append(1)
a.extend([3,5])
print(a)
print(a.__eq__([1,3,5])) #true
# infinite loop
for i in a:
 print(i)

It works fine, it does not store an auxiliary list, so it does not consume twice as much memory, like itertools.cycle. But I'm not sure about the possible caveats when it comes to handling attributes. And yeah, I had to implement iterator within the InfiniteList class itself, so I had to redirect the calls to iterator's methods (like __length_hint__) to the saved iterator self.it.

• 301_Moved_Permanently: You know that by doing a = list(range(50)); b = InfiniteList(a) you are effectively copying your initial list, right? So what is the exact advantage over itertools.cycle?

Answer 1

Simplifying the class

def __init__(self, arg):
 super(InfiniteList, self).__init__(arg)

Is unnecessary. Not only it doesn't add any value over the default constructor of lists (which accept any iterable to initialize a list from it) but it prevent from building an empty list by using a constructor without argument, such as empty_list = list().

---

def __getattr__(self, name):
 if name == "it":
 # get iterator
 result = object.__getattribute__(self, name)
 else:
 try:
 result = super(InfiniteList, self).__getattribute__(name)
 except AttributeError:
 try:
 result = self.it.__getattribute__(name)
 except AttributeError:
 # Initialize iterator cuz it's not initialized
 self.__iter__()
 result = self.it.__getattribute__(name)
 return result

Contrary to __getattribute__, __getattr__ is only called if the lookup already failed. So you are assured that, if self.it has already been initialized, __getattr__ won't be called to access it. You also don't really need to store the iterator directly. You know that, given the use cases of your class, __getattr__ will most likely be called when trying to access an iterator method. You can thus build on that to create an iterator on demand and try to access its methods:

def __getattr__(self, name):
 """Redirect missing attributes and methods to
 those of the underlying iterator.
 """
 iterator = super(InfiniteList, self).__iter__()
 return getattr(iterator, name)

---

def __iter__(self):
 it = super(InfiniteList, self).__iter__()
 self.it = it
 return self
def __next__(self):
 try:
 result = next(self.it)
 except StopIteration:
 self.__iter__()
 try:
 result = next(self.it)
 except StopIteration:
 raise InfiniteListEmptyError("Could not iterate. List is empty!")
 return result

Is overly complicated as implementing __iter__ that returns self and having a __next__ method can be simplified to turning __iter__ into a generator most of the time:

def __iter__(self):
 if not self:
 raise InfiniteListEmptyError("Could not iterate. List is empty!")
 while True:
 iterator = super(InfiniteList, self).__iter__()
 yield from iterator

---

So the class can be defined as:

class InfiniteList(list):
 def __getattr__(self, name):
 """Redirect missing attributes and methods to
 those of the underlying iterator.
 """
 iterator = super(InfiniteList, self).__iter__()
 return getattr(iterator, name)
 def __iter__(self):
 if not self:
 raise InfiniteListEmptyError("Could not iterate. List is empty!")
 while True:
 iterator = super(InfiniteList, self).__iter__()
 yield from iterator

Differences with itertools.cycle

Building an InfiniteList out of an iterable will inevitably create memory to store the list and, as such, is no different from the memory used by itertools.cycle. Even building an InfiniteList out of an existing list will duplicate memory.

The only "advantage" being that you could build the list from scratch using append or extend and not use as much memory than itertools.cycle. However, such approaches are generally better handled using a list-comprehension or a generator expression. And feeding the generator expression to either itertools.cycle or InfiniteList will give the same memory footprint.

Except itertools.cycle:

1. is optimized in C;
2. doesn't consume the iterable upfront and, as such, can handle infinite generators (meaning itertools.cycle(itertools.count()) is fine albeit unneccessary whereas InfiniteList(itertools.count()) will eat up all your memory).

In short, unless you have a very specific use-case, most of the time your InfiniteList will have the exact same memory footprint than itertool.cycle with worse performances.

Answer 2

This seems kind of complicated compared to something like this:

def cycle(seq):
 "Generator that yields the elements of a non-empty sequence cyclically."
 i = 0
 while True:
 yield seq[i]
 i = (i + 1) % len(seq)

Or, as Joe Wallis points out in comments:

def cycle(seq):
 "Generator that yields the elements of a non-empty sequence cyclically."
 while True:
 yield from seq

• Peilonrayz: You can remove the need for the i = if you use for item in seq: yield item like itertools.cycle does. But nice answer