homepage

The json library's encoder includes a function called 'iterencode'. iterencode allows for encoding to be streamed; as tokens are produced they are yielded. This allows for the encoded object to be streamed to a file, over a socket, etc. without being placed all into memory.
Unfortunately, iterencode cannot encode general iterators. This significantly limits the usefulness of the function. For my use case I wish to convert a large stream (iterator) of objects into json. Unfortunately, I currently have to:
A. Bring all the objects into memory by encasing the iterator in a list()
B. Make a hack where I subclass list and making that object's __iter__ function return my desired iterator.
The problem is that the json library explicitly checks for something being a list:
 if isinstance(value, (list, tuple)):
 chunks = _iterencode_list(value, _current_indent_level)
It would work just as well (and be more pythonic) to see if the value supports the iterator protocol:
 if isinstance(value, collections.Iterable):
 chunks = _iterencode_list(value, _current_indent_level)
Erroring example:
>>> import json
>>> e = json.JSONEncoder()
>>> r = xrange(20)
>>> gen = e.iterencode(r)
<generator object _iterencode at 0x14a5460>
>>> next(gen)
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/usr/lib/python3.2/json/encoder.py", line 419, in _iterencode
 o = _default(o)
 File "/usr/lib/python3.2/json/encoder.py", line 170, in default
 raise TypeError(repr(o) + " is not JSON serializable")
TypeError: xrange(0, 20) is not JSON serializable