homepage

If you pass an iterable of tokens and none of them are an ENCODING token, tokenize.untokenize() returns a string. This is contrary to what the docs say:
 It returns bytes, encoded using the ENCODING token, which is the
 first token sequence output by tokenize().
Either the docs should be clarified or untokenize() fixed. My vote is to fix it. It could check that the first token is an ENCODING token and raise an exception. Alternately it could fall back to using 'utf-8' by default.
[1] http://docs.python.org/py3k/library/tokenize.html#tokenize.untokenize