homepage

The documentation for the email.Generator module claims
that the flatten() method is idempotent (i.e., output
identical to the input, if email.Parser.Parser was used
on the input), but it is not in all cases. The most
obvious example is that you need to disable mangle_from
and set maxheaderlen=0 to disable header wrapping. 
This could be considered common sense, but the
documentation should mention it, as both are enabled by
default. (unixfrom can also create differences between
input and output, but is disabled by default.) More
importantly, whitespace is not preserved in headers: if
there are extra spaces between the header name and the
header contents, it will be collapsed to a single space.
This little snippet will demonstrate the problem:
 parser = email.Parser.Parser()
 msg = parser.parse(sys.stdin)
 print msg
 gen = email.Generator.Generator(sys.stdout,
mangle_from_=False, maxheaderlen=0)
 gen.flatten(msg, unixfrom=False)
Feed it a single message with extra spaces beween field
name and field contents in one or more fields, and diff
the input and the output.
It is probably not worth actually making these routines
idempotent, as preserving whitespace is not important
in most applications and would require extra
bookkeeping. However, as long as the documentation
claims the routines are idempotent, it is a bug not to
be. In my particular application, it was important to
be truly idempotent, so this was a problem. Had the
documentation not made false claims, I would have known
from the start that I needed to write my own versions
of the routines in the email module.

I think we should update the documentation to mention these exceptions.

Does this belong with RDM or docs@python?

Here is a patch that adds a footnote explaining the issue.

New changeset 8bd30967bc4b by R David Murray in branch '3.2':
#1440472: Explain that email parser/generator isn't *quite* "idempotent"
http://hg.python.org/cpython/rev/8bd30967bc4b
New changeset f534e6363bfb by R David Murray in branch 'default':
merge #1440472: Explain that email parser/generator isn't *quite* "idempotent"
http://hg.python.org/cpython/rev/f534e6363bfb
New changeset 9d99273c2f74 by R David Murray in branch '2.7':
#1440472: Explain that email parser/generator isn't *quite* "idempotent"
http://hg.python.org/cpython/rev/9d99273c2f74
New changeset 180d16af22e9 by R David Murray in branch '2.7':
#1440472: reflow
http://hg.python.org/cpython/rev/180d16af22e9
New changeset d1fbfd9af5c5 by R David Murray in branch '3.2':
#1440472: reflow
http://hg.python.org/cpython/rev/d1fbfd9af5c5
New changeset 4ec95207281c by R David Murray in branch 'default':
merge #1440472: reflow
http://hg.python.org/cpython/rev/4ec95207281c