homepage

This is a catch-all issue for improving the email package's handling of unicode. For now, please add issues/problems you find with email & unicode to this tracker item.
For example:
MIMEText()'s first argument should accept a unicode if _charset is also given. It should not be necessary to manually encode the first argument into an 8-bit string.

Link to #1681333, #4487, #1443875, #1555842, #4661, #1078919, #963906,
#1379416 and #1368247.

Probably these are related too. #5259 #5304 

In Python 3.1.1, email.mime.text.MIMEText accepts an 8-bit charset, but not utf-8.
I think you should not have to specify a charset. All strings are unicode now, so I think the package should choose an appropriate charset based on the characters in the text, us-ascii, some iso-8859 charset, or utf-8, whatever fits.
Python 3.1.1 (r311:74480, Oct 2 2009, 11:50:52) 
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2 
Type "help", "copyright", "credits" or "license" for more information. 
>>> from email.mime.text import MIMEText 
>>> text = 'H\u00e9' 
>>> msg = MIMEText(text, 'plain', 'iso-8859-1') 
>>> print(msg.as_string()) 
Content-Type: text/plain; charset="iso-8859-1" 
MIME-Version: 1.0 
Content-Transfer-Encoding: quoted-printable 
 
H=E9 
>>> msg = MIMEText(text, 'plain', 'utf-8') 
Traceback (most recent call last): 
 File "/my/opt/Python-3/lib/python3.1/email/message.py", line 269, in set_charset 
 cte(self) 
TypeError: 'str' object is not callable 
 
During handling of the above exception, another exception occurred: 
 
Traceback (most recent call last): 
 File "<stdin>", line 1, in <module> 
 File "/my/opt/Python-3/lib/python3.1/email/mime/text.py", line 30, in __init__ 
 self.set_payload(_text, _charset) 
 File "/my/opt/Python-3/lib/python3.1/email/message.py", line 234, in set_payload 
 self.set_charset(charset) 
 File "/my/opt/Python-3/lib/python3.1/email/message.py", line 271, in set_charset 
 self._payload = charset.body_encode(self._payload) 
 File "/my/opt/Python-3/lib/python3.1/email/charset.py", line 380, in body_encode 
 return email.base64mime.body_encode(string) 
 File "/my/opt/Python-3/lib/python3.1/email/base64mime.py", line 94, in body_encode 
 enc = b2a_base64(s[i:i + max_unencoded]).decode("ascii") 
TypeError: must be bytes or buffer, not str 
>>>

Now that we are primarily focused on Python3 development, collecting "unicode" issues is not really all that useful (at least not to me, and I'm currently doing the email maintenance), so I'm closing this. All the relevant issues are assigned to me anyway, so I'll be dealing with them by and by.

I am having hard time parsing all the text/html and text/plain parts of a message, concatenating them into a string. I am thinking of writing some custom code to do manual handling of this...
If this could be fixed that would be great. The issues are converting from and to ascii/unicode or whatever encoding/charset the part uses.

That particular problem will get fixed in the next version of the email package (hopefully in Python3.3), but that isn't ready yet.