Python, mysqldb and unicode

Question

I'm having trouble to get unicode values out of mysql queries.

Here is how I do it now:

>>> from MySQLdb import connect
>>> from MySQLdb.cursors import DictCursor
>>> con = connect(
 passwd = "****",
 db = 'my_db',
 user = "db_user",
 host = "localhost",
 cursorclass = DictCursor,
 use_unicode=True,
 charset="utf8"
)
>>> cursor = con.cursor ()
>>> cursor.execute (u'Select * from basic_applet')
>>> res = cursor.fetchall()
>>> print(res)
({'Title_de': 'test title', .... })
>>> type(res[0]['Title_de'])
<type 'str'>

As you can see, it is not returning unicode.

In my table structure, Title_de is set as unicode.

IDbasic_applet int(10)...
Title_de varchar(75) utf8_bin

I really don't know what I'm doing wrong, any help would be really welcome.

Thanks in advance,

Simon

Answer 1

What you get is a bytestring. You must decode it in order to get a unicode string. It basically comes down to this:

>>> byte_string = 'F\xe9vrier'
>>> byte_string.decode('UTF-8')
u'Février'

• Simon Rolin: I understand that, the fact is that I thought it was possible to ask for unicode string instead of byte string. Do you know what is the effect of use_unicode=True?
• pvoosten: It has the desired effect for me. My column definition contains CHARACTER SET UTF8 instead of utf8_bin. Afaik that's the only difference.
• mcrisc: It's been some time since @SimonRolin asked his question, but just for the record MySQLdb.connect(use_unicode=True, charset='utf8', ...) makes MySQL return unicode.
• fede.evol: Be aware that until MySQLdb 1.2.3 (which is still the current version in many packet based sitros) there was a bug which prevented the utf8_bin columns working properly. Check here for the bug description: sourceforge.net/p/mysql-python/bugs/289