Confusion about Python Decode method

Question

I'm trying to run the command u'\xe1'.decode("utf-8") in python and I get this error:

Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/encodings/utf_8.py", line 16, in decode
 return codecs.utf_8_decode(input, errors, True)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 0: ordinal not in range(128)

Why does it say I'm trying to decode ascii when I'm passing utf-8 as the first argument? In addition to this, is there any way I can get the character á from u'\xe1' and save it in a string?

• Padraic Cunningham: what exactly are you trying to do?
• BrockLee: The python script I'm running takes text, processes it, and prints a JSON string containing a categorized version of the original text. What I've noticed is that characters like this sometimes end up as their unicode values in the printed JSON string.
• Padraic Cunningham: when you print your string you will see á
• BrockLee: So I was able to solve the problem. Thank you guys for the help. But I'm still confused on why the error says it's an ascii encoding problem when I'm using utf-8 instead.

Answer 1

decode will take a string and convert it to unicode (eg: "\xb0".decode("utf8") ==> u"\xb0")

encode will take unicode and convert it to a string (eg: u"\xb0".encode("utf8") ==> "\xb0")

neither has much to do with the rendering of a string... it is mostly an internal representation

try

print u"\xe1"

(your terminal will need to support unicode (idle will work ... dos terminal not so much))

>>> print u"\xe1"
á
>>> print repr(u"\xe1".encode("utf8"))
'\xc3\xa1'
>>> print repr("\xc3\xa1".decode("utf8"))
u'\xe1'

• Hackaholic: hey can we do this too??. >>> chr(ord("\xe1")) 'á'
• Terry Jan Reedy: The rule given in the answer is mostly true for 2.x, perhaps always for 3.x. The example output is for 2.x, slightly different in 3.x.
• Joran Beasley: in python 2 it is >>> unichr(ord("\xe1")) 'á' @Hackaholic