Python UTF-8 conversion

Question

I would like to ask how do the following conversion (source->target) by Python program.

>>> source = '\\x{4e8b}\\x{696d}'
>>> print source
\x{4e8b}\x{696d}
>>> print type(source)
<type 'str'>
>>> target = u'\u4e8b\u696d'
>>> print target.encode('utf-8')
事業

Thank you.

Answer 1

Taking advantage of Blender's idea, you could use re.sub with a callable replacement argument:

import re
def touni(match):
 return unichr(int(match.group(1), 16))
source = '\\x{4e8b}\\x{696d}'
print(re.sub(r'\\x\{([\da-f]+)\}', touni, source))

yields

事業

• root: I suppose you really wouldn't want restrict the matches only to codepoints that have lenght 4 if you are writing a conversion program. Otherwise +1.
• unutbu: @root: Yes, I suppose that was overly restrictive.

Answer 2

You can use int and unichr to convert them:

>>> int('4e8b', 16)
 20107
>>> unichr(int('4e8b', 16))
 u'\u4e8b'
>>> print unichr(int('4e8b', 16))
事

Answer 3

import re
p = re.compile(r'[\W\\x]+')
print ''.join([unichr(int(y, 16)) for y in p.split(source) if y != ''])
事業

also stole idea from @Blender...