Reading a binary file as plain text using Python

Question

A friend of mine has written simple poetry using C's fprintf function. It was written using the 'wb' option so the generated file is in binary. I'd like to use Python to show the poetry in plain text.

What I'm currently getting are lots of strings like this: ��������

The code I am using:

with open("read-me-if-you-can.bin", "rb") as f:
 print f.read()
f.close()

• Tagc: Your original question was slightly unclear. I've attempted to clarify it but if I've inadvertently changed the meaning, you can roll it back.
• John Coleman: Just read in as a bytearray and then convert it to a string.
• João Pedro: Could you give me an example?
• Fruitspunchsamurai: Are you using the 'rb' option while reading it in?
• João Pedro: Yeah, I'm using this option. I have updated the question adding the code.

Answer 1

The thing is, when dealing with text written to a file, you have to know (or correctly guess) the character encoding used when writing said file. If the program reading the file is assuming the wrong encoding here, you will end up with strange characters in the text if you're lucky and with utter garbage if you're unlucky.

Don't try to guess, try to know: you need to ask your friend in what character encoding he or she wrote the poetry text to the file. You then have to open the file in Python specifying that character encoding. Let's say his/her answer is "UTF-16-LE" (for sake of example), you then write:

with open("poetry.bin", encoding="utf-16-le") as f:
 print(f.read())

It seems you're on Python 2 still though, so there you write:

import io
with io.open("poetry.bin", encoding="utf-16-le") as f:
 print f.read()

You could start by trying UTF-8 first though, that is an often used encoding.

• João Pedro: I used both UTF-8 and ASCII and it haven't worked... how may I get the used encoding? Asking my friend? Using an UNIX command?
• Irmen de Jong: try file poetry.bin in your shell, if it doesn't tell you what you need to know, ask your friend. Alternatively, open the file in a good text editor. It should tell you the file's encoding somewhere.
• João Pedro: read-me-if-you-can.bin: ISO-8859 text, with very long lines, with no line terminators. However, when I put "ISO-8859" into encoding variable, LookupError: unknown encoding: ISO-8859
• Irmen de Jong: ISO-8859 is a family of encodings. Try ISO-8859-1 instead. See en.wikipedia.org/wiki/Character_encoding However, this file should be trivially readable in any text editor and even in Python without using an encoding. It seems to me that the file you got is wrong or corrupted or your friend didn't write it correctly at all.
• João Pedro: UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-131071: ordinal not in range(128)