reading bytes literally in python

Question

I am reading three bytes from a file in python the bytes in the hex format exact byte values are \x00 \x07 \x71

but when I read these bytes using fp.read method python converts the last byte to q

Is there any way to avoid the conversion when python reads the bytes to ASCII and literally read the byte value

• L3viathan: Do you read the file in byte-mode, i.e. open("filename", "rb")? In that case Python doesn't really "convert" anything, it just chooses to display 0x71 as "q", but that is identical. Note for example that b"q"[0] == 113.
• SyntaxVoid: But the result from your fp.read is a bytes object, isn't it? 71 is the ASCII hex value for q which is why it is displayed as q. It is not converting it to q, it is q. If you re-write your data to a new file, it will write \x71. If a = b"\x00\x07\x71", then print(a[2]) will print 113, which is the decimal value for hex 71.
• staticdev: Can you post the code you are having issues for us to try to help?
• abhishake: @SyntaxVoid thanks for the reply , the issue is that I want to convert it to it's integer value , 000771 is 1905 but when i try to convert the hexstring 0007q to int using int(0007q, 16) I get an error ValueError: invalid literal for int() with base 16
• abhishake: @SyntaxVoid int('000771', 16) gives 1905

Answer 1

From the comments, it sounds like you need to convert an arbitrary number of bytes to a single integer. So \x00\x07\x71 should be treated as one binary number. If you wanted to write your own function (which isn't too hard), you can iterate through each byte and bit-shift it appropriately dependent on its position.

For your example of a = b"\x00\x07\x71", this expands to the binary number a_b = 0b00000000_00000111_01110001. We start with the right-most byte. This doesn't need to be bitshifted at all - we add it to the result. We move on to the 2nd byte (\x07). This needs to be leftshifted 1 byte and then added to the result. Finally, the leftmost byte needs to be leftshifted 2 bytes and then added to the result. So our by-hand calculation yields (113 + 7*2^8 + 0*2^16 =わ 113 +たす 1792 +たす 0 =わ 1905)

def b_concat(bs):
 """
 Performs binary concatenation of an arbitrary number of 8-bit values.
 b_concat stands for binary-concatenation.
 Input:
 bs: bytes - A bytes string to convert to a single integer
 Returns: The concatenated result as an integer. See examples below
 """
 result = 0
 n = len(bs)
 for i in range(n):
 result += (bs[n - i - 1] << (8*i)) # or you can use bs[::-1][i]
 return result
tests = [b"\x00", b"\x01", b"\x02", b"\x00\x01", b"\x14\xa2", b"\x00\x07\x71"]
for test in tests:
 print(f"{test} --> {b_concat(test)}")

or a 1-liner, if you prefer

def b_concat(bs):
 return sum([bs[::-1][i] << (8*i) for i in range(len(bs))])

In either case, the output is

b'\x00' --> 0
b'\x01' --> 1
b'\x02' --> 2
b'\x00\x01' --> 1
b'\x14\xa2' --> 5282
b'\x00\x07q' --> 1905

Answer 2

I found the solution I use the binascii.b2a_hex to convert binary values to their hexadecimal representation and the use int(converted_val, 16) to get integer value