reading structured binary data in python3.6 using struct

Question

I really tried to look for this doubt in many ways but maybe since I never worked with binary files before I don't have any idea what the keywords to search similar things to help me out. That's why I am asking here.

So, I have a file:

path = 'myPath/file.pd0'
in_file = open(path, "rb")
read_file = in_file.read()
type(read_file) 

when I try to check what is inside read_file I get:

b'\x7f\x7f\xcc\x05\x00\x0f$\x00`\x00\xa2\x00$\x02\xe6\x02\xa8\x03\xd0\x032\x04d\x04\x96\x04\xa6\x04\xe0\x04'

The type of read_file is bytes. When I try to use struct since it is the function people suggest I get the following error:

import struct
struct.unpack('hhl', read_file[0:30])
error: unpack requires a buffer of 16 bytes

No matter what fmt I get unpack requires a buffer of n bytes.

The file structure that I am trying to read is defined as follow:

HEADER (6 BYTES + [2 x No. OF DATA TYPES])

FIXED LEADER DATA (60 BYTES)

VARIABLE LEADER DATA (66 BYTES)

CORRELATION MAGNITUDE (2 BYTES + 4 BYTES PER DEPTH CELL)

Any idea how I could start reading these bytes using struct or something similar in python?

Thank you

Answer 1

unpack() expects bytes which are the exact length of the format described by its first argument. The format string 'hhl' describes data of 16 bytes (on your machine - see below), so you must pass a byte string of 16. If you want to parse only part of the bytes, you can do this:

fmt = 'hhl'
size = struct.calcsize(fmt)
struct.unpack(fmt, data[:size])

Additionally, your format string doesn't have a byte order, size and alignment specifier. It is assumed to be "native" by default. This means your code is system-dependent, which is probably not what you want for parsing a file format. You might need different alignments for different parts of the file.