Python - Reading and Writing Structured Binary Files

Question

Currently I'm trying to manipulate a kind of binary file, which structure is like this:

FileHeader + [SectionHeader + Binary(1D-array)] * NumSetions

After searching on the internet, I came up with the following code to read it:

import numpy as np
# Always BIG Endian
#file_header bytes: 12
file_header_dtype = np.dtype([
 ('nsection', '>i4'), # number of sections
 ('nsample', '>i4'), # number of samples for each section
 ('dt', '>f4'), # sampling rate
 ])
#section_header bytes: 28
section_header_dtype = np.dtype([
 ('rec', '>i4'), # record number
 ('x', '>f8'), # x, in meter
 ('y', '>f8'), # y, in meter
 ('z', '>f8'), # z, in meter
 ])
def dtype_section(nt):
 return np.dtype(section_header_dtype.descr + [('binary', ('>f4',nt))])
def read_file_header(_file):
 with open(_file, 'rb') as f:
 file_header = np.fromstring(f.read(12), dtype=file_header_dtype)
 return file_header
def readFile(filename):
 raw = open(filename, 'rb')
 file_header = np.fromstring(raw.read(12), dtype=file_header_dtype)
 nt = file_header['nsample'][0]
 dtype_file_sections = dtype_section(nt)
 data = np.fromstring(raw.read(), dtype=dtype_file_sections)
 return (file_header, data)

With this way, I can easily call the header-striped binary part and perform plt.imshow or anything else.

data = readFile('site1.bin')
data_arr = data[1]['binary']
#plt.imshow(data_arr)

The problem is, I cannot find a way to output the data while maintaining same data structure

ndarray.tofile() only works for one array per time

And np.array((data[0],data[1])).tofile('test') would cause IOError

IOError: cannot write object arrays to a file in binary mode

I'm pretty new to Python and I'm not sure if I made any mistake. Or should I consider another way to read this kind of file, rather than using numpy.dtype? Please help me.

• juanpa.arrivillaga: Shouldn't you be passing the result of file_header['nsample'][0] to dtype_section(nt)?
• Pakox.Wang: Oops, my bad. Let me correct it

Answer 1

The straightforward way would be to simply write to a binary file:

with open('test','wb') as f:
 f.write(data[0].tobytes())
 f.write(data[1].tobytes())

• Pakox.Wang: I think you're referring to numpy.ndarray.tobytes(). To think of the solution is so simple and obvious, I'm feeling embarrassed now... Anyway, thank you with the answer.
• juanpa.arrivillaga: @Pakox.Wang yes, I meant tobytes(), sorry, there are many similarly named methods :)