Skip to main content
Stack Overflow
  1. About
  2. For Teams

Return to Answer

Post Timeline

Add mmap option
Source Link
Davis Herring
  • 42.3k
  • 4
  • 59
  • 92

Seeking, possibly several times, within just 50 kB is probably not worthwhile: system calls are expensive. Instead, read each message into one bytes and use slicing to "seek" to the offsets you need and get the right amount of data.

It may be beneficial to wrap the bytes in a memoryview to avoid copying, but for small amounts of dataindividual reads it probably doesn’t matter much. If you can use a memoryview, definitely try using mmap, which exposes a similar interface over the whole file. If you’re using struct, its unpack_from can already seek within a bytes or an mmap without wrapping or copying.

Seeking, possibly several times, within just 50 kB is probably not worthwhile: system calls are expensive. Instead, read each message into one bytes and use slicing to "seek" to the offsets you need and get the right amount of data.

It may be beneficial to wrap the bytes in a memoryview to avoid copying, but for small amounts of data it probably doesn’t matter much. If you’re using struct, its unpack_from can already seek within a bytes without wrapping or copying.

Seeking, possibly several times, within just 50 kB is probably not worthwhile: system calls are expensive. Instead, read each message into one bytes and use slicing to "seek" to the offsets you need and get the right amount of data.

It may be beneficial to wrap the bytes in a memoryview to avoid copying, but for small individual reads it probably doesn’t matter much. If you can use a memoryview, definitely try using mmap, which exposes a similar interface over the whole file. If you’re using struct, its unpack_from can already seek within a bytes or an mmap without wrapping or copying.

Source Link
Davis Herring
  • 42.3k
  • 4
  • 59
  • 92

Seeking, possibly several times, within just 50 kB is probably not worthwhile: system calls are expensive. Instead, read each message into one bytes and use slicing to "seek" to the offsets you need and get the right amount of data.

It may be beneficial to wrap the bytes in a memoryview to avoid copying, but for small amounts of data it probably doesn’t matter much. If you’re using struct, its unpack_from can already seek within a bytes without wrapping or copying.

lang-py

AltStyle によって変換されたページ (->オリジナル) /