-
Notifications
You must be signed in to change notification settings - Fork 4.3k
[C++][Parquet] Speed up DELTA_BINARY_PACKED decoding #51268
Open
Assignees
Description
Describe the enhancement requested
DeltaBitPackDecoder decodes noticeably slower than the other integer encodings, for two
reasons that are both in the decoder rather than the format:
- The prefix-sum loop reloads the frame and stores the running value on every value, because
both members may alias the caller-owned output buffer. - The bit unpacker is called once per miniblock -- one call per 32 values with the default
geometry -- even though miniblocks sharing a bit width are packed contiguously and can be
unpacked in a single call.
Together these are 1.5x-1.7x on the DELTA_BINARY_PACKED decode benchmarks already in the tree,
on the arms where the running sum and per-call setup are a meaningful share of the work.
Component(s)
C++