Skip to content

Navigation Menu

Sign in
Sign up

[AURON #2518] Cache maximum accumulator indices to avoid repeated scans - #2519

Open
lyne7-sc wants to merge 5 commits into
apache:master from
lyne7-sc:perf/agg-idx-cache
Open

[AURON #2518] Cache maximum accumulator indices to avoid repeated scans #2519
lyne7-sc wants to merge 5 commits into
apache:master from
lyne7-sc:perf/agg-idx-cache

Conversation

@lyne7-sc

@lyne7-sc lyne7-sc commented Sep 9, 2026

Copy link
×ばつ 8,192 rows and Int64 SUM expressions. | Benchmark | Before | After | Time reduction | | --- | ---: | ---: | ---: | | hash_agg_1_sum_1024_groups | 6.171 ms | 5.321 ms | 13.8% | | hash_agg_4_sums_1024_groups | 12.140 ms | 8.221 ms | 32.3% | | hash_agg_8_sums_1024_groups | 20.022 ms | 12.374 ms | 38.2% | | hash_agg_1_sum_8192_groups | 5.541 ms | 5.108 ms | 7.8% | | hash_agg_4_sums_8192_groups | 11.001 ms | 8.280 ms | 24.7% | | hash_agg_8_sums_8192_groups | 18.004 ms | 12.889 ms | 28.4% | # Was this patch authored or co-authored using generative AI tooling? - [x] Yes - [ ] No If yes, include: `Generated-by: openai codex gpt-6-astra` ASF guidance: https://www.apache.org/legal/generative-tooling.html " data-view-component="true"> Copy Markdown
Contributor

Which issue does this PR close?

Closes #2518

Rationale for this change

Accumulator capacity checks repeatedly compute the maximum over the same group-index array. Caching this maximum allows the checks to reuse it across aggregate expressions, reducing redundant scans while preserving capacity guarantees.

What changes are included in this PR?

  • Add cached index selections for usize and u32 indices.
  • Use private fields to prevent inconsistent cached values.
  • Reuse the cached maximum across table-level and individual accumulator capacity checks.
  • Cache shared indices at update and merge entry points, including per-aggregate FILTER selections.
  • Add regression tests for capacity growth, index iteration, NULL values, FILTER, and multi-batch aggregation.
  • Add operator-level benchmarks with different numbers of SUM expressions.

Are there any user-facing changes?

No. This is an internal performance optimization.

How was this patch tested?

UTs passed.

Benchmarks

Native Partial → Final HashAgg with 32 batches ×ばつ 8,192 rows and Int64 SUM expressions.

Benchmark Before After Time reduction
hash_agg_1_sum_1024_groups 6.171 ms 5.321 ms 13.8%
hash_agg_4_sums_1024_groups 12.140 ms 8.221 ms 32.3%
hash_agg_8_sums_1024_groups 20.022 ms 12.374 ms 38.2%
hash_agg_1_sum_8192_groups 5.541 ms 5.108 ms 7.8%
hash_agg_4_sums_8192_groups 11.001 ms 8.280 ms 24.7%
hash_agg_8_sums_8192_groups 18.004 ms 12.889 ms 28.4%

Was this patch authored or co-authored using generative AI tooling?

  • Yes
  • No

If yes, include: Generated-by: openai codex gpt-6-astra

ASF guidance: https://www.apache.org/legal/generative-tooling.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Avoid repeated maximum-index scans in aggregate capacity checks

1 participant

AltStyle によって変換されたページ (->オリジナル) /