Skip to content

Navigation Menu

Sign in
Sign up

GH-36472: [Ruby] Add run-end encoded array values and raw records - #51276

Open
emecii wants to merge 1 commit into
apache:main from
emecii:gh-36472-ruby-run-end
Open

GH-36472: [Ruby] Add run-end encoded array values and raw records #51276
emecii wants to merge 1 commit into
apache:main from
emecii:gh-36472-ruby-run-end

Conversation

@emecii

@emecii emecii commented Sep 10, 2026
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor

Rationale for this change

GH-36472 requests the same native Ruby conversion support for run-end encoded arrays as other arrays. Currently RunEndEncodedArray#values resolves to a GObject child property, while chunked values and record conversion raise NotImplemented.

What changes are included in this PR?

  • Map logical positions to physical runs with Arrow's run-end utility, then convert the physical value through the existing native converters, including null and nested values.
  • Add REE dispatch to array values, bulk/streaming record conversion, and the existing nested-array visitors.
  • Remove the shadowing GObject values getter so Array#values is inherited. The physical child remains available through the existing values_raw method; logical_values is unchanged.

Are these changes tested?

Built the C++/GLib libraries from main at 68243c4ba and compiled the changed native Ruby extension on macOS arm64 with Ruby 4.0.6.

  • New regression matrix: 43 tests, 339 assertions, covering all three run-end widths, a logical offset above 32 bits, sliced physical children, slices within runs, null runs, empty arrays/chunks, nested values and REE children, and both Table/RecordBatch bulk and streaming paths. Against the unchanged extension, these tests produce 6 failures and 34 NotImplemented errors.
  • Full red-arrow suite: 2,710 tests, 3,014 assertions, zero failures/errors; two ORC omissions.
  • GLib suite in the locally built core configuration: 2,238 tests, 2,299 assertions, zero failures/errors; 243 optional-component/platform omissions. A local preload excludes installed Arrow 25 optional typelibs from the Arrow 26 process. Both native GLib C examples pass.
  • Repository-configured RuboCop 1.71.0, Ruby syntax, git diff --check, and the repository RAT license audit pass.

Are there any user-facing changes?

REE arrays now support native Ruby values and raw-record conversion.

This PR includes breaking changes to public APIs. RunEndEncodedArray#values returns a Ruby array of logical values instead of the GObject physical-child property. Callers needing that child should use the already available values_raw method.

AI assistance

OpenAI Codex was used to generate the implementation and regression tests and to run the validation described above.

Convert logical run positions through the native value converters, including nested values and null runs. Restore Array#values while retaining values_raw for physical child access.
Generated-by: OpenAI Codex
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kou kou Awaiting requested review from kou kou is a code owner

Assignees

No one assigned

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

AltStyle によって変換されたページ (->オリジナル) /