Skip to content

Navigation Menu

Sign in
Sign up

GH-51042: [Python] Reject invalid Arrow wrapper types - #51163

Open
1fanwang wants to merge 1 commit into
apache:main from
1fanwang:1fannnw/fix-pyarrow-type-segfaults
Open

GH-51042: [Python] Reject invalid Arrow wrapper types #51163
1fanwang wants to merge 1 commit into
apache:main from
1fanwang:1fannnw/fix-pyarrow-type-segfaults

Conversation

@1fanwang

@1fanwang 1fanwang commented Sep 5, 2026 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor

Rationale for this change

Several PyArrow APIs accept values that are not the documented Arrow wrapper type, pass a null native pointer into C++, and terminate the Python process. Invalid inputs should raise TypeError.

Fixes #51042.

What changes are included in this PR?

The affected Parquet, sparse tensor, Substrait, and compute option entry points now validate their wrapper objects before native calls. The same guard covers the matching sibling APIs.

Are these changes tested?

The reported calls were run independently against PyArrow 25.0.1, then covered by focused tests against the patched source.

Raw logs 
$ python -c 'import pyarrow.dataset as ds; ds.ParquetReadOptions(binary_type=0)'
Segmentation fault: 11
$ python -c 'import pyarrow as pa; pa.SparseCOOTensor.from_tensor(0)'
Segmentation fault: 11
$ python -c 'import pyarrow.dataset as ds; ds.ParquetReadOptions(binary_type=0)'
TypeError: DataType expected, got <class 'int'>
$ python -c 'import pyarrow as pa; pa.SparseCOOTensor.from_tensor(0)'
TypeError: Argument 'obj' has incorrect type (expected pyarrow.lib.Tensor, got int)
$ pytest -q python/pyarrow/tests -k 'parquet_read_options or parquet_file_rejects_invalid_binary_type or sparse_tensor_from_tensor_rejects_invalid_type or serializing_schema or option_class_equality'
........ [100%]
8 passed, 4 skipped, 8262 deselected

Are there any user-facing changes?

Yes. Invalid wrapper arguments now raise TypeError instead of crashing the interpreter.

 

Generated-by: GitHub Copilot CLI (GPT-5.6 Sol)
Signed-off-by: 1fanwang <1fannnw@gmail.com>
Copilot AI lite review requested due to automatic review settings September 5, 2026 04:21

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown

⚠️ GitHub issue #51042 has been automatically assigned in GitHub to PR creator.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The changes directly address the reported segfaults by enforcing wrapper-type validation at the Cython boundary and add targeted regression tests for each affected API.

Pull request overview

This PR hardens several PyArrow entry points (Parquet read options, sparse tensor conversion, Substrait serialization, and compute IndexOptions) against invalid wrapper arguments that previously could propagate null native pointers into C++ and crash the interpreter; invalid inputs now raise TypeError instead.

Changes:

  • Add explicit wrapper-type validation for Parquet binary_type in both dataset and parquet reader paths.
  • Enforce Tensor, Schema, and Scalar wrapper types at key Cython boundaries (sparse from_tensor, Substrait serialization, and IndexOptions).
  • Add focused regression tests covering the previously-crashing invalid inputs.
File summaries
File Description
python/pyarrow/_parquet.pyx Reject non-DataType binary_type before unwrapping and native calls.
python/pyarrow/_dataset_parquet.pyx Reject non-DataType binary_type in ParquetReadOptions setter.
python/pyarrow/_compute.pyx Require Scalar for IndexOptions / _set_options to prevent invalid unwraps.
python/pyarrow/tensor.pxi Require Tensor for sparse from_tensor conversions across sparse tensor types.
python/pyarrow/_substrait.pyx Require Schema for Substrait serialize_schema / serialize_expressions.
python/pyarrow/tests/test_dataset.py Add regression test for ParquetReadOptions(binary_type=0) raising TypeError.
python/pyarrow/tests/parquet/test_parquet_file.py Add regression test for ParquetFile(..., binary_type=0) raising TypeError.
python/pyarrow/tests/test_compute.py Add regression test for IndexOptions(0) raising TypeError.
python/pyarrow/tests/test_sparse_tensor.py Add regression test ensuring sparse from_tensor(0) raises TypeError.
python/pyarrow/tests/test_substrait.py Add regression tests for Substrait serialization rejecting invalid schema inputs.
Review details
  • Files reviewed: 10/10 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@AlenkaF AlenkaF left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, the PR looks good to me.

Might you be willing to grep the Cython bindings more broadly for the same pattern? (also commented on #51161 (review)).

1fanwang reacted with eyes emoji

1fanwang commented Sep 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Thanks, the PR looks good to me.

Might you be willing to grep the Cython bindings more broadly for the same pattern? (also commented on #51161 (review)).

Thanks @AlenkaF

Swept the rest of the Cython bindings for the same pattern. Passing None to every public module-level callable across the main modules turned up one reachable crash, dataset.get_partition_keys, which killed the interpreter with a bus error. Filed as #51293 and fixed in #51294.

That PR also adds a guard test so the class stays covered: it passes None to each public callable in a child process and fails naming the offender if the interpreter dies by signal. Most typed parameters handle None deliberately, so it asserts that nothing crashes rather than requiring not None everywhere.

AlenkaF commented Sep 11, 2026

Copy link
Copy Markdown
Member

Thanks. I think the only missing thing from the connected issue is acero.Declaration.from_sequence([0.0]) case, then this can be merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AlenkaF AlenkaF AlenkaF left review comments
Copilot code review Copilot
Copilot review effort, defaults to Lite
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@raulcd raulcd Awaiting requested review from raulcd raulcd is a code owner
@rok rok Awaiting requested review from rok rok is a code owner

Assignees

No one assigned

Labels

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Python] Several APIs segfault instead of rejecting arguments of the wrong type

AltStyle によって変換されたページ (->オリジナル) /