GH-51293: [Python] Reject a null Expression in two public APIs - #51294

This PR fixes a hard crash in the PyArrow Dataset Python API by rejecting a None partition expression at the Cython boundary, converting an interpreter "bus error" into a normal Python TypeError and adding a regression test.

Changes:

• Mark get_partition_keys’s partition_expression parameter as not None in python/pyarrow/_dataset.pyx so None is rejected before dereferencing.
• Add a regression test ensuring ds.get_partition_keys(None) raises TypeError.

• Files reviewed: 2/2 changed files
• Comments generated: 0
• Review effort level: Lite

• Files reviewed: 3/3 changed files
• Comments generated: 2
• Review effort level: Lite

Suppressed comments (4)

Previously missed (1) — in code that hasn't changed since the last review.

python/pyarrow/tests/test_misc.py:319

• This subprocess sweep has no timeout, so a single blocking callable can hang the entire test run. Consider adding a reasonable timeout (and include stderr in the failure) to make CI failures deterministic and easier to debug.

This issue also appears on line 377 of the same file.

python/pyarrow/tests/test_misc.py:280

• The comment says passing None to any public callable must produce a Python exception, but the test intentionally allows callables to accept None (it only asserts the subprocess doesn't crash). Please update the comment to match the actual contract (no crash).

 # GH-51293: a typed Cython parameter that is not declared "not None"
 # lets None reach code that dereferences it, killing the interpreter
 # instead of raising. Passing None to any public callable must produce
 # a Python exception, never a fatal signal.

python/pyarrow/tests/test_misc.py:381

• Same issue as the callable sweep above: this subprocess.run has no timeout and the failure message drops stderr, which can lead to hung CI jobs and harder debugging when something goes wrong.

 res = subprocess.run([sys.executable, "-c", code],
 capture_output=True, text=True)
 lines = res.stdout.splitlines()
 if not lines or lines[-1] != "DONE":
 culprit = lines[-1] if lines else "<no output>"

python/pyarrow/_compute.pyx:2687

• The equals() docstring parameter type references pyarrow.dataset.Expression, but this is pyarrow.compute.Expression (and the method signature enforces that). This mismatch can confuse users reading the docs/help().

 Parameters
 ----------
 other : pyarrow.dataset.Expression

• Files reviewed: 4/4 changed files
• Comments generated: 0 new
• Review effort level: Lite

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

python/pyarrow/_compute.pyx:2687

• The Expression.equals docstring references pyarrow.dataset.Expression, but this type lives in pyarrow.compute (and this method is defined in pyarrow/_compute.pyx). Updating the docstring avoids confusing API users.

 def equals(self, Expression other not None):
 """
 Parameters
 ----------
 other : pyarrow.dataset.Expression

python/pyarrow/tests/test_misc.py:363

• The subprocess method sweep uses inspect._empty, which is a private inspect implementation detail. Prefer the public inspect.Parameter.empty to reduce risk of breakage across Python versions.

• Files reviewed: 4/4 changed files
• Comments generated: 0 new
• Review effort level: Lite