Skip to content

Navigation Menu

Sign in
Sign up

[AURON #1264] Add BatchExecCalc shadow for Flink BATCH mode Calc - #2516

Draft
argoyal2212 wants to merge 1 commit into
apache:master from
argoyal2212:argoyal/flink-batch-calc-shadow
Draft

[AURON #1264] Add BatchExecCalc shadow for Flink BATCH mode Calc #2516
argoyal2212 wants to merge 1 commit into
apache:master from
argoyal2212:argoyal/flink-batch-calc-shadow

Conversation

@argoyal2212

@argoyal2212 argoyal2212 commented Sep 8, 2026
edited
Loading

Copy link
Copy Markdown

Which issue does this PR close?

TBD

Rationale for this change

Flink's BATCH mode currently has no native execution: DefaultPlannerFactory routes it to the stock, unmodified BatchPlanner. NativePlanFusionBuilder and FlinkAuronCalcOperator are already engine-agnostic (no streaming-specific types), so the smallest correct first step is a BatchExecCalc shadow reusing both, mirroring the existing StreamExecCalc shadow.

What changes are included in this PR?

  • New org.apache.flink.table.planner.plan.nodes.exec.batch.BatchExecCalc in auron-flink-planner, matching Flink 1.18.1's stock constructor signature exactly.
    • Native conversion via NativePlanFusionBuilder.buildNativeCalcPlan, falling back to Flink's codegen Calc (or throwing if fallback is disabled), same as streaming.
    • No graph-level source fusion yet (no batch counterpart to AuronOperatorFusionProcessor); supportFusionCodegen() returns false.
    • No @ExecNodeMetadata/@JsonCreator, matching stock batch BatchExecCalc at this Flink version (compiled-plan restoration is streaming-only here).
  • Registered in shadowed-flink-execnodes.txt; added matching shade <exclude> in auron-flink-assembly/pom.xml.

No changes to any streaming file or to DefaultPlannerFactory.

Are there any user-facing changes?

Yes: BATCH queries now get native execution for Calc (projection/filter) when auron-flink-assembly is ahead of stock flink-table-planner on the classpath. Falls back to Flink's codegen Calc when not convertible, matching existing streaming behavior.

How was this patch tested?

  • ShadowedExecNodeRegistryTest and AssemblyJarStructureIT pass with the new class registered and shaded correctly.
  • Verified constructor signature and lack of @ExecNodeMetadata/@JsonCreator against the real flink-table-planner_2.12-1.18.1.jar.
  • Manually verified end-to-end on a local Flink 1.18.1 cluster: a BATCH SQL query's Calc step executes via Auron's native engine (native ProjectExec/FFIReader in task logs).

Draft while a dedicated BatchExecCalcTest (mirroring StreamExecCalcTest) is added.

Was this patch authored or co-authored using generative AI tooling?

  • Yes
  • No

ASF guidance: https://www.apache.org/legal/generative-tooling.html

Mirrors the existing StreamExecCalc shadow so native execution also covers BATCH mode's Calc operator, reusing NativePlanFusionBuilder and FlinkAuronCalcOperator unchanged. Fusion codegen and source fusion are out of scope for this first step.
Relates to apache#1264 
argoyal2212 force-pushed the argoyal/flink-batch-calc-shadow branch from 4efc256 to ee603ca Compare September 8, 2026 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

AltStyle によって変換されたページ (->オリジナル) /