Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

feat: Stop Flink job with drain in test executions#2129

Open
ferenc-csaky wants to merge 4 commits into
main from
feat/drain-flink-job-in-test
Open

feat: Stop Flink job with drain in test executions #2129
ferenc-csaky wants to merge 4 commits into
main from
feat/drain-flink-job-in-test

Conversation

@ferenc-csaky

@ferenc-csaky ferenc-csaky commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

Key Changes

  • Instead of Flink cancel, use stop with advancing watermark to the end of event time to make sure no event is stuck in the pipeline
  • Since the Flink API only exposes stopWithSavepoint, we cannot omit taking a savepoint, which gets saved to a temp dir, and then deleted right away
  • This stop action will now trigger a checkpoint failure (expected, cause the job is already stopping), added some logic to exclude those warnings with stacktraces from the output
  • Removed the now unnecessary watermark advance record from the banking test data, and updated every snapshot that used it

@ferenc-csaky ferenc-csaky force-pushed the feat/drain-flink-job-in-test branch from 283141c to 58503b5 Compare June 12, 2026 11:04

codecov Bot commented Jun 12, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 18.27957% with 76 lines in your changes missing coverage. Please review.
✅ Project coverage is 14.07%. Comparing base (0d9795d) to head (e1c6715).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...i/src/main/java/com/datasqrl/cli/DatasqrlTest.java 26.41% 36 Missing and 3 partials ⚠️
...li/src/main/java/com/datasqrl/cli/DatasqrlRun.java 7.50% 36 Missing and 1 partial ⚠️
Additional details and impacted files 
@@ Coverage Diff @@
## main #2129 +/- ##
============================================
+ Coverage 14.03% 14.07% +0.04% 
- Complexity 864 866 +2 
============================================
 Files 606 606 
 Lines 17359 17401 +42 
 Branches 2104 2107 +3 
============================================
+ Hits 2436 2449 +13 
- Misses 14688 14714 +26 
- Partials 235 238 +3

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@ferenc-csaky ferenc-csaky added this to the 0.10.6 milestone Jun 12, 2026

Copy link
Copy Markdown
Contributor

@ferenc-csaky I am surprised that we don't have more tests that are affected by this. The only one seems to be banking, right?

ferenc-csaky commented Jun 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Even that was not affected, cause it ran before I changed the execution flow to only execute the test queries after the Flink job exited.

The changes were necessary cause the record i removed affected the calculations (which points to the same direction, it was already encountered before this change).

Copy link
Copy Markdown
Contributor

@ferenc-csaky I would suggest we add a time window query to our test cases - those should definitely be affected because the last window should not have closed before this change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mbroecheler mbroecheler Awaiting requested review from mbroecheler

@mateczagany mateczagany Awaiting requested review from mateczagany

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

enhancement New feature or request

Projects

None yet

Milestone

0.10.6

Development

Successfully merging this pull request may close these issues.

AltStyle によって変換されたページ (->オリジナル) /