Group Window Aggregate function is causing my checkpoint size to increase dramatically

Question

I'm experiencing an issue with state management in my Flink job:

I have two Kafka sources that I'm unioning together and the second source uses a tumbling window based on processing time. I'm using RocksDB as the state backend In the Flink UI dashboard, I notice there are no watermarks enabled for this particular union/group by and I suspect the windows are never expiring from state My event time isn't critically important, so I don't want to use watermarks or tumble based on event time. While I could take savepoints periodically as a workaround, I'd prefer to understand what's happening first.

Any suggestions on how to debug this state expiration issue with processing time windows?

Answer 1

What makes you think there's an issue with the state? With RocksDB, state that has been deleted can take some time to actually be compacted and freed. By default, I believe compaction begins somewhere around 400MB.

• BackgroundChecker1994: The cpu utilization hits 100% maximum (70% average) as the state goes over 200mb. I actually thought compaction starts around 200mb. I think there is an issue with state because my sources, and keyed operation take kbs whereas my group window aggregate (union using proc time) takes close to 99% of the state size. I am not sure how do I expire state for proc time windows though i did verify the start and end of window is whatever I set the tumble proc time to be
• David Anderson: You don't have to do anything to expire state for proc time windows. Their state is automatically freed when the window ends, unless you have done something with globalState.