Search code examples
apache-flinkflink-streaming

Does side outputs get checkpointed?


I'm looking to utilize Flink side outputs to collect data that match a criteria and do extra processing. My main concern is fault tolerance: does Flink also checkpoint side outputs?

I was not able to find a conclusive answer from reading the Flink 1.16 docs https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/dev/datastream/fault-tolerance/checkpointing/

https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/dev/datastream/side_output/


Solution

  • Flink checkpoints state, which can be explicit (e.g. keyed state in your custom functions) or implicit (e.g. operator state in Kafka sources). It doesn't checkpoint streams, including side output streams - these are just the connections between sources, functions, and sinks.