In the Flink logic as below, I am wondering the mechanism of the checkpointing.
Suppose that there is simple dataflow as below.
source (kafka) -> process -> sink (db)
As far as I know, the checkpoint barrier is injected and go through the entire operators.
when barrier arrives at the source, the offset and partition are stored to external storage by snapshot and finishing the checkpointing at the source operator and go next step. Also, do the same thing in the following operators.
At this sink operator, suppose that the exception occurred, and the job restarted.
In the scenario, the entire checkpoint is not finished and can see the checkpoint is failed in the dashboard.
As I mentioned I know that the entire checkpoint is failed but wondering that the state is rollback or is maintained in the before operators (source, process)
Or the state is rollback because the one of the operators (sink) has exception and restarts the job?
Thanks.
The state in every operator is rolled back to the state recorded in the checkpoint, and the job is restarted.