I am trying to deploy a job to Flink from Jenkins. Thus far I have figured out how to submit the jar file that is created in the build job. Now I want to find any Flink jobs running with the old jar, stop them gracefully, and start a new job utilizing my new jar.
The API has methods to list the jobs, cancel jobs, and submit jobs. However, there does not seem to be a stop job endpoint. Any ideas on how to gracefully stop a job using API?
Even though the stop endpoint is not documented, it does exist and behaves similarly to the cancel one.
Basically, this is the bit missing in the Flink REST API documentation:
Stop Job
DELETErequest to/jobs/:jobid/stop.Stops a job, result on success is
{}.
For those who are not aware of the difference between cancelling and stopping (copied from here):
The difference between cancelling and stopping a (streaming) job is the following:
On a cancel call, the operators in a job immediately receive a
cancel()method call to cancel them as soon as possible. If operators are not not stopping after the cancel call, Flink will start interrupting the thread periodically until it stops.A “stop” call is a more graceful way of stopping a running streaming job. Stop is only available for jobs which use sources that implement the
StoppableFunctioninterface. When the user requests to stop a job, all sources will receive astop()method call. The job will keep running until all sources properly shut down. This allows the job to finish processing all inflight data.