When using spring cloud dataflow to schedule jobs, does the application supporting the job get deployed and then undeployed on the schedule, or is it continually running?
I'm trying to figure out if it's more resource efficient to deploy/undeploy a stream on a schedule to support running complex daily tasks, or if we automatically get that if we use the job scheduling functionality.
Let's start with a non-scheduler workflow in SCDF.
When you launch a batch-job as a Task, SCDF resolves the registered Task application, and it deploys to run the Task as a short-lived application in the targetted platform.
For instance, in Local-server, The Task app performs its operation, and the Java process shuts down cleanly afterward. In CF-server, the app runs as a short-lived CF-task in Cloud Foundry, and when the app operation completes, the container running the app is gracefully shutdown. Likewise, on K8s-server, the pod/job running the Task is shut down when the app finishes its operation.
Now, if you schedule a Task from SCDF, all of the above still applies to the Task app. The only difference being, the Scheduler will launch the Task instead of SCDF on the recurring cadence (i.e., cron-job expression evaluation); however, the lifecycle of the app would be the same.
In other words, the Task app will be run a short-lived resource with or without a scheduler involved. Those apps won't continue to chew up the resources when it is not actively running.