I am writing a SCDF spi implementation for supporting stream and task application. As part of this we need to perform some clean up operations once the task finishes.
Could someone provide info on whether SCDF will be getting a callback on task completion. If not then what are the alternative ways to perform cleanup.
A Task is a short-lived and finite operation. Depending on what you're trying to accomplish, you can do one of the following to invoke any custom cleanup routine.
1) A task running a batch-job and in that job, you can define "n" number of steps as part of a workflow, and upon successful upstream steps, the last step could invoke the cleanup routine.
2) You can have a stream in SCDF listening to Task-complete events (a batch-job example here), which can finally kickoff another task/job to invoke the cleanup routine.
3) You can define a composed-task graph (via Dashboard/shell) where each of the steps (aka tasks) can run its intended operation, and upon successful transition or failure event, you get the opportunity to kick off the cleanup routine.