Search code examples
spring-cloud-streamspring-cloud-dataflowspring-cloud-task

Spring Cloud Task for functional programming - how really to achieve it?


We have a need to develop on-demand applications on the lines of FRP (something similar to AWS Lambda) on our private Cloud Foundry stack. This is in order to save cost on otherwise always running low volume applications.

The need is to trigger/start the application only when a message comes on our JMS based messaging system (e.g. IBM MQ). Being a low volume application, it should remain stopped all other times.

Based upon the detailed study and search through Spring documents, it appears to me that, a desired solution that triggers/start my microservice whenever a message appears on the source queue and shuts it down when done, will need a combination of: Spring Cloud Data Flow + Spring Cloud Stream + Spring Cloud Task as the technology stack.

Out of the various samples, below combination looks like the one needed to start PoC on local machine:

https://github.com/spring-cloud/spring-cloud-task/tree/master/spring-cloud-task-samples/tasksink https://github.com/spring-cloud/spring-cloud-stream-samples/tree/master/sink https://github.com/spring-cloud/spring-cloud-dataflow/tree/master/spring-cloud-dataflow-server-local

However am still unsure how to plug it all and achieve the results.

Some progress is made but the full clarity of architecting this solution is still unclear.

To best of my understanding it appears that we need to...

  1. run a Spring Cloud DataFlow Server
  2. run Spring Cloud DataFlow Shell or UI
  3. create Spring Cloud Stream sink application and run it (do we need to install it in Maven as well?)
  4. create Spring Cloud Task sink application and install it in Maven Repo
  5. do I need a Spring Cloud Task as well created?
  6. register modules (but what are modules for this case?)
  7. create Stream in DF Server and deploy on it
  8. anything more?

...to achieve the results.

Can someone help me in this and suggest is any building block is missed out from here? And if the answers to questions above are known.

Also are there any gaps in my understanding, and am I assuming it correctly enough that @EnableTask will make my application run only when triggered and keep it stopped otherwise?


Solution

  • There is an important distinction you need to make when choosing the Spring Cloud Stream vs Spring Cloud Task applications.

    Spring Cloud Stream -> Long lived micro service application that is event driven Spring Cloud Task -> Short lived micro service application that runs on demand

    am I assuming it correctly enough that @EnableTask will make my application run only when triggered and keep it stopped otherwise

    Yes, but the task application is not stopped instead it dies after completing its run.

    create Spring Cloud Stream sink application and run it (do we need to install it in Maven as well?)

    Not sure why you need a sink application. You would need a source application here that triggers the TaskLaunchRequest to the sink application. Your source application needs to be configured to receive the messages out of your JMS broker that cause the triggers.

    create Spring Cloud Task sink application and install it in Maven Repo

    This is the Spring Cloud stream sink application that launches the task you create at 5. This is one such example. This application is a long-lived one that looks for incoming TaskLaunchRequests

    do I need a Spring Cloud Task as well created?

    Yes, this is the actual task application that is the short-lived one that you want to run on demand.