Search code examples
spring-batchspring-cloudspring-cloud-streamspring-cloud-dataflowspring-cloud-task

Spring Cloud components confusion


How do these Spring components relate/differ to/from each other? What does each represent conceptually? Would one use them together or are they competing projects?

  • Spring Cloud Data Flow
  • Spring Cloud Stream
  • Spring Cloud Task
  • Spring Cloud Task App Starters
  • Spring Batch

From my understanding, SC Tasks are just "units of work" to execute, a processing unit in the form of a short-lived/task-based microservice. SC Data Flow is orchestration for the tasks. These two I (think I) understand how they relate and what they represent conceptually, but a lot of documentation and examples talk about the other projects in the same context.

I also thought that SC Task was a replacement for Spring Batch but in some examples they seem to imply that Spring Batches are executed inside SC Tasks


Solution

  • Thanks for your interest in Spring Cloud projects! Find below the high-level introductions for the primary projects involved in Spring Cloud Data Flow (SCDF) ecosystem. The launch blog covers the backstory and among other details.

    • Spring Cloud Stream is a lightweight event-driven microservices framework to quickly build applications that can connect to external systems (eg: Kafka, Cassandra, MySQL, Hadoop, ..).

    • Spring Cloud Task is a short-lived microservices framework to quickly build applications that perform finite amounts of data processing (eg: batch-jobs, ..). The connection with Spring Batch framework is explained in the launch blog linked above.

    • Spring Cloud Data Flow provides the orchestration mechanics to deploy applications built with Spring Cloud Stream and Spring Cloud Task programming model to a variety of runtime platforms including Cloud Foundry, Apache Yarn, Apache Mesos and Kubernetes. There's community developed SCDF implementations for OpenShift and Nomad, too. More details here.

    The building blocks visual from the project site should cover the high-level interaction between the various projects in SCDF's ecosystem.