Search code examples
javaapache-sparkarchitecturestreamingrestful-architecture

How can i create a RESTFul application which uses Apache Spark, specially Streaming at the back end

I am quiet new to Apache Spark. I want to have a RESTFul service which receives a file (Large file) with Apache Spark to process this file in near real-time at the back-end. I want to implement this in Java.

  1. What are the suggested light weight framworks?

  2. how can i pack this application (distribution). Say i have a server (e.g. Tomcat). Should i pack Spark inside the Webservice ?

  3. How can i run the Spark cluster programmatically? Like a service which is always up and down on demand.

  4. Is there any samples which follow this kind of architecture or implementation?

Thanks

Solution

  • Use Akka HTTP to implement REST services. Complete working example of how to integrate Akka and Spark is killrweather.