Do I need spring batch if I have to process API services instead of CSV jobs?

Question

I'm writing an app where its primary objective is to pull data from a service provider(XXX), convert and push it to another service provider(YYY) via HTTP REST.

I can do this with a simple for-loop and I'm storing the last retrieved item creation date in a separate table. So I can continue pull items from last converted item whenever I needed and I'll be maintaining separate table to hold failed records and re-try those failed records with periodic intervals via CRON as well.

Do I still need to use SPRING Batch. From what I can see is, it helps modular the code with a Reader, Processor and Writer.

Am I missing anything? Can I use spring batch and benefit for syncing data over HTTP services from point A - point B?

Answer 1

You can still use Spring Batch for many reasons. Here is a non exhaustive list with an example related to your use case for each feature:

• Declarative I/O: Spring Batch provides a rich library of readers/writers and you don't have to write the I/O code yourself (open/close resources, read raw data, iterate on items, etc). Spring Batch is based on Spring Framework, you can leverage all Spring Framework features like AOP, transaction management, etc. In your case, you can leverage the RestTemplate to GET/POST resources
• Fault tolerance: The retry/skip mechanisms allow you to introduce some resilience in your batch jobs. For example, in your case, retry processing an item if the rest service is temporarily down.
• Chunk-oriented processing: This model is perfect for huge amounts of data. Data is read in chunks to avoid reading all the data set in memory. In your case, you can have a call per page to get data from your rest end point
• Transaction management: Spring Batch manages transactions for you so you don't lose data when errors happen. In your case, you mentioned saving items to a database table, Spring Batch can manage the transaction commit/rollback for you