Search code examples
fileapache-camelidempotent

Camel File component - whats the difference between a IdempotentRepository and a InProgressRepository

What's exactly the difference between the IdempotentRepository and the InProgressRepository? I have following definitions from the File component page:

  • IdempotentRepository: "Option to use the Idempotent Consumer EIP pattern to let Camel skip already processed files"
  • InProgressRepository: "The in-progress repository is used to account the current in progress files being consumed."

For me these are the same definitions, only slightly differently phrased. They can also both use the same idempotent repository.

So I'm slightly confused, do I need both? Or is the idempotentRepository good enough?

Solution

  • Before you read below info, make sure you have read and understand the concept of idempotent.

    IdempotentRepository - The place used to store the cache of already processed file (i.e. files have been consumed and handled by your route). In use when you check on idempotent feature.

    InProgressRepository - The place used to store the cache of current in progress file (i.e. files to be consumed in current batch). Always in use for file consumer.

    IMO, one always need InProgressRepository and with default setup (memory based repository) in general. One might need IdempotentRepository if idempotent is required and choose their own setup (file-based, JPA-based, ...) to against app restart.