Search code examples
javaapache-camelcluster-computingidempotent

Delete processed file apache-camel in a cluster


I use apache camel to process files received on a ftp channel. My application is deployed in a cluster (4 nodes), for this I use RedisIdempotentRepository to ensure that a single node processes the file. My problem is that I want to delete the file after processing, if I use delete=true, the node A that processed the file when it finishes and will delete the file, node B already deleted it because the node B will not go through the filter and therefore it will directly access the deletion.

I would like to know how to only allow node A to delete the file?

from("sftp://host:port/folder?delete=true)
 .idempotentConsumer(simple("${file:onlyname}"),
     RedisIdempotentRepository.redisIdempotentRepository(redisTemplate, "camel-repo"))
 .bean("orderTrackingFileProcessor");

Solution

  • I workaround this using pollEnrich:

    adding delete step at end of processing:

    .pollEnrich(remoteLocation + "?delete=true&fileName=${file:name}");
    

    Full Example Route:

    String remoteLocation = "sftp://host:port/folder";
    
    from(remoteLocation)
     .idempotentConsumer(simple("${file:onlyname}"),
         RedisIdempotentRepository.redisIdempotentRepository(redisTemplate, "camel-repo"))
     .bean("orderTrackingFileProcessor")
     .pollEnrich(remoteLocation + "?delete=true&fileName=${file:name}");