Search code examples
javaspring-bootfileapache-camel

camel pollEnrich is not working for the second time

I am reading and processing 2 files from 2 different file locations and comparing the content. If 2nd file is not available , the rest of the process execute with 1st file. If 2nd file is available, comparison process should happen. For this I am using camel pollEnrich, but here the problem is that, camel is picking the 2nd file at first time only. Without restarting the camel route 2nd file is not getting picked up even if it is present there. After restarting the camel route it is working fine, but after that its not picking the 2nd file.

I am moving the files to different locations after processing it.

Below is my piece of code,

from("sftp:" + firstFileLocation + "?privateKeyFile=" + ppkFileLocation + "&username=" + sftpUsername
            + "&readLock=changed&idempotent=true&move=" + firstFileArchiveLocation)
    .pollEnrich("sftp:" + secondFileLocation + "?privateKeyFile=" + ppkFileLocation + "&username=" + sftpUsername
            + "&readLock=changed&idempotent=true&fileExist=Ignore&move="+ secondFileLocationArchive ,10000,new FileAggregationStrategy())
    .routeId("READ_INPUT_FILE_ROUTE")

Need help.

Solution

  • You're setting idempotent=true in the sftp consumer, which means camel will not process the same file name twice. Since you're moving the files, it would make sense to set idempotent=false.

    Quoted from camel documentation

    Option to use the Idempotent Consumer EIP pattern to let Camel skip already processed files. Will by default use a memory based LRUCache that holds 1000 entries. If noop=true then idempotent will be enabled as well to avoid consuming the same files over and over again.