Apache Camel FTP goes into endless loop when deleting files which are less than the specified amount

Question

I have setup following Apache Camel Route which reads XML files from an FTP server, transforms the content into Java Objects, aggregates the created Java Objects in a List, and then dumps the List into a Direct object to be read by a ConsumerTemplate. The amount of to be processed files is 10.

This is my Route:

JAXBContext jaxbContext = JAXBContext.newInstance(new Class[]{Dokumente.class});
JaxbDataFormat jaxbDataFormat = new JaxbDataFormat();
jaxbDataFormat.setContext(jaxbContext);

from("ftp://tstusr@localhost/uploads/Replies?delete=true&password=ftpServPW&passiveMode=true&binary=true&disconnect=true")
        .unmarshal(jaxbDataFormat)
        .aggregate(AggregationStrategies.flexible(Dokumente.class).accumulateInCollection(LinkedList.class))
        .constant(true)
        .completionSize(10)
        .to("direct:files");

The Route is doing what it is supposed to be doing, except the "delete=true" part. When I have less than 10 files in my directory, the program goes into an infinite loop. In my local testserver for FTP (FileZilla Server) I can see that it logs in, changes the directory, executes the LIST command and then quits. It does this repeatedly in an infinite loop.

How can I fix the Route to stop it from doing that? I do not know how many files there are in the directory. Which means there can always be less than the batch size. I am using version 3.11.4 of Apache Camel.

Answer 1

The problem is that the route keeps waiting for the missing files to be present, if there are less files available than the specified file amount.

So if it is told to download 10 files, but only 2 are present it will wait for the 8 missing files.

I had to terminate the route manually by first giving it an ID. Then I had to set the route option sendEmptyMessageWhenIdle=true and attach a processor which stops the route when an empty message is received. Since the empty message does not do anything I filter it out afterwards and then force com The adjusted route looks like this

        from("ftp://dm@localhost/uploads/Replies?delete=true&password=ftpPass&passiveMode=true&binary=true&disconnect=true&sendEmptyMessageWhenIdle=true") 
            .routeId("ftpRoute")
            .process(new Processor() {

                @Override
                public void process(Exchange exchange) throws Exception {
                    if (exchange.getMessage().getBody() == null) {
                        exchange.getContext().getRouteController().stopRoute("ftpRoute");
                    }
                }
            })
            .filter(simple("${body} != null"))
            .unmarshal(jaxbDataFormat)
            .aggregate(AggregationStrategies.flexible(Dokumente.class).accumulateInCollection(LinkedList.class)) 
            .constant(true) 
            .completionSize(10) 
            .forceCompletionOnStop()
            .to("direct:files");