Search code examples
groovyapache-camelesbmiddleware

Polling a folder and deleting files that do not differ from last update


I have a process that is generating xml updates in a folder regardless whether any change has occurred. I want to move those files if they differ from the last update to a processed folder, otherwise delete them. Does it make sense to use a FileIdempotentRepository with cacheSize of 1 to filter duplicate files out, or am I on the wrong track? How do I tell the route below to use the file contents as the idempotentKey?

<bean id="fileStore" class="org.apache.camel.processor.idempotent.FileIdempotentRepository">
    <property name="fileStore" value="target/fileidempotent/.filestore.dat"/>
    <property name="cacheSize" value="1" />
</bean>

<camelContext xmlns="http://camel.apache.org/schema/spring">
    <route>
        <from uri="file://inputFolder" />
        <idempotentConsumer messageIdRepositoryRef="fileStore">
            <!-- I DO NOT want to use the messageId, how do I use the entire file contents? -->
            <header>messageId</header>
            <to uri="file://outputFolder"/>
        </idempotentConsumer>
    </route>
</camelContext>

To avoid confusion, here's the pseudo code for what I'm trying to do:

current = ''
watch(inputFolder).onNewFile(f):
  if (f.contents == current)
    delete f
  else
    mv f to outputFolder
    current = f.contents

Update (really want someone to get this bounty!)

If you know how to use inline Groovy/Simple script within this context file that will work for me as well.


Solution

  • The expression for the idempotent consumer can be anything that Camel supports such as any of these languages: http://camel.apache.org/languages

    So what you want is the message body as the correlation expression, and for that you can use

    <simple>${body}</simple>
    

    But as the file component routes the file as a java.io.File handle, you would need to load the content into memory for comparision, so you can do

      <from uri="file://inputFolder" />
      <convertBodyTo type="String" />
      <idempotentConsumer messageIdRepositoryRef="fileStore">
         <simple>${body}</simple>
         ...