I am new to flink (java)and trying to move xml files on a netapp file server mounted as a file path onto server that flink is installed.
How to do batch or stream processing in real time to fetch files coming to the folder and sink it with s3.
I couldn't find any examples in flink-starter to read files from local file system, is flink atleast a right choice to this use case? If so where can I find resources to listen to folder and manage checkpoints/ save points?
Full working code for this question is in the following link . You need to enable checkpointing to move the .inprogress files to actual files
// start a checkpoint every 1000 ms env.enableCheckpointing(1000);