Search code examples
pentahopentaho-spoonpdi

pentaho spoon/pid: how to move files to folders with different name everytime?


I have new text files every month from where I extract the data and do some transformations. In the end of every month, I need to move these files to a folder with current date in name. Which means, the destination folder's name is different everytime. I made a step before move files that creates a folder and its name is current date (exp: 2019-06-01, 2019-07-01), but then on move files step, I don't know how to specify the destination folder. Guess "wildcard" is only used for source...

Otherwise, on move files - Destination file - Move to folder, there is create folder option, thought it could be a solution but as on the screenshot, this section can't be modified. What is the reason?

Does anyone have an idea ? Thanks for reading :))


Solution

  • You can utilize this approach for this task.

    Basically you'll have to extract the date you want from each file before doing the transformations, and after all the transformations are done, you'll use the JOB step 'Move files', but instead of passing a fixed destination, you'll use variables in the path, much like this answer, but you don't need to create the folder before, you can just use the option on the 'Move files' step, so the folders are created at run time.

    I have attached an example of this approach with some commentary in the KTR/KJB. You can download here

    Let me know if the link is working, this is my first time posting an external file in StackOverflow