Search code examples
pentahokettlespoon

Loop over file names in sub job (Kettle job)


The task is to get file names from the folder and then loop the same task (job) over all the files one by one.

I created a simple job with transformation (get files names) and then job with flag "Execute for each row" (now is just logging the name of the file).

Did it the same way it is described here: http://ramathoughts.blogspot.ch/2010/08/processing-group-of-files-with-kettle.html

However, the path of the received files is not passed to the sub-job (logging doesn't display variable value). But the sub-job is executed as many times as there is number of files in the input folder. So it looks like it is passed to some extent, but for some reason is not available as a variable.

Image with log details, as seen the variable is displayed as ${path} instead of value of the path: https://i.sstatic.net/lfEVX.png

The sample code is below as archive with jobs and transformation and also sample input files. Any help is appreciated, as I may be missing something simple here https://www.hightail.com/download/bXBhL0dNcklCMTVsQXNUQw


Solution

  • The issue is the 2nd Job (i.e. j_log_file_names.kjb) is unable to detect the parameter path. Just try defining the parameter to this Job; like the image below:

    enter image description here

    This will make sure that the parameter that is coming from the prev. step is correctly fetched into the Job. Rest of your job looks absolutely fine.

    Hope this helps :)