Search code examples
talend

Talend: Triggering subjobs in an iterate

I have a Talend job that I am trying to refactor to run iteratively. The original job would put a value into the global map in a prejob->tJava, and then a fileInput uses that value to open a file (tFileInputDelimited) and execute rowMain within the subjob. The fileInput also has a OnSubjobOk that triggers another fileInput, which has a series of rowMain processes.

I am now trying to refactor the job so that the tJava prejob sets up a list and adds it to the globalMap. I am then using an iterator with iterate to tJavaFlex, which then has an iterate to the fileInput triggering the first subjob. However, I can't trigger the 2nd subjob from the first subjob, and wondering how I could iterate a whole series of subjobs from 1 iterator.

How could I design the job as such?

Here's a sample of the original job. It took a context parameter to specify a file name which was then used by "read demo". I want to iterate through all subjobs for each file. The job needs to finish the first subjob, and then complete 2nd, 3rd and 4th subjobs before doing the next iteration with the next filename. The prejob just set a default on the file name if it wasn't supplied, and isn't necessary.

I've tried a way to use a tIterate through into "read demo" component, but not able to get it working as needed.

original job

Solution

  • Instead of OnSubJobOK simply use the tFlowToIterate -> Iterate link to trigger the read price CSV.

    Keep in mind that the Read price CSV filename can't be changed easily in the job. When you add an iterate link, the whole readCSV will be part of the subjob, and it is a pain to change it dynamically. Still you have a few options:

    • Move the read CSV part to a separate subjob that accepts the filename.
    • Use a fileInputStream to read the files, this fileInputStream can be changed dynamically.
    • Iterate on a Tjava and use an On*OK component to trigger your current read CSV.