Search code examples
talend

TfileList catches one of the 6 files only


I tried to display some results from several files in a directory. I use TFileList, and 2 tFileInputDelimited which are both linked to TFileList. I don't know why but at the end of the processing my results are lugged from just one of the 6 files I want. It appears that there are results from the list file of the directory.

schemaTalend

Each tFileInputDelimited has ((String)globalMap.get("tFileList_1_CURRENT_FILEPATH")) as name of the flow.

Here is my TMap: enter image description here


Solution

  • Your job is set up so your lookup is iterative which causes some issues as Talend only seems to use the last iteration rather than doing what you might expect and iterating through every step for everything it needs (although this might be more complicated than you first think).

    One option is to rework the job so you use your iterate part of the job as the main input to the tMap rather than the lookup.

    Switch the lookup parts of the job

    Alternatively, you could iterate the data into a tBufferOutput component and then OnSubjobOk you could link the job as before but replace the iterative part with a tBufferInput component as it will store all of the data from all of the files iterated through.

    Buffer the iterated data and then read it back in as the lookup