Search code examples
etltalend

Talend loop for each record


Hi i am designing a data generation job. my job is something like this

tRowGenerate --> tMap --> tFileOutputDelimited.

Lets say my tRowGenerate produces 5 columns with 2 records. I want to iterate for this records i.e for each record I want to iterate certain number of times.

for record 1 iterate 5 times to produce further data.

for record 2 iterate 3 times to produce further data.

enter image description here

Please suggest how to apply this multiply by xi logic. where xi for each record can change.

Thanks!


Solution

  • If you want to loop on the data generated from the tRowGenerator you can use a tLoop where you put the call to your business rule to determine the number of loops or when stop looping.

    An example job might look like:

    Example job layout and tLoop configuration

    Logic of flow:

    1. row1 is a main connection taking the generated values to the tFlowtoIterate that stores them in global variables;
    2. the iterate link activates the tLoop that can use the values stored in the global vars to activate your business rule (to have the number of loops or tho ask if continue or stop);
    3. the tLoop activate the tJavaFlex that uses the stored global vars to produce the output you like and pass it to the tFileOutputDelimited with a main link (row2).

    You have to activate the append flag on the tFileOutputDelimited to keep the data from the different loops. If you need you can add a tFileDelete at the beginning to empty the output file before a new processing round.