Search code examples
jsonpaginationtalend

Loop over tfilefetch while passing different paging token in Talend


Well, this scenario may be quite familiar but I couldn't find the solution. Scenario: I am making a REST API call through tFileFetch and I get a json out of it. I parse it to get paging token and more result through tflowtoiterate. Now if more result is equals true, I have to call the same tFileFetch component to get the new set of json using new pagination result.

I have to loop through tFileFetch until the 'more result' is false.

My approach:

Access token-pagination-tFileFetch_1->JSON->tflowtoIterate->more result=true->IF [moreresult=true]->tFileFetch_2->JSON->tFlowtoIterate->more result=true->tLOOP [moreresult.equals{true)->tFileFetch_2->

After tFileFetch2 I have used tSetGlobalVar to put pagination as common var to pass to tFileFetch2

I am not sure whether this approach is appreciable or not, please suggest any improvements if any?


Solution

  • I have actually covered it by myself. 'code' Add SubJob Ok from tLoop to tFileFetch and add condition in tLoop which say until your condition matches 'more result' is false. Finally added delimited output with append mode