Search code examples
pentaho

run transformation kettle for each row xml data dynamically


How dynamically can I get each element from get data from xml step separately to be an input to other transformation which do the parsing of message(value node xml), my main idea is how to run transformation kettle for each row xml data dynamically) .

*dynamically means that the number of elements is unknown.

question in forum pentaho community: http://forums.pentaho.com/showthread.php?204226-run-transformation-kettle-for-each-row-xml-data-dynamically&highlight=How+dynamically+can+I+get+each+element+from+get+data+from+xml+step+separately+to+be+an+input+to+other+transformation+which+do+the+parsing+of+message%28value+node+xml%29%2C+my+main+idea+is+how+to+run+transformation+kettle+for+each+row+xml+data+dynamically%29+.++%2Adynamically+means+that+the+number+of+elements+is+unknown.


Solution

  • It's a bit dated, but it sounds like this is what you're looking for:

    Run Kettle Job for each Row

    Essentially, you get the data from your XML file with a transform (Get data from XML) and flow it into a Copy rows to result step. Then in your job, add a Transformation step and in its options, on the advanced tab, check the "Copy previous results to parameters" and "Execute for every input row" check boxes.

    You will have to setup parameters for the Transformation step to match the metadata of your XML data row.

    Note that this will be pretty slow if you have a large number of message IDs and relatively little child data for each message. If that's the case, you might want to try a lookup from the XML data in the first Transformation instead.