Search code examples
javapentahokettlepdi

Process multiple inputs to a Kettle step

I am trying to create a step where I have two inputs. Both of them should be in the same format. I need to process using these two input streams and output one stream.

Below is the code I have so far in processRow():

List<StreamInterface> infoStreams = meta.getStepIOMeta().getInfoStreams();

data.oneRowSet = findInputRowSet(infoStreams.get(0).getStepname());
data.twoRowSet = findInputRowSet(infoStreams.get(1).getStepname());

if its the first row and the following for all rows:

data.one = getRowFrom(data.oneRowSet);
data.two = getRowFrom(data.twoRowSet);

if (data.one == null && data.two == null) {
      setOutputDone();
      return false;
}

However on the line data.oneRowSet = findInputRowSet(infoStreams.get(0).getStepname());, I get the following error: The source step to read from [null] couldn't be found.

Your inputs on this would be really helpful. Thanks.

EDIT: I have added an example transformation and the source code of my step here: https://github.com/adeydas/KettleMultipleStreams/tree/develop

Solution

  • I seem to have found a solution. I wrote a small "how-to" on it here: http://abhis.ws/display/KKB/Kettle+transformation+step+with+multiple+input+streams and the code is on GitHub: https://github.com/adeydas/KettleMultipleStreams.