Search code examples
web-servicesetlkettle

Kettle hangs on post data using web service step


I am using Kettle to bulk load data and i am facing issue in dealing with web service step.As per the inspection after few thousands of call web service becomes unresponsive only time counter is increased but no progress is done.I can notice that all previous steps are finished and the transformation is stuck at web service step.Looking for the solution or workaround to overcome from this problem.


Solution

  • I have found the solution to this problem.The Web service step was becoming a bottle neck step and therefor requires some tweaks to be done to transformation. I have followed the solution found on the following link :

    http://type-exit.org/adventures-with-open-source-bi/2010/06/parallel-processing-in-pentaho-kettle-jobs/

    and now job is done.All just need to set No of rows in rowset to deal with bottle neck steps to maintain the buffer. Refer the screenshots and multiple threads on bottleneck step to overcome from it.

    Check the attached screenshots. Adjust no rows in rowset Job is done with better performance