Search code examples
djangoetlkettle

Django - How to conect Pentaho Kettle ETL with wep app

I have a Django application where users can upload big dataset in PostgreSQL db. I would like to use an ETL tool like Pentaho Kettle to do the upload job. Later, I would like to use Kettle for data mining and transformations.

However, I have no idea how I to connect my Django app to Kettle. I am not sure what version of Kettle is needed, do I need spoon, or carte?

How can I trigger the job from django python and how to pass the request.FILES of my app to Kettle?

Can you explain the general steps or point to documentation or examples?

Solution

  • 1) make your data transformation on kettle (spoon.sh).

    2) do a shellscript launching kitchen or pan and this transformation.

    3) trigger this shellscript from your django app after the data is uploaded to the server.