Search code examples
mqttpentahopentaho-data-integrationpentaho-cde

Receive and Display MQTT Stream at Pentaho Dashboard

I'm trying to disdplay (and process) incoming MQTT-messages in a Pentaho dashboard, using their CCC-Charts.

I tried setting up a kettle transformation, which I then uploaded to my Pentaho server an use it as a datasource.(I tested running the transformation within spoon and it worked, so the issue shold be somewhere else). When I try to display the dashboard, it tells me 'Error processing component(component name)' while the server throws this exception:

Caused by: org.pentaho.reporting.libraries.resourceloader.ResourceCreationException: Unable to parse the document: ResourceKey{schema=pt.webdetails.cda.settings.CdaRepositoryResourceLoader:, identifier=/home/admin/MqttReceive2.ktr, factoryParameters={}, parent=null}
...

What am I doing wrong? Is there a proper way to do this?

cheers

Edit: I found solutions working arround this problem using a websocket implementation, but this isn´t quite what I need.

Solution

  • The proper way to solve this, is to make use of a data service.

    Here is what I came up with:

    You can use Kettle to receive MQTT-Data and process it within your transformation: kettle transformation

    You can attach a data service to any step, by right clicking->Data Services->New The service name needs to be unique among all data services and the data service type needs to be set to streaming:

    dataServiceSetup

    The whole transformation needs to be saved to a Pentaho repository(basically the internal filesystem of Pentaho)

    With this you can create a dashboard by following these instructions:

    https://help.pentaho.com/Documentation/8.2/Products/CTools/Create_Streaming_Service_Dashboard

    To break it down for you:

    1. In the datasource Panel, you need to select 'streaming over data service' and enter a streaming data service name, as well as the query.

    2. The parameters need to be set to:

      • Window Mode: Row Based

      • Window Size: 50 (If you want to work with 50 datapoints in your dashboard)

      • Window Every: 1 (This determines the frequency, in which your graph will be updated)

      • Window Limit: 0

    3. In the component panel, you need to select the component of your choice and use the datasource you just created as 'Datasource'. The parameters need to be set as follows:

      • Don't Block the UI While Executing: True

      • Clears Before Pre Execution: False

    4. Create a fronted layout and bind the component to a layout HTML object.