Search code examples
time-seriesgrafanafiwarecrate

Ask for FIWARE project recommendations: 3D plot monitoring of entity attrs

The goal of the project is to plot the x,y,z coordinates (attrs from an entity) in a 3D graph which updates as they change.

Note: it's not important how the value of x,y,z changes, it can be for example by hand through the prompt, using curl.

At first, I thought about using QuantumLeap, CrateDB and Grafana, but when I have deployed it I have realised that Grafana doesn't support the crate plugin anymore (https://community.grafana.com/t/plugin-cratedb-not-available/17165), and I got errors (I have tried it using PostgreSQL as it is explained here: https://crate.io/a/pair-cratedb-with-grafana-6-x/)

At this point, I would like to ask for some recommendations: Do you think I need to work with time-series data? If not, how should I address the problem? If yes, can I use another database manager with QuantumLeap and supported by Grafana that works with this time-series format? Or maybe do not use Grafana and accessing the time-series data from the Crate database manually via any frontend software which shows the 3D graph?

Solution

  • This is all a matter of question framing. Because the data format is well defined you can indirectly use any tool with with any NGSI Context Broker.

    The problem can be broken down into the following steps:

    • What Graphing/Business Intelligence tools are available?
    • What databases do they support?
    • Which FIWARE Components can push data into a supported Database?

    Now the simplest answer (given the user's needs) and as proposed in the question is to use Grafana - the PostGres plugin for Grafana will read from a CrateDB database and the QuantumLeap component can persist time-series data into CrateDB which is compatible with the PostGres format. An example on how to do this can be found in the QuantumLeap documentation

    However you could use a component such as Draco or Cygnus to persist your data to a database (Draco is easier here since you could write a custom NIFI step to push in your preferred format.

    Alternatively you could use the Cosmos Spark or Flink connectors to listen to an incoming stream of context data and persist something to a database

    Or you could write a custom microservice which listens to the NGSI notification endpoint (which is raised by a subscription) interpret the payload and push to the database of your choice.

    Once you have the data in a database there as well as Grafana there are plenty of other tools available - consider using the Knowage Engine or Apache Superset for example.