Search code examples
solace

Monitor Solace Queues with Grafana


I have a couple of microservices that communicate between them using Solace queues. My task is to find a solution to inspect the messages that are flowing within thouse queues and based or the content to push some data data in a InfluxDB that will funder be interpreted by Grafana.

So long story short I need to know if there is any technical solution to get a copy of each message (intercept the message) that is passed around in a queue without actually consuming it.

Also there is a requirement not to modify the code of any of the existing services so my plan is to create an additional that connects to this queues based on their name some how sniff form messages.

I what to know if this is technically possible.


Solution

  • There's a couple different possible solutions to your situation, which may or may not be appropriate depending on your exact requirements.

    If the InfluxDB listener component can operate in parallel to your microservices, then the easiest way to make this work would be for the publishing microservice(s) to publish on a topic, have the consuming microservice connected to a queue that is subscribed to the same topic, and the InfluxDB listener component would have its own queue subscribed to the same topic(s)... which allows it to receive a copy of every message sent by the publishing microservice (standard pub/sub pattern). This is a typical best-practice for Solace is to publish on Topics, consume from Queues, and have the topic-to-queue mapping configured/controlled within the infrastructure... it allows you to change the routing of messages between your apps (e.g. adding a new listener) without affecting your existing code/flows.

    If you can't change your publisher code to publish to a Topic, then you could still achieve the same functionality by using a 2nd queue for the InfluxDB consumer, and subscribing it to the first queue's "network topic", which looks something like #P2P/QUE/queue-name ... you can see this special topic in the queue's details.

    Alternatively, if you need to have the InfluxDB listener component act inline to your microservice apps (instead of parallel via a copy of the messages), then you'd need to use one queue to for your first microservice to publish into, and your InfluxDB listener to consume from... then it does its content routing / decision making, and then publishes the message back into the 2nd queue where the 2nd microservice can consume from. This essentially makes the InfluxDB listener just another microservice that is in the processing chain between the microservices.

    Note that for the first two options (having a copy of the message land on two queues), Solace only keeps one copy of the message on disk and maintains a reference count. So disk usage is not impacted by "fan-out".