Search code examples
mqtt

MQTT - single message with an array of records vs message per record


I have a small LED display that should be scheduled to display various messages through out the day based of CRON. It keeps schedules internally for the case when the connection to the server is lost. It creates/updates/removes schedules based of received MQTT messages.

I can see two approaches and I am wondering which one of them would be considered best practice.

  1. Using single schedule message holding all schedules:
    Topic: led_display/12EB0770/schedule
    Payload:
    {
        [
            {"id":1, "cron":"0 0 * * *", "AM"},
            {"id":1, "cron":"0 12 * * *", "PM"}
        ]
    }

Downside of this solution in my eyes is that I have to publish larger payload any time I make a change. This could be solved by 2nd approach.

  1. Using message per schedule with schedule ID as part of the topic:
    Topic: led_display/12EB0770/schedule/1
    Payload:
    {"cron":"0 0 * * *", "AM"}

    Topic: led_display/12EB0770/schedule/2
    Payload:
    {"cron":"0 12 * * *", "PM"}

Downside of this solution is the fact that after subscription to led_display/12EB0770/schedule/+ I will never be sure if I have already received all the schedules or not. For that I might need to add another topic such as led_display/12EB0770/schedule containing this information.


Solution

  • This is a great question. I had to solve the same when implementing a data-synchronization protocol for robotics over mqtt (https://github.com/transitiverobotics/transitive-utils). In the end this depends a lot on the need for atomicity of your data. If it is critical that no client ever receives a partial update, then your first option (called atomic in our protocol) is of course the right choice.

    However, the second option can be made to work. It comes down to the question of whether everything has been received. There is no native support in MQTT for that, however the approach described in https://stackoverflow.com/a/49636546/1087119 works in practice. The way we implemented this is to subscribe to the brokers heartbeat topic ($SYS/broker/uptime) and use this logic:

    • subscribe to topic in question (in your case the partial update topic)
    • wait for next heartbeat
    • consider all partial updates received