Search code examples
mqtt

Can a MQTT QoS 2 message on persistent session be delivered twice

I was reading about MQTT message order, on two SO questions and answers here and here. So basically we used to have a belief that QoS 2 messages are delivered in order, and only once. Those two SO discussions corrected the belief to that in some cases QoS 2 messages can be delivered out-of-order, namely if they are on different topics, or when the first delivery is not acknowledged properly.

To think further, here is the question: Would a QoS 2 message be delivered more than once?

At least, think about a persistent session case: If the PUBREL is received by the subscriber, then the subscriber will deliver the message and send a PUBCOMP to the publisher. The subscriber keeps PUBREL and other information in memory so if the PUBCOMP is lost the publisher will send PUBREL again, or resend the message from beginning, the subscriber will know it is a duplicate of the same message and will not deliver the same message twice.

If the PUBREL and other information is lost due to a lost connection, if the subscriber still has the message record in memory, the same message will not be delivered twice after a reconnect.

But if the subscriber has lost the record, then it won't be able to detect a duplicate. In the latter case, the same message will be delivered again. Is this is the case or is it my misunderstanding? Is there other cases in which a QoS 2 message will be delivered more than once? What does the spec say about this? Are individual implementations different with this respect?

enter image description here

Solution

  • TLDR: QOS 2 = "Exactly once delivery". However MQTT requires that the session state be retained, if this is lost then the guarantees don't apply.

    I'll answer based on the MQTT V5 spec as you did not specify a version. V3 and V5 are similar, but V5 is a much later spec and benefits from experience gained from V3 deployments.

    Would a QoS 2 message be delivered more than once?

    The MQTT spec states that QOS2 = "Exactly once delivery", so the answer to your question is "No". However, this assumes compliance with the specification, especially:

    The Client and Server MUST store Session state for the entire duration of the Session [MQTT-4.1.0-1].

    Note that this is a "MUST":

    Must: This word, or the terms "REQUIRED" or "SHALL", mean that the definition is an absolute requirement of the specification.

    So loss of the session state violates this condition, meaning that any guarantees made in the spec would be invalidated in the event such loss occurs. The spec authors realised this requirement may be breached and provide non-normative comment/examples that provide some guidance (basically you need to assess the impact of session state loss on your use case, and take appropriate measures to protect the state).

    So onto your questions:

    if the PUBCOMP is lost the publisher will send PUBREL again, or resend the message from beginning.

    The spec the sender:

    MUST NOT re-send the PUBLISH once it has sent the corresponding PUBREL packet [MQTT-4.3.3-6].

    So the situation you mention above "or resend the message from beginning" would be a violation of the spec (of course the entire message could be resent as a new message, but QOS guarantees would not apply in that situation).

    But if the subscriber has lost the record, then it won't be able to detect a duplicate

    As mentioned in my introduction this would be a breach of the spec, so the QOS guarantees no longer apply (and the message may be lost/duplicated). Should the application choose to resend the message, then the message content could be received twice (but that is outside of the scope of MQTT).

    One further thing to note is that the MQTT spec covers the protocol. Normally this protocol will be implemented by a library and designing the library interface can be tricky (you want it to be simple/easy to use, but this may make some edge cases difficult to handle). If you are really concerned about message delivery then it's important you check the library source (and store the session state to disk if appropriate).