We need a lightweight client based messaging solution. We used AMQP, RabbitMQ before, but in C++ we have problems.
We would like to choose ZeroMQ with malamuteserver or MQTT ? Our IoT will publish data (45 kb) almost every 5 min.
We need to deliver this message 100% and do not want to lose any message.
We tried MQTT QoS level 2, but when server disconnected or main server client has a problem we are losing published messages.
We need exactly RabbitMQ task / worker model. Messages should queued in the server until consumers connected if anything happens.
Any suggestion, direction and examples welcome.
P.S.: This will be production so we want to chose less problematic way :)
I think MQTT is overhyped , sure the result I think is due to available open source servers. Zeromq does offer a lot of features to build something that will meet the requirements. The more I look at the options available , the more i am leaning towards zeromq. In our case, we will be receiving data at random intervals from a very large number of devices ( gateway's in a mesh network or End nodes themsevles ). Our finalized message size is very simple , a delimited string , binary encoded , zipped ( <100 Bytes) and sent over the wire. I am decided on zeromq on the server to receive messages. Reasons are not merely based on zeromq as a broker but also how We can utilize its curveZmq features to make provisioning of devices easy and allowing for a simple ZTP (Zero Touch Provisioning system) and key manageability . I am at the preset time debating on using pub/sub vs push /pull patterns where in each end device is a publisher or pusher with a proxy subscriber on the cloud server. while typically pub in a pub/sub there are fewer publishers and more subscribers in a typical large scale IOT deployment there are more publishers and fewer subscribers so it makes me wonder if i should go with pub/sub and there there is the problem of loosing messages due to slow joining subscriber - what if we bring down the server for maintenance , devices in the field would keep publishing messages until HWM is reached. Guess there is always a risk of loosing messages irrespective of anything ex - backhaul network is down and the device Hits the HWM - this is out of control.
Malamute does not have much documentation else i would have explored it a bit more.
So , have you decided on what to use ? If you want to persist messages until they are consumed , i strongly suggest zeromq as a proxy with workers pushing messages into a persistent store .. you can get creative here as well with including a sequence # etc..and allowing clients to request messages given a sequence range etc if they are lost .