After much reading of the Microsoft Azure IoT Hub documentation and playing with the samples, I am still in the dark over whether the technology is suitable for devices that connect over intermittent/unreliable and expensive networks (e.g. GSM), and where minimising cost is much more important than minimising latency.
In particular, I notice that in all of the samples, no attention is given to the protocol overhead on messages. Telemetry data is always sent as small and simple messages, e.g.
{
"time": "2016-01-26T20:47:53.0000000",
"dspl": "sensorE",
"temp": 123,
"hmdt": 34
}
presumably with the assumption that real-time delivery is such a high priority that cost is not really a consideration. I also notice that the topic/endpoint names used by the IoT Hub are quite verbose, which must add to the overhead.
The C SDK documentation makes passing mention of "Batches messages to improve communication efficiency", but there are no further details, and it's not clear if this applies only to HTTP, or also to AMQP. There is also no mention of how the library decides which messages to batch together.
There is also a mention of a "SetBatching" option to IoTHubClient_LL_SetOption (off by default), but it doesn't say if this applies to HTTP only or also AMQP. And when I looked at the source code this option didn't seem to exist, so that linked document might be out of date.
Update: "more about IoTHubClient" also mentions SetBatching, but it still isn't clear if this is HTTP-only. (Perhaps batching wouldn't confer any advantage with AMQP - I want to understand this better and this is very much the heart of my question.)
I would like to know, specifically with respect to the Azure IoT C SDK:
What is the typical protocol overhead on an Azure IoT Hub device-to-cloud message, using AMQP?
What is included in the C SDK for batching messages when AMQP is used? For example, if the application sends 3 messages in rapid succession (while the connection is up), will the SDK combine them into a single packet over the network? How much time must elapse between the application submitting messages to the SDK before the SDK will decide to send them rather than waiting to see if there are more?
How can a device that is shutting down determine which messages are still buffered by the SDK (not yet sent), in order to save these messages and try sending them again next time it is started? (This one is easy - there's a callback parameter to IoTHubClient_LL_SendEventAsync() that tells you when a message has actually been sent.)
The protocol overhead of AMQP is quite low - it was designed with this in mind. After the links are negotiated, end point strings are not sent with every data message so it does not matter that these are quite verbose in Azure IoT Hub.
Chuck Rolke has written up an example, AMQP Illustrated, showing packet captures of generic AMQP traffic (not specifically IoT Hub). In the example, the Transfer frame containing the "Hello World!" message has a total size of 47 bytes - so the protocol overhead is 35 bytes, at least in this case. (This doesn't include the TCP, IP, and Ethernet headers.)
Olivier Bloch of Microsoft has confirmed that the SetBatching option in the Microsoft Azure IoT Hub C SDK is for HTTP transport only. If the option is set, then the SDK will send as many buffered messages in a single HTTP request as it can. When using the HTTP transport it is intended that requests should not be made too often, so in all likelihood several outgoing messages will be buffered between HTTP requests.
The ultimate conclusion is that batching isn't supported with AMPQ, but it's not really needed either.