I am mqtt-ing a string from a Rasbperry Pi(sitting in a field, supported by LTE internet, costing me 10$/500MB/month) to an MQTT broker. I am using paho-mqtt client in python to do this for me. The string looks something like "MM:DD:YYY HH:MM:SS, X1, X2, X3, , , , X24", and I am sending a new string every 30 seconds. X1 to Xn are floating point numbers 0 to 700 with 2 digit precision. I think this will cost me a lot of internet when I deploy it to 24/7 use. Is my data format good? What other data formats should I look at?
You can represent the Unix time with a 4-byte float. And you can represent a float with an IEEE754 float in 4 bytes. So your time and 24 floats can be packed into 100 bytes with Python struct.pack(). That looks like this:
import struct
import time
import random
# Synthesize some sample data - a time and 24 floats 0..700
data = [time.time()] + [ random.uniform(0, 700) for _ in range(24)]
# Pack as 25 IEEE754 floats of 4 bytes each
payload = struct.pack('!25f', *data)
print(len(payload)) # prints 100 (bytes)
Currently, you seem to be using:
So, that's around 180 bytes as you currently have it.
If you multiplied your floats by 100 and made them integer you could maybe encode as 16-bit unsigned values (i.e. half the space of a 4-byte float) which would go from 0..65535 to represent 0..655 which is close to your data range of 0..700. So that would be 4 bytes for the time, plus 24 samples of 2 bytes each, for a total of 52 bytes.
So, rather than 100, use 65535/700 or 93.62:
# Scale the data to the range 0..65535 and make into integers
smallerData = [data[0]] + [ int(93.62*data[i]) for i in range(1,25)]
payload = struct.pack('!f24H', *smallerData)
print(len(payload)) # prints 52 (bytes)
Obviously all the numbers above exclude MQTT protocol overhead.