Connection management mqtt broker kubernetes?
We are using Mosquitto MQTT broker, an IoT server implementation and devices which connect to the Broker as an Client. Both device and IoT server will be publishing to the broker.
The devices publish requests to the topic : /req/<device-id>/<server-id>
The devices subscribe to responses : /resp/<device-id>/<server-id>
And if IoT server sends a notification to a particular device, it publish to notification topic to which device also subscribes:
/req/<server-id>/<device-id>
So this scenario is working fine uptill now, but we want to shift our implementation on Kubernetes such that both MQTT broker and IoT server will have multiple pods running.
So now any device can connect to any mosquitto pod instance and any IoT server pod would be connecting to mosquitto pod instance.
So a device connected to mosquitto pod1 and IoT server pod1 might not receive notifications if pod2 of IoT server generates notification and sends it to mosquitto pod instance to which our device wasn't connected to.
So IoT server will need the awareness that which device is connected to which pod instance to send notifications.
How to achieve this in Kubernetes environment??
The short answer to this is: You don't easily.
The longer answer is:
You will probably need to pick a different MQTT broker than mosquitto, mosquitto does not support clustering, so there is no (simple) way to run multiple instances that sessions and messages can be distributed across.
You can setup bridging between multiple broker instances to ensure messages end up on all instances, but the best way to do this is with a star formation, with a "central" broker that then redistributes all the messages to the points of the star. The devices would then connect to the these star instances. This does not solve any of the problems with distributed sessions.
You will also probably need to look at shared subscriptions so that messages are only consumed by a single instance of the IoT Server.
There are several other MQTT broker implementations that support proper clustering, iirc things like HiveMQ and emqx
- How can I close over variables in kdb/Q?
- When is the EACH operator extension necessary in K besides mod/rotate?
- Handling single-character strings - in a function or in its caller? ssr()
- Kdb+ data fomat when writing to a file
- How to convert a symbol to a string in kdb+?
- Sum of each two elements using vector functions
- A dictionary with a single value and multiple keys
- Table transformation, table as list of dicts
- Accumulator gives different result then direct function applying
- Reshape [cols;table]
- FK field over IPC
- Protected execution, 2 cases
- Enums for tables
- Converge (fixed point) syntax difference in q and k
- .Q.trp and bt handling
- NULLs in q and in k.h
- Strange view declaration behaviour
- How to build a parse-tree of projections?
- Could not evaluate manually created equial ~ parse tree
- Select distinct for all columns from keyed table
- Parallel execution: blocking receive, deferred synchronous
- Multiple variable assignment in q
- Select a table from the inside of external select
- Select when one of filter-column may not exists
- What is the meaning of `s attribute on a table?
- On parallel execution - which side reports about an error?
- Validate if a keyed table have unique keys
- Applying dictionary to dictionary
- About xkey implementation
- Parse tree built on values from vars