Enable WebSocket Connections with multiple Pods in Spring Boot Application

Question

I'm using a web-socket protocol in my spring boot application. There are multiple pods used, to handle heavy traffic. Now, having multiple pods is causing an issue. Let me brief it a bit,

Let's assume there are 2 pods (Pod 1, Pod 2). Angular UI is subscribing to spring boot application on the web-socket protocol, let's say via Pod 1. Now, the spring boot application sends a message to the UI, let's say its send via Pod 2, and this message is getting dropped (never reach the UI) since the web-socket connection was established via Pod 1.

Because of this, some messages are getting dropped, which are being sent to UI by other Pods (which were not used for the initial subscription process), and messages send via Pod which was used initially for subscription, only those messages are received at UI.

How to tackle this scenario, so that every message is send to UI in this multiple pods environment?

Answer 1

The solution to multiple pod issues is by using an external message broker (like RabbitMq, ActiveMq), instead of an in-memory message broker (default behavior).

You may face the below issues while implementing this (writing them down in one place so that you don't have to struggle much as I did 🙂),

• Creating Auto-Delete Queues
  When using external message broker, you might observe that the queues are created for every websocket connection, but they are not deleted when the websocket connection is over.
  
  We don't even need these queues. Hence come the need of Auto-Delete Queues. These auto-queues are automatically deleted when websocket connection is closed.
  
  How to declare auto-delete queues, its easy peasy

When using user destinations with an external message broker, check the broker documentation on how to manage inactive queues, so that when the user session is over, all unique user queues are removed. For example, RabbitMQ creates auto-delete queues when destinations like /exchange/amq.direct/position-updates are used. So in that case the client could subscribe to /user/exchange/amq.direct/position-updates. Similarly, ActiveMQ has configuration options for purging inactive destinations.

In simple terms, websocket client and websocket server should use /exchange/amq.direct/<anything> this exchange destination.

For more info, read the official docs

• ssl/stomp protocol on Cloud instance
  Another issue you might face when you are hosting you application to AWS or Azure or Google Cloud, is that they use ssl/stomp protocol, so you code which works fine in your local machine (since it uses stomp protocol) doesn't work fine in Cloud.

• Broadcasting message from one pod to other pods
  This issue is the same as written in this Stackoverflow question. [refer the question for clearance]

Now, lemme put up the code snippet and will add comments to indicate which part of snippet fixes which issue. Add it inside your configureMessageBroker method,

val tcpClient = new ReactorNettyTcpClient<> 
  (TcpClient.create()
    .port(yourRabbitmqCloudStompPort)
    .host(yourRabbitmqCloudHost)
    .secure(SslProvider.defaultClientProvider()), 
  new StompReactorNettyCodec());

messageBrokerRegistry
  // enables stompbroker, instead of in-memory broker
  .enableStompBroker("/queue", "/topic", "/exchange")
  .setClientLogin(yourRabbitmqCloudClientLogin)
  .setClientPasscode(yourRabbitmqCloudClientPasscode)
  .setSystemLogin(yourRabbitmqCloudSystemLogin)
  .setSystemPasscode(yourRabbitmqCloudSystemPasscode)
  // broadcast msg to every pod
  .setUserDestinationBroadcast("/topic/unresolved-user-destination")
  .setUserRegistryBroadcasr("/topic/user-registry")
  // enables ssl/stomp protocol
  .setTcpClient(tcpClient);