best approach for in-memory storage (multi region on GCP/cloud memorystore)

Question

I'm building a chat app in react-native, with a nodejs backend. I'm using the google cloud platform.

I'm using websockets to create a continuous connection between the app and the backend. Because users can send messages to specific clients, I'm storing the sockets in nodejs:

var sockets = {}
io.on('connection', socket => {
    console.log('user connected')
    let userId = socket.handshake.query.userId
    sockets[userId] = socket
    socket.on('message', msgData => {
        let msg = JSON.parse(msgData)
        sockets[msg.userId].emit('message', JSON.stringify(msg.message))
    }
    socket.on('disconnect', () => {
        console.log('user disconnected')
        delete sockets[userId]
    }
})

Please note that this is a simplified example.

The problem is: I'm planning on having multiple instances in different regions behind a load balancer. When you connect to a specific instance, other instances can't reach the sockets object. So when 2 different users are connected to 2 different instances, they can't chat with each other.

To solve this, I was thinking of storing the sockets in a redis cache (cloud memorystore). But the redis instance must be in the same region as the VM instance. But like I said, I have multiple VM instances in multiple regions.

My questions are:

1) Is this solution the best way to go? Or are there any other possibilities, like just storing the sockets in a database?

2) How can I solve the issue of not being able to connect VM instances to a redis instance when they are not in the same region. Should I create a redis instance for each region I use (asia-east1, europe-north1, us-central1), and mirror those 3 redis instances so they all have the same content?

If you on the other hand have a total different approach, please let me know! I'm still learning nodejs and google cloud platform, and I'm open to new input.

Edit: All instances (instancegroups) are ofc in the same VPC.

Edit 2: What if I create a VM in the same region as the redis instance, and use this as a proxy? Would there be any performance issues?

Edit 3: I got it working by creating a proxy server using haproxy. The instance is located in the same region as the redis instance. One question: will there be any performance issues? And is this really the way to go?

Answer 1

Focusing on your first question, I would say that this architecture is not the best way of implementing a chat application. Google Cloud Platform provides a very strong messaging service - Pub/Sub. By using this service all the issues regarding load balancing, concurrency, connections and efficiency would be solved by the default.

Here you can find a really nice article about how to create a chat application wiht Cloud Pub/Sub. It is C# based, but the idea is the same, but using the Nodejs client libraries Take a look on a general schema on how Pub/Sub works :

The architecture of this app will have the following advantages:

• One-to-One (Direct) and One-to-Many messaging functionality

• Transmission method that did not require a full server to be
  developed

In case you do not want to use Pub/Sub, I would still think that you will need a centralized server application, which will be able to communicate with the users, process their messages and send them to the proper destination and reverse.

Regarding your second question, that may work, but I think it may affect the performance and, more important than that, the clearness of the system itself. It would be a nightmare to maintain, debug something like this.