I would like to design the best architecture for my following project: I have an application running on any device (desktop, mobile...) where users can publish or receive notifications with other users they share data.
Basically, a user can share with other users what he is doing on the application, other users being notified in real-time of the changes and vice-versa. And users are only able to receive notifications they are allowed by other users.
For example, when a user moves a widget on the screen, the application must store the new widget position, and also notify in real-time other users of this new position to perform the change on their screen. For this need, I would see an event-driven architecture with a publish-subscribe pattern. However, I guess I would also need to handle sync request-response pattern when the application needs to retrieve the list of users to share a widget for example.
I had a quick look at Streaming Data book by Manning where a streaming data architecture is described, but I don't know if this kind of architecture would fit my needs. One difference for example in the implementation part is that the event source producer can also be an event consumer in my application (in the book, the event source producer is a separate public streaming API and the real application is the only consumer)
My idea if I follow a bit the book would be the following: WebSocket for data ingestion and data access, a broker-like Kafka as message repository and a separate analysis service consuming Kafka topics and persisting data in DB. One interrogation is if I could use only one WebSocket for both data ingestion and data access.
Which detailed architecture and tools would you use to fit these needs?
For the implementation, I would consider javascript for the client part, and Java for the server part.
This is a pretty common use case for Kafka (leveraging both the broadcast and storage elements). There are some examples here which should help, although the context is slightly different:
https://www.confluent.io/blog/building-a-microservices-ecosystem-with-kafka-streams-and-ksql/
In this example the CQRS pattern is used, so changes you make to the screen position would create events sent to kafka, then you create a view service that other application instances can (long) poll to get changes.
You could also implement this with a websocket. There are a few implementations of this on github but I haven't tried any of them personally. The one complexity is that, if you want to scale out to many nodes, you need some way to map messages in Kafka to open websockets (whereas mapping requests to kafka partitions in the REST example is handled automatically). Getting started with a single server implementation wouldn't require this complexity though.