I have a web app which on the back end has a series of events going on in the back end which a currently logged in user should be aware of. Originally we could long poll for these events going on behind the scenes when there were only a few event types such as getNewNotifications, getNewComments but having more than a couple of long polls going on is a huge waste of resources and seems time time to rethink this process.
Looking at most social networks it seems they long poll for new updates which will return an array of hashs. Presumably if they process each of these either the front end or backend processes the hash and redirects the process to request the necessary function.
This technique only needs one long poll but it requires a queue on the backend to post new messages to as events happeb on the backend such as a comment being posted or some activity contributing to leaderboard movement.
This is where I am puzzled, I have considered having a redis queue for each user which will need to be cleared on page refresh, relogin or user becoming active again. In order to keep a record of users who are active I was considering having a redis set of active users, which have to be cleared out if active user object they are linked becomes expired.
I feel like this must be a common problem in modern web apps with a lot of information going back and forth and polling each service a real waste of resources, yet I haven't seen any examples (at least that I can see the correlation). Any help would be much appreciated.
Using the PubSub pattern with Redis and WebSockets is used by many popular services, http://blog.newsblur.com/post/20371256202/building-real-time-feed-updates-for-newsblur