I have Django server that servers HTTP requests and updates Postgres DB. There is also Tornado TCP server that should push new data from Database.
The problem is: best way to notify Tornado about DB changes. That's it.
What options did I think of:
NOTIFY
and LISTEN
that are also asynchronous and can be easily added to Tornado IOLoop (io_loop.add_handler
with connection file descriptor). This is low level too, but more important is that this solution violates Data Independency Principle.Now I need to mention a subtle detail that can make a big difference with what I've described:
99% of changes in DB occur not in Django process, but in separate scripts (and hence separate processes) that are launched by scheduler and use Django ORM.
Now this description more or less reflects the actual state of project. So is Celery really not suited for solving this kind of problem? Is Redis the best solution? Are there any other that I don't know about? Any thoughts and suggestions are appreciated.
After evaluating all these options, I decided to go with Redis+Tornado approach. It showed itself quite robust and stable (it was 5 years ago, just for the reference). I laid out the whole project with a working example here: https://www.databrawl.com/2017/04/27/real-time-python-via-tcp/