I want to create beanstalkd jobs when a django model is saved. For this I am listening to the model's post_save signal and the associated handler is creating a beanstalkd job like below for example (in myapp/signals.py
):
@receiver(post_save, sender=MyModel)
def create_job(sender, **kwargs):
beanstalk = beanstalkc.Connection(
host='example.com',
port=11300
)
beanstalk.use('test')
beanstalk.puts('job');
However I do not like the fact that I am creating a new connection each time I receive a signal. My questions are:
Finally I ended up following this approach:
I created a class inside common/util/beanstalkd_client.py:
from django.conf import settings
import beanstalkc
class _BeanstalkdClient:
def __init__(self):
self.beanstalk = beanstalkc.Connection(
host=settings.BEANSTALKD.get('host'),
port=int(settings.BEANSTALKD.get('port'))
)
self.job = None
def watch(self, tube, ignore_default=True):
self.beanstalk.watch(tube)
if ignore_default:
self.beanstalk.ignore('default')
def use(self, tube):
self.beanstalk.use(tube)
def reserve(self):
self.job = self.beanstalk.reserve()
return self.job
def put(self, payload):
self.beanstalk.put(payload)
beanstalkd_client = _BeanstalkdClient()
This allows me to reuse the beanstalkd_client and handles the connection inside. Since python will always create one instance of a module, beanstalkd_client module will always have one connection to beanstalkd.
Then in my signals.py:
from django.db.models.signals import post_save, post_delete
from django.dispatch import receiver
from django.conf import settings
from common.util.beanstalkd_client import beanstalkd_client
beanstalkd_client.use(settings.BEANSTALKD.get('my_tube_name'))
@receiver(post_save, sender=Address)
def es_create_index(sender, **kwargs):
beanstalkd_client.put('my awesome job!')
However this still does not handle the 2nd point: What if the connection to beanstalkd is broken!