django -celery worker not receiving tasks

Question

I am trying to send emails via Django celery but the tasks are not getting received by the celery.

settings.py

CELERY_BROKER_URL = 'redis://127.0.0.1:6379'
CELERY_RESULT_BACKEND = 'redis://127.0.0.1:6379'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TASK_SERIALIZER = 'json'
CELERY_TIMEZONE = "UTC"

tasks.py

@app.task(name="send_activation_email")
def send_activation_email(user):
    
    to = user.email
    user_id = user.id
    subject = 'Activate Your %s Account' % user.get_user_role_display()
    text_content = ''
    uid = urlsafe_base64_encode(
        force_bytes(user_id))

    token = account_activation_token.make_token(user)
    c = ({'user': user,
          'user_type': user.user_role,
          'base_url': base_url, 'token': token, 'uid': uid, 'text_content': text_content})
    html_content = get_template('mail/user/invite.html').render(c)
    msg = EmailMultiAlternatives(
        subject, text_content, from_email, [to])
    msg.attach_alternative(html_content, "text/html")
    msg.send()

views.py

class SignUpView(generics.ListAPIView):
    authentication_classes = ()
    permission_classes = ()
    renderer_classes = (JSONRenderer, )

    def post(self, request, user_role="employee",format=None, version=None):
        user = UsersSerializer(data=request.data,context={"user_role": user_role})
        if user.is_valid():
            t_user = user.save()
            send_activation_email.delay(t_user)
            message_data = custom_http_messages(
                code=200, message='You have successfully created your account.',data=user.data)
            return Response(message_data, status=message_data['code'])        
        else:
            message_data = custom_http_messages(
                    code=400, message='', data=user.errors)
            return Response(message_data, status=message_data['code'])

Getting error

it is working without the .delay function ie without celery. also the celery terminal is not receiving any tasks.

Answer 1

t_user is of type User. When celery is initiating a task via a broker, it has to serialize all parameters that the tasks need so that the task (which may be on a different server) can deserialize its arguments from the broker and operate on them in a stateless manner.

The error that you are seeing is because celery does not know how to serialize the t_user instance to json. You can do one of two things: modify both the producer and consumer to operate on json-serialized User dictionaries, or modify the task_serializer and accept_content settings to include pickle.

Of course, using pickle introduces security concerns if a non-trusted producer application can initiate tasks by putting task requests on the broker. If you are not concerned with the issue of non-trusted producers, you should be fairly secure (but you may want to read up on all of the vulnerabilities that you may be susceptible to using pickle).

You can read more about accept_content and task_serializer in the celery docs (links provided).