Search code examples
djangoexceptioncelerylocals

django passing locals() to celery task

I am using python 2.7 and django 1.27, i am using celery for tasks.

I have this view

def my_view(request):
    do_stuff()
    local_1 = 1
    local_2 = 4
    celery_delayed_task(locals())
    return HttpResponse('OK')

this resulted in this excpetion

Passing locals() fails: a class that defines slots without defining getstate cannot be pickled

so i thought maybe i need to create a copy of the locals() dictionary since the task will be called when the view does not exist any more.

i try this instead:

def my_view(request):
    do_stuff()
    local_1 = 1
    local_2 = 4
    locals_dict = copy.deepcopy(locals())
    celery_delayed_task(locals_dict)
    return HttpResponse('OK')

and now i got this error:

Deepcopy of object fails: object.new(cStringIO.StringO) is not safe, use cStringIO.StringO.new()

obviously i am doing it wrong, any thoughts ?

Solution

  • The task arguments must be serialized.

    Celery uses the Python pickle protocol by default, but also supports json, yaml, msgpack or custom serializers.

    The objects you are trying to send cannot be pickled. There's a chance you could make them pickleable, but in the end -- passing locals as task arguments is not a good practice.

    See: http://docs.python.org/library/pickle.html