How to use Celery to upload files in Django

Question

I was wondering how I can use Celery workers to handle file uploads. So I tried implementing it on a simple class. I overrided the create class in my ModelViewSet. But apparently Django's default json encoder does not serialize ImageFields (Lame). I'll really appreciate it if you guys could tell me how I can fix this. Here is what I came up with:

serializers.py:

class ProductImageSerializer(serializers.ModelSerializer):
    class Meta:
        model = ProductImage
        fields = ['id', 'image']

tasks.py:

from time import sleep
from celery import shared_task
from .models import ProductImage

@shared_task:
def upload_image(product_id, image):
    print('Uploading image...')
    sleep(10)
    product = ProductImage(product_id=product_id, image=image)
    product.save()

views.py:

class ProductImageViewSet(ModelViewSet):
    serializer_class = ProductImageSerializer

    def get_queryset(self):
        return ProductImage.objects.filter(product_id=self.kwargs['product_pk'])

    def create(self, request, *args, **kwargs):
        product_id = self.kwargs['product_pk']
        image = self.request.FILES['image']
        image.open()
        image_data = Image.open(image)
        upload_image.delay(product_id, image_data)

        return Response('Thanks')

and here's the my model containing my ImageField:

class ProductImage(models.Model):
    product = models.ForeignKey(Product, on_delete=models.CASCADE, related_name='images')
    image = models.ImageField(upload_to='store/images', validators=[validate_image_size])

Answer 1

Hello everyone earlier I posted a solution for this question and even though that solution worked properly, I found a better solution. Encoding and Decoding binary files using base64 makes them larger and that is not something we want. So a better solution is to temporarily save the uploaded file on the disk, pass the path to our celery worker to upload it and create a ProductImage instance in our database and then delete the file we saved on the disk .

Here’s how to implement it:

tasks.py:

from time import sleep
from celery import shared_task
from .models import ProductImage
from django.core.files import File
from django.core.files.storage import FileSystemStorage
from pathlib import Path

@shared_task
def upload(product_id, path, file_name):

    print('Uploading image...')

    sleep(10)
    
    storage = FileSystemStorage()

    path_object = Path(path)

    with path_object.open(mode='rb') as file:
        
        picture = File(file, name=path_object.name)

        instance = ProductImage(product_id=product_id, image=picture)

        instance.save()


    storage.delete(file_name)

    print('Uploaded!')

In serializers.py you should override the create method of the ProductImage serializer like this:

    def create(self, validated_data):
        product_id = self.context['product_id']
        image_file = self.context['image_file']
        storage = FileSystemStorage()
        
        storage.save(image_file.name, File(image_file))

        return upload.delay(product_id=product_id, path=storage.path(image_file.name), file_name=image_file.name)

You should also override the create method in ProductImage’s ViewSet to provide the image file for your serializer’s context:

    def create(self, request, *args, **kwargs):
        product_id = self.kwargs['product_pk']
        image_file = self.request.FILES['image']
        serializer = ProductImageSerializer(
            data=request.data,
            context={
                'product_id': product_id,
                'image_file': image_file
            }
        )
        serializer.is_valid(raise_exception=True)
        serializer.save()
        return Response('Upload Started...')