Why does len() execute __set__ of my class?

Question

I've implemented a custom model field in Django. It is an image field that allows assiging an URL string to load an image from, in addition to directly assigning a file.

import uuid
import urllib.request

from django.core.files.base import ContentFile
from django.db import models
from django.db.models.fields.files import ImageFileDescriptor


class UrlImageFileDescriptor(ImageFileDescriptor):
    def __set__(self, instance, value):
        # If a string is used for assignment, it is used as URL
        # to fetch an image from and store it on the server.
        if isinstance(value, str):
            try:
                response = urllib.request.urlopen(value)
                image = response.read()
                name = str(uuid.uuid4()) + '.png'
                value = ContentFile(image, name)
            except:
                print('Error fetching', value)
                pass
        super().__set__(instance, value)


class UrlImageField(models.ImageField):
    descriptor_class = UrlImageFileDescriptor

In general, the field works. But for some reason, Django itself internally assigns string values to it. Every time a query set of models using the field gets filtered, __set__ is called with a string so that the print statement in the except clause fires Error fetching upload/to/50e170bf-61b6-4670-90d1-0369a8f9bdb4.png.

I could narrow down the call to django/db/models/query.py from Django 1.7c1.

def get(self, *args, **kwargs):
    """
    Performs the query and returns a single object matching the given
    keyword arguments.
    """
    clone = self.filter(*args, **kwargs)
    if self.query.can_filter():
        clone = clone.order_by()
    clone = clone[:MAX_GET_RESULTS + 1]
    num = len(clone) # This line causes setting of my field
    if num == 1:
        return clone._result_cache[0]
    # ...

Why is the line causing my field's __set__ to get executed? I could validate the input value to be a valid URL to work around this, but I'd like to know the reason first.

Answer 1

The story is there in your traceback. To get the length of the query:

File "C:\repository\virtualenv\lib\site-packages\django\db\models\query.py" in get
  350.         num = len(clone)

It fetches all the query results into a list:

File "C:\repository\virtualenv\lib\site-packages\django\db\models\query.py" in __len__
  122.         self._fetch_all()
File "C:\repository\virtualenv\lib\site-packages\django\db\models\query.py" in _fetch_all
  966.             self._result_cache = list(self.iterator())

For each query result, it creates a model object using the data from the db:

File "C:\repository\virtualenv\lib\site-packages\django\db\models\query.py" in iterator
  275.                     obj = model(*row_data)

To create the model object, it sets each field of the model:

File "C:\repository\virtualenv\lib\site-packages\django\db\models\base.py" in __init__
  383.                 setattr(self, field.attname, val)

Which winds up calling __set__ on your custom model field:

File "C:\repository\invoicepad\apps\customer\fields.py" in __set__
  18.                           response = urllib.request.urlopen(value)

It's hard to say more about the larger-scale reasons behind this, both because I don't know that much about Django and because I don't know what your db structure is like. However, essentially it looks whatever database field populates your UriImageField has data in it that is not actually valid for the way you implemented the descriptor. (For instance, judging from your error, the db has 'upload/to/50e170bf-61b6-4670-90d1-0369a8f9bdb4.png' but there is not actually such a file.)