Django - Call model's method on demand to filter and prefetch data from a reversed foreignkey

Question

Im currently trying to improve the performance on my server, and I have been working a few days with this, and so far i havent found a solution. Im using django 1.5 and there is for no no chance to upgrade to a newer version.

Given these models:

class Company(models.Model):
    name = models.CharField()

class Store(models.Model):
    name = models.CharField()
    closed = models.BooleanField()
    store = models.ForeignKey(Company)

According to the docs, if i want to prefetch a reversed foreign key i can do this:

companies = Company.objects.filter(
    name__startswith = "stackoverflow",
    ).prefetch_related("store_set")

for company in companies:
    open_stores = company.store_set.all() 

This works just fine, and its all done in 1 query. But what happens is that i only want the stores that are open. Doing this will result in extra queries being done:

for company in companies:
    open_stores = company.store_set.filter(open = True)

So, this is not useful at all. How can i prefetch all the stores that are open?

One of my ideas was to do something similar to this:

class Company(models.Model):
    name = models.CharField()
    prefetched_stores = None
    def _prefetch_stores(self):
        return self.store_set.filter(closed = False)
    def __init__(self, *args, **kwargs):
        force_prefetch = kwargs.pop("force_prefetch" or None)
        if force_prefetch:
            self.prefetched_stores = self._prefetch_stores()

class Store(models.Model):
    name = models.CharField()
    closed = models.BooleanField()
    store = models.ForeignKey(Company)

One issue with this is that i dont always want to preload the stores, so the init is not the correct place (and will most likely cause a bunch of side problems) but i hope you get the idea.

Is this possible at all or have i reached a dead end? If possible, how can i do this?

Answer 1

Just filter the stores in python code:

for company in companies:
    open_stores = [s for s in company.store_set.all() if not s.closed]

Or manually prefetch the stores:

companies = Company.objects.filter(name__startswith="stackoverflow")

open_stores_d = {}
for store in Store.objects.filter(closed=False, company__in=companies):
    open_stores_d.setdefault(store.company, []).append(store)

for company in companies:
    open_stores = open_stores_d.get(company, [])