How to avoid repeated django prefetch related database hits

Question

When i fetch a client i want to also prefetch all related probes and probe channels, and when i fetch a probe i also want to fetch all related probe channels.

But it seems that when i get a client object, or view the admin, it runs the get_queryset methods from both the client and the probe manager. so it will prefetch the probechannels twice. This can be seen both in the number of sql queries in debug toolbar, and by adding print statements in the get_queryset methods. so i've got

class ClientManager(models.Manager):
    def get_queryset(self, *args, **kwargs):
        return super(ClientManager, self).get_queryset(*args, **kwargs).prefetch_related('probe_set__probechannel_set')

class Client(models.Model):
    objects = ClientManager()

class ProbeManager(models.Manager):
    def get_queryset(self, *args, **kwargs):
        return super(ProbeManager, self).get_queryset(*args, **kwargs).prefetch_related('probechannel_set')

class Probe(models.Model):
    Client = models.ForeignKey('Client')
    objects = ProbeManager()

class ProbeChannel(models.Model):
    Probe = models.ForeignKey('Probe')

Is there a way to avoid this? I have read about _base_managers being used for related objects, but the _base_manager here is just models.Manager. I'm using django 1.11 and this trendy new thing called python 2.7.

Answer 1

I'm still not sure if this is desired Django behaviour, but the prefetches are definitely firing twice. The answer i've worked out to avoid ProbeManager prefetches to be fired as well was simple- avoid using this manager. So in the Probe class I've added

objects_without_prefetch = models.Manager()

and then used a prefetch object Prefetch('probe_set',Probe.objects_without_prefetch.all()) instead of simply using 'probe_set' which was calling its own prefetch.

Thanks anyway those that took the time to think about this and answer!