Search code examples
djangopostgresqldjango-modelsormdjango-queryset

How to optimize nested loop query of models with one-to-many relationship


I have two models with a one-to-many relationship. There are several objects of each model class in the database. I created a custom queryset that iterates over all parent objects and all child objects then loads them into a dictionary that is then used as the queryset.

My question is how can I optimize my nested_query_loop function to hit the database less times? I tried incorporating select_related() and prefetch_related() but could never get it to work since I need to query it as a backward relationship in order to get each child object to be with its parent object.

Second part of the question: I tried making the function reusable by creating parent_object and child_object arguments, but this did not work due to the fact that I have to use the ObjectName_set() method, which requires the objects name to be lower cased. Also the fact that ObjectName_set() has an underscore made it so child_object_set() and even ChildObject_set() was not recognized and threw an error. In my case the child objects model name is first letter upper case. So my question is: is there any way to manipulate the models name inside the function to be all lower cased? or is the only way around this to change the models name at the source to lower case? Or even better question: is there a django class or function that I'm just not seeing that would do all of this for me? I'd hate to reinvent the wheel and make it worse.

Please see the models and views below. view Models

''' python

class MasterListView(ListView):
    template_name = 'passdown/master_list.html'
    context_object_name = 'entries'

    def nested_query_loop(self):
        object_list = []
        all_objects = PassDown.objects.all() #queryset containing all passdown objects
        for object in all_objects:
            object_list.append(object.pk) #list containing all passdown primary keys (object_list)

        query_dict={}
        for i in object_list:
            temp_obj = PassDown.objects.get(pk=i) #ith passdown object itself
            sub_object_pk_list=[]
            all_i_entries = temp_obj.entry_set.all() #queryset containing all entry objects of i
            for j in all_i_entries:
                sub_object_pk_list.append(j.pk) #list containing all entry primary keys of i (sub_object_pk_list)
                
            sub_object_list = []
            for k in sub_object_pk_list:
                sub_object_list.append(all_i_entries.get(pk=k)) #list containing all ith entry objects (sub_object_list)
            query_dict.__setitem__(temp_obj, sub_object_list)

        return query_dict

    def get_queryset(self):
        queryset = PassDown.objects.all()
        return queryset
    
    def get_context_data(self, **kwargs):
        context = super(MasterListView, self).get_context_data(**kwargs)
        queryset = self.nested_query_loop()
        context.update({
            "parent_child_dict": queryset
        })
        return context

'''

''' python

class PassDown(models.Model):
    shiftList = [("Days", "Days"),
                 ("Nights", "Nights"),
                 ("Mids", "Mids")]
    shift = models.CharField(max_length=10, choices=shiftList)
    date_time = models.DateTimeField(default=timezone.now)
    notes = models.TextField()
    entered_by = models.ForeignKey(User, on_delete=models.DO_NOTHING)
    #entry = models.ForeignKey(Entry, on_delete=models.DO_NOTHING)

    def __str__(self):
        return f"{self.shift} {self.date_time}"

class Entry(models.Model):
    modex = models.IntegerField()
    discrepancy = models.CharField(max_length=50)
    text_body = models.TextField()
    passdown = models.ForeignKey(PassDown, on_delete=models.CASCADE, default=PassDown.objects.last)

    def __str__(self):
        return f"{self.modex} {self.discrepancy} {self.passdown.date_time}"

'''


Solution

  • I do not getthe difference between your code and the following

    def nested_query_loop(self):
            passdowns =  PassDown.objects.all()
            return {passdown: passdown.entry_set.all() for passdown in passdowns}
    

    If I did not miss anything. You can use a simple prefetch to reduce the number of queries to two.

    def nested_query_loop(self):
            passdowns =  PassDown.objects.prefetch_related("entry_set")
            return {passdown: passdown.entry_set.all() for passdown in passdowns}
    

    However I do not see what your dict could be used for since you can call related_manager in the template.

    Here is what a classical list view is supposed to look like

    class MasterListView(ListView):
        model PassDown
        template_name = 'passdown/master_list.html'
        context_object_name = 'passdowns'
    
        def get_queryset(self):
            # Only define for performance reasons
            queryset = super().get_queryset()
            queryset = queryset.prefetch("entry_set")
            return queryset
    
    {{ for passdown in passdowns }}
      # Render your passdown object
      {{ for entry in passdown.entry_set.all }}
        # Render your related entry