Django - queryset split to 4, causes lots of hits to the database

Question

Here is my template tag:

@register.filter
def split_to_4_columns(queryset):
    split = int(ceil(queryset.count()/4.))
    columns = list()
    for i in range(4):
        columns.append(queryset[i*split:(i+1)*split])
    return columns

And here is my pseudo usage in templates:

{% for column in queryset|split_to_4_columns %}
<div class="list">
    {% for object in column %}
    <a href="{{ object.get_absolute_url }}" class="item">{{ object }}</a>
    {% endfor %}
</div>
{% endfor %}

It basically splits my long querysets (few hundreds objects) into 4 columns for a better html presentation. But it also appears to hit hard on my database. First it evaluates the full queryset, and then every column gets it's own evaluation, how can I optimize it?

I am using caching, but still, I'd like to know how can I optimize it on a lower level than just caching.

Answer 1

As you process all the data, at least one select returning all the data must be processed. To query data once, you can on first step request all the data (evaluate queryset) and on the second split it with pure python (not hitting DB):

@register.filter
def split_to_4_columns(queryset):
    values = list(queryset.all())
    split = int(ceil(len(values)/4.))
    columns = [values[i*split:(i+1)*split] for i in range(4)]
    return columns