I need to get UNIQUE data from EACH column I have a list of fields:
MY_FIELDS = ['id', 'created', 'And_more_28']
My HORRIBLY INEFFECTIVE code that produces the correct result
qs = Sheets.objects.all()
v = []
v += [list(qs.values(v).distinct()) for v in MY_FIELDS]
I get what I need.... But at what cost? (28 requests is painful) Tell me how to make one query out of this....
This is how it produces non-unique data for EACH column (tried it)
qs = Sheets.objects.all().values(*MY_FIELDS).distinct()
Your second approach does produce unique values, but these are unique combinations. So if there are 10 fields with each 5 values, that already can result in at most 510 results, so it scales dramatically.
But the good news is, we can solve this, by aggregating all columns into distinct lists. Indeed, with ArrayAgg
[Django-doc]:
MY_FIELDS = ['id', 'created', 'And_more_28']
from django.contrib.postgres.aggregates import ArrayAgg
qs = Sheets.objects.aggregate(
**{'values_{f}': ArrayAgg('f', distinct=True) for f in MY_FIELDS}
)
v = [qs[f'values_{f}'] for f in MY_FIELDS]
This will produce lists of distinct values for each column, and all in one single query.