Django Datatables taking too long to load

Question

I am trying to load 25600 rows into datatables but it's taking around 10 seconds. The request is via an ajax API call.

views.py

@api_view()
def get_all_data(request):


get_all_data_ = Data.objects.values("name","contact_number","email_address","address","age",
"location_type","sector","phase","total_data","total_usage","city","district")
                 

return JsonResponse(list(get_all_data_), safe=False)

template.html

var table = $('#data-table').DataTable({

                serverSide: true,

                "ajax": {
                "url": "/alldata/",
                "dataSrc": ""
                },

              "columns": [

                  {"data": "name"},
                  {"data": "contact_number"},
                  {"data": "email_address"},
                  {"data": "address"},
                  {"data": "age"},
                  {"data": "location_type"},
                  {"data": "sector"},
                  {"data": "phase"},
                  {"data": "total_data"},
                  {"data": "total_usage"},
                  {"data": "city"},
                  {"data": "district"}
                 
              ],

        
          });

How can i make it instantaneous?

Answer 1

What I would do:

• I would add pagination (as @Willem Van Onsem suggested) just load the first 100 (for example) samples and add link to paginate over the table (next/previous). Take a look at Django Paginator.
• If you want to show some information about the whole dataset, I would compute statistics describing the data and display it instead of whole data. For example mean values, or median values ...

If you really want to make it faster with the whole table:

• You can try to use a faster server (maybe will help a little),
• You can try to cache it, maybe store the data in the memory so you don't need to run DB query for each request (but you need to remember to refresh/validate the cache).