Search code examples
pythondjangoapiormtastypie

Tastypie DB level Field Selection


Given PostgreSQL 9.2.10, Django 1.8, python 2.7.5 and django-tastypie 0.12.1:

I am trying to figure out how to do a dynamic queryset for tastypie based off of a custom param (field selection) a end user may filter for via something like this:

https://example.com/api/v1/resource/?fields=field1 (list view)
https://example.com/api/v1/resource/1/?fields=field1 (detail view)

def dehydrate() (not using):
Implementing this in dehydrate is fine, but since the data has already been 'dehydrated' the only gain I achieve is that the response is smaller in size due to removing unselected fields. Code for this HERE.

def full_dehydrate() (currently using):
Implementing this in full_dehydrate is better, as I am skipping the initial 'dehydate' on fields I did not select. This is faster and more effecient than the dehydrate method above. Code for this HERE.

These two methods do not implement the filter at the db level though (I am still doing a model.objects.all()). This is fine in a dev instance where my db is MORE than likely local to the application/api itself, however in prod (in my case) the db is on a separate server. So my question is this, is there a way that I can dynamically change my tastypie query filter for a given resource based off of params passed in the API request? get_object_list looks promising but I am unsure if this is the method to pursue or if I should be writing a custom filter to add in via build_filters (I am assuming a build_filter is better,, though every example I see with custom build filters also requires a appy_filter for some reason.). Also I am unsure as to what that would look like for the particular scenario, as the method used to get request params is bundle.request which is not present in these two methods.

Thank you for anyone who took the time to read this and a huge thanks to anyone who provides a working solution. I greatly appreciate the help and thought process behind answers given.

** EDIT 1 **
HERE is a good visualization of the process in which this filter will need to occur (hopefully this helps stimulate some idea's)

** TL;DR **
I want to allow for db level filtering of a tastypie resrouce queryset based off a custom param a end user may use (if not used Tastypie should behave normally) in an API request (read: dynamically modify the resource queryset).


Solution

  • It seems you want to set specific fields on the generated Django SELECT clause. For that to happen you need to use the only() and defer() methods from Django's queryset. The use of one or another depends uniquely on your needs.

    Now, you need to call that methods on the queryset before items are accessed. Tastypie Resource's build_filters() is useless in this case because it has to do with the arguments that queryset's filter() will be receiving. apply_filters() sounds like a nice option, but i would not place it there because we are not modifying filtering (WHERE) but selection (SELECT). Finally i would do the respective modification in obj_get_list(), just after the apply_filters() call.

    I am providing you with a Gist that should fit your needs HERE. As explained in the Gist:

    This has to be done in two parts:

    1) In model queryset for database selection. This need to be in some code section where the queryset is ready, but still not evaluated. obj_get_list and obj_get are perfect for Tastypie GET requests.

    2) In full_dehydrate method, to prevent from Tastypie fetching all the fields from the model.

    The gist for dehydrate() that you propose does not seem to be necessary, as full_dehydrate() is already picking only the selected fields.

    NOTE: For the record, using only() or defer() on a queryset, and later trying to access an unselected field on the model causes an individual SQL call for the requested extra field. So be careful to always select the needed fields, specially in list of objects, because not doing so would generate n*m calls being n the number of unselected fields later-accessed and m the number of objects queried. This is the justification for the full_dehydrate() Gist modification.