Increasing time before 504 time-out using python requests

Question

I am using the requests module to query a relatively large PostgRES database (~14 GB), accessible via a PostgREST API, hosted on a simple server set up by a peer.

When I make simple API calls, such as

requests.get('https://api.nycdb.info/real_property_legals?bbl=eq.4043210050')

I get a 200 response and everything works fine. However, when I do more computationally intensive searches (such as text searches), I get 504 responses the vast majority of the time.

This is an example of an API call that almost always returns 504 time-outs:

requests.get('https://api.nycdb.info/real_property_parties?address1=@@."MADISON"')

I suspect this is because the server is not responding fast enough, due to its simple processing power and the size of the database. But, I look to the community for help here.

Is there a way to increase the amount of time a request waits for a response before a 504 response is provided? What else can I do to try to improve the reliability of more complicated queries?

Answer 1

You can add ,timeout= where seconds is a float. (See pydoc requests.request for available opts which can be supplied directly to pydoc.get().) There may also be limits on the API server side in quick case you may need to bust down your query into bite size chunks.