python3 - requests to OSM API take at least the configured connect timeout duration

Question

I am requesting the OSM reverse geocode API (https://nominatim.org/release-docs/develop/api/Reverse/) from app engine. When configuring no timeout, the requests run infinitely. When configuring a connect and read timeout, the request returns successfully with an overall roundtrip time of $connect_timeout plus a small amount of milliseconds. This is the same with:

• different libraries (requests, urllib3, urllib builtin)
• Cloud Functions environment

This does not happen when:

• I request another API than OSM from App Engine/Cloud Functions (e.g. https://api.opentopodata.org/v1/test-dataset)
• I request from a Compute Engine VM
• I request from my local machine

What can be the reason for this behaviour?

Answer 1

A workaround is to not request via the domain name, but request with an ip address (turning ssl cert validation off).

This request is slow:

requests.get(url="https://nominatim.openstreetmap.org/reverse?lat=0&lon=0", headers={"Referrer": "https://www.example.org"}, timeout=4)

Whereas this request is fast:

requests.get(url="https://130.117.76.9/reverse?lat=0&lon=0", headers={"Referrer": "https://www.example.org"}, timeout=4, verify=False)

Google seems to do something to openstreetmap requests internally.

I have opened an issue for it: https://issuetracker.google.com/issues/171904232