How do you perform HTTP requests from App Engine to internal services?

Question

I have an App Engine Standard (1st Gen) Python app, set up with serverless VPC access. I use requests 2.3 (I have issues with later versions on GAE) to do HTTP requests.

When I try doing an HTTP request to an internal IP address (10.x), it refuses to connect. However, serverless VPC access is working properly: I tested connecting to the same IP address on the same port using a non-HTTP client library (e.g. redis) from App Engine, and this works fine. It's just HTTP requests that fail.

I suspect the URL Fetch service is trying to do the HTTP requests, and fails on internal addresses. If so, is there a way to use requests without triggering the URL Fetch service? Or do you have any other clue what might be going wrong?

Answer 1

I think I found the answer by looking at the SDK source code:

from python_std_lib import httplib

gets you an HTTPLib that uses sockets, without having to override the global httplib (causing sockets to be used everywhere).