Connect to Host machine using Python Docker SDK

I have a script in Python, that builds and starts a Docker container using the docker Python SDK. I am setting a custom option extra_hosts, as well as specifying the network my container is running in (it is a specific custom network). I create the container as follows:

import docker

docker_client = docker.DockerClient(base_url='unix://var/run/docker.sock')

container_config = {
    'image': self.data.build.tag,  # a string
    'ports': self.data.run.ports,  # a dict of ports
    'network': self.data.run.network  # a string
    'extra_hosts': {'host.docker.internal': 'host-gateway'},
    'detach': True,
}
container = docker_client.containers.create(
    **container_config
)

But, when trying to connect to the host machine in a service, I tried both:

import os
import requests

host = os.getenv("HOST", "172.17.0.1")
info = requests.get(f"http://{host}:8000/cluster/feeds/node/yahoo-finance/").json()

and

import os
import requests

host = os.getenv("HOST", "host.docker.internal")
info = requests.get(f"http://{host}:8000/cluster/feeds/node/yahoo-finance/").json()

But am never able to connect to the endpoint. What could I be doing wrong? I get the following error:

requests.exceptions.ConnectionError: HTTPConnectionPool(host=\'localhost\', port=8000): Max retries exceeded with url: /cluster/feeds/node/yahoo-finance/ (Caused by NewConnectionError(\'<urllib3.connection.HTTPConnection object at 0x75f3c2a12c20>: Failed to establish a new connection: [Errno 111] Connection refused\'))

I tried setting the host_mode, which works, but then removes my container from its network, breaking the rest of the application. I was expecting it to be able to connect to the host machine service.

Solution

For anyone that comes across this, make sure to allow connections through your firewall to the host service, in my case the host service was running on port 8080, so I just allowed it with:

sudo ufw allow 8080

When bounding a service to 0.0.0.0 and accessing it using the host.docker.internal hostname, connections will pass through the firewall.