VPC Private Google API access for mqtt.googleapis.com (Cloud IOT) using a proxy

Question

I have enabled Private Google API access for a VPC and I use this HTTP proxy solution described to connect my offsite datacenter to the Google Cloud backend.

Using the solution, I have verified that the Google object storage api's work, by using gsutil to move files across the offsite network.

However I am unable to connect to mqtt.googleapis.com that is required for cloud IOT.

I think this is because the MQTT broker running at mqtt.googleapis.com cannot be accessed via a private network unless it is also proxied like the HTTP proxy solution described above.

Meanwhile actual gsutil IOT commands work fine because I presume they are running over the Google HTTP API.

To solve this I see we'd need any one of the below, unless someone has different way to do this?

1. Run an MQTT broker proxy in the private VPC and route MQTT packets to the mqtt.googleapis.com . Is there a suitable MQTT proxy broker that we can use in this case?

2. If we get a range of public IP's that the mqtt bridge (mqtt.googleapis.com) is running at then we can simply build the network routes for this one use case. Is this available?

Answer 1

I managed to get this to work using NGINX as a reverse proxy and stream the TCP traffic directly to mqtt.googleapis.com. Here are the steps to achieve this

1. Install Nginx with the --with-stream configuration flag . This builds Nginx with the functionality of a TCP streaming proxy
2. My Nginx conf file contains the following to point to Google's broker. The Nginx server is running in an instance in the VPC

/etc/nginx/nginx.conf

 stream {
    upstream google_mqtt {
            server mqtt.googleapis.com:8883;
    }
    server {
            listen 8883;
            proxy_pass google_mqtt;
    }
}

3. The internal private VPC has a DNS Server that resolves mqtt.googleapis.com to the IP of the Nginx server