I'm trying to create an SSH tunnel to run Jupyter on a Google Cloud Dataproc cluster.
The tutorial gives the following template for creating the tunnel. It says:
"Replace port1 with the Cloud Shell port you will use (8080 - 8084), and port2 with the Web interface port on the cluster master node."
gcloud compute ssh master-host-name \
--project=project-id --zone master-host-zone -- \
-4 -N -L port1:master-host-name:port2
What I am stuck on is port 2. I've looked in VPC settings, in the VM settings, etc but I cannot find any "web interface port". Is there someplace specific that I should be looking?
You may find this article helpful: https://cloud.google.com/dataproc/docs/concepts/accessing/cluster-web-interfaces
You're likely looking for Yarn resource manager on port 8088. However, if you want to drill down into your applications (which do not run on master node) you may find setting up SSH tunnel easier to work with:
gcloud compute ssh master-host-name ... -- -D 1080 -N
Then start the browser pointed at the proxy:
/usr/bin/google-chrome \
--proxy-server="socks5://localhost:1080" \
--host-resolver-rules="MAP * 0.0.0.0 , EXCLUDE localhost" \
--user-data-dir=/tmp/master-host-name
You can now enter http://master-host-name:8088 in browser's address bar.