How do I configure to run my Cloud dataflow job using Internal IP? Our policy doesn't allow to use external IP to spawn the workers. So, looking for options that would disallow external IP. I ran and got the below error.
Startup of the worker pool in zone XXX failed to bring up any of the desired 1 workers. Please check for errors in your job parameters, check quota, and retry later, or please try in a different zone/region. Add instance projects to use external IP with it.
You can use the --usePublicIps=false flag. Here you can look at some examples.