Right now, I am running into the following issue exactly. Specifically, spark-submit is attempting to connect to the yarn.resourcemanager at location 0.0.0.0/0.0.0.0.
I have checked all of the logs delineated in the stack overflow thread above. They all seem to be correct. I have also added in a yarn.resourcemanager.address=... line to the default settings files at the top of the spark configuration directories, exported YARN_CONF_DIR and all of the other fixes listed on that thread.
At the bottom of the comments in the top rated answer, a commentator pointed out that if none of the above fixes work, then spark is not using the correct configurations.
At this point, I am pretty sure that my spark install is not using the correct configurations (I did not install it).
How does one go about determining which configurations spark is using, and how does one change them to the correct configurations? (or maybe I just need to reboot the machine?)
In spark-shell for example, I can do this:
scala> getClass.getClassLoader.getResource("yarn-site.xml")
res1: java.net.URL = file:/etc/spark2/conf.cloudera.spark2_on_yarn/yarn-conf/yarn-site.xml
...and the result shows the exact resolved location of a config file from my current classpath. Same could be easily translated to Java (almost verbatim) if your application is Java-based.