Search code examples
clouderaapache-sparkhadoop-yarn

spark-submit: application master can not talk back to driver

I submitted a spark job to Cloudera cluster using spark-submit in YARN cluster mode.

I get the following exception:

org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:187)
Caused by: org.jboss.netty.channel.ChannelException: Failed to bind to: /myip:0
    at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272).

Any one knows why application master can not bind to the driver IP?

Solution

  • I found it!

    remove

    .set("spark.driver.host", "driver computer ip address")

    from your driver code.

    also posted on spark-submit to cloudera cluster can not find any dependent jars