Search code examples
apache-sparkpysparkhadoop-yarnspark-submit

Spark submitted application not shown in YARN web ui

I have node where I have installed spark in yarn mode. When I run an application with 

sudo ./usr/bin/spark-submit --master yarn --deploy-mode client MySparkCode.py

it runs fine.

When I connect in spark history server at http://localhost:18089/ I can see my submitted application.

But When I go at Yarn recourse manager web ui at http://localhost:8088/cluster/apps my application is not showing at all. Did I do something wrong? Shouldn't my application be shown there?

Solution 
  • sudo ./usr/bin/spark-submit --master yarn --deploy-mode cluster MySparkCode.py

    Try mode Cluster