Search code examples
apache-sparkhuecloudera-manager

See the number of executors and partitions for a spark job in cloudera manager

How can i see the number of executors and partitions for a spark ingestion job in cloudera manager. I just cant seem to find this information.

I looked under yarn/applications found the job but did not find the information also looked under the log files.

I also looked in hue under the workflows for the specific job I just couldnt find it maybe im just missing it somewhere.

Thanks

Solution

  • You can find all the details in Spark history webUI ex: www.hostname:18088/history