Search code examples
hadoopapache-sparkazkaban

Failed Azkaban Spark Job has Success final status in stead of Failure

Use case:

  1. Azkaban starts Spark job
  2. Sparks Job Fails somewhow

Expected Result:

Hadoop ResourceManager said job final status is FAILED

Actual Result:

Hadoop RM said job final status is SUCCESSFUL

Does anybody know how can it be fixed?

Solution

  • Spark was running in default (client) mode.

    As a solution mode was changed to cluster.