Search code examples
azure-hdinsight

listing submitted jobs in HDInsight

Is there any option to get a list of all submitted jobs in HDInsight without using remote desktop?? (thro .net SDK or web interface)

  1. There is a function 'ListJobs'.

But it shows exception '502 - Web server received an invalid response while acting as a gateway or proxy server'

2 Also I am not able to access the jobtracker portal with the namenode IP(IP of remote desktop).

(http://:50030/jobtracker.jsp)

Solution

  • You can use WebHCat for this. Send a GET request to https:///templeton/v1/jobs with showall set to True.

    Documentation of the API is here

    This will return all job ids submitted and visible to the user Id you authenticate in with.

    Providing the curl command just for completeness of solution.

    curl -i -u <clusterUserName>:<clusterPassword> -d user.name=<JobSubmissionUserName> -d showall=true -s "https://<clusterDnsName>..azurehdinsight.net/templeton/v1/jobs"