Search code examples
apache-pigooziehuecloudera-cdh

Pig Jobs are stucked when they run concurrently

My CDH 5.2 is configured for single node.

Using Hue 3.6.0, I submitted first workflow which has 2 pig script. Right after that, I submitted the second workflow which has 1 pig script .

My second workflow waited for the first workflow to complete the first pig script, but it did not wait the first workflow to complete the second pig script. The second executed it's script while the first workflow begin executed it's second script. And they were stucked.

I tried run those pig script concurrently in shell mode. It worked !.

Any guide how to fix it ?

Thanks very much

Solution

  • This could really be the YARN gotcha #5 of http://blog.cloudera.com/blog/2014/04/apache-hadoop-yarn-avoiding-6-time-consuming-gotchas/ ?