H2o Number of Executors not working

Question

I start the sparkling-shell with the following command.

./bin/sparkling-shell --num-executors 4 --executor-memory 4g --master yarn-client

I only ever get two executors. Is this an H2o problem, YARN problem, or Spark problem?

Mike

Answer 1

I got over the problem by changing the following four values in cloudera manager

Setting                                  Value
yarn.scheduler.maximum-allocation-vcores 8  
yarn.nodemanager.resource.cpu-vcores     4 
yarn.nodemanager.resource.cpu-vcores     4 
yarn.scheduler.maximum-allocation-mb     16 GB