i started to do tests on aws emr some weeks ago, but i'm having a random problem. I start a cluster from console like this:
aws emr create-cluster --name "Prueba zookeeper y giraph" --use-default-roles --ec2-attributes KeyName=keyPairAmazonJose --applications Name=MapR,Args=--edition,m3,--version,4.0.2 --ami-version 3.3.2 --instance-groups InstanceGroupType=MASTER,InstanceCount=1,InstanceType=m3.xlarge InstanceGroupType=CORE,InstanceCount=1,InstanceType=m3.xlarge
And the cluster starts successfully. I can login into the master through the master public dns name. The problems come when i do a basic operation on the cluster, like this one:
hdfs dfsadmin -report
Half of the times, the cluster throw a exception like this:
Call From ip-172-31-29-147.sa-east-1.compute.internal/172.31.29.147 to ip-172-31-29-147.sa-east-1.compute.internal:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
As far as I know, I’m using the cluster "as is", without major changes on it, so, this command should always work, right?
It appears to be that is just a matter of time. When the cluster started,
hdfs dfsadmin -report
didn't work. But if we expect a little longer, it works perfectly fine (we can watch this through jps).
The answer from Amazon forums, that didn't add any information, just suggest me to switch the type of cluster that i was using, here.