AWS Data Pipeline is not creating all slave / core instance nodes

Question

I have tried creating AWS Data pipelines using the CLI and also using the GUI. Either way, when I specify more than one slave node, it doesn't get created properly. Here is an example definition:

{
    "name": "EmrClusterForLoad",
    "coreInstanceCount": "16",
    "coreInstanceType": "r3.xlarge",
    "releaseLabel": "emr-5.13.0",
    "id": "EmrClusterForLoad",
    "masterInstanceType": "r3.xlarge",
    "region": "#{myDDBRegion}",
    "type": "EmrCluster"
},

Any suggestions or thoughts?

Answer 1

Weird. I think this may be a bug. The "fix" was to change the value of the "Resize Cluster Before Running:" from true to false. If it's not a bug, then I am not sure I understand the option.

If you are creating the pipeline via CLI, then the entry is:

"resizeClusterBeforeRunning": "false"

When I changed this value, all of the sudden the EC2 instances started to be created.