error running aws emr create-cluster from powershell

Question

The following command fails when run in a PowerShell script:

aws emr create-cluster --steps file://./monthly_step.json --ec2-attributes KeyName=ff_test,InstanceProfile=EMR_EC2_DefaultRole --release-label emr-4.0.0   --instance-groups InstanceGroupType=MASTER,InstanceCount=1,InstanceType=m3.xlarge InstanceGroupType=CORE,InstanceCount=2,InstanceType=m3.xlarge --auto-terminate

The error messages are:

Parameter validation failed:
Missing required parameter in [0]: "InstanceCount"
Missing required parameter in [0]: "InstanceType"
Missing required parameter in [1]: "InstanceGroupType"
Missing required parameter in [1]: "InstanceType"
Missing required parameter in [2]: "InstanceCount"
Missing required parameter in [2]: "InstanceGroupType"
Missing required parameter in [3]: "InstanceCount"
Missing required parameter in [3]: "InstanceType"
Missing required parameter in [4]: "InstanceGroupType"
Missing required parameter in [4]: "InstanceType"
Missing required parameter in [5]: "InstanceCount"
Missing required parameter in [5]: "InstanceGroupType"

This same command executes successfully from a standard Windows cmd prompt.

aws s3 commands work fine from PowerShell. What causes this aws emr to fail?

Answer 1

I don't have access to a similar environment for testing but from the looks of it I think PowerShell is seeing arrays in some of your parameters. Even if I am wrong about the specifics of the problem I think the solution is clear.

When you said that it works fine in cmd that is telling me that we need to tell PowerShell to stop interpreting what you are writing. Therefore I think you need to be using the stop parsing operator --%. From TechNet:

The stop-parsing symbol (--%), introduced in Windows PowerShell 3.0, directs Windows PowerShell to refrain from interpreting input as Windows PowerShell commands or expressions.

When calling an executable program in Windows PowerShell, place the stop-parsing symbol before the program arguments. This technique is much easier than using escape characters to prevent misinterpretation.

Call operator might not be required here but wouldn't hurt either. Give this a shot:

& aws emr --% create-cluster ....