Search code examples
hadoopamazon-web-servicesargumentshiveelastic-map-reduce

AWS EMR CLI - Pass Arguments to HIVE


I am using aws' emr ruby cli to generate Hadoop clusters and I am trying to include arguments to use within a HIVE script hosted elsewhere like so:

./elastic-mapreduce --create ... --args -d,DT=2013-01-26

'DT' shows up satisfactorily in my HadoopJarStep.Args array, so I try to include it in the HIVE script like so:

...

tblproperties(
  'dynamodb.table.name' = ${DT},
  ...

but I quickly get this:

Parse Error: line 8:28 mismatched input '$' expecting StringLiteral near '=' in specifying key/value property

How should I properly include the argument in my HIVE script?


Solution

  • I'm not exactly sure why your current approach isn't working, but I've been successful with:

    ./elastic-mapreduce --create ... --args "-hiveconf,DT=2013-01-26"
    

    and in the hive script:

    tblproperties(
        "dynamodb.table.name" = "${hiveconf:DT}",
        ...
    )
    

    Hope this helps.