As part of EMR Cluster version upgrade to 5.20.0 . All the frameworks in the cluster Big Data Frameworks and AWS Services versions got upgraded to latest versions.
For example :
Spark 2.3.2 to Spark 2.4.0
Presto 0.212 to 0.214
While testing the stability and version compatibility with the application we got 1 major issue.
i.e., aws cli is not working because of aws config file in the cluster is getting created with region name as null.
Error in the config file:
s3 = signature_version = s3v4
region = null
Given i am a Developer or Tester when i create a new EMR Cluster with Jenkins pipeline the error above should be resolved and aws cli should work without any failure.
I am having the same problems, on this thread on the aws forum they say that this error could be fixed running:
aws configure set default.region eu-west-1
in the bootstrap script.
See: https://forums.aws.amazon.com/thread.jspa?messageID=883372#883372