botocore.exceptions.NoRegionError: You must specify a region for EmrServerlessCreateApplicationOperator

Question

I am trying to create a emr-serverless application through the EmrServerlessCreateApplicationOperator but I keep facing the error botocore.exceptions.NoRegionError: You must specify a region.

I am passing the region like below:

 create_app = EmrServerlessCreateApplicationOperator(
        task_id="create_spark_app",
        job_type="SPARK",
        release_label="emr-6.6.0",
        config={"aws_access_key_id":args["aws_access_key_id"], 
                "aws_secret_access_key": args["aws_secret_access_key"], 
                "aws_session_token": args["aws_session_token"],
                "region_name":'us-east-1'}
    )

Any help is appreciated. Thank you.

Answer 1

I was able to figure it out. I passed them as environment variables.

os.environ["AWS_ACCESS_KEY_ID"]= YOUR VALUE
os.environ["AWS_SECRET_ACCESS_KEY"]= YOUR VALUE
os.environ["AWS_SESSION_TOKEN"]= YOUR VALUE
os.environ["AWS_DEFAULT_REGION"]= YOUR VALUE

This worked.