Search code examples
pythonamazon-web-servicesaws-glue

Optional job parameter in AWS Glue?


How can I implement an optional parameter to an AWS Glue Job?

I have created a job that currently have a string parameter (an ISO 8601 date string) as an input that is used in the ETL job. I would like to make this parameter optional, so that the job use a default value if it is not provided (e.g. using datetime.now and datetime.isoformatin my case). I have tried using getResolvedOptions:

import sys
from awsglue.utils import getResolvedOptions

args = getResolvedOptions(sys.argv, ['ISO_8601_STRING'])

However, when I am not passing an --ISO_8601_STRING job parameter I see the following error:

awsglue.utils.GlueArgumentError: argument --ISO_8601_STRING is required


Solution

  • Porting Yuriy's answer to Python solved my problem:

    if ('--{}'.format('ISO_8601_STRING') in sys.argv):
        args = getResolvedOptions(sys.argv, ['ISO_8601_STRING'])
    else:
        args = {'ISO_8601_STRING': datetime.datetime.now().isoformat()}