Endpoint request timed out when starting AWS ECS task from Lambda

Question

I am trying to run an ECS task from Lambda Api gateway. But frequently getting Endpoint request timed out. I have changed the default Labmda timeout to 5 min. But still getting timeout exception sometimes. Is there any way to run the ECS task through Lambda without getting timeout ?

Here is the core python code from Lambda to run an ECS task

ecs = boto3.client(
    'ecs',
    region_name=config.AWS_REGION,
    aws_access_key_id=config.AWS_ACCESS_KEY_ID,
    aws_secret_access_key=config.AWS_SECRET_ACCESS_KEY
)

request_id = str(uuid.uuid1())

ecs.run_task(
    cluster='test-cluster',
    taskDefinition='test-task',
    startedBy=request_id,
    launchType='FARGATE',
    overrides={
        'containerOverrides': [
            {
                'name': 'test-container',
                'environment': [
                    {
                        'name': 'request_id',
                        'value': request_id
                    }
                ]
            }
        ]
    },
    networkConfiguration={
        'awsvpcConfiguration': {
            'securityGroups': [
                'sg-XXXXXXXX',
            ],
            'subnets': [
                'subnet-XXXXXXXX',
                'subnet-XXXXXXXX',
                'subnet-XXXXXXXX'
            ],
            'assignPublicIp': 'ENABLED'
        }
    }
)

Answer 1

AWS API Gateway has a hard timeout limit of 29 seconds. So regardless of the timeout limit of your lambda, the API Gateway will return a timeout if it doesn't get a response after 29 seconds.

Your lambda should carry on regardless though, this should be simple enough to verify in cloudwatch logs.

I'm not sure what you want API Gateway to return in this scenario, but it may be enough to have lambda begin the job and return a response to API gateway that the job has begun.

To do this I would have API Gateway call a Lambda that invoked the Lambda running the ECS task. To make this asynch, the invocation from Lambda to Lambda would need to be an 'Event' invocation type.

Example:

import boto3
import json

def lambda_handler(event, context):
response = client.invoke(
    FunctionName='<ecs_lambda>',
    InvocationType='Event',
    Payload=json.dumps(event)
)
return { "result": "OK" }