I have an array of objects which I want to pass to a Lambda function iteratively. However, I also need to have an ECS Task running for each Lambda function I start.
I found that I would need to have an AWS Step Function where I iterate through an array of JSON inputs. For each input, I would have to start an ECS Task, wait for it to be in a RUNNING state, then move to next step where I invoke a Lambda function. In my case, the ECS Task itself does not return anything. It is supposed to stay running because the Lambda function uses it.
Currently, I have it so that the ECS Task starts but it stays stuck in the starting the ECS Task step because it does not return anything. How would I be able to wait for it be in a RUNNING state before moving to the next step?
Current Step Function definition:
{
"StartAt": "Iterate",
"States": {
"Iterate": {
"Type": "Map",
"Iterator": {
"StartAt": "Start ECS Task",
"States": {
"Start ECS Task": {
"Type": "Task",
"Resource": "arn:aws:states:::ecs:runTask",
"Parameters": {
"LaunchType": "FARGATE",
"Cluster": "<cluster-arn>",
"TaskDefinition": "<task-definition-arn>",
"NetworkConfiguration": {
"AwsvpcConfiguration": {
"Subnets": [
"<subnet-id>"
],
"AssignPublicIp": "ENABLED"
}
}
},
"Next": "Invoke Lambda function"
},
"Invoke Lambda function": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"FunctionName": "<lambda-function-arn>",
"Payload": {
"Input.$": "$"
}
},
"End": true
}
}
},
"End": true
}
}
}
Instead of using an integrated AWS service (i.e. AWS Step Function), I created my own script for starting ECS Tasks using aws-sdk v3 for JavaScript. I specifically used the waitUntilTasksRunning function from the @aws-sdk/client-ecs package to wait for the ECS Task to be in a RUNNING state.
Read more about it from the documentation
Sample function for running and waiting for the ECS Task to be in a RUNNING state (TypeScript) note: not all parameters in the functions are required, check the documentation for that:
import { ECSClient, RunTaskCommand, waitUntilTasksRunning } from '@aws-sdk/client-ecs'
const startAndWaitUntilECSTaskRunning = async (region: string, clusterARN: string, launchType: string, subnets: Array<string>, taskDefinition: string, securityGroups: Array<string>, assignPublicIp: string) => {
var ecsClient = new ECSClient({ "region": region })
var runTaskCommand = new RunTaskCommand({
cluster: clusterARN,
taskDefinition: taskDefinition,
launchType: launchType,
networkConfiguration: {
awsvpcConfiguration: {
assignPublicIp: assignPublicIp,
subnets: subnets,
securityGroups: securityGroups
}
}
})
var ecsTask = await ecsClient.send(runTaskCommand)
var taskArn: string | undefined = ecsTask.tasks?.[0].taskArn
if (typeof taskArn !== "string") {
throw Error("Task ARN is not defined.")
}
var waitECSTask = await waitUntilTasksRunning({"client": ecsClient, "maxWaitTime": 600, "maxDelay": 1, "minDelay": 1}, {"cluster": clusterARN, "tasks": [taskArn]})
// note: there are multiple waitECSTask states, check the documentation for more about that
if (waitECSTask.state !== 'SUCCESS') {
// your code to handle this
} else {
// your code to handle this
}
}