Search code examples
amazon-web-servicesamazon-ecs

AWS ECS Service Concurrency control


We want to run max of 10 long running task in parallel and if the 11, 12 task is invoked it will have to wait for the ongoing execution and must be replaced if any of 10 running once complete.

From documentation of ECS it is not clear this behavior is achievable - if task invoked will be queued and then triggered once if any 1 of 10 completes, can any one please share if this is achievable ? Our load of request is spiky at intervals and we want to limit only 10 task to execute so downstream systems don't get impacted.

Since it is long running lambda is not option -- we want to achieve lambdas async concurrency config via ECS is it possible ?


Solution

  • ECS doesn't support this directly, without involving some other AWS services. I suggest doing the following:

    • Add your tasks to an SQS queue
    • Configure ECS service auto-scaling, with a scaling configuration that tracks the SQS queue's ApproximateNumberOfMessagesVisible metric, and a Max count of 10.
    • Configure your ECS task to pull messages from SQS and process them.