Search code examples
amazon-web-servicesdockercontainersamazon-ecsecs-taskdefinition

How can i fix this error message on ECS "STOPPED (VolumeError: Agent could not create task's volume resources)"

I am trying to update my ECS cluster by starting a new task but I keep having this message

STOPPED (VolumeError: Agent could not create task's volume resources).

Anybody has an idea of what does it mean? or how can I debug this ?

Solution

  • For this ECS task my launching method was EC2 which means my container use the real ec2 instance to run. So the problem was that those instances were full of dangling docker containers, docker volumes, docker images. To be certain of this I went on the instances (via ssh) and check memory (by using a build in Linux command)

    df

    the "df" command showed that my disk( /dev/xvda1 ) was used at 85% there was no way i could a single file on this instance

    this i clean it by using those commands

    docker ps -q --filter "status=exited" | xargs --no-run-if-empty docker rm ;
docker volume ls -qf dangling=true | xargs -r docker volume rm;
docker rmi $(docker images -f "dangling=true" -q)

    and then the rate went from 85% to 15% and everything just went fine from there

    enter image description here

    if you need more help check this article about the same issue AWS ECS “no space left on device”