Search code examples
dockerstorage

Docker failed to copy files: copy file range failed: no space left on device on Rebuild, only a few lines changed

Hello everyone I am struggling with an issue when rebuilding a project that I have hosted on a VPS.

It is a platform where I upload videos, and I just made a few changes to the platform (after a few months of use), so I changed only a few lines of code (my point is that I didn't dramatically increase the size of my project), However after trying to build my project again I get a message telling me that I have no space left.

sudo docker compose up --build -d

And I get this message:

failed to copy files: copy file range failed: no space left on device

As you can see I have a "decent" amount of storage left, so if I changed a few lines then I shouldn't have any problem rebuilding my project and adding a few lines of code right?

enter image description here

I've already pruned my system as I saw that was a very common "solution" for similar questions. enter image description here

I think the problem is related to my docker-compose.yaml file and the volumes. I am persisting the videos uploaded to my app by using a volume and from my understanding every time I run: docker compose build the data stored in that volume is copied to the new container and there could be the problem.

If that is true, then If I'm using about 45GB of storage then when creating a new container I'll be copying those 45GB of storage to it and I'm going to run out of my storage (about 78GB).

Is there any solution to this? Like maybe copying my volume's content to a different folder and emptying the one in use, then rebuild with 0GB of storage to copy, and finally copy the content to the new container's volume?

That's what I'm thinking about right now but it sounds like a lot of trouble because:

  • Active users will think they lost their videos (downtime or need for a maintenance mode)
  • I will have to repeat the process even if I just updated a line
  • Its very slow

I have also been thinking about amazon S3 or something like that in order to not use volumes and have fast re-build times but I don't think my budget will allow me to pay for my VPS + Amazon's S3 + other expenses.

Another "solution" that I can think of is increasing the storage but then what will I do when I have 500GB of storage in videos? will I have to upgrade to 1000GB even though I am just using half of the storage?

enter image description here

Solution

  • it turns out that the issue was caused by a mistake I made. Recently I noticed that transferring context size was so large and it shouldn't.

    As I shared before, my volumes are stored in the same folder as my code (in ./docker-data) so when the Dockerfile runs

    COPY . .

    Everything is copied incluing docker-data (which I was expecting to be ignored because its a volume), now I know its not ignored and all I needed to do was add that folder to .dockerignore

    .dockerignore file:

    node_modules/*
docker-data/*

    I've tested it and now I don't run out of space and my build time has increased by a lot and my volumes are not lost.

    Hope this helps anyone in the same situation.