I am using docker for many different services and tools. I run a docker stack deploy -c docker-compose.yml --with-registry-auth stack_name. On the swarms themselves, only one or two of the nodes will have them images pulled and not on the others. I thought that the deploy causes all nodes to pull so that the images exist everywhere. The error that then occurs is a no such image because it wasnt pulled on that particular node. I have been looking around for help and i see many pages about how it already does this normally. Am I missing something that is causing this, any help is helpful.
I finally figured out what the problem is. When deploying a job the token it uses only stays active for however long the jobs is running. So in my script in my gitlab-ci file, i always at least pulled the image on the first node so it always worked there. This made it so at least one node had the image. To get them on the other nodes i had to add a sleep so that the other nodes had enough time to pull the image. It was a race condition, the token became useless after the job ended and couldnt pull any images.