I use docker-compose for running my containers in docker. I have two services - one of celerybeat and other the web (I have many others but considering only these services because they contain my problem).
docker-compose.yml file looks like this:
.
.
.
celerybeat:
image: web-image
volumes:
- /home/ubuntu/celerybeat:/code/celerybeat
command: >
/bin/ash -c "su -m celery -c 'celery -A <application_here> beat -s /code/celerybeat/celerybeat-schedule'"
web:
image: web-image
volumes:
- /home/ubuntu/celerybeat:/code/celerybeat
command: >
<some_command_to_run_server>
In my Dockerfile I have added these commands for proper permissions
RUN mkdir celerybeat
RUN touch celerybeat/celerybeat-schedule
RUN chown -R celery:celery celerybeat
Note: In my compose file structure written above I have provided volume mounts for both containers (but actually I am using one at a time), for the ease of not writing the compose files again and again.
The problem is actually here only. Technically the volume mount should be provided only in the celerybeat service. When I write volume mount for celerybeat-schedule in the celerybeat docker service I get permission denied. Whereas when I write the volume mount command in the web service celerybeat service starts happily. What is happening here can anyone explain me? I need a fix for this.
Order of operations - docker build then docker run (with docker-compose up counting as a docker run.
When you mount a volume, the files and folders in that volume are owned by root. Having RUN chown -R celery:celery celerybeat would work if you didn't mount a volume. When you bind mount the volume in your docker run/docker-compose up, anything that exists at /code/celerybeat is overwritten, including permissions.
So when you run celerybeat as root, you're good in this case. If you run it as the celery user as you have tried, that user can't access /code/celerybeat because as a bind mount volume, it's owned by root.
Instead of chowning the directory in your Dockerfile, run chown as part of an entrypoint script. Something like:
#!/bin/bash
chown -R celery:celery celerybeat
/bin/ash -c "su -m celery -c 'celery -A <application_here> beat -s /code/celerybeat/celerybeat-schedule'"
This script, and thus chown, execute after the bind mount, where RUN chown -R celery:celery celerybeat executes before the bind mount, and is overwritten by it.