Docker: create requirements.txt in dockerfile utility container

Question

Summary: used a bind mount to populate a requirements.txt file using a python version I don't have locally, but the requirements.txt file on my local machine always remains empty.

Details:

I wanted to start a Python project using the docker hub image for Python 3.12.

I tried to use my requirements.txt file from a previous project, but the requirements.txt file is incompatible with Python 3.12.

So, I wanted to install the modern versions of the packages using Python 3.12, which I don't have on my computer. I thought I could build a utility container to:

1. install python in the container
2. install the dependencies in the container
3. create a requirements.txt file in the container
4. freeze the dependency list to the requirements.txt
5. Populate the requirements.txt on my host computer using a bind mount.

But the requirements.txt file remains empty. I've tried a few things, but my current dockerfile is

FROM python:3.12.0-bookworm
WORKDIR /backend
# Install dependencies:
RUN pip3 install --upgrade pip
RUN pip3 install pandas
<install more packages>

ENTRYPOINT [ "python3", "-m", "pip", "freeze", "-r", "requirements.txt"]

and my docker-compose file is

version: '3.8'
services:
  python-deps:
    working_dir: /backend
    build:
      context: ./backend
      dockerfile: python.Dockerfile
    volumes:
        - ${PWD}/backend/requirements.txt:/backend/requirements.txt
    tty: true 
    stdin_open: true
    profiles:
      - startup-backend

The output of docker-compose run python-deps is:

docker-compose run python-deps
[+] Building 0.0s (0/0)  docker:desktop-linux
[+] Building 0.0s (0/0)  docker:desktop-linux
## The following requirements were added by pip freeze:                                                              
pandas==2.1.1
<more packages>

I can copy the content of the requirements.txt file as it's displayed in the terminal. I can paste the content into the requirements.txt file I have on my local machine. Then I can run other docker containers which copy that requirements.txt file. So it's useful to me in its current form :P

But, the requirements.txt file is empty once the container finishes.

After reading this, I switched the bind mount to

volumes:
  - ${PWD}/backend/:/backend/`

but still the same issue. Any ideas on what concept/syntax I've got wrong here?

Answer 1

When you run pip freeze, you need to redirect its output somewhere. The pip freeze -r option causes Pip to read a requirements file, rather than write one. You can't do this using the exec (JSON-array) form of ENTRYPOINT or CMD, but you can do this using the shell form, and particularly for this setup there's not a specific reason to avoid this.

CMD python3 -m pip freeze > requirements.txt

---

Hopefully you have your list of dependencies written out somewhere else as well, like a pyproject.toml file. (Other tools like Poetry or Pipfile would work similarly.) To create the lock file, you fundamentally need to do three things:

1. Create a new empty virtual environment;
2. Install your package dependencies, with non-specific versions; and
3. Dump out the new lock file.

This is few enough steps that you could imagine doing it at the command line, and if this is only a one-off process then I might consider doing it with a temporary container and not a Dockerfile. This might look like

docker run \
  --rm \
  -v "$PWD:/app" \
  -w /app
  python:3.12 \
  sh -c 'python3 -m venv /venv; /venv/bin/pip install .; /venv/bin/pip freeze > requirements.txt'

This runs a container, --rm deleting itself on completion, -v mounting the current directory into the container and -w making that directory the current directory when the container runs. We bundle the three commands into a single sh -c invocation to be the one process the container runs. The virtual environment is in a /venv directory in the container temporary filesystem and is deleted with the container.