Docker execute can't find file within container

Question

I've set up a container to run some python scripts, but when I attempt to run them from the host with 'docker exec' it throws an error -

OCI runtime exec failed: exec failed: container_linux.go:370: starting container process caused: no such file or directory: unknown

but, if I open a shell to the container, I can see my script is clearly there, and can I can even run it.

DockerFile:

FROM python:3.7.1

WORKDIR /regex_locator
COPY ./py_code/req.txt ./proj/req.txt
RUN pip install -r ./proj/req.txt

CMD ["/bin/bash"]

Makefile:

    MAJOR?=0
MINOR?=1

VERSION=$(MAJOR).$(MINOR)

#vars
IMAGENAME=regex_locator
IMAGEFULLNAME=${IMAGENAME}-${VERSION}
CONTAINERNAME=${IMAGEFULLNAME}-TESTCONT

.PHONY: help build run all

help:

        @echo ""
        @echo "Makefile commands:"
        @echo "build"
        @echo "push"
        @echo "all"

.DEFAULT_GOAL := all

build:
        @docker build -t regex_locator-img .

run:
        @docker run -dit --name regex_locator-test_cont regex_locator-img
        @docker cp ./py_code/  regex_locator-test_cont:/regex_locator/proj/
        @docker exec -it regex_locator-test_cont /regex_locator/proj/py_code/run_tests.py


all: build run

Thanks in advance!

Answer 1

I'd recommend using an ordinary Python virtual environment here. This uses tooling built into the standard Python distribution (so it's available on almost every Unix-like host; Docker would be an additional installation) and doesn't require administrator-level permissions. Since this uses the host filesystem normally, you don't need to copy files around, manually map directories, worry about wrong-architecture native libraries, and so on.

VENV := venv

# Create the virtual environment
$(VENV)/bin/python3:
        python3 -m venv "$(VENV)"

# Any application stub installed by a package
$(VENV)/bin/pytest: requirements.txt $(VENV)/bin/python3
        "$(VENV)/bin/pip" install -r "$<"

# Run the tests
.PHONY: test
test: $(VENV)/bin/pytest
        "$(VENV)/bin/python3" ./py_code/run_tests.py

If you need to build a Docker image, it generally should be self-contained. Do not bind-mount or docker cp the code into the image; COPY it in, so that it can be docker run without also having a copy of the source code separately. You can then docker run an alternate command, without needing the docker exec debugging tool.

REGISTRY := docker.io
IMAGE := me/regex-locator
TAG := latest
DOCKER_IMAGE := $(REGISTRY)/$(IMAGE):$(TAG)

.PHONY: docker-build docker-push docker-run docker-test

docker-build: .docker-build
.docker-build: Dockerfile requirements.txt app/main.py ...
        docker build -t "$(DOCKER_IMAGE)" .
        touch "$@"

docker-push: docker-build
        docker push "$(DOCKER_IMAGE)"

docker-run: docker-build
        docker run --rm -p 8000:8000 "$(DOCKER_IMAGE)"

docker-test: docker-build
        docker run --rm "$(DOCKER_IMAGE)" ./proj/run_tests.py

This requires you to make sure your image actually is self-contained -- it shouldn't be an empty shell that could eventually run something.

FROM python:3.7.1

WORKDIR /regex_locator
COPY ./py_code/req.txt .  # no particular need for a subdirectory
RUN pip install -r req.txt
COPY ./py_code .          # make sure to copy the rest of the application too
CMD ./main.py             # and set a useful command for the container to run

Your unit tests shouldn't have complex dependencies. (Use an in-memory database like SQLite or mock other dependencies: tests that are 80% functional and 100% easy to run are much better than tests that are 100% functional but very hard to run.) It shouldn't make a different whether they run in the container or in a virtual environment. In that case you can run unit tests on the host, and if they pass, build and publish your image.

make test         # in a virtual environment
make docker-push  # building the image along the way