Search code examples
pythondockerdocker-volumefile-copying

Docker with python to copy file specified, unzip and do actions


Setup

  • Project
    • src
      • main.py
    • Dockerfile

Dockerfile (raw, needs to be revamped)

FROM python:3
ADD src/main.py /
RUN chmod +x main.py
RUN /usr/local/bin/python -m pip install --upgrade pip
COPY . /opt/app
RUN pip install -r /opt/app/requirements.txt
ADD / /usr/local
ENTRYPOINT [ "python",  "./main.py" ]

main.py

if __name__ == '__main__':
   if len(sys.argv) == 2:
      main(sys.argv[1])

def main(logs_file_archive):
    unzip_logs(logs_file_archive)  # unzips all logs from the provided path to the folder with the same name as archive/Extracted directory
    create_csv_files()  # creates CSV files needed to plot graph
    process_files()  # populate CSV files generated with the right data
    plot(logs_file_archive)  # build this data representation

Actual/desired behaviour

Actual:

2022-01-17T22:05:31.547047838Z   File "//./main.py", line 214, in <module>
2022-01-17T22:05:31.547210046Z     main(sys.argv[1])
2022-01-17T22:05:31.547259438Z   File "//./main.py", line 187, in main
2022-01-17T22:05:31.547670294Z     unzip_logs(logs_file_archive)
2022-01-17T22:05:31.547732344Z   File "//./main.py", line 54, in unzip_logs
2022-01-17T22:05:31.548296998Z     with zipfile.ZipFile(file_path, "r") as zip_ref:
2022-01-17T22:05:31.548350898Z   File "/usr/local/lib/python3.10/zipfile.py", line 1240, in __init__
2022-01-17T22:05:31.549638566Z     self.fp = io.open(file, filemode)
2022-01-17T22:05:31.549692977Z FileNotFoundError: [Errno 2] No such file or directory: '/Users/user/PerfReader/misc/archive.zip'

No such file or directory: '/Users/user/PerfReader/misc/archive.zip' is expected well... because there is no such file in the Docker machine.

Desired: container runs using Dockerfile, data processes, the plot is displayed real-time or saved as a file and transferred to host

Question/issue description

  1. I am not entirely sure how to transfer the file specified to the Docker container. I read https://docs.docker.com/storage/volumes/ but it doesn't provide any examples so I seek examples of how volumes can be mounted.
  2. Provided that my plot() in main.py does plot data properly, what are my options on displaying this data (output of the whole exercise is the plot)? Can I display the plot in real-time from Docker? Or is my only option is to generate a plot and then transfer it back to the host machine using matplotlib.pyplot.savefig?

Solution

  • First Question:

    There are multiple ways to access the host files. You can use the copy or add commands like what you are doing in your Docker file and the file will be copied during building the image. Furthermore, you can use the binding mount, it allows you to access the host directory that was binded, something like you are running the container in this directory.

    Second Question:

    Docker doesn't support GUI or accessing the host display. However, you can allow it to do so using Xauth. Consider the following link for the steps.

    Either way, I don't encourage using Docker for what you are doing especially the plotting part, python virtual environment would be more than enough.