Search code examples
pythonmultithreadingdocker

Thread long running python task, doesn't work inside Docker


I am trying to deploy my website. More or less I made everything connected inside a docker and db is working fine with my python code.

Inside my code, though, I have a long-running thread. That should update db every 15 min. this is the code.

import time
import threading
import traceback


class PeriodicTask:
    def __init__(self, delay, task):
        self.delay = delay
        self.task = task
        self.stop_event = threading.Event()
        self.task_thread = None

    def signal_handler(self, sig, frame):
        print(f"Signal {sig} received. Stopping...")
        self.stop_event.set()

    def every(self):
        next_time = time.time() - self.delay
        # signal.signal(signal.SIGINT, self.signal_handler)
        try:
            while not self.stop_event.is_set():

                while time.time() < next_time and not self.stop_event.is_set():
                    time.sleep(5)  # Check stop_event every 5 seconds
                if self.stop_event.is_set():
                    break

                try:
                    self.task()
                except Exception:
                    traceback.print_exc()
                    # in production code you might want to have this instead of course:
                    # logger.exception("Problem while executing repetitive task.")
                time.sleep(max(0, next_time - time.time()))

                # Skip tasks if we are behind schedule:
                next_time += (time.time() - next_time) // self.delay * self.delay + self.delay
                # print(next_time)
        finally:
            # signal.signal(signal.SIGINT, signal.SIG_DFL)
            print(f"Loop {self.delay} has been stopped safely.")

    def start(self):
        self.task_thread = threading.Thread(target=self.every)
        self.task_thread.start()

    def stop(self):
        self.stop_event.set()
        if self.task_thread is not None:
            self.task_thread.join()

It doesn't work properly inside docker for some reason. Just running on my computer, it works, but inside docker it doesn't.

I tied to insert less waiting time, like 5 seconds, but even though after it starts, it doesn't print me right away requested data, and every 5 seconds it not working. It's not printing anything. It only starts to work a bit when I make a request to the link. (this is the backend on fastapi) it prints like 5 - 9 data logs and then nothing.

I have another 2 containers running. 1 with nginx and another with PostgreSQL.

What can I try to do? Thank you very much!


Solution

  • Consider setting the PYTHONUNBUFFERED=1 environment variable set in your Docker Compose environment, or invoking your script with the -u option, like python -u script.py.

    References:

    1. https://docs.python.org/3/using/cmdline.html#envvar-PYTHONUNBUFFERED
    2. https://docs.python.org/3/using/cmdline.html#cmdoption-u
    3. https://stackoverflow.com/a/59812588/2635528