Search code examples
pythonpython-logging

python logging module output not working in multiprocessing process


I can't get logging to work correctly until I initialize logging for a second time in my python module. There is no output (printing works fine), and if I manage to get output it is not applying the formatting I specified with

logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')

I've tried adding test log entries in my __init__ which display as expected. However, I cannot log anywhere else in my class. After trying different locations for my test entries and log init, I can log in my __init__ but not any method afterwards.


Solution

  • On a whim I tried moving initialization from __init__ to the run method specified in my calling parents mp.Process() command which worked.

    self._application = Application(self.outgoing_queue, self.incoming_queue, self.username, application_cores, log_level=logging.DEBUG)
    self._application_process = mp.Process(target=self._application.run)
    self._application_process.start()
    

    My conclusion is that when the class object is handed off to mp.Process the self.log variable seems to either get clobbered or needs a "fresh" handle in the new process.

    Since moving the log initialization to take place after the new process is spawned, all logging is working as expected.