Search code examples
pythonloggingpython-logging

Merge python logging handler output


I am implementing python.logging for a project that relies on a 3rd-party software that also uses logging.

The issue is we want console output, and the logs from the two are getting double printed. Is there a way to set up a handler to "combine" the output of sub-handlers?

import logging
import sys


def third_party_use():
    print_log = logging.StreamHandler(stream=sys.stdout)
    print_log.setLevel(logging.INFO)
    print_log.name = "SubPrintLog"
    print_log.setFormatter(logging.Formatter("SubPrintLog %(name)s - %(levelname)s: %(message)s"))

    third_party_logger = logging.getLogger("ThirdParty")
    third_party_logger.addHandler(print_log)
    third_party_logger.info("I don't want this to print twice")


if __name__ == '__main__':
    print_log = logging.StreamHandler(stream=sys.stdout)
    print_log.setLevel(logging.INFO)
    print_log.name = "MyPrintLog"
    print_log.setFormatter(logging.Formatter("MyPrintLog %(name)s - %(levelname)s: %(message)s"))

    logging.basicConfig(level=0, handlers=[print_log])
    logger = logging.getLogger(__name__)

    logger.debug("This shouldn't appear")
    logger.info("This should")
    logger.warning("This definitely should")

    third_party_use()

    logger.info("Just my stuff again")

yields:

MyPrintLog __main__ - INFO: This should
MyPrintLog __main__ - WARNING: This definitely should
SubPrintLog ThirdParty - INFO: I don't want this to print twice
MyPrintLog ThirdParty - INFO: I don't want this to print twice
MyPrintLog __main__ - INFO: Just my stuff again

Is there a way to set up MyPrintLog to not reproduce output from ThirdParty (assume we have no access to modify the configuration of ThirdParty. I was hoping that the propagate flag would fix this problem, but it flows the other direction and we can't modify ThirdParty)

One solution that "works" is to effectively disable MyPrintLog when calling ThirdParty:

def turn_off_console():
    for idx, handler in enumerate(logger.root.handlers):
        if handler.name == "MyPrintLog":
            old_level = handler.level
            handler.setLevel(logging.CRITICAL)
            return idx, old_level


def turn_on_console(handle_index, level):
    logger.root.handlers[handle_index].setLevel(level)

And then wrapping each access to the third party:

    idx, old_level = turn_off_console()
    third_party_use()
    turn_on_console(idx, old_level)

Which yields the correct output:

MyPrintLog __main__ - INFO: This should
MyPrintLog __main__ - WARNING: This definitely should
SubPrintLog ThirdParty - INFO: I don't want this to print twice
MyPrintLog __main__ - INFO: Just my stuff again

But, that means I have to toggle the logger every time, which is prone to error.

EDIT:::SOLVED

class NoThirdParty(logging.Filter):
    def filter(self, record):
        return not record.name == "ThirdParty"

if __name__ == '__main__':
    print_log = logging.StreamHandler(stream=sys.stdout)
    print_log.setLevel(logging.INFO)
    print_log.addFilter(NoThirdParty()) # Do Not Reproduce Third Party!
    print_log.name = "MyPrintLog"
    print_log.setFormatter(logging.Formatter("MyPrintLog %(name)s - %(levelname)s: %(message)s"))
    
    ...

yields:

MyPrintLog __main__ - INFO: This should
MyPrintLog __main__ - WARNING: This definitely should
SubPrintLog ThirdParty - INFO: I don't want this to print twice
MyPrintLog __main__ - INFO: Just my stuff again

This Solution solves the same problem by applying the filter to the 3rd Party logger


Solution

  • class NoThirdParty(logging.Filter):
        def filter(self, record):
            return not record.name == "ThirdParty"
    
    if __name__ == '__main__':
        print_log = logging.StreamHandler(stream=sys.stdout)
        print_log.setLevel(logging.INFO)
        print_log.addFilter(NoThirdParty()) # Do Not Reproduce Third Party!
        print_log.name = "MyPrintLog"
        print_log.setFormatter(logging.Formatter("MyPrintLog %(name)s - %(levelname)s: %(message)s"))
        
        ...
    

    yields:

    MyPrintLog __main__ - INFO: This should
    MyPrintLog __main__ - WARNING: This definitely should
    SubPrintLog ThirdParty - INFO: I don't want this to print twice
    MyPrintLog __main__ - INFO: Just my stuff again