Logs are printed twice using faust

Question

Using python Faust, all of my logs are doubled.

As loggers are hierarchal, i want to prevent the downstream logger father.son from logging anything and propagate all the logs to upstream father logger.

I tried:

1 - setting disable_existing_loggers to True in the dictConfig when main is called

2 - adding filter (inside a handler) to a specific logger name using the dictConfig:

class LogPreventingFilter(logging.Filter):
    def filter(self, record):
        return False

logging_config = {
    'version': 1,
    'disable_existing_loggers': True,
    'filters': {
        'log_preventing_filter': {
            '()': LogPreventingFilter
        }
    },
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
            'level': 'DEBUG',
            'filters': ['log_preventing_filter']
        }
    },
    'loggers': {
        'father.son': {
            'level': 'DEBUG',
            'handlers': ['console']
        }
    }
}

dictConfig(logging_config)

3 - adding the same logging_config to faust's app configurations logging_config (with and without 'disable_existing_loggers': True) - in that case i could see logs from the logger i mentioned (father.son) but not from the father, even after i added the father logger to to faust config), nor logs from any other non-hierarichial logger ( like getLogger("some_name_not_related_to_father")).

ill mention that i tried to remove the filter and handler and it didnt make any change.

Worth to mention that the logs start to double only after i call faust's app.main().

also worth mentioning that the loggers that i want to actually log data are all from imported modules, not part of the code that is visible for faust. only father.son logger is.

my code:

some.imported.module

father_logger = logging.getLogger("father")
some_other_logger = logging.getLogger("not_hierarchiali_related")

class MyGenericClass:
   // add filters and handlers and do some generic stuff 

stream_processing_module

son_logger = logging.getLogger("father.son")

async def processing_function():
   // do some stuff
   son_logger.info("log this and that")

the module using faust:

son_logger = logging.getLogger("father.son")

@app.agent
async def stream_processing():
   // call the processing_function() for the other module for each record
   son_logger.info("log some data")

if __name__ == "main":
   son_logger.info("start")
   with MyGenericClass:
      app.main()

notice that only son_logger is visible to faust. still - i want the father logger to do the printing and not the son logger.

any ideas? thanks!

Answer 1

I found in the Faust logging manuals that the application writes to an independent root_logger (you cant get it when calling logging.getLogger() on your python app) and must be set with at least one handler in the application configuration (logging_config) BEFORE starting the faust app (app.main()). If you dont set the root logger up, Faust will do that for you, and might print both the log in the python app context AND the same log in the Faust app context.