Logging from multiple modules in package not working from __main__

Question

I am trying to implement logging in my project and want all modules to log to the same file. I managed to get this to work if I set up the logger in init, but not if I set it up in main. When I set it up in main it only logs the statements from main and not other modules.

I want to set it up in main so that I can place the configuration of the logger in dictConfig in a config.py file. When I do this from init something goes wrong with the import statements.

Here is what I have in ___main____.py:

import logging
from logging.config import dictConfig
import config as cfg

if __name__ == '__main__':
    dictConfig(cfg.logging)
    logger = logging.getLogger()
    logger.info('Completed configuring logger()!')
    main()

In config.py:

logging = dict(
    version = 1,
    formatters = {
        'f': {'format':
              '%(asctime)s %(name)-12s %(levelname)-8s %(message)s'}
        },
    handlers = {
        'h': {'class': 'logging.StreamHandler',
              'formatter': 'f',
              'level': logging.INFO},
        'r': {'class': 'logging.handlers.RotatingFileHandler',
                'filename': 'data/logger.log',
                'formatter':'f',
              'maxBytes': 10000,
              'backupCount':5},
        },
    root = {
        'handlers': ['h', 'r'],
        'level': logging.INFO,
        },
)

In backend.py:

import logging
logger = logging.getLogger(__name__)

class Backend(object):

    def __init__(self, dbi):
        self._dbi = dbi

    def getDimensionTableColumns(self, table_name):
        logger.warning('still not working')

The output in my logger.log file and terminal is:

2018-03-07 09:48:00,858 root         INFO     Completed configuring logger()!

And I know that the getDimensionTableColumns is running because if I put a print statement it outputs to the terminal.

Could someone please explain what is going wrong and why?

Answer 1

you are using two different loggers: the root logger (which is configured) in your __main__ module (the one you get with logger = logging.getLogger()) and a logger called __name__ = 'backend'(which is not).

you can either use the default logger in backend.py using

logger = logging.getLogger()  # no __name__ !

or you could configure a named logger and use that in both modules.