How to use Python's RotatingFileHandler

Question

I'm trying to do a test run of the logging module's RotatingFileHandler as follows:

import logging
from logging.handlers import RotatingFileHandler

# logging.basicConfig(filename="example.log", level=logging.DEBUG)

logger = logging.getLogger('my_logger')
handler = RotatingFileHandler("my_log.log", maxBytes=2000, backupCount=10)
logger.addHandler(handler)

for _ in range(10000):
    logger.debug("Hello, world!")

However, with logging.basicConfig line commented out, the resulting my_log.log file contains no data:

If I comment in the line with logging.basicConfig(filename="example.log", level=logging.DEBUG), I get the expected my_log.log files with numbered suffixes. However, there is also the example.log which is a (relatively) large file:

How can I set up the logging so that it only generates the my_log.log files, and not the large example.log file?

Answer 1

Python provides 5 logging levels out of the box (in increasing order of severity): DEBUG, INFO, WARNING, ERROR and CRITICAL. The default one is WARNING. The docs says, that

Logging messages which are less severe than lvl will be ignored.

So if you use .debug with the default settings, you won't see anything in your logs.

The easiest fix would be to use logger.warning function rather than logger.debug:

import logging
from logging.handlers import RotatingFileHandler

logger = logging.getLogger('my_logger')
handler = RotatingFileHandler('my_log.log', maxBytes=2000, backupCount=10)
logger.addHandler(handler)

for _ in range(10000):
    logger.warning('Hello, world!')

And if you want to change logger level you can use .setLevel method:

import logging
from logging.handlers import RotatingFileHandler

logger = logging.getLogger('my_logger')
logger.setLevel(logging.DEBUG)
handler = RotatingFileHandler('my_log.log', maxBytes=2000, backupCount=10)
logger.addHandler(handler)

for _ in range(10000):
    logger.debug('Hello, world!')