Is there a way to get the logger name (e.g. module name in getLogger(__name__) ) when creating custom logging TimedRotatingFileHandler?

Question

Im trying to create a custom TimedRotatingFileHandler. For loggers that should be in a single log file, I create custom file handlers like this, to automatically create directory for log files if it doesnt exist:

class CustomTimedRotatingFileHandler(logging.handlers.TimedRotatingFileHandler):
    def __init__(self, filename: str, *args, **kwargs):
        log_path = os.path.dirname(filename) # filename passes full path, seperate path from filename
        pathlib.Path(log_path).mkdir(parents=True, exist_ok=True) # Use the seperated path to create directories for logfile
        logging.handlers.TimedRotatingFileHandler.__init__(self, log_name, *args, **kwargs)

What I wanted to do know was create a special handler, that would create a log file not from predefined name, but from the module name that I pass when creating a logger with logging.getLogger(name). Loggers with this handler would create their own log file with their own module name:

class CustomTimedRotatingFileHandler(logging.handlers.TimedRotatingFileHandler):
    def __init__(self, filename: str, *args, **kwargs):
        log_path = os.path.dirname(filename) # filename passes full path, seperate path from filename
        pathlib.Path(log_path).mkdir(parents=True, exist_ok=True) # Use the seperated path to create directories for logfile
        logger_name = # Somehow get the logger name e.g module name from __name__
        new_filename  f"{log_path}/{logger_name}.log" # create new filename for the __init__
        logging.handlers.TimedRotatingFileHandler.__init__(self, new_filename, *args, **kwargs)

Currently, to achieve same functionality, I have to create seperate handler for each logger to have them be logged in seperate files.

So the question is: how do I get the logger name inside the Handler?

Answer 1

You can add whatever parameters you need to your custom handler's __init__ method, so why not just add a logger_name parameter, which you then set when calling logger.addHandler(CustomTimedRotatingLogHandler(filename, __name__))?

If you still want to try to get the __name__ of the calling module, you can get that with the inspect module as detailed here, but this is a fragile solution and breaks encapsulation. In general a function or method shouldn't know or care who the caller was.

As an additional change, I also updated the last line to call super().__init__() instead of directly referencing the parent class.

custom_logger.py

import inspect
import logging
import logging.handlers
import os
import pathlib


class CustomTimedRotatingFileHandler(logging.handlers.TimedRotatingFileHandler):
    def __init__(self, filename: str, logger_name=None, *args, **kwargs):
        if logger_name is None:
            logger_name = inspect.currentframe().f_back.f_locals.get("__name__", "<unknown>")
        log_path = os.path.dirname(filename) # filename passes full path, seperate path from filename
        pathlib.Path(log_path).mkdir(parents=True, exist_ok=True) # Use the seperated path to create directories for logfile

        new_filename = f"{log_path}/{logger_name}.log" # create new filename for the __init__
        super().__init__(new_filename, *args, **kwargs)

my_module.py

import logging

from custom_logger import CustomTimedRotatingFileHandler

logger=logging.getLogger("")
# Use default logger_name in handler
logger.addHandler(custom.CustomTimedRotatingFileHandler("./logs/custom_log_from_module.log"))
logger.critical("Message from Module")

# Submit a logger_name to handler
second=logging.getLogger("SecondLogger")
second.addHandler(custom.CustomTimedRotatingFileHandler("./logs/custom_log_2_from_module.log", logger_name="Second_logger_Override"))
logger.debug("Message from Module")

main.py

import logging

import custom_logger
import my_module

logger = logging.getLogger()

def main():
    # Send override name to handler
    logger.addHandler(custom_logger.CustomTimedRotatingFileHandler("./logs/Log_From_Main.txt", logger_name="From_main"))
    logger.warning("Logging a message from main")
    
    # Have handler use default
    second_main = logging.getLogger("2ndMain")
    second_main.addHandler(custom_logger.CustomTimedRotatingFileHandler("./logs/FromMain2.log"))
    second_main.warning("Second message from main")

if __name__=="__main__":
    main()

And you can see the passed in logger_name value was used for the filename when available. The default case in my_module.py used "my_module", while the default case from my __main__ script was unable to get the name from the framestack and used the fallback of "_unknown_" instead.

In my opinion, it would be better to just make logger_name a required parameter, instead of trying to pull info from the framestack.