Search code examples
pythonloggingmkdirpathlib

pathlib.Path(path).mkdir(), not creating directory


I am trying to implement logging for my flask project on a virtual machine (running Ubuntu-16.04). I have the following function for creating a new directory.

def mkdir_p(path):
    try:
        pathlib.Path(path).mkdir(parents=True, exist_ok=True)
    except FileExistsError as exc:
        raise

And the following file handler inheriting from RotatingFileHandler.

class MyRotatingFileHandler(RotatingFileHandler):
    def __init__(self, filename, mode='a', maxBytes=0, backupCount=0, encoding=None, delay=0):
        mkdir_p(os.path.dirname(filename))
        RotatingFileHandler.__init__(self, filename, mode, maxBytes, backupCount, encoding, delay)

This all works fine when registering a new logger at runtime, on my local machine, but when I try and run this same code on an azure instance, the path and file I am passing in ('log/error.log'), isn't created.

I have made sure the user running the code has write permissions set on the directory. I really can't think of any other reason why this might be happening though.


Solution

  • For anyone it might help, the following link may provide some useful explanation, but in essence, when using external application server to serve your application (as opposed to the dedicated Flask runtime, Werkzeug) you need to make sure your logger definitions occur before you register your Flask app, as follows.

    # add logger
    logger = logging.getLogger('flask.app')
    ...
    # register flask
    app = Flask(__name__)
    
    ...
    
    if __name__ == '__main__':
        app.run("0.0.0.0", threaded=True)
    

    When you want to configure logging for your project, you should do it as soon as possible when the program starts. If app.logger is accessed before logging is configured, it will add a default handler. If possible, configure logging before creating the application object. - Flask Docs - Logging

    I was using uWSGI to serve my application, but I was instantiating my logger after my Flask app, and inside the if __name__ = '__main__': blocks, which is why it worked fine in my local configuration but stopped working the minute I pushed it to production.