is there any way to modify the current logging to json format with few other fields added python logging

Question

I have many application code written in python django and every application is using standard python logger module and just a simple message strings are getting logged.

Now is there any way where I can add my own customer middleware or django app where when ever the logging is called in their methods it should reach my function and I will add more value to the logs, make a proper json format and then write to the log file. So that in current application developer don't need to do lots of changes except that he need to add my django app or a middle ware

Answer 1

You can use json_log_formatter. You can install it by:

pip install JSON-log-formatter==0.2.0

In django, you need to put update your logs settings like this:

LOGGING = {
    ...
    'formatters': {
        "json": {
            '()': 'json_log_formatter.JSONFormatter',
        }
    },
    ...
}

And use it in your code like this:

import logging

logger_name = 'some_name'
logger = logging.getLogger(logger_name)

logger.info('Sign up', extra={'referral_code': '52d6ce'})

Here the extra parameter sent through here will be rendered in the log like this(from documentation):

{
    "message": "Sign up",
    "time": "2015-09-01T06:06:26.524448",
    "referral_code": "52d6ce"
}

Override Json Log Formatter

You can override json_log_formatter.JSONFormatter class to add extra info like IP address if needed. Like this:

import json_log_formatter


class CustomJsonFormatter(json_log_formatter.JSONFormatter):

    def json_record(self, message, extra, record):
        request = extra.pop('request', None)
        if request:
            extra['IP_ADDRESS'] = request.META.get('HTTP_X_FORWARDED_FOR')  # or other ways to get ip
        return super(CustomJsonFormatter, self).json_record(message, extra, record)

# usage
logger.info('Sign up', extra={'referral_code': '52d6ce', 'request': request })  # django request object
# Django Settings
     'formatters': {
        "json": {
            '()': 'path.to.CustomJsonFormatter',
        }