django database query log linenumber

Question

I am logging my database queries in Django along with the pathname and linenumber.

Right now i am getting these logs:

07/Dec/2018 14:25:00 DEBUG django.db.backends utils **/Users/XXXXX/.idea/lib/python2.7/site-packages/django/db/backends/utils.py:89** (0.340) SELECT "metadata"."metaname", "metadata"."description", "metadata"."attributes" FROM "metadata" WHERE "metadata"."metaname" = 'date_type'; args=('date_type',)

For all queries, I am getting the same path and line number. Is there any way I can capture the line number from my main application instead of the one from utils.

Current logging Implementation:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'console': {
            'level': 'DEBUG',
            'class': 'logging.StreamHandler',
            'formatter': 'color'
        },
    },
    'loggers': {
        'django.db.backends': {
            'handlers': ['console'],
            'level': 'DEBUG',
            'propogate': True,
        }
    }
}

Using python 2.7 and django 1.9

Answer 1

I'm guessing that you're trying to determine which lines in your application are responsible for running which queries.

One way to achieve this would be to create a custom handler that prints out the current stack at the point where Django logs the query. That would allow you to see which line in your application is executing.

You could create a custom handler such as:

import logging
import traceback

class StackInfoHandler(logging.StreamHandler):

    trim = 5

    def emit(self, record):
        super(StackInfoHandler, self).emit(record)
        stack = ''.join(
            str(row) for row in traceback.format_stack()[:-self.trim]
        )
        self.stream.write(stack)

And then in your logging config, you could just switch the handler class to use the StackInfoHandler:

'handlers': {
    'console': {
        'level': 'DEBUG',
        'class': 'my.package.StackInfoHandler',  # Reference the custom handler
        'formatter': 'color'
    },
},

Note that the StackInfoHandler trims 5 lines off the stack so that it doesn't show you stack frames from the logging framework itself. You might need to tweak this number (5 works for me locally).