In my Python service I'm using a logging library that's built in-house and not compatible with Python's logging module. While using this library, I want to add additional info to the logs like lineno and funcName. I have been using the inspect library to get this info like below:
frame = inspect.currentframe().f_back
frame_info = inspect.getframeinfo(frame)
frame_info.lineno
However as per discussions in Python inspect.stack is slow, these methods of using inspect library are somewhat expensive since they access filesystem. We notice a performance decrease in our service because of this (although not nearly as bad as when we were using inspect.stack()[0] earlier).
I noticed Python's native logging module is able to get funcName and lineno methods efficiently. These are exposed through LogRecord attributes: https://python.readthedocs.io/en/latest/library/logging.html#logrecord-attributes.
My question is, how does Python's logging module work to achieve efficient collection of this info? I tried to dig into its source code but had a hard time finding the right spot.
You don't need a full inspect.getframeinfo call. The frame object's f_lineno and its code object's co_name can be retrieved much more cheaply than going through all the work of inspect.getframeinfo.
lineno = frame.f_lineno
funcname = frame.f_code.co_name
In my timings, this is about 100 times faster than using inspect.getframeinfo.