Extracting complete log statements from python source code

Question

Can you please share how can I extract logger statement from python source code.

For instance I have below snippet:

import logger
def some_function(some_list):
    #multi-line logger
    logger.info("Inside 
    some function")
    for item in some_list:
        logger.debug("item is {}}.format(item)

The output should be:

logger.info("Inside some function") #converted to single line 
logger.debug("item is {}}.format(item)

I have used the AST module in python which gives me source code lines that has logger variable

#tree is a AST parse tree of python source file
for node in ast.walk(tree):
    #previous = node.parent
    try:
        for child in ast.iter_child_nodes(node):
            #print("The length of child nodes is {}".format(len(ast.iter_child_nodes(node))))
            if  child.id == "logger":
                print("The start line no is {} and end line no is {}".format(child.lineno,child.end_lineno))
    except AttributeError:
        pass

Below is the output of the AST script:

The start line no is 150 and end line no is 150
logger = logging.getLogger(__name__)

The start line no is 226 and end line no is 226
logger.info

The start line no is 232 and end line no is 232
logger.info

The start line no is 294 and end line no is 294
logger.info

The start line no is 300 and end line no is 300
logger.info

The start line no is 303 and end line no is 303
logger.info

As you can see I am not able to retrieve the entire logger source code lines, any help is much appreciated, thanks!

Answer 1

The astor module makes this easy to do:

import ast
import astor

class LogPrinter(astor.TreeWalk):
    def pre_Call(self):
        if (isinstance(self.cur_node.func, ast.Attribute) and
            isinstance(self.cur_node.func.value, ast.Name) and
            self.cur_node.func.value.id == "logger"
           ):
            print(astor.to_source(self.cur_node), end="")

You'd call it with something like LogPrinter(ast.parse(some_source_code)).

It works by looking for any ast.Call node in the ast tree and then checking if the call is being made on an attribute of something named logger. You could substitute in some other logic if you want, this was just a quick and dirty version that works on your specific example code (if it's syntax errors are fixed):

code_text = """import logger
def some_function(some_list):
    #multi-line logger
    logger.info(
        "Inside some function"
    )
    for item in some_list:
        logger.debug("item is {}".format(item))
"""

LogPrinter(ast.parse(code_text))

The output is:

logger.info('Inside some function')
logger.debug('item is {}'.format(item))