Pyparsing: changing default debug actions for all parser elements

I want to improve the readability of pyparsing's debugging output by adding indentation. For example, instead of this:

Match part at loc 0(1,1)
Match subpart1 at loc 0(1,1)
Match subsubpart1 at loc 0(1,1)
Matched subsubpart1 at loc 10(2,1) -> ...
Matched subpart1 at loc 20(3,1) -> ...
Match subpart2 at loc 20(3,1)
Match subsubpart2 at loc 20(3,1)
Matched subsubpart2 at loc 30(4,1) -> ...
Matched subpart2 at loc 40(5,1) -> ...
Matched part at loc 50(6,1) -> ...

I would like to have it indented like this to better understand what's going on during parsing:

Match part at loc 0(1,1)
    Match subpart1 at loc 0(1,1)
        Match subsubpart1 at loc 0(1,1)
        Matched subsubpart1 at loc 10(2,1) -> ...
    Matched subpart1 at loc 20(3,1) -> ...
    Match subpart2 at loc 20(3,1)
        Match subsubpart2 at loc 20(3,1)
        Matched subsubpart2 at loc 30(4,1) -> ...
    Matched subpart2 at loc 40(5,1) -> ...
Matched part at loc 50(6,1) -> ...

So in pyparsing.py, I just changed _defaultStartDebugAction, _defaultSuccessDebugAction and _defaultExceptionDebugAction to:

pos = -1
def _defaultStartDebugAction( instring, loc, expr ):
    global pos
    pos = pos + 1
    print ("\t" * pos + ("Match " + _ustr(expr) + " at loc " + _ustr(loc) + "(%d,%d)" % ( lineno(loc,instring), col(loc,instring) )))

def _defaultSuccessDebugAction( instring, startloc, endloc, expr, toks ):
    print ("\t" * pos + "Matched " + _ustr(expr) + " -> " + str(toks.asList()))
    global pos
    pos = pos - 1

def _defaultExceptionDebugAction( instring, loc, expr, exc ):
    print ("\t" * pos + "Exception raised:" + _ustr(exc))
    global pos
    pos = pos - 1

(I just added the pos expressions and "\t" * pos to the output to get my desired result)

However, I don't like tampering directly with the pyparsing library. On the other hand, I don't want to use the .setDebugActions method on every parser element I define, I want them all to use my modified default debug actions.

Is there a way I can achieve this without having to tamper with the pyparsing.py library directly?

Thanks!

Solution

Python modules are just like any other Python object, and you can manipulate their symbols using standard Python function decorating methods. Often referred to as "monkeypatching", these can be done entirely from your own code, without modifying the actual library source.

The simplest way to implement this change is to just overwrite the symbols. In your code, write:

import pyparsing
# have to import _ustr explicitly, since it does not get pulled in with '*' import
_ustr = pyparsing._ustr

pos = -1
def defaultStartDebugAction_with_indent( instring, loc, expr ):
    global pos
    pos = pos + 1
    print ("\t" * pos + ("Match " + _ustr(expr) + " at loc " + _ustr(loc) + "(%d,%d)" % ( lineno(loc,instring), col(loc,instring) )))

def defaultSuccessDebugAction_with_indent( instring, startloc, endloc, expr, toks ):
    global pos
    print ("\t" * pos + "Matched " + _ustr(expr) + " -> " + str(toks.asList()))
    pos = pos - 1

def defaultExceptionDebugAction_with_indent( instring, loc, expr, exc ):
    global pos
    print ("\t" * pos + "Exception raised:" + _ustr(exc))
    pos = pos - 1

pyparsing._defaultStartDebugAction = defaultStartDebugAction_with_indent
pyparsing._defaultSuccessDebugAction = defaultSuccessDebugAction_with_indent
pyparsing._defaultExceptionDebugAction = defaultExceptionDebugAction_with_indent

Or a cleaner version is to wrap the original methods with your code as a decorator:

pos = -1

def incr_pos(fn):
    def _inner(*args):
        global pos
        pos += 1
        print ("\t" * pos , end="")
        return fn(*args)
    return _inner

def decr_pos(fn):
    def _inner(*args):
        global pos
        print ("\t" * pos , end="")
        pos -= 1
        return fn(*args)
    return _inner

import pyparsing
pyparsing._defaultStartDebugAction = incr_pos(pyparsing._defaultStartDebugAction)
pyparsing._defaultSuccessDebugAction = decr_pos(pyparsing._defaultSuccessDebugAction)
pyparsing._defaultExceptionDebugAction = decr_pos(pyparsing._defaultExceptionDebugAction)

This way, if you update pyparsing and the original code changes, your monkeypatch will get the updates without your having to modify your copies of the original methods.

To make your intentions even clearer, and to avoid duplicating those function names (DRY), this will replace those last 3 lines:

def monkeypatch_decorate(module, name, deco_fn):
    setattr(module, name, deco_fn(getattr(module, name)))

monkeypatch_decorate(pyparsing, "_defaultStartDebugAction", incr_pos)
monkeypatch_decorate(pyparsing, "_defaultSuccessDebugAction", decr_pos)
monkeypatch_decorate(pyparsing, "_defaultExceptionDebugAction", decr_pos)