Search code examples
pythonarcpy

How to use a decorator to log all function calls from a particular module?


Some background:

I am trying to write a decorator for logging. Specifically, I work with arcpy a lot and the way to get messages back from tools is to use arcpy.GetMessages(). However, this is kind of an annoying function because it only holds the most recent message and must be called after every tool. A pseudo-code example

import arcpy
import logging
log = logging.getLogger(__name__)

def test_function(in_data):
    out_data = 'C:/new_path/out_data'
    arcpy.MakeFeatureLayer_management(in_data, out_data)
    log.info(arcpy.GetMessages())

    arcpy.Delete_management(in_data)
    log.info(arcpy.GetMessages()) 

    # If you did log.info(arcpy.GetMessages()) again here you'd just get 
    # the message from the Delete tool again

It would be much better to write a decorator that could identify any time an arcpy function was called and log it. Like:

def log_the_arcpy(fn):
    @functools.wraps(fn)
    def inner(*args, **kwargs):
        result = fn(*args, **kwargs)
        # Some magic happens here?!
        if module_parent == arcpy: #module_parent is obviously fake, is there a real attribute?
            log.info(arcpy.GetMessages())
        return result
    return inner

However, I am quite stuck in two places: (1) how to identify the "arcpy-ness" (or whatever package) of an individual function, and (2) the overall approach to dig inside of a function with a decorator and determine the package membership of potentially many function calls.

Bits and pieces that have seemed useful are:

None of these are ideas that are very well fleshed out - this because many of these topics are quite new to me. I would appreciate any direction - I'm trying to ask early so I don't get locked into asking a bunch of XY Problem questions later.


Solution

  • If you're going to be calling methods directly on arcpy, wrapping the module would probably be the easiest and least performance-affecting approach:

    # arcpy_proxy.py
    import arcpy as _arcpy
    import logging
    
    class _proxy(object):
    
        def __getattr__(self, item):
            ref = getattr(_arcpy, item)
            if callable(ref):  # wrap only function calls
                return self._wrap(ref)
            return ref
    
        @classmethod
        def _wrap(cls, func):
            def inner(*args, **kwargs):
                val = func(*args, **kwargs)
                logging.info(_arcpy.GetMessages())  # log the messages
                return val
            return inner
    
    arcpy = _proxy()
    

    Then you can just do from arcpy_proxy import arcpy as a drop-in replacement. You can even add sys.modules["arcpy"] = arcpy in your main script (after the import of course) so you don't have to replace it anywhere else to have it proxied.