Search code examples
pythonpython-3.xpython-internalspython-importliblinecache

`inspect.getsource` from a function defined in a string? `s="def f(): return 5"`


Given a function defined inline, how do I get getsource to provide the output? - This is for a test, here's the kind of thing I'm trying:

from importlib.util import module_from_spec, spec_from_loader

_locals = module_from_spec(
    spec_from_loader("helper", loader=None, origin="str")  # loader=MemoryInspectLoader
)
exec(
    'def f(): return "foo"',
    _locals.__dict__,
)
f = getattr(_locals, "f")
setattr(f, "__loader__", MemoryInspectLoader)

With my attempt, as it looks like a linecache issue:

from importlib.abc import Loader

class MemoryInspectLoader(Loader):
    def get_code(self): raise NotImplementedError()

But the error is never raised. From getsource(f), I just get:

In [2]: import inspect
   ...: inspect.getsource(f)
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-3-1348c7a45f75> in <module>
----> 1 inspect.getsource(f)

/usr/lib/python3.8/inspect.py in getsource(object)
    983     or code object.  The source code is returned as a single string.  An
    984     OSError is raised if the source code cannot be retrieved."""
--> 985     lines, lnum = getsourcelines(object)
    986     return ''.join(lines)
    987 

/usr/lib/python3.8/inspect.py in getsourcelines(object)
    965     raised if the source code cannot be retrieved."""
    966     object = unwrap(object)
--> 967     lines, lnum = findsource(object)
    968 
    969     if istraceback(object):

/usr/lib/python3.8/inspect.py in findsource(object)
    796         lines = linecache.getlines(file)
    797     if not lines:
--> 798         raise OSError('could not get source code')
    799 
    800     if ismodule(object):

OSError: could not get source code

How do I make getsource work with an inline-defined function in Python 3.6+?


Solution

  • Here's my solution to this:

    import os.path
    import sys
    import tempfile
    from importlib.util import module_from_spec, spec_from_loader
    from types import ModuleType
    from typing import Any, Callable
    
    class ShowSourceLoader:
        def __init__(self, modname: str, source: str) -> None:
            self.modname = modname
            self.source = source
    
        def get_source(self, modname: str) -> str:
            if modname != self.modname:
                raise ImportError(modname)
            return self.source
    
    
    def make_function(s: str) -> Callable[..., Any]:
        filename = tempfile.mktemp(suffix='.py')
        modname = os.path.splitext(os.path.basename(filename))[0]
        assert modname not in sys.modules
        # our loader is a dummy one which just spits out our source
        loader = ShowSourceLoader(modname, s)
        spec = spec_from_loader(modname, loader, origin=filename)
        module = module_from_spec(spec)
        # the code must be compiled so the function's code object has a filename
        code = compile(s, mode='exec', filename=filename)
        exec(code, module.__dict__)
        # inspect.getmodule(...) requires it to be in sys.modules
        sys.modules[modname] = module
        return module.f
    
    
    import inspect
    func = make_function('def f(): print("hi")')
    print(inspect.getsource(func))
    

    output:

    $ python3 t.py 
    def f(): print("hi")
    

    there's a few subtle, and unfortunate points to this:

    1. it requires something injected into sys.modules (inspect.getsource always looks there for inspect.getmodule)
    2. the __loader__ I've built is bogus, if you're doing anything else that requires a functioning __loader__ this will likely break for that
    3. other oddities are documented inline

    an aside, you're probably better to keep the original source around in some other way, rather than boomeranging through several globals (sys.modules, linecache, __loader__, etc.)