Why don't I see errors from readline.set_completion_display_matches_hook?

Question

Consider this code:

#!/usr/bin/env python3

from cmd import Cmd
import readline

class mycmd(Cmd):
    def match_display_hook(self, substitution, matches, longest_match_length):
        someNonexistentMethod()
        print()
        for match in matches:
            print(match)
        print(self.prompt, readline.get_line_buffer(), sep='', end='', flush=True)

    def do_crash(self, s):
        someNonexistentMethod()

    def do_quit(self, s):
        return True

if __name__ == '__main__':
    obj = mycmd()
    readline.set_completion_display_matches_hook(obj.match_display_hook)
    obj.cmdloop()

I expect to see NameError: name 'someNonexistentMethod' is not defined when I run that and hit TabTab. However, nothing actually seems to happen at all (the error does occur, so the other functions that would print the completion don't run; I just don't see the error). I do see the expected error when I run crash, so I know error handling works fine in the program overall, but is just broken inside of the set_completion_display_matches_hook callback. Why is this, and can I do something about it?

Answer 1

Why?

I would guess that this is by design. According to rlcompleter docs:

Any exception raised during the evaluation of the expression is caught, silenced and None is returned.

See the rlcompleter source code for the rationale:

• Exceptions raised by the completer function are ignored (and generally cause the completion to fail). This is a feature -- since readline sets the tty device in raw (or cbreak) mode, printing a traceback wouldn't work well without some complicated hoopla to save, reset and restore the tty state.

Workaround

As a workaround, for debugging, wrap your hook in a function that catches all exceptions (or write a function decorator), and use the logging module to log your stack traces to a file:

import logging
logging.basicConfig(filename="example.log", format='%(asctime)s %(message)s')

def broken_function():
    raise NameError("Hi, my name is Name Error")

def logging_wrapper(*args, **kwargs):
    result = None
    try:
        result = broken_function(*args, **kwargs)
    except Exception as ex:
        logging.exception(ex)
    return result

logging_wrapper()

This script runs successfully, and example.log contains both the log message and the stack trace:

2020-11-17 13:55:51,714 Hi, my name is Name Error
Traceback (most recent call last):
  File "/Users/traal/python/./stacktrace.py", line 12, in logging_wrapper
    result = function_to_run()
  File "/Users/traal/python/./stacktrace.py", line 7, in broken_function
    raise NameError("Hi, my name is Name Error")
NameError: Hi, my name is Name Error