Python, "filtered" line editing, read stdin by char with no echo

Question

I need a function that reads input into a buffer as raw_input() would, but instead of echoing input and blocking until returning a full line, it should supress echo and invoke a callback every time the buffer changes.

I say "buffer changes" instead of "character is read" because, as raw_input(), I'd like it to be aware of special keys. Backspace should work, for example.

If I wanted to, for example, use the callback to simulate uppercased echo of input, the code would look like this:

def callback(text):
    print '\r' + text.upper()

read_input(callback)

How can I achieve this?

NOTE: I've been trying to use readline and curses to meet my ends, but both Python bindings are incomplete. curses cannot be made to start without clearing the whole screen, and readline offers a single hook before any input begins.

Answer 1

Well, I wrote the code by hand. I'll leave an explanation for future reference.

Requirements

import sys, tty, termios, codecs, unicodedata
from contextlib import contextmanager

Disabling line buffering

The first problem that arises when simply reading stdin is line buffering. We want single characters to reach our program without a required newline, and that is not the default way the terminal operates.

For this, I wrote a context manager that handles tty configuration:

@contextmanager
def cbreak():
    old_attrs = termios.tcgetattr(sys.stdin)
    tty.setcbreak(sys.stdin)
    try:
        yield
    finally:
        termios.tcsetattr(sys.stdin, termios.TCSADRAIN, old_attrs)

This manager enables the following idiom:

with cbreak():
    single_char_no_newline = sys.stdin.read(1)

It's important to perform the clean up when we're done, or the terminal might need a reset.

Decoding stdin

The second problem with just reading stdin is encoding. Non-ascii unicode characters will reach us byte-by-byte, which is completely undesirable.

To properly decode stdin, I wrote a generator that we can iterate for unicode characters:

def uinput():
    reader = codecs.getreader(sys.stdin.encoding)(sys.stdin)
    with cbreak():
        while True:
            yield reader.read(1)

This may fail over pipes. I'm not sure. For my use case, however, it picks up the right encoding and generates a stream of characters.

Handling special characters

First off, we should be able to tell printable characters apart from control ones:

def is_printable(c):
    return not unicodedata.category(c).startswith('C')

Aside from printables, for now, I only want to handle ← backspace and the CtrlD sequence:

def is_backspace(c):
    return c in ('\x08','\x7F')

def is_interrupt(c):
    return c == '\x04'

Putting it together: xinput()

Everything is in place now. The original contract for the function I wanted was read input , handle special characters, invoke callback. The implementation reflects just that:

def xinput(callback):
    text = ''

    for c in uinput():
        if   is_printable(c): text += c
        elif is_backspace(c): text = text[:-1]
        elif is_interrupt(c): break

        callback(text)

    return text

Trying it out

def test(text):
    print 'Buffer now holds', text

xinput(test)

Running it and typing Hellx← backspaceo World shows:

Buffer now holds H
Buffer now holds He
Buffer now holds Hel
Buffer now holds Hell
Buffer now holds Hellx
Buffer now holds Hell
Buffer now holds Hello
Buffer now holds Hello 
Buffer now holds Hello w
Buffer now holds Hello wo
Buffer now holds Hello wor
Buffer now holds Hello worl
Buffer now holds Hello world