Wrapping stdin/stdout causes IPython to lose auto completion and history features

Question

I'm working on a script that uses the embedded shelling capabilities of IPython and as a requirement, it has to log all the data from stdin/stdout into a file. For this reason I decided to write wrappers for them; however, after switching out the streams my embedded IPython shell loses its auto completion and history capabilities, outputting something like this when I press the arrow buttons:

In [1]: ^[[A^[[B^[[A^[[C...

I'm guessing that the wrapper somehow prevents IPython from recognizing the ANSI escape sequences used for the UP, DOWN, LEFT, and RIGHT arrows (ESC[#A, ESC[#B, ESC[#C, ESC[#D).

Here is a code that demonstrates my issue:

import sys
from time import strftime
import IPython

# Custom IO class for file logging
class StdinLogger (object):
    def __init__(self, wrapped, file):
        # double-underscore everything to prevent clashes with names of
        # attributes on the wrapped stream object.
        self.__wrapped = wrapped
        self.__file = file

    def __getattr__(self, name):
        return getattr(self.__wrapped, name)

    def readline(self):
        str = self.__wrapped.readline()
        self.__file.write(str)
        self.__file.flush()
        return str


# Custom IO class for file logging
class StdoutLogger (object):
    def __init__(self, wrapped, file):
        # double-underscore everything to prevent clashes with names of
        # attributes on the wrapped stream object.
        self.__wrapped = wrapped
        self.__file = file

    def __getattr__(self, item):
        return getattr(self.__wrapped, item)

    def write(self, str):
        self.__file.write(str)
        self.__file.flush()
        self.__wrapped.write(str)
        self.__wrapped.flush()


f = open("LOG-" + strftime("%Y-%m-%d-%H-%M-%S") + ".txt", 'w')
# Initialize the file logger
sys.stdin = StdinLogger(sys.stdin, f)
sys.stdout = StdoutLogger(sys.stdout, f)


# Embed IPython shell
IPython.embed(banner1="", banner2="")

Any ideas on how this can be addressed?

Thanks in advance.

Answer 1

@Carreau's (IPython core developer) response to this issue at Github:

The issue is that with prompt toolkit stdout in particular is more than a stream, as you literally draw/erase/redraw on the screen, and as it uses an eventloop and async completions, you cannot consider the stdin/out/err to be enough to know the current state.

So you will (likely) need to hook into the prompt_toolkit and overwrite the vt100-input/output and/or the windows ones.

We most-likely will need extra hooks to set that up in IPython at startup.