python python-3.x matplotlib plot user-input

Matplotlib: Can't Manipulate plot while script is waiting for `input()`

I am trying to get user input via input() after the user manipulates a plot using the standard zoom controls. Eg. User plays with the plot, figures out the desired X-value, and types it into the command-line prompt.

Plot can be either in a separate window (Spyder/Python) or in-line (in Jupiter Notebook).

After the user types in the value, the script continues (eg. asks for another value from the plot, or does some calculation with the values).

However, I can't get the plot to actually display and be responsive while the command-line is waiting for user-input. I have tried:

plot() statement first, input() statement second.
Spyder with Python 3.6 (I think), from source via MacPorts (updated Spyder as far as I could)
Spyder via Python 3.7 from ContinuumIO's Anaconda package, in IPython
Jupiter Notebook also from Anaconda
Numerous backends: macosx, qt, etc.
Notebook %matplotlib, notebook, inline, qt etc.
separate figure windows (Spyder & Python) vs. in-line figures (Jupyter Notebook)
fig.show( block=False ) and variations of this, eg. plt.show( block=False )
two different MacBooks (2017 and 2010 MacBook Pro's)

I did get the plot to actually update (previously it was either a blank space in a Notebook, or a blank separate figure window) by adding a matplotlib.pyplot.pause(0.5) between the plot() and input() statements. This was major progress, but once the script hits the input() statement, I get a spinning beachball on the Figure window (preventing zooming etc.) until I complete the input() statement by entering something, and then the script completes. At that point the plot is interactive.

It seems like the python console(s) can't handle more than one user-interaction simultaneously? Ie. input() is freezing all other user-interactivity?

I've been searching SO, google etc. for days now and haven't figured this out! The idea was to use this as a "quick and dirty" way to get user input from the plot, prior to undertaking the theoretically more complex task of acquiring user-clicks directly from the plot (which would have to snap to plotted data like data cursors).

Solution

Theory

The main execution thread blocks on user input, effectively pausing all other operations including rendering. You can mitigate this by doing plotting in another thread and passing UI input to that thread through a queue so that thread never blocks and stays responsive.

The docs have a great section on interactive figures, including ipython integrations.

Here are some examples:

Use non-blocking plot: plt.show(block=False)
Use matplotlib.animation
Use more complex multithreading and queues (good for integrating into UIs)

Some of the code below is from an old project of mine.

Example using `input()` with `matplotlib.animation`

Updates starting x location on input(), quits with q. Note that you can zoom and pan on the plot while waiting for user input. Also note the use of non-blocking plt.show() in mainloop():

import queue
import numpy as np  # just used for mocking data, not necessary
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation

fig = plt.figure()
ax = fig.add_subplot(1, 1, 1)
animation_queue = queue.Queue()
update_rate_ms = 50

xdata = np.linspace(0, 2 * np.pi, 256)
ydata = np.sin(xdata)
zdata = np.cos(xdata)

def normal_plot_stuff():
    """Some run of the mill plotting."""
    ax.set_title("Example Responsive Plot")
    ax.set_xlabel("X")
    ax.set_ylabel("Y")
    ax.plot(xdata, ydata, "C0", label="sin")
    ax.plot(xdata, zdata, "C1", label="cos")
    ax.legend(loc="lower right")

def animate(_, q):
    """Define a callback function for the matplotlib animation. 
       This reads messages from the queue 'q' to adjust the plot.
    """
    while not q.empty():
        message = q.get_nowait()
        q.task_done()
        x0 = float(message)
        ax.set_xlim([x0, x0 + 5])

def mainloop():
    """The main loop"""
    _ = FuncAnimation(fig, animate, interval=update_rate_ms, fargs=(animation_queue,))
    normal_plot_stuff()
    plt.show(block=False)
    while True:
        try:
            uinput = input("Type starting X value or 'q' to quit: ")
            if uinput == "q":
                break
            animation_queue.put_nowait(float(uinput))
        except ValueError:
            print("Please enter a valid number.")

mainloop()

Example with a live plot embedded in a UI

The window starting X and window size update as a user enters it in the text field. The matplotlib canvas is tied to the UI rendering for responsiveness.

"""
Imbed a live animation into a PySimpleGUI frontend.

The animation fires on a timer callback from matplotlib and renders to
a PySimpleGUI canvas (which is really just a wrapped tk canvas).
"""

import queue
import numpy as np  # just used for mocking data, not necessary
import PySimpleGUI as sg  # used just for example
import matplotlib
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation
from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg  # used just for example

matplotlib.use("TkAgg")


fig = plt.figure()
ax = fig.add_subplot(1, 1, 1)
animation_queue = queue.Queue()
update_rate_ms = 50

xdata = np.linspace(0, 2 * np.pi, 256)
ydata = np.sin(xdata)
zdata = np.cos(xdata)


def animate(_, q):
    """Define a callback function for the matplotlib animation."""
    message = None
    while not q.empty():
        message = q.get_nowait()
        q.task_done()
    if not message:  # ignore empty UI events
        return

    ax.clear()
    if message[1]["sin"]:  # if SIN enable checkbox is checked
        ax.plot(xdata, ydata, "C0", label="sin")
        ax.legend(loc="lower right")
    if message[1]["cos"]:  # if COS enable checkbox is checked
        ax.plot(xdata, zdata, "C1", label="cos")
        ax.legend(loc="lower right")

    x0 = float(message[1]["x_start"])
    size = float(message[1]["w_size"])
    ax.set_xlim([x0, x0 + size])
    ax.set_title("Example Responsive Plot")
    ax.set_xlabel("X")
    ax.set_ylabel("Y")


layout = [
    [
        sg.Text("Start X:"),
        sg.Input(size=(5, 0), default_text=0, key="x_start"),
        sg.Text("Window Size:"),
        sg.Input(size=(10, 0), default_text=6.28, key="w_size"),
        sg.Button("Exit"),
    ],
    [
        sg.Frame(
            title="SIN",
            relief=sg.RELIEF_SUNKEN,
            layout=[
                [sg.Checkbox("Enabled", default=True, key="sin", enable_events=True)],
            ],
        ),
        sg.Frame(
            title="COS",
            relief=sg.RELIEF_SUNKEN,
            layout=[
                [sg.Checkbox("Enabled", default=True, key="cos", enable_events=True)],
            ],
        ),
    ],
    [sg.Canvas(key="-CANVAS-")],
]


def plot_setup():
    """MUST maintain this order: define animation, plt.draw(), setup
    window with finalize=True, then create, draw and pack the TkAgg
    canvas.
    """
    _ = FuncAnimation(fig, animate, interval=update_rate_ms, fargs=(animation_queue,))
    plt.draw()
    window = sg.Window(
        "Responsive Plot Example",
        layout,
        font="18",
        element_justification="center",
        finalize=True,
    )
    # tie matplotlib renderer to pySimpleGui canvas
    canvas = FigureCanvasTkAgg(fig, window["-CANVAS-"].TKCanvas)
    canvas.draw()
    canvas.get_tk_widget().pack(side="top", fill="both", expand=1)
    return window


def mainloop():
    """Main GUI loop. Reads events and sends them to a queue for processing."""
    window = plot_setup()
    while True:
        event, values = window.read(timeout=update_rate_ms)
        if event in ("Exit", None):
            break
        animation_queue.put_nowait([event, values])
    window.close()


mainloop()

Example with live data streaming

Specifically, notice that you can type different values into the window field at the top of the UI and the plot immediately updates without blocking/lagging. The ADC controls at the bottom are pretty meaningless for this example, but they do demonstrate more ways of passing UI data to the plotting thread.

"""
Imbed a live animation into a PySimpleGUI frontend, with extra plotting
and sensor control.

Live sensor data gets read from a separate thread and is converted to
PSI using calibration coefficients from a file.

The animation fires on a timer callback from matplotlib and renders to
a PySimpleGUI canvas (which is really just a wrapped tk canvas).
"""

import time
import queue
import random
import threading
from datetime import datetime
import numpy as np  # just used for mocking data, not necessary
import PySimpleGUI as sg
import matplotlib
import matplotlib.pyplot as plt
from matplotlib import animation
from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg

matplotlib.use("TkAgg")


fig = plt.figure()
ax = fig.add_subplot(1, 1, 1)
animation_queue = queue.Queue()  # to pass GUI events to animation
raw_data_queue = queue.Queue()  # to pass raw data to main thread
update_rate_ms = 50  # refresh time in ms
ts, adc0, adc1 = [], [], []  # live data containers


def get_sensors(msg):
    """Return the names of the currently selected sensors from the GUI."""
    names = np.array(["A", "B", "C"])
    s0 = [msg[2], msg[3], msg[4]]  # adc0 sensor
    s1 = [msg[6], msg[7], msg[8]]  # adc1 sensor
    return (names[s0][0], names[s1][0])  # boolean index to the names


def data_collection_thread(data_queue):
    """Simulate some live streamed data that and put it on a queue."""
    t = 0
    while True:
        t += 1
        x = np.sin(np.pi * t / 112) * 12000 - 10000
        y = random.randrange(-23000, 3000)
        line = f"{t}:{x}:{y}"
        data_queue.put(line)
        time.sleep(0.001)


def process_data(data_queue, message, t, x, y):
    """Consume and process the data from the live streamed data queue."""
    while not data_queue.empty():
        line = data_queue.get()
        try:
            t0, v0, v1 = line.split(":")
            t.append(float(t0))
            x.append(float(v0))
            y.append(float(v1))
        except ValueError:
            pass  # ignore bad data
        data_queue.task_done()
    try:  # truncate to appropriate window size
        n = int(message[0])
        return t[-n:], x[-n:], y[-n:]
    except (ValueError, TypeError):
        return t, x, y  # don't truncate if there is a bad window size


# draws live plot on a timer callback
def animate(_, q):
    # get last message on event queue
    message = None
    while not q.empty():
        message = q.get_nowait()
        q.task_done()

    # plot last n datapoints
    try:
        n = int(message[1][0])  # parse window size
        adc0_window = adc0[-n:]
        adc1_window = adc1[-n:]
        ts_window = [i for i in range(len(adc0_window))]
        ax.clear()
        if message[1][1]:  # if adc0 enable checkbox is checked
            ax.plot(ts_window, adc0_window, "C0", label="adc0")
            ax.legend(loc="lower right")
        if message[1][5]:  # if adc0 enable checkbox is checked
            ax.plot(ts_window, adc1_window, "C1", label="adc1")
            ax.legend(loc="lower right")
        ax.set_title("Live Sensor Readings")
        ax.set_xlabel("Time (ms)")
        ax.set_ylabel("Pressure (psi)")

        # save displayed data
        if message[0] == "Save":
            basename = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
            plt.savefig(basename + ".png")
    except (ValueError, TypeError):
        pass  # ignore poorly formatted messages from the GUI


layout = [
    [  # row 1, some control buttons
        sg.Text("Window Size (ms):"),
        sg.Input(size=(5, 0), default_text=100),
        sg.Button("Start"),
        sg.Button("Pause"),
        sg.Button("Save"),
        sg.Button("Exit"),
    ],
    [sg.Canvas(key="-CANVAS-")],  # row 2, the animation
    [  # row 3, some frames for the ADC options
        sg.Frame(
            title="ADC 0",
            relief=sg.RELIEF_SUNKEN,
            layout=[
                [sg.Checkbox("Enabled", default=True)],
                [
                    sg.Radio("Sensor A", 1, default=True),
                    sg.Radio("Sensor B", 1),
                    sg.Radio("Sensor C", 1),
                ],
            ],
        ),
        sg.Frame(
            title="ADC 1",
            relief=sg.RELIEF_SUNKEN,
            layout=[
                [sg.Checkbox("Enabled", default=True)],
                [
                    sg.Radio("Sensor A", 2),
                    sg.Radio("Sensor B", 2, default=True),
                    sg.Radio("Sensor C", 2),
                ],
            ],
        ),
    ],
]

# MUST maintain this order: define animation, plt.draw(), setup window
# with finalize=True, then create, draw and pack the TkAgg canvas
ani = animation.FuncAnimation(
    fig, animate, interval=update_rate_ms, fargs=(animation_queue,)
)
plt.draw()  # must call plot.draw() to start the animation
window = sg.Window(
    "Read Pressure Sensors",
    layout,
    finalize=True,
    element_justification="center",
    font="18",
)

# tie matplotlib renderer to pySimpleGui canvas
canvas = FigureCanvasTkAgg(fig, window["-CANVAS-"].TKCanvas)
canvas.draw()
canvas.get_tk_widget().pack(side="top", fill="both", expand=1)

# kick off data collection thred
threading.Thread(
    target=data_collection_thread, args=(raw_data_queue,), daemon=True
).start()
data_collection_enable = True

# main event loop for GUI
while True:
    event, values = window.read(timeout=update_rate_ms)
    # check for button events
    if event in ("Exit", None):
        break
    if event == "Start":
        data_collection_enable = True
    if event == "Pause":
        data_collection_enable = False
    # send GUI events to animation
    animation_queue.put_nowait((event, values))
    # process data when not paused
    if data_collection_enable:
        ts, adc0, adc1 = process_data(raw_data_queue, values, ts, adc0, adc1)
    else:  # if paused, throw away live data
        while not raw_data_queue.empty():
            raw_data_queue.get()
            raw_data_queue.task_done()

window.close()

Matplotlib: Can't Manipulate plot while script is waiting for `input()`

Theory

Example using input() with matplotlib.animation

Example with a live plot embedded in a UI

Example with live data streaming

Example using `input()` with `matplotlib.animation`