I am using pygtgraph and pyqt5 to draw lines with crosshair and then to color them with in ten different colors according to an algorithm. I also use slider to choose how many first lines I want to color. Each time the slider value changes, the GUI freezes to calculate colors which I don't want to happen.
I tried to put the calculation in a different thread and it sort of worked but now I want to use processes. What I am trying to do is to use multiprocessing to create a pool of threads to process algorithm calls in a queue and one thread to recolor drawn lines.
self.threads = max(1, multiprocessing.cpu_count() - 3)
self.queue_in = multiprocessing.Queue()
self.queue_out = multiprocessing.Queue()
self.the_pool = multiprocessing.Pool(self.threads, algorithm_worker, (self.queue_in, self.queue_out))
self.recolor_thread = multiprocessing.Process(target=assign_and_recolor_worker, args=(
self.queue_out, self.color_numbers, self.canvas.getPlotItem(), self.pens))
self.recolor_thread.start()
These are the functions:
def algorithm_worker(queue_in, queue_out):
print(os.getpid(), "working")
while True:
lines = queue_in.get(block=True)
result = list(get_color_nums_algo(lines)) if lines else []
print(result)
queue_out.put(result)
def assign_and_recolor_worker(queue_in, color_nums, plot_item, pens):
print(os.getpid(), "working")
while True:
color_nums = queue_in.get(block=True)
for plotdataitem, pen_i in zip(plot_item.items, color_nums):
plotdataitem.setPen(pens[pen_i % len(pens)])
The first part seems to be working fine but I struggle quite a bit with the second one since I am new to multiprocessing.
self.recolor_thread mainly: the list of assigned color numbers self.color_numbers.self.canvas.getPlotItem().items1 - I tried using manager but it doesn't update the value
self.manager = multiprocessing.Manager()
self.color_numbers = self.manager.list()
2 - I pass PlotItem and mkPen but get errors like
TypeError: cannot pickle 'PlotItem' object
How do I solve or circumvent these issues? Here is the link to the necessary code https://pastebin.com/RyvNpP67
First, you need to keep the GUI event loop running. The GUI runs an event loop which freezes/hangs when you stay in an event/signal for a long time. If you click a button and stay in the button clicked signal the GUI cannot process other events while you are in that function.
Threading only helps with this if the thread is waiting on I/O or time.sleep is called. I/O is when you call something like socket.recv() or queue.get(). These operations wait for data. While the thread is waiting for data the main thread can run and process events on the Qt event loop. Threads that only run calculations do not let the main thread process events.
Multiprocessing is used to send data to a separate process and wait for the results. While waiting for the results your Qt event loop can process events if the waiting is in a separate thread. Sending data to a separate process and receiving data from the process may take a long time. In addition to this some items cannot be sent to a separate process easily. It may be possible to send QT GUI items to a separate process, but it is difficult and probably has to use window handles with the operating system.
Multiprocessing
def __init__(self):
...
self.queue_in = multiprocessing.Queue()
self.queue_out = multiprocessing.Queue()
# Create thread that waits for data and plots the data
self.recolor_thread = threading.Thread(target=assign_and_recolor_worker, args=(
self.queue_out, self.color_numbers, self.canvas.getPlotItem(), self.pens))
self.recolor_thread.start() # comment this line
self.alg_proc = multiprocessing.Process(target=algorithm_worker, args=(self.queue_in, self.queue_out))
self.alg_proc.start()
def calc_color(self):
lines_to_recog = self.current_lines[:self.color_first_n]
self.queue_in.put(lines_to_recog) # Send to process to calculate
Process Events
If you just want a responsive GUI call QtWidgets.QApplication.processEvents(). Although "processEvents" is not always recommended it can be very useful in certain situations.
def get_color_nums_algo(lines: list, proc_events=None) -> list:
li = []
for _ in lines:
li.append(np.random.randint(0, 10))
try:
proc_events()
except (TypeError, Exception):
pass
return li
class MainWindow(QtWidgets.QMainWindow):
....
def calc_color(self):
lines_to_recog = self.current_lines[:self.color_first_n]
# Color alg
proc_events= QtWidgets.QApplication.processEvents
color_nums = list(get_color_nums_algo(lines_to_recog, proc_events)) if lines_to_recog else []
# Plot data
for plotdataitem, pen_i in zip(self.canvas.getPlotItem().items, color_nums):
plotdataitem.setPen(self.pens[pen_i % len(self.pens)])
QtWidgets.QApplication.processEvents()
Mouse Moved
Also the mouseMoved event can happen very fast. If update_drawing takes too long you may want to use a timer to call update_drawing periodically, so it is called 1 time when 10 events may have happened.