I am working on a python project that is polling for data on a COM port and also polling for user input. As of now, the program is working flawlessly but seems to be inefficient. I have the serial port polling occurring in a while loop running in a separate thread and sticking data into a Queue. The user input polling is also occurring in a while loop running in a separate thread sticking input into a Queue. Unfortunately I have too much code and posting it would take away from the point of the question.
So is there a more efficient way to poll a serial or raw_input() without sticking them in an infinite loop and running them in their own thread?
I have been doing a lot of research on this topic and keep coming across the "separate thread and Queue" paradigm. However, when I run this program I am using nearly 30% of my CPU resources on a quad-core i7. There has to be a better way.
I have worked with ISR's in C and was hoping there is something similar to interrupts that I could be using. My recent research has uncovered a lot of "Event" libraries with callbacks but I can't seems to wrap my head around how they would fit in my situation. I am developing on a Windows 7 (64-bit) machine but will be moving the finished product to a RPi when I am finished. I'm not looking for code, I just need to be pointed in the right direction. Thank you for any info.
You're seeing the high CPU usage because your main thread is using the non-blocking get_nowait call to poll two different queues in an infinite loop, which means most of the time your loop is going to be constantly looping. Constantly running through the loop uses CPU cycles, just as any tight infinite loop does. To avoid using lots of CPU, you want to have your infinite loops use blocking I/O, so that they wait until there's actually data to process before continuing. This way, you're not constantly running through the loop, and therefore using CPU.
So, user input thread:
while True:
data = raw_input() # This blocks, and won't use CPU while doing so
queue.put({'type' : 'input' : 'data' : data})
COM thread:
while True:
data = com.get_com_data() # This blocks, and won't use CPU while doing so
queue.put({'type' : 'COM' : 'data' : data})
main thread:
while True:
data = queue.get() # This call will block, and won't use CPU while doing so
# process data
The blocking get call will just wait until it's woken up by a put in another thread, using a threading.Condition object. It's not repeatedly polling. From Queue.py:
# Notify not_empty whenever an item is added to the queue; a
# thread waiting to get is notified then.
self.not_empty = _threading.Condition(self.mutex)
...
def get(self, block=True, timeout=None):
self.not_empty.acquire()
try:
if not block:
if not self._qsize():
raise Empty
elif timeout is None:
while not self._qsize():
self.not_empty.wait() # This is where the code blocks
elif timeout < 0:
raise ValueError("'timeout' must be a non-negative number")
else:
endtime = _time() + timeout
while not self._qsize():
remaining = endtime - _time()
if remaining <= 0.0:
raise Empty
self.not_empty.wait(remaining)
item = self._get()
self.not_full.notify()
return item
finally:
self.not_empty.release()
def put(self, item, block=True, timeout=None):
self.not_full.acquire()
try:
if self.maxsize > 0:
if not block:
if self._qsize() == self.maxsize:
raise Full
elif timeout is None:
while self._qsize() == self.maxsize:
self.not_full.wait()
elif timeout < 0:
raise ValueError("'timeout' must be a non-negative number")
else:
endtime = _time() + timeout
while self._qsize() == self.maxsize:
remaining = endtime - _time()
if remaining <= 0.0:
raise Full
self.not_full.wait(remaining)
self._put(item)
self.unfinished_tasks += 1
self.not_empty.notify() # This is what wakes up `get`
finally:
self.not_full.release()