I have a python application where thread1 calls an api to see 'what reports are ready to download' and sends that report_id to thread2 which 'downloads/processes those reports. these threads iterate over a dictionary and then wait 5 minutes. Even while code is 'doing nothing' it is pegging on CPU.
I am not sure if the CPU is being pegged in a thread or in main. In main I have some handlers to check for a stop signal so I posted most of that code. I have a few threads that do similar tasks with similar ways they wait at end of loop. Areas that I suspect that code relate to pegging of cpu a)In main - the while run: pass
. b)In each of the threads is_killed = self._kill.wait(600)
and that being a theading.Event()
Any idea what is pegging cpu.
if __name__ == '__main__':
t2 = ProcessReport()
t2.start()
t1 = RequestReport(t2)
t1.start()
t3 = report_test()
t3.start()
t4 = run_flask()
t4.start()
run = True
signal.signal(signal.SIGINT, handler_stop_signals)
signal.signal(signal.SIGTERM, handler_stop_signals)
signal.signal(signal.SIGHUP, handler_stop_signals)
while run:
pass # Stay here until kill
print("About to kill all threads in clean order")
t3.kill()
t3.join()
t1.kill()
t1.join()
t2.kill()
t2.join()
print("Clean Exit")
sys.exit()
Signal Handler
def handler_stop_signals(signum, frame):
global run
run = False
One of the threads
class report_test(Thread):
def __init__(self):
Thread.__init__(self)
self._kill = threading.Event()
def kill(self):
self._kill.set()
def run(self):
while True:
cursor.execute("SELECT * FROM tbl_rpt_log where cron='1'")
reports_to_run = cursor.fetchall()
for row in reports_to_run:
report_on_row(row)
is_killed = self._kill.wait(600)
if is_killed:
print("Killing - ReportCheckTable")
break
The problem (the main one, at least) is:
while run:
pass # Stay here until kill
This is because the only operation here is evaluating loop's condition and the CPU "can't catch a break".
I didn't look through the whole code, to understand it high level, but the quickest way to work around it is:
import time
# ...
while run:
time.sleep(0.1) # Stay here until kill