This is my first threading program. I'm facing a strange issue here. I'm building a simple scheduler like application in Django, where function names (to be executed periodically) will be stored in Django Model along with their next execution time.
A management command is executed to start a thread which runs continuously to check if execution of any function is due, if yes, a new thread is started for executing this function. This way, separate thread for each function is created (at least, that's the idea!).
class Command(BaseCommand):
def __init__(self):
super(Command, self).__init__()
self.lock = None
def handle(self, *args, **kwargs):
self.lock = threading.RLock()
t1 = threading.Thread(target=self.cron_thread)
t1.start()
t1.join()
def cron_thread(self):
while True:
# Fetch only Active records
scheduled_actions = Scheduler.objects.filter(active=True)
for scheduled_action in scheduled_actions:
# check if execution is due
if scheduled_action.next_execution_time == datetime.now():
# creating a new thread
function_thread = threading.Thread(target=eval(scheduled_action.function_name), args=[self.lock])
function_thread.start()
function_thread.join()
scheduled_action.next_execution_time = local_timezone.localize(datetime.now() + relativedelta(minutes=scheduled_action.interval))
scheduled_action.run_now = False
scheduled_action.save()
def somefunction(self):
self.lock.acquire()
# function body
self.lock.release()
The command that I have created to start execution of whole program is: python3 manage.py runcrons-debit
As soon as I execute this command, I can see in the htop results that two process are running and consuming almost 80% of CPU, as shown in the following image:
View Image
Please note here that no scheduler records are active yet.
When scheduler records are made active, and when the function actually runs, the processes displayed in htop increase to three and CPU usage decreases drastically to 0.0%. As shown in the following image:
View Image
There are two things here that I can't understand,
cron_thread has an infinite loop. This loop first retrieves scheduled actions, then loops over them. For each action, if the action is scheduled for the exact current time, the action is executed.
If no actions are scheduled, the loop will simply keep retrieving the scheduled actions, over and over again. If there is an action, it will check whether now is the time to execute it. Here's another problem: datetime.datetime.now() has very high precision (nearest microsecond), so the chances that it'll match the scheduled time for an action are very low. This means your loop will retrieve all scheduled actions, loop over all the actions, and then go back to the top.
On the off-chance that a scheduled action's time does match the current time, that action will be executed, then the inner loop moves onto the next action. When it's looped through all the actions, it will go back to the top and retrieve all actions once again.
Basically, your program is constantly comparing any scheduled actions to the current time. That takes processing power. A better way to execute these actions would be to check the time for each new action as it's added to the list of tasks, calculate the necessary delay until that action needs to be executed, and then set a timer to execute that action after the necessary delay (time.sleep in a thread, after calls in tkinter, that sort of thing).
