Locust stalled after temporarily getting Error 500

Question

When the loadbalancer in front of the tested https web site fails-over, this generates some HTTPError 500 for few seconds, then Locust hangs:

• The response time graph stops (empty graph)
• The total requests per second turns to a wrong green flat line. If I just stop & start the test, locust restart monitoring properly the response time.
• We can see some HTTPError 500 in the Failures tab

Is this a bug ? How can I make sure Locust kills and restarts users, either manually or when timeout ?

My attempt to regularly "RescheduleTaskImmediately" did not help.

My locustfile.py:

#!/usr/bin/env python
import time
import random
from locust import HttpUser, task, between, TaskSet
from locust.exception import InterruptTaskSet, RescheduleTaskImmediately

URL_LIST = [
"/url1",
"/url2",
"/url3",
]

class QuickstartTask(HttpUser):
    wait_time = between(0.1, 0.5)
    connection_timeout = 15.0
    network_timeout = 20.0

    def on_start(self):
        # Required to use the http_proxy & https_proxy
        self.client.trust_env = True
        print("New user started")
        self.client.timeout = 5
        self.client.get("/")
        self.client.get("/favicon.ico")
        self.getcount = 0

    def on_stop(self):
        print("User stopped")

    @task
    def track_and_trace(self):
        url = URL_LIST[random.randrange(0,len(URL_LIST))]
        self.client.get(url, name=url[:50])
        self.getcount += 1
        if self.getcount > 50 and (random.randrange(0,1000) > 990 or self.getcount > 200):
            print(f'Reschedule after {self.getcount} requests.')
            self.client.cookies.clear()
            self.getcount = 0
            raise RescheduleTaskImmediately

Answer 1

Each locust runs in a thread. If the thread gets blocked, it doesn't take further actions.

self.client.get(url, name=url[:50], timeout=.1)

Something like this is probably what you need, potentially with a try/except to do something different when you get an http timeout exception.