Looking at load testing tools, I found Locust and immediately found it appealing as a Python guy, but I'm not entirely sure if I can achieve the following scenario with it...
I'm tasked with load testing a REST API with real-life traffic. I've extracted 5 minutes worth of GET traffic from production Apache logs, and the idea would be to run those same requests with the same time distribution (between 1 and 36 requests per second) using the load testing tool. To this end, I've built a Python dictionary where a relative timestamp (xx:xx, i.e. mins:secs) is the key, and a list of URL paths to request on that second is the value.
We have a dump of the production database from a certain point in time restored to our test environment, and the requests are from the next five minutes following the moment when the dump was created. Between test runs, I'm changing the parameters of how the REST API connects to the database, so the test runs need to be as identical as possible for the metrics to be comparable.
I've looked at custom load shapes in the Locust documentation, and it seems like they might work, but I'm not positive. Can the custom tick method implementation achieve this:
At 0 seconds, make a set of 4 requests.
At 1 second, make a set of 2 requests.
At 2 seconds, make a set of 12 requests.
...time passes, a set of predefined requests happens each second...
At 4 minutes 59 seconds, make a set of 27 requests.
At 5 minutes, make a set of 14 requests.
How would this map onto Locust's capabilities? It wouldn't matter how many users are spawned, all that matters is how many requests are made, and at what point in time.
I really like Locust for its ease of use and familiar syntax, but is it suitable for testing with a static, repeatable request load like this?
EDIT: as it seems this approach may not be achievable without great difficulty (or at all), I've come up with an alternate approach, different enough to warrant a separate question: Getting Locust to send a predefined distribution of requests per second
In short, no. Locust is centered around running a certain number of users and those users being fairly independent from each other.
You can control the throughput for each user using wait_time = constant_throughput(x) (https://docs.locust.io/en/stable/writing-a-locustfile.html#wait-time-attribute). You can also check out locust-plugins's constant_total_ips that targets a global throughput (https://github.com/SvenskaSpel/locust-plugins/blob/master/examples/constant_total_ips_ex.py).
But if you want to do an exact number of requests at an exact time, I dont think there is anything readily available.
You could use python to "sync up" all the users before firing the requests. I dont have a nice solution for it, but this kinda works (you need to be running exactly 10 users though, otherwise you will run in to concurrency issues - pretty sure there are better solutions out there)
users_waiting = 0
lock = Semaphore(10)
#...
@task
def t(self):
global users_waiting
with lock:
users_waiting = users_waiting + 1 if users_waiting < 10 else 1
while self.environment.runner and users_waiting < 10:
time.sleep(0.1)
# actually do your requests here