How to load test an asynchronous pipeline?

Question

I have the following pipeline in AWS

API-Gateway -> Lambda -> Kafka(Amazon MSK) -> Consumers

I need to load test the entire pipeline and each component in the pipeline (to identify bottlenecks)

I don't have any prior experience in load testing. So, didn't know where to start. Some blogs mentioned JMeter can be used to do the load testing. But later got to know that the pipeline is asynchronous and it can't be done using JMeter.

How can I load test the pipeline? Is there any standard way to do it?

Any help is greatly appreciated. Thanks!

Answer 1

You can use any load testing tool which is capable of sending message to the API gateway and then reading it from Kafka.

When it comes to JMeter it's capable of both:

1. API Gateway: Building a WebService Test Plan
2. Kafka consumer: Apache Kafka - How to Load Test with JMeter
3. If you want to measure the cumulative duration of the request from API till it gets "consumed" - Transaction Controller