What is the difference between Scalability test and the other performance tests?

Question

Are there any clear ideas that define the Scalability test? I have designed Load, Stress, Spike and Soak tests using JMeter Ultimate Thread Group but, i have not any idea about Scalability test differs from these tests. How to design a good scalability test with ultimate thread group in Jmeter for maximum user count is equal to 500.

Answer 1

As per the Wikipedia article on Scalability testing:

Scalability testing, is the testing of a software application to measure its capability to scale up or scale out in terms of any of its non-functional capability.

So basically you can use the same approach as for the Stress Testing, something like this:

then you need to pay attention to the following charts/KPIs:

• Active Threads Over Time - to show number of active virtual users
• Transactions per Second - to show the system throughput
• Charts of system resources consumption - to show usage of CPU, RAM, etc.

Ideally the charts should be more or less the same/linear, i.e. if you increase the load by factor of 2x the throughput should increase by the same factor and resource consumption should increase proportionally.

If the charts are not equal/similar/proportional - then at some point the system won't be able to keep threads/transactions per second ratio and this will indicate the bottleneck