Search code examples
web-serviceshttpsapache-axisjmetersoapui

Why I get higher TPS (and more stable) when using JMeter instead of SoapUI?


I am doing a load testing for a web services server (based on Tomcat+Axis2).

In the test, I invoke one method with HTTPS:

  • When using SoapUI, with simple strategy and set the thread number, when TPS go above 60 for some time, there will be some error (about 0.1-0.3%) shown in SoapUI ("target server failed to respond"). And on the web services server, seems didn't receive those requests at all. 99.7-99.9% of requests are correctly received and processed.

  • When using JMeter, the TPS is easily go higher (100-200TPS) and no error shown in JMeter aggregation report. And on the web services server, every request is received and correctly processed.

Why the test result looks different from JMeter to SoapUI? Any special configuration on the tool?

BTW, when invoke the method with HTTP, then there is no error on SoapUI.


Solution

  • Are you using a single server for generating load?

    There is an option to increase socket timeout in soapUI preference, see if that helps.

    The only reason i can think of why the web server is not getting some of the request is that the port being used is getting bogged down. In addition to increasing the socket timeout, try to use different load generator or better yet use loadUI.

    Also, check soapUI's memory mgmt page

    http://www.soapui.org/Load-Testing/memory-management.html

    Speaking from personal experience, i have found jmeter to be more reliable for performance testing WS than soapUI.