I have to test my website API (in worse case it load 50 API on the page). And I need to test such case for around 5000+ users. From single host, I run it for 350 users (350 * 50 api) from one machine and it worked well without much of the error. Now I was to test this case for 6000 users. I have 17 slave who are running test, and 1 master which is collecting the stats. I want to know, if this is right way. Many time i see socket related error which I believer is due to thread limitation of my master. What is the right way to test this scenarios. How much slave one master can have.
My website is hosted on google app engine server and it adds new server on the fly by monitoring the traffic, however none of in my team know at what point google adds the server. They says it does on its own and cant be controlled. I want to find out how much servers we are going to need on regular basis to support 5000+ users.
How people generally test such scenario.
I suggest you start with less slaves, maybe 2 or 4, and determine the load they can achieve by monitoring the resources on the master, all slaves, and the server under test.
Use a slow ramping test to your target of 5000 users. Slow ramping tends to be more realistic of typical load on servers, and creates inherent timing randomness between threads.
Depending on the complexity of the tests and response times of the server, each slave can be capable of thousands of transactions per second.
More slaves mean more load on the master, which has to collate all the results. I've never reached a limit, but i make my slaves work hard.
Using the above scenario, when you see the resources of master or slave reach limits, you can add more resources (ie, more memory, cpu, etc) or more slaves.