Relationship between dataprovider in parallel mode and thread parallel "methods" setting

There are two test classes with each having one test method using a datatprovider in parallel mode.

public class FirstNg {

    @Test(dataProvider = "dp11", description="f one")
    public void f11(Integer n, String s) throws InterruptedException {
        System.out.println("DP FIR ----" + Thread.currentThread().getId() + "----" + System.currentTimeMillis());
        Thread.sleep(4000);
    }

    @DataProvider(parallel = true)
    public Object[][] dp11() {
        return new Object[][] { new Object[] { 1, "a" }, new Object[] { 2, "b" }, new Object[] { 3, "c" },
                new Object[] { 4, "d" }};
    }

}

public class SecondNg {

    @Test(dataProvider = "dp22", description="f two")
    public void f22(Integer n, String s) throws InterruptedException {
        System.out.println("DP SEC ----" + Thread.currentThread().getId() + "----" + System.currentTimeMillis());
        Thread.sleep(4000);
    }

    @DataProvider(parallel = true)
    public Object[][] dp22() {
        return new Object[][] { new Object[] { 1, "a" }, new Object[] { 2, "b" }, new Object[] { 3, "c" },
                new Object[] { 4, "d" }};
    }       
}

The duration of test run is calculated with times determined in BeforeSuite and AterSuite.

Case 1 - Run without any thread parallel setting.

<suite name="suite" data-provider-thread-count="2"> 
  <test name="test"> 
    <classes> 
      <class name="FirstNg"/> 
      <class name="SecondNg"/> 
    </classes> 
  </test>
</suite>

This gives the following result.

DP FIR ----12----1552410839748
DP FIR ----11----1552410839752
DP FIR ----12----1552410843753
DP FIR ----11----1552410843756
DP SEC ----13----1552410847763
DP SEC ----14----1552410847764
DP SEC ----13----1552410851767
DP SEC ----14----1552410851768
DURATION - 16.936 secs

The first two rows are started belonging to the dataprovider of the FirstNg class. This is repeated in pairs of two (equals the value of data-provider-thread-count), then the dataprovider of SecondNg class is used.

Case 2 - Run with thread parallel setting to methods.

<suite name="Surefire suite" data-provider-thread-count="2" parallel="methods"> 
  <test name="Surefire test"> 
    <classes> 
      <class name="testngparallel.FirstNg"/> 
      <class name="testngparallel.SecondNg"/> 
    </classes> 
  </test>
</suite>

This gives the following result.

DP SEC ----14----1552412828961
DP FIR ----13----1552412828962
DP FIR ----16----1552412828964
DP SEC ----15----1552412828966
DP FIR ----13----1552412832972
DP FIR ----16----1552412832977
DP SEC ----15----1552412832979
DP SEC ----14----1552412832980
DURATION - 8.161 secs

The first four rows are started together. Two rows belong to the FirstNg dataprovider and other two belong to the SecondNg. This is repeated till all the rows of all dataproviders are exhausted.

From this discussion it appears there are 2 pools, one for dataprovider and one for other tests - https://groups.google.com/forum/#!topic/testng-users/BKfSgHoAChU

Adding the parallel="methods" setting, reduces the time of the test execution. Also the sequence of tests are changed, tests from both the dataprovider are mixed up. What is the connection between the two settings?

Solution

Adding the parallel="methods" setting, reduces the time of the test execution. Also the sequence of tests are changed, tests from both the dataprovider are mixed up. What is the connection between the two settings?

Basically the two settings control different execution aspects. All test methods can be bucketed into two categories.

Regular test methods
Test methods that are backed by a data provider.

TestNG has two specific settings to cater to the needs of the above mentioned two categories via

thread-count - which controls at any given point how many regular test methods can run concurrently.
data-provider-thread-count - which controls at any given point, how many data driven test methods can run concurrently.

When you enable both these two settings (and when you have a mix of regular and data driven test methods) what happens is, TestNG runs all the test methods concurrently and amongst them if there are test methods that are data driven in nature, those data driven iterations are executed in parallel as well.

Its like you spinning off a bunch of threads concurrently but one or more of those threads are internally spinning off additional threads.

This setting gives you the maximum throughput in terms of execution speed, but if you have a larger value (obtained by adding both these settings value) you can affect the overall performance, because now the JVM would start doing more context switching instead of scheduling the threads and getting the work done.

The rule of thumb for number of threads in simplistic terms be taken as 2N-1 (where N represents the number of cores in your processor. So if you have a quad core processor, your max thread count should be 7). This is an over-simplistic way of calculating the thread count, but being aware of this has helped me.