Cucumber ExtentReport generation with TestNG during multiple thread execution

Question

I'm running tests in parallel on mobile devices using Cucumber and TestNG. My TestNG runner class is given below.

@CucumberOptions(
        features="src/test/resources/features",
        glue={"org.cucumber.stepdefs"},
        plugin = {
        "com.cucumber.listener.ExtentCucumberFormatter:" }, monochrome = true)

public class TestRunner extends BaseTest {

    private static TestNGCucumberRunner testRunner;

    @BeforeClass
    public void setUP() {   
        System.out.println("Thread = " + Thread.currentThread().getId() + " - object hash = " + this.hashCode());
        testRunner = new TestNGCucumberRunner(TestRunner.class);
        ExtentProperties extentProperties = ExtentProperties.INSTANCE;
        extentProperties.setReportPath("output/" + this.hashCode() + "-report.html");
    }

    @Test(description="Tests",dataProvider="features")
    public void setUpClass(CucumberFeatureWrapper cFeature) {
        testRunner.runCucumber(cFeature.getCucumberFeature());
    }

    @DataProvider(name="features")
    public Object[][] getFeatures() {
        return testRunner.provideFeatures();
    }

    @AfterClass
    public static void teardown() {
        testRunner.finish();
    }

}

Which is working as expected - my Cucumber tests are run in parallel on separate threads for each connected device (defined in my testng.xml file).

I'm using ExtentReports to generate reports after each run, though at the moment i'm getting only a single report, where I would like a separate report for each running thread.

What I can't understand is why this is happening. I've added console prints to check if there is an instant of the TestRunner class for each running thread, and indeed there is - my console log after a run contains the following (assuming 2 mobile devices are connected):

Thread = 33 - object hash = 923219673
Thread = 32 - object hash = 280884709

So this is working as i expect, except when it comes to generating the report, only 1 report containing all the test data is created. I was under the assumption that because i am setting the report path to a unique file name in each thread in the setUp() method that a report for each thread should be created?

It seems that when it comes to generating the report that the Cucumber ExtendReport plugin waits for all tests to finish and generates a bulk report by default. Is this correct? And if so, how might i change this?

Below is the main part of my pom.xml file for additional configuration information.

<properties>
      <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
   </properties>
   <dependencies>
      <!-- https://mvnrepository.com/artifact/info.cukes/cucumber-java -->
      <dependency>
         <groupId>info.cukes</groupId>
         <artifactId>cucumber-java</artifactId>
         <version>1.2.5</version>
      </dependency>
      <!-- https://mvnrepository.com/artifact/io.cucumber/cucumber-jvm -->
      <dependency>
         <groupId>io.cucumber</groupId>
         <artifactId>cucumber-jvm</artifactId>
         <version>4.2.0</version>
         <type>pom</type>
      </dependency>
      <!-- https://mvnrepository.com/artifact/info.cukes/cucumber-testng -->
      <dependency>
         <groupId>info.cukes</groupId>
         <artifactId>cucumber-testng</artifactId>
         <version>1.2.5</version>
         <scope>compile</scope>
      </dependency>
      <!-- https://mvnrepository.com/artifact/info.cukes/cucumber-picocontainer -->
      <dependency>
         <groupId>info.cukes</groupId>
         <artifactId>cucumber-picocontainer</artifactId>
         <version>1.2.5</version>
         <scope>test</scope>
      </dependency>
      <!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-server -->
      <dependency>
         <groupId>org.seleniumhq.selenium</groupId>
         <artifactId>selenium-server</artifactId>
         <version>3.4.0</version>
      </dependency>
      <!-- https://mvnrepository.com/artifact/org.testng/testng -->
      <dependency>
         <groupId>org.testng</groupId>
         <artifactId>testng</artifactId>
         <version>6.11</version>
      </dependency>
      <!-- https://mvnrepository.com/artifact/io.github.bonigarcia/webdrivermanager -->
      <dependency>
         <groupId>io.github.bonigarcia</groupId>
         <artifactId>webdrivermanager</artifactId>
         <version>1.7.1</version>
      </dependency>
      <!-- https://mvnrepository.com/artifact/com.aventstack/extentreports -->
      <dependency>
         <groupId>com.aventstack</groupId>
         <artifactId>extentreports</artifactId>
         <version>3.1.5</version>
      </dependency>
      <dependency>
         <groupId>com.vimalselvam</groupId>
         <artifactId>cucumber-extentsreport</artifactId>
         <version>3.0.1</version>
      </dependency>
      <!-- https://mvnrepository.com/artifact/info.cukes/cucumber-junit -->
      <dependency>
         <groupId>info.cukes</groupId>
         <artifactId>cucumber-junit</artifactId>
         <version>1.2.5</version>
         <scope>test</scope>
      </dependency>
      <!-- https://mvnrepository.com/artifact/io.appium/java-client -->
      <!-- https://mvnrepository.com/artifact/io.appium/java-client -->
      <dependency>
         <groupId>io.appium</groupId>
         <artifactId>java-client</artifactId>
         <version>6.0.0-BETA3</version>
      </dependency>
   </dependencies>
   <build>
      <plugins>
         <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <configuration>
               <source>1.8</source>
               <target>1.8</target>
            </configuration>
         </plugin>
         <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.19</version>
            <configuration>
               <parallel>tests</parallel>
               <threadCount>10</threadCount>
               <suiteXmlFiles>
                  <suiteXmlFile>testng.xml</suiteXmlFile>
               </suiteXmlFiles>
            </configuration>
         </plugin>
      </plugins>
   </build>

Answer 1

@BeforeClass
public void setUP() {   
    System.out.println("Thread = " + Thread.currentThread().getId() + " - object hash = " + this.hashCode());
    testRunner = new TestNGCucumberRunner(TestRunner.class);
    ExtentProperties extentProperties = ExtentProperties.INSTANCE;
    extentProperties.setReportPath("output/" + this.hashCode() + "-report.html");
}

This setup method is invoked twice in parallel. It you are changing setReportPath on ExtentProperties.INSTANCE which is shared between these threads. So the threads are overwriting each others results. You can verify this by printing out the hash of ExtentProperties.INSTANCE.