I have a TestListener class, which implements ITestListener, which is responsible for taking pictures and saving the html if a selenium test fails.
My selenium tests have a parent class BaseTest, which helps do common tasks, like initiating each driver.
The content of each important area:
TestListener
public class TestListener implements ITestListener {
WebDriver driver=null;
ITestContext context = null;
String filePath = "artifacts/";
@Override
public void onTestFailure(ITestResult result) {
ITestContext context = result.getTestContext();
driver = (WebDriver) context.getAttribute("WebDriver");
System.out.println("***** Error "+result.getName()+" test has failed *****");
String methodName=result.getName().toString().trim();
String currentTime = getCurrentTime();
saveScreenShot(methodName, currentTime);
savePageSource(methodName, currentTime);
saveConsoleLog(methodName, currentTime);
}
...
BaseTest (each test calls this separately)
private void testngSetup(ITestContext context){
context.setAttribute("WebDriver", driver);
TestRunner runner = (TestRunner) context;
runner.setOutputDirectory(artifactLocation);
}
pom.xml
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.2.5</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>testng.xml</suiteXmlFile>
</suiteXmlFiles>
<useSystemClassLoader>false</useSystemClassLoader>
</configuration>
</plugin>
testng.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="esm4hpc">
<listeners>
<listener class-name="selenium.listeners.TestListener"/>
</listeners>
<test name="sample-Test" verbose="2" parallel = "classes">
<packages>
<package name=".*"/>
</packages>
</test>
</suite>
I have found this thread (How to get the current class driver in ItestListener), which suggested to change getting the driver in ITestListener the following way: (WebDriver)result.getTestClass().getRealClass().getDeclaredField("driver").get(result.getInstance()), but it throw java.lang.NoSuchFieldException : driver
public class TestListener implements ITestListener {
WebDriver driver=null;
ITestContext context = null;
String filePath = "artifacts/";
@Override
public void onTestFailure(ITestResult result) {
ITestContext context = result.getTestContext();
driver = (WebDriver)result.getTestClass().getRealClass().getDeclaredField("driver").get(result.getInstance());
System.out.println("***** Error "+result.getName()+" test has failed *****");
String methodName=result.getName().toString().trim();
String currentTime = getCurrentTime();
saveScreenShot(methodName, currentTime);
savePageSource(methodName, currentTime);
saveConsoleLog(methodName, currentTime);
}
...
I have also tried to set the parallel tag in testng.xml to false, but did not seem to help.
I also tried to list each testing class separately, but it also gave back wrong pictures.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="esm4hpc">
<listeners>
<listener class-name="selenium.listeners.TestListener"/>
</listeners>
<test name="sample-Test" verbose="2" parallel = "false">
<classes>
<class name="selenium.esm4hpc.AuthenticationTest"/>
<class name="selenium.esm4hpc.FileEditorTest"/>
<class name="selenium.esm4hpc.FileManagerTest"/>
<class name="selenium.esm4hpc.JobTest"/>
<class name="selenium.esm4hpc.ProjectTest"/>
</classes>
</test>
</suite>
The only thing seemed to help is to move each test class into different test-cases:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="esm4hpc">
<listeners>
<listener class-name="selenium.listeners.TestListener"/>
</listeners>
<test name="AuthenticationTest" verbose="2" parallel = "false">
<classes>
<class name="selenium.esm4hpc.AuthenticationTest"/>
</classes>
</test>
<test name="FileEditorTest" verbose="2" parallel = "false">
<classes>
<class name="selenium.esm4hpc.FileEditorTest"/>
</classes>
</test>
<test name="FileManagerTest" verbose="2" parallel = "false">
<classes>
<class name="selenium.esm4hpc.FileManagerTest"/>
</classes>
</test>
<test name="JobTest" verbose="2" parallel = "false">
<classes>
<class name="selenium.esm4hpc.JobTest"/>
</classes>
</test>
<test name="ProjectTest" verbose="2" parallel = "false">
<classes>
<class name="selenium.esm4hpc.ProjectTest"/>
</classes>
</test>
</suite>
But this causes the result files to be generated separately, which makes it cumbersome to debug failed cases.
Let's start with clearing out a few details.
ITestContext is TestNG's API way of pointing to a <test> tag (from the suite lingo)ITestResult is TestNG's API way of pointing to the result of the execution of a particular @Test or a configuration method.ITestContext. So one ITestContext can have one or more ITestResult objects in it.@Test method, if you invoke org.testng.Reporter.getCurrentTestResult() you will always have access to the currently running test method's result.ITestResult.setAttribute() method lets you attach attributes to a particular test method, similar to how ITestContext.setAttribute() allows you to attach attributes at the <test> tag level and ISuite.setAttribute() attaches attributes at the <suite> tag level.To fix issues that you are experiencing you would need to do the following:
driver = (WebDriver) context.getAttribute("WebDriver"); to driver = (WebDriver) testResult.getAttribute("WebDriver");ITestResult object of the currently running test, as an attribute using its setAttribute() method. You would need to ensure that the setup method is explicitly called via the @Test method or implicitly called via its run() method (If your base class is implementing IHookable interface that TestNG provides)The easiest way of ensuring that there's absolute thread-safety when it comes to sharing WebDriver instances and also in getting access to it in both your test classes as well as in your listeners is to do something like shown in the below sample.
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.ITestListener;
import org.testng.ITestResult;
import org.testng.Reporter;
import java.util.Optional;
public class WebDriverLifeCycleManager implements ITestListener {
public static final String DRIVER = "driver";
public static RemoteWebDriver driver() {
return Optional.ofNullable(Reporter.getCurrentTestResult().getAttribute(DRIVER))
.map(it -> (RemoteWebDriver) it)
.orElseThrow(() -> new IllegalArgumentException("Could not find driver"));
}
@Override
public void onTestStart(ITestResult result) {
ChromeDriver driver = new ChromeDriver();
result.setAttribute(DRIVER, driver);
}
@Override
public void onTestSuccess(ITestResult result) {
cleanupDriverReference(result);
}
@Override
public void onTestFailure(ITestResult result) {
//Add logic to capture screenshot etc.,
cleanupDriverReference(result);
}
@Override
public void onTestFailedButWithinSuccessPercentage(ITestResult result) {
cleanupDriverReference(result);
}
@Override
public void onTestSkipped(ITestResult result) {
cleanupDriverReference(result);
}
private void cleanupDriverReference(ITestResult result) {
Optional.ofNullable(result.getAttribute(DRIVER))
.map(it -> (RemoteWebDriver) it)
.ifPresent(RemoteWebDriver::quit);
result.removeAttribute(DRIVER);
}
}
Here's a test class that explicitly uses the above listener (You can have this changed to use the service loading approach of automatically wiring in listeners as explained in the documentation )
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;
@Listeners(WebDriverLifeCycleManager.class)
public class ThreadSafeWebDriverSample {
@Test
public void test() {
WebDriverLifeCycleManager.driver().get("https://www.google.com");
}
}