Search code examples
javamavenselenium-webdrivergithub-actionstestng

Github Actions test run shows 0 tests, running locally all tests run properly

I've setup a repository for building/running selenium tests with TestNG and Maven. Running the tests locally on my machine the test run goes through with no issues, but after setting up Github actions for the repository, the runs there show 0/0 tests ran. There are no errors with the build process otherwise.

Here's a link to an example Actions run where this is happening.

Any ideas as to what is preventing the tests from running in actions?

project repository: https://github.com/camescasse/java-automation/

here's my actions.yml:

name: Selenium Tests

on:
  push:
    branches:
      - master
      - 38-fix-github-actions-test-run
  pull_request:
    branches:
      - master

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4
      - uses: actions/[email protected]
        with:
          distribution: 'zulu'
          java-version: '22'
          cache: 'maven'
      - name: Build with Maven, run tests
        run: mvn -B clean test

and my pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.example</groupId>
    <artifactId>java-automation</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <maven.compiler.source>22</maven.compiler.source>
        <maven.compiler.target>22</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencies>

        <!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-java -->
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>4.19.1</version>
        </dependency>

        <!-- https://mvnrepository.com/artifact/org.testng/testng -->
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>7.10.1</version>
            <scope>test</scope>
        </dependency>

        <!-- https://mvnrepository.com/artifact/com.google.truth/truth -->
        <dependency>
            <groupId>com.google.truth</groupId>
            <artifactId>truth</artifactId>
            <version>1.4.2</version>
            <scope>test</scope>
        </dependency>

    </dependencies>

</project>

I have so far tried manually installing chromedriver dependencies, even though they should be installed already with Selenium. Tried also downgrading to older java versions all the way down to 17.

Solution

  • I have tried to clone your project and run them locally. Here's what I have noticed:

    1. When you run them via GitHub actions, it looks like you are using maven-surefire-plugin:2.12.4:test. See here
    2. When you run locally you may not necessarily use the same version and so you won't see the same failure locally. The tests are definitely failing. But because of something Maven surefire plugin is perhaps just gobbling up stuff and reporting as if nothing ran. See below for console output from my local execution.
    [INFO] --- surefire:2.12.4:test (default-test) @ java-automation ---
[INFO] Surefire report directory: /Users/kmahadevan/githome/playground/so/java-automation/target/surefire-reports
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running BaseTest
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Configuring TestNG with: org.apache.maven.surefire.testng.conf.TestNG652Configurator@326de728
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.43 sec

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

    Now when you enhance your pom file to include the latest released version of surefire plugin (snippet looks like below)

    <build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.2.5</version>
        </plugin>
    </plugins>
</build>

    and then run the same test, you should see a failure such as below

    [INFO] --- surefire:3.2.5:test (default-test) @ java-automation ---
[INFO] Using auto detected provider org.apache.maven.surefire.testng.TestNGProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
.properties file not found
Apr 22, 2024 11:46:39 AM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 124, returning the closest version; found: 123; Please update to a Selenium version that supports CDP version 124
Apr 22, 2024 11:46:46 AM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 124, returning the closest version; found: 123; Please update to a Selenium version that supports CDP version 124
Apr 22, 2024 11:47:02 AM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 124, returning the closest version; found: 123; Please update to a Selenium version that supports CDP version 124
Apr 22, 2024 11:47:06 AM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 124, returning the closest version; found: 123; Please update to a Selenium version that supports CDP version 124
Apr 22, 2024 11:47:13 AM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 124, returning the closest version; found: 123; Please update to a Selenium version that supports CDP version 124
[ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 50.33 s <<< FAILURE! -- in TestSuite

    You might also want to consider adding one of the slf4j implementations to your pom file ( slf4j-simple.jar or slf4j-log4j12.jar or slf4j-jdk14.jar or logback-classic.jar ) as a dependency so that you start seeing logs