Selenium Grid always executes an unwanted extra instance of my test

Question

Background

I have a Selenium Grid project set up to execute my test in two different browsers, Chrome and Firefox. I'm using Gradle to execute my tests. The test will successfully execute twice, once in Chrome and once in Firefox, as expected, and then a third instance will execute in the default browser and fail.

Expected Outcome

1. A Chrome instance will open, the spec will run, and it will pass.
2. A Firefox instance (using geckodriver) will open, the spec will run, and it will pass.
3. The Gradle task will finish successfully.

Actual Outcome

1. A Chrome instance will open, the spec will run, and it will pass.
2. A Firefox instance (using geckodriver) will open, the spec will run, and it will pass.
3. A new Firefox instance (using the firefoxdriver) will open, the spec will not run, and it will fail.
4. The Gradle task will fail as the last test execution failed.

My Thoughts

• I've run into issues with Gradle executing Spock tests twice before. to fix this, I had to add the following code:

  test {
      actions = []
  }
  

• I've also noticed that when my Selenium test gets executed again, it will open it up using the default firefox driver, not the gecko or marionette driver.

  • The firefox driver is old and doesn't support the latest version of Firefox, but it's Selenium's "default" browser when you don't specify a browser to execute your test in.
• I have two Selenium Grid nodes set up, so I'm wondering if Gradle is executing a third version of my test that doesn't match up with one of the nodes, but I'm only telling it to run two tests.

Code

I created a sample project that reproduces this issue on Bitbucket. Instructions on how to run the sample test is included in the readme.

As a snippet, this is the sample spec I have:

class W3SchoolsFormExampleSpec extends Specification {

    def 'Test form submission is successful on W3Schools'() {
        when: 'Name info is submitted into the form'
            open('https://www.w3schools.com/html/html_forms.asp')

            $(byName('firstname')).setValue('Clark')
            $(byName('lastname')).setValue('Kent')

            $x('//*[@id="main"]/div[3]/div/form/input[3]').click()

        and: 'Switch to newly opened tab'
            switchTo().window(1)

        then: 'New page should display the passed-in request params'
            $x('/html/body/div[1]').shouldHave(text('firstname=Clark&lastname=Kent'))
    }
}

and this is a snippet of my build.gradle file:

test {
    // Prevent Gradle from strangely executing Spock tests twice
    actions = []
}

task testW3SchoolsForm(type: Test) {
    outputs.upToDateWhen { false }

    doFirst {
        // Check to see that the Selenium drivers are installed
        if (!file("C:/Selenium/chromedriver.exe").exists()) {
            throw new GradleException(
                    'ERROR: Please install the web drivers in the correct location.'
            )
        }

        // Register the hub
        GridLauncherV3.main('-role', 'hub')

        // Register the Chrome and Firefox nodes
        GridLauncherV3.main('-role', 'node',
                '-browser', 'broswerName=chrome,platform=WINDOWS',
                '-hub', 'http://localhost:4444/grid/register',
                '-port', '4446'
        )
        GridLauncherV3.main('-role', 'node',
                '-browser', 'broswerName=firefox,platform=WINDOWS',
                '-hub', 'http://localhost:4444/grid/register',
                '-port', '4446'
        )
    }
}

enum BrowserType {
    CHROME('chrome'),
    FIREFOX('gecko')

    def browserString

    BrowserType(browserString) {
        this.browserString = browserString
    }
}

BrowserType.values().each { browserType ->
    tasks.create("testW3SchoolsForm${browserType}", Test) {
        // Force the tests to run every time
        outputs.upToDateWhen { false }

        // Allow parallel execution
        maxParallelForks = 3
        forkEvery = 0

        def drivers = [
                (BrowserType.CHROME): 'chromedriver.exe',
                (BrowserType.FIREFOX): 'geckodriver.exe'
        ]

        def browserProperty = browserType.browserString
        def webdriverPath = file("C:/Selenium/${drivers[browserType]}")

        // Set the respective system properties for each browser
        systemProperties["webdriver.${browserProperty}.driver" as String] = webdriverPath
        systemProperties['selenide.browser'] = browserType.browserString

        filter {
            include 'com/example/dummy/W3SchoolsFormExampleSpec.class'
        }

        testLogging {
            events 'PASSED', 'FAILED', 'STARTED', 'SKIPPED'
        }

        testW3SchoolsForm.dependsOn "testW3SchoolsForm${browserType}"
    }
}

Any ideas as to why a third instance of my test would execute in the default Selenium browser?

Answer 1

After a couple more days of trial and error and finding resolution by posting on the Gradle forums, I figured out the cause, and this may help future readers.

Take caution when creating a custom Test task.

By default, applying the Java or Groovy plugin will automatically create a default Test task which executes all tests under your test source directories.

When you create a custom test task by doing something like:

task testABC(type: Test) {
    filter {
        include "ABCTest"
    }
}

...executing this test task will also execute the default test task. If you wish to disable execution of the default test task, then set test.enabled = false.