Gradle is executing tests terribly slow because it is adding to much tests to the suite

Question

I have about 8 end-to-end-test classes that extend my abstract SpringContextLoadingTest class, which looks like this:

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
public abstract class SpringContextLoadingTest extends AbstractTestNGSpringContextTests {
}

I have main Application class with the @SpringBootApplication annotation.

As I use TestNG, I have some classes in one group ("channel A") and some in the other ("channel B").

I made gradle tasks for running separate groups:

task runChannelA(type: Test) {
    forkEvery = 1
    useTestNG() {
        includeGroups "channel A"
    }
}

Without "forkEvery = 1", there is a problem with busy ports when running more than 1 tests.

Thanks to this simple config below, I receive much more verbose output from gradle task execution:

tasks.withType(Test) {
    testLogging.showStandardStreams = true
}

Without it, it would looked like after the tests are executed, application hangs for 2 minutes at closing the EntityManagerFactory, but this flag revealed that gradle picked up tests it wasn't asked to. For every test, no matter in which group it is in, gradle is logging:

Gradle Test Executor 22 STANDARD_OUT
2016-12-21 17:10:00.115  INFO   --- [    Test worker] .b.t.c.SpringBootTestContextBootstrapper : Neither @ContextConfiguration nor @ContextHierarchy found for test class [mypackage.OtherTest], using SpringBootContextLoader
2016-12-21 17:10:00.141  INFO   --- [    Test worker] o.s.t.c.support.AbstractContextLoader    : Could not detect default resource locations for test class [mypackage.OtherTest]: no resource found for suffixes {-context.xml, Context.groovy}.
2016-12-21 17:10:00.143  INFO   --- [    Test worker] t.c.s.AnnotationConfigContextLoaderUtils : Could not detect default configuration classes for test class [mypackage.OtherTest]: DbCongestionExploratoryTest does not declare any static, non-private, non-final, nested classes annotated with @Configuration.
2016-12-21 17:10:00.455  INFO   --- [    Test worker] .b.t.c.SpringBootTestContextBootstrapper : Found @SpringBootConfiguration mypackage.Application for test class mypackage.OtherTest
2016-12-21 17:10:00.466  INFO   --- [    Test worker] .b.t.c.SpringBootTestContextBootstrapper : Using TestExecutionListeners: [org.springframework.test.context.web.ServletTestExecutionListener@9404cc4, org.springframework.test.context.support.DirtiesContextBeforeModesTestExecutionListener@46876feb, org.springframework.test.context.support.DependencyInjectionTestExecutionListener@dd46df5, org.springframework.test.context.support.DirtiesContextTestExecutionListener@49e2c374]

And it takes so much time because I have a lot of other tests. This is happening after I can see in IntelliJ that tests that I wanted to execute have passed. For example, I see after 25 seconds that the tests have passed, but because it is doing whatever the hell it is doing with every other test set up this way in my project, runChannelA takes more than 3 minutes. Funny thing is I can just stop the process during this strange behaviour, and the progress bar in IntelliJ just fills up to the end, and it is as nothing was going on, everything green and great.

Can someone help me with this ?

Answer 1

I have resolved my problem thanks to answer to this question- Reuse spring application context across junit test classes.

"Because Spring does not know when JUnit is done, it caches all context forever and closes them using JVM shutdown hook."

Thats why for different contexts you need different jvm, because context is removed only when shutting down the jvm.

Unfortunately turns out that if you use 'forkEvery =1' in gradle task, then Spring is forking jvm for every context you have in your classpath, including contexts not used in any of the tests your are currently executing.

I had many different contexts because I was trying to load for every test only minimum required configuration.

I have resolved my problem grouping tests by contexts as much as possible. I have expanded SpringContextLoadingTest to load more stuff. It now covers most of my tests. Tests using this context are run in one gradle task, without forkEvery = 1. Tests where I need to load different context, (because I have changed some property or loaded different version of a bean) I run in another gradle task, this time with forkEvery = 1 specified.