Note that I'm happy with the maven-surefire-plugin sitting in the main project build as it gives me the successful builds I need.
But I still have a question on something that is puzzling me...
I'm running a build of a Maven project. There are two kinds of tests. The integration tests. These tests run against an external and totally empty database, before the war archive is being created and deployed. There are 12 such tests. And the acceptance tests. These tests run against an external and non empty database, using the war archive after it has been being created and deployed. There is 1 such test.
Now, when running with the command:
mvn clean install -Denv="test"
I can see the 12 integration tests have passed and the build is successful.
But when running the command:
mvn clean install -Denv="acceptance"
I can see 13 tests are considered and the build fails with the console log showing:
Tests in error:
testUser(it.kahoot.robot.rest.acceptance.UserControllerTest): I/O error on POST request for "http://localhost:8080/kahoot-rest/api/users/login":Connection refused; nested exception is java.net.ConnectException: Connection refused
Tests run: 13, Failures: 0, Errors: 1, Skipped: 0
The test results indicates the build attempts to run the acceptance test before the war archive has been created and deployed.
-------------------------------------------------------------------------------
Test set: it.kahoot.robot.rest.acceptance.UserControllerTest
-------------------------------------------------------------------------------
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 20.388 sec <<< FAILURE!
testUser(it.kahoot.robot.rest.acceptance.UserControllerTest) Time elapsed: 0.407 sec <<< ERROR!
org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://localhost:8080/kahoot-rest/api/users/login":Connection refused; nested exception is java.net.ConnectException: Connection refused
Here is how the packages are split: stephane@stephane-ThinkPad-X301:rest> tree
.
├── acceptance
│ ├── AbstractControllerTest.java
│ ├── BaseControllerTest.java
│ └── UserControllerTest.java
├── assertion
│ ├── PartResourceAssert.java
│ ├── UserResourceAssert.java
│ └── UserRoleResourceAssert.java
└── integration
├── AbstractControllerTest.java
├── BaseControllerTest.java
├── UserControllerTest.java
├── UserExceptionTest.java
└── WebSecurityTestConfiguration.java
My pom.xml file has a "test" profile for the integration tests, and an "acceptance" profile for the acceptance tests.
<profile>
<id>test</id>
<activation>
<property>
<name>env</name>
<value>test</value>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
<configuration>
<includes>
<include>**/integration/*.java</include>
</includes>
<excludes>
<exclude>**/acceptance/*.java</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>acceptance</id>
<activation>
<property>
<name>env</name>
<value>acceptance</value>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>failsafe-maven-plugin</artifactId>
<version>2.4.3-alpha-1</version>
<configuration>
<includes>
<include>**/acceptance/*.java</include>
</includes>
<excludes>
<exclude>**/integration/*.java</exclude>
</excludes>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
<version>9.3.4.RC0</version>
<executions>
<execution>
<id>start-jetty</id>
<phase>pre-integration-test</phase>
<goals>
<goal>start</goal>
</goals>
</execution>
<execution>
<id>stop-jetty</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
<configuration>
<stopKey>stop</stopKey>
<stopPort>8081</stopPort>
</configuration>
</execution>
</executions>
<configuration>
<scanIntervalSeconds>10</scanIntervalSeconds>
<webApp>
<contextPath>/kahoot-rest</contextPath>
</webApp>
<daemon>true</daemon>
<connectors>
<connector
implementation="org.eclipse.jetty.nio.SelectChannelConnector">
<port>8080</port>
<maxIdleTime>300000</maxIdleTime>
</connector>
</connectors>
</configuration>
</plugin>
</plugins>
</build>
</profile>
The interesting bit is this: when I move the maven-surefire-plugin out of the "test" profile and into the main project build, both commands end up with successful builds. The following plug-in:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
<configuration>
<includes>
<include>**/integration/*.java</include>
</includes>
<excludes>
<exclude>**/acceptance/*.java</exclude>
</excludes>
</configuration>
</plugin>
is moved out of the "test" profile and placed in the main project build, which results in both commands giving successful builds.
The naming schemas are documented at their appropriate plugin pages:
I would make the acceptance tests into a separate project and define the execution by using a profile.
The integration tests could be run by the appropriate life cylcle phase like integration-test
But based on your information you would like to start a jetty for the integration tests so it would make sense to make a separate module for the integration tests as well which can be activated by a profile.
The other important point of separating them is having a separate class path for integration test and acceptance tests which could mean having different dependencies and may be some kind of configuration (property files etc.) based on the tests.