Distributed build with maven?

Question

Currently, we have a Maven project with a few thousand tests that takes 2 hours to run.

We've tried running these in parallel, but since they are functional tests, each test configures the system in a specific way, which leads to race conditions and random test failures.

I'd like to spin up N servers on AWS, then have Maven divide my tests up, and run them on these servers (each server would run its tests sequentially, but all servers would run in parallel), then aggregate the results.

Is there any plug-in that does something like this?

I've seen something close to what I want implemented in Jenkins, but I'd prefer it to be Maven-driven, so developers can use it locally without having Jenkins installed.

Answer 1

I don't know any plugin that does exactly all of this out of the box, but I believe that the work can be acomplished.

I find myself using an iterator plugin when having to split work up. See examples here: http://khmarbaise.github.io/iterator-maven-plugin/

To do the actual upload and execution, one suggestion could be to use the ant plugin to scp the project over, and subsequently use sshexec to execute it.

   <build>
    ...
    <plugins>
    ...
      <plugin>
        <artifactId>maven-antrun-plugin</artifactId>
        <configuration>
          <tasks>
            <scp todir="${scp.user}:${scp.password}@${scp.host}:/${scp.dirCopyTo}" trust="true" failonerror="false">
              <fileset dir="${bundle.dir}" />
            </scp>
            <sshexec host="${scp.host}"
                username="${scp.user}"
                keyfile="${user.home}/.ssh/id_dsa"
                command="touch somefile"/>
          </tasks>
        </configuration>
        <dependencies>
          <dependency>
            <groupId>ant</groupId>
            <artifactId>ant-jsch</artifactId>
            <version>1.6.5</version>
          </dependency>
          <dependency>
            <groupId>com.jcraft</groupId>
            <artifactId>jsch</artifactId>
            <version>0.1.42</version>
          </dependency>
        </dependencies>
      </plugin>
...