Search code examples
javamavenjunitconcurrencymaven-surefire-plugin

Running multiple test classes with maven surefire plugin

I'm trying to use the parallel option in the maven-surefire-plugin to run multiple test classes at once. Below is my configuration:

<configuration>
    <argLine>-Xmx1024m -XX:MaxPermSize=1024m</argLine>
    <excludes>
        <exclude>**/Test*.java</exclude>
    </excludes>
    <runOrder>hourly</runOrder>
    <parallel>classes</parallel>
    <threadCountClasses>5</threadCountClasses>
</configuration>

It very clearly is doing parallel test classes but it's doing far more than what I would expect. I see it jumping up to approximately 40 test classes being executed at once, despite the threadCountClasses being set to 5. Am I misunderstanding how these options work? My desire would be for only five test classes to be executing at any one time. As it is, it's choking my system with 40.

I'm currently running version 2.17 of the plugin.

Edit: Is there any chance the number of CPU cores would come into play here? After more careful accounting, I realize I am running exactly 40 test classes with this configuration and I have 8 cores. It seems like it would be quite a coincidence if the CPU cores don't play a part.

Solution

  • As per official documentation on the threadCountClasses configuration entry

    This attribute allows you to specify the concurrency in test classes, i.e.:

    • number of concurrent classes if threadCount is 0 or unspecified
    • limited classes concurrency if useUnlimitedThreads is set to true
    • if threadCount and certain thread-count parameters are > 0 for parallel, the concurrency is computed from ratio. For instance parallel=all and the ratio between threadCountSuites:threadCountClasses:threadCountMethods is 2:3:5, there is 30% of threadCount in concurrent classes.
    • as in the previous case but without this leaf thread-count. Example: parallel=suitesAndClasses, threadCount=16, threadCountSuites=5, threadCountClasses is unspecified leaf, the number of concurrent classes is varying from >= 11 to 14 or 15. The threadCountSuites become number of threads.

    Only makes sense to use in conjunction with the parallel parameter. The default value 0 behaves same as unspecified one.

    In your case you are not specifying the threadCount element, hence you should fall into the first case (number of concurrent classes) which however it doesn't seem to be the case.

    The third and fourth case instead define a ratio (and not a specific number), as it instead seems to happen in your case. Ratio however depends on your test classes (suites, if any, classes and methods) and not on your CPU cores.

    However, the perCoreThreadCount configuration entry:

    Indicates that threadCount, threadCountSuites, threadCountClasses, threadCountMethods are per cpu core.

    Is set to true by default, effectively influencing the dynamics described above and matching the behavior you described (according to option one above).

    Hence, a suggestion would be to set it to false.