I'm trying to use the Maven Build Cache Extension to cache the results from one Maven command for use in another. For example:
./mvnw clean compile
./mvnw test # Should use cached compile
./mvnw package # Should use cached compile and test
./mvnw verify # Should use cached compile, test, and package
./mvnw install # Should use cached compile, test, package, and verify
./mvnw deploy # Should use cached compile, test, package, verify, and install
However, when I run the commands consecutively like this, I notice that it resumes where the previous command left off to start out with, but then it goes back and runs the Maven lifecycle from the beginning. Consider this output of ./mvnw install
after running ./mvnw verify
. It uses the previously cached steps and jumps straight to the install, as one would expect it to, but after that install finishes, it starts the Maven lifecycle over again from the compilation phase.
[INFO] --------------< com.example:foo-core >---------------
[INFO] Building foo-core 5.0.0-SNAPSHOT [2/27]
[INFO] from core/foo-core/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] Going to calculate checksum for project [groupId=com.example.foo, artifactId=foo-core]
[INFO] Scanning plugins configurations to find input files. Probing is enabled, values will be checked for presence in file system
[INFO] Found 421 input files. Project dir processing: 42, plugins: 6 millis
[INFO] Project inputs calculated in 77 ms. XX checksum [0a64c649f3c93a45] calculated in 151 ms.
[INFO] Attempting to restore project com.example.foo:foo-core from build cache
[INFO] Local build found by checksum 0a64c649f3c93a45
[INFO] Found cached build, restoring com.example.foo:foo-core from cache by checksum 0a64c649f3c93a45
[INFO] Project com.example.foo:foo-core restored partially. Highest cached goal: verify, requested: install
[INFO] Skipping plugin execution (cached): resources:resources
[INFO] Skipping plugin execution (cached): flatten:flatten
[INFO] Skipping plugin execution (cached): compiler:compile
[INFO] Skipping plugin execution (cached): resources:testResources
[INFO] Skipping plugin execution (cached): compiler:testCompile
[INFO] Skipping plugin execution (cached): surefire:test
[INFO] Skipping plugin execution (cached): jar:jar
[INFO] Skipping plugin execution (cached): source:jar-no-fork
[INFO]
[INFO] --- install:3.1.2:install (default-install) @ foo-core ---
[INFO] Installing /home/me/sources/foo-5.0.x/core/foo-core/pom.xml to /home/me/.m2/repository/com/example/foo/foo-core/5.0.0-SNAPSHOT/foo-core-5.0.0-SNAPSHOT.pom
[INFO] Installing /home/me/.m2/build-cache/v1/com.example.foo/foo-core/0a64c649f3c93a45/local/foo-core.jar to /home/me/.m2/repository/com/example/foo/foo-core/5.0.0-SNAPSHOT/foo-core-5.0.0-SNAPSHOT.jar
[INFO] Installing /home/me/.m2/build-cache/v1/com.example.foo/foo-core/0a64c649f3c93a45/local/foo-core-sources.jar to /home/me/.m2/repository/com/example/foo/foo-core/5.0.0-SNAPSHOT/foo-core-5.0.0-SNAPSHOT-sources.jar
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ foo-core ---
[INFO] Copying 1 resource from src/main/java/com/example/foo/core/spring/schema to target/classes/com/example/foo/core/spring/schema
[INFO] Copying 8 resources from src/main/resources to target/classes
[INFO]
[INFO] --- flatten:1.6.0:flatten (flatten) @ foo-core ---
[INFO] Generating flattened POM of project com.example.foo:foo-core:jar:5.0.0-SNAPSHOT...
[INFO]
[INFO] --- compiler:3.13.0:compile (default-compile) @ foo-core ---
[INFO] Recompiling the module because of changed source code.
[INFO] Compiling 354 source files with javac [debug deprecation target 17] to target/classes
This behavior is not limited to this example either; I see the same behavior no matter which Maven command I pass to it: test
, package
, verify
, install
, or deploy
.
Is this something that the Maven Build Cache Extension supports, or am I misunderstanding how it is meant to be used? If it does support this usage, is this a bug or is it something I'm doing incorrectly? For reference, I'm using Java 17, Maven v3.9.6 (via the Maven Wrapper Script), Maven Build Cache Extension v1.1.0 and a multi-module project structured using Maven CI-Friendly Versions.
This issue should be fixed when version 1.2.0
of the extension is released. So to answer my questions:
As background information to this answer: to debug the issue myself, I cloned the source and built the extension locally to add some logging to see where the issue was. Much to my surprise, there was no issue on the latest version of the source code (1.1.1-SNAPSHOT
). I went through the commits between version 1.1.0
and the latest commit and found commit 5af2549, which fixes issue MBUILDCACHE-80 with the following commit message:
Fix Incremental builds with a higher goal than the highest cached goal is rebuilding the full project from scratch
- fix incremental build cache replay problem
- fix Issue67Test regression for restoration error is handled properly
- Add simple IncrementalRestoreTest unit test
- Fix restore cached artifact using build's final artifact name
- Resolve the dual meaning of
restored
variable usage in cache restore
The reported issue describes the error as I had observed it:
When executing a build for a higher goal (i.e. deploy) then the currently highest cached goal (i.e. verify), the extension skips cached executions and runs mojos between cached and current goals while missing to restore cached final artifacts into the project build directory. After that, it runs the full build again from the begining to rebuild the artifacts and save build cache. Instead of reducing the build time by reusing already packaged artifacts and executions, it almost doubles the time to re-run the deploy for release from scratch. It also causes the Maven source plugin (3.3.0) to fail due to a duplicate sources artifact error, causing the deploy build to fail.