Search code examples
javagradlebuild.gradleartifact

Gradle is unable to find zip artifact in composite build if java plugin is applied

I have a Gradle project which creates a zip artifact. I define the artifact via artifacts.add('default', zipTask). I add this project to another project via includeBuild and use the zip as dependency (dependencies { myConfiguration 'org.example:testA:+@zip' }). So far so good. It works.

The problem starts when I add the plugin java to the first project. For some reason it prevents Gradle from finding the zip artifact. The error is:

Execution failed for task ':doubleZipTask'.
> Could not resolve all files for configuration ':myConfiguration'.
   > Could not find testA.zip (project :testA).

Why? How to fix it?

Complete example:

Project testA

settings.gradle:

rootProject.name = 'testA'

build.gradle:

plugins {
    id 'base'
    // Uncomment the line below to break the zip artifact
    //id 'java'
}

group = 'org.test'
version = '0.0.0.1_test'

task zipTask(type: Zip) {
    from './settings.gradle' // just so the zip isn't empty
}

artifacts.add('default', zipTask)

Project testB

settings.gradle:

rootProject.name = 'testB'

// This line may be commented out in some cases and then the artifact should be downloaded from Maven repository.
// For this question it should be always uncommented, though.
includeBuild('../testA')

build.gradle:

plugins {
    id 'base'
}

configurations {
    myConfiguration
}

dependencies {
    myConfiguration 'org.test:testA:0.0.0.+@zip'
}

task doubleZipTask(type: Zip) {
    from configurations.myConfiguration
}

Update 1

I've added some diagnostic code at the end of the build.grade:

configurations.default.allArtifacts.each() {
    println it.toString() + ' -> name: ' + it.getName() + ', extension: ' + it.getExtension()
}

and in the version with java plugin it prints:

ArchivePublishArtifact_Decorated testA:zip:zip: -> name: testA, extension: zip
org.gradle.api.internal.artifacts.dsl.LazyPublishArtifact@2c6aaa5 -> name: testA, extension: jar

However, I'm not sure if an additional artifact can break something.

It doesn't seem to be a problem when I add a second artifact myself.

Update 2

Maybe the zip file isn't the best representation of my intentions. After all, I could build java related files in one project and zip them in another. However, the problem also applies to war files. (War plugin internally uses the Java plugin so it cannot be run separately.)

Solution

  • The following setup should work with Gradle 5.6 (when using another attribute it will likely also work with previous versions). It mostly corresponds to your original setup with the exception of the changes indicated by XXX.

    Project testA

    settings.gradle:

    rootProject.name = 'testA'

    build.gradle:

    plugins {
    id 'base'
    // Uncomment the line below to break the zip artifact
    //id 'java'
}

group = 'org.test'
version = '0.0.0.1_test'

task zipTask(type: Zip) {
    from './settings.gradle' // just so the zip isn't empty
}

// XXX added an attribute to the configuration
configurations.default.attributes {
    attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE,
              project.objects.named(LibraryElements, 'my-zipped-lib'))
}

artifacts.add('default', zipTask)

    Project testB

    settings.gradle:

    rootProject.name = 'testB'

// This line may be commented out in some cases and then the artifact should be downloaded from Maven repository.
// For this question it should be always uncommented, though.
includeBuild('../testA')

    build.gradle:

    plugins {
    id 'base'
}

configurations {
    // XXX added the same attribute as in the testA project
    myConfiguration {
        attributes {
            attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE,
                      project.objects.named(LibraryElements, 'my-zipped-lib'))
        }
    }
}

dependencies {
    myConfiguration 'org.test:testA:0.0.0.+@zip'
}

task doubleZipTask(type: Zip) {
    from configurations.myConfiguration
}

    I have tested this setup both with and without the java plugin. I’ve also tested publishing to a Maven repository and letting testB take its dependency from there instead of from the included build of testA. Adding an additional dependency from testB on the JAR artifact of testA (myConfiguration 'org.test:testA:0.0.0.+@jar') worked, too.

    Some explanation on what (I believe) is going on: Gradle needs a way of determining automatically which local component/artifact in testA it can use for substituting the external dependency of testB.

    • Without applying the java plugin, there is only a single component/artifact and my guess is that Gradle then just selects that single one without further ado.
    • As you saw, by applying the java plugin, another artifact is added to testA. Now which one should Gradle take? One would expect that it would look at the file extension specified on the dependency in testB but that doesn’t seem to be the case. It seems that Gradle isn’t actually working at the artifacts level when substituting dependencies but rather at the module/component level. You might say we only have one component with two artifacts, so selecting the one component should be straightforward. But it seems that we actually have two variants of the same component and Gradle wants to select one of these variants. In your own testB setup, there is no clue given that would tell Gradle which variant to select; so it fails (with an admittedly bad/misleading error message). In my changed testB setup I provide the clue: I tell Gradle that we want the variant that has a specific attribute with the value my-zipped-lib. Since I’ve added the same attribute on the published configuration of testA, Gradle is now able to select the right variant (because there is only one that has the required attribute). The file extension on the dependency is still relevant in a second step: once Gradle has selected the component variant, it still needs to select the right artifact – but only then.

    Note that we’re actually working on the rim of what is supported with composite builds today. See also Gradle issue #2529 which states that “projects that publish non-jar artifacts” were not terribly well supported. When I first saw your question I honestly thought we’d be out of luck here … but it seems there’s a way to again step a little closer to the rim ;-)

    In the comments, the question came up why adding multiple custom artifacts works while applying the java plugin breaks the build. As I’ve tried to explain above, it’s not a question of multiple artifacts but of multiple component variants. IIUIC, these variants originate from different attributes on configurations. When you don’t add such attributes, then you will not have different components variants. However, the Java plugin does add such attributes which consequently leads to different component variants in your project(/component). If you’re interested, you can see the different attributes by adding something like the following to your build.gradle:

    configurations.each { conf ->
    println "Attributes of $conf:"
    conf.attributes.keySet().each { attr ->
        println "\t$attr -> ${conf.attributes.getAttribute(attr)}"
    }
}

    Now where and when to add which attributes? That depends on your setup. I wouldn’t blindly add attributes to all configurations in the hope that that’ll solve things magically. While it might work (depending on your setup), it’d certainly not be clean. If your project setup is as complex and/or special as your question suggests, then it’ll probably make sense to think more deeply about which configurations you need and what attributes they should carry. The first sections of the Gradle documentation page that I have linked above are probably a good starting point if you are not intricately familiar with these nuances of configurations in Gradle, yet. And yes, I agree that such logic would best live in a Gradle plugin.