Search code examples
javagradlemaven-centralmaven-publish

How to generate all-projects-in-one Gradle project?

I have a gradle monolithic project with too many dependencies.

I'd like to explode it into many sub-projects and publish all sub-projects (build + sources + javadoc) + an extra project being the merge of all sub-projects.

This extra project should be like a virtual artifact with all my projects in a single jar like it is today because I don't want a too big change for my users.

The jar must not include dependencies (it is not an uber-jar) but the resulted pom.xml must contain the dependencies of all sub-projects (the generated pom.xml of the maven artifact must contain all dependencies).

The virtual artifact will include the merge of javadoc and sources too in order to respect Maven Central conventions.

Current state:

  • Project Main, generate
    • pom.xml
    • main.jar
    • main-sources.jar
    • main-javadoc.jar

Expected state:

  • Subproject A, generate
    • A-pom.xml
    • A.jar
    • A-sources.jar
    • A-javadoc.jar
  • Subproject B, generate
    • B-pom.xml
    • B.jar
    • B-sources.jar
    • B-javadoc.jar
  • virtal-Project Main, generate
    • pom.xml=A-pom.xml+B-pom.xml
    • main.jar=A.jar+B.jar
    • main-sources.jar=A-sources.jar+B-sources.jar
    • main-javadoc.jar=A-javadoc.jar+B-javadoc.jar

How can I manage it?

Solution

  • We have been in exactly the same situation for some time now. We want to publish a single artifact for our clients to depend on, although internally the product is developed through a few separate component projects. I got it done eventually (with compromises), and here is what I learned:

    1. Merging jars is not as straightforward as it looks like because there could be things like resource files within a jar that are not always namespace-ed. It is possible that two of your jars have a resource file with the same name, in which case you will have to merge the content of those files.

    2. Javadoc is very hard to merge without accessing the original source files because it has summary pages (index pages).

    So my advice would be:

    • Think twice, maybe what you really want is NOT a single jar, but a single dependency for your clients? These are different. You can easily have a pom only artifact. Depending on this pom only artifact will simply translates transitively into depending on individual artifacts of your component sub projects. To your client, practically, nothing is changed. Spring Boot takes this approach. To do it, you can create an empty java-library project, make all your component projects its api dependency. You don't even need any source code in this project.

    • If you really want to merge into a single jar, you can try building a fat jar with customization. The customization is not to pull in 3rd party dependencies.

    We use the Gradle Shadow plugin for merging jars. Its original purpose was to build a fat jar, which will include all the transitive dependencies. But it also has a special "shadow" configuration, to which you can add dependencies if you want the dependencies to be exported into POM rather than bundled. So what you need to do:

    1. Define a non-transitive configuration (say bundler) to which you will add your sub-project as dependencies. This is going to be the target configuration for the Gradle Shadow plugin.
    2. Define a transitive configuration (bundlerTransitive) that extends from your non-transitive one. This will be manually resolved in order to find the 3rd party dependencies
    3. in your build.gradle, register an afterEvaluate closure, where you find the level two dependencies of the resolved transitive configuration, add them to the shadow configuration. The reason for level-two is that level one dependencies will be your sub-project artifacts.
    4. After all the above, the artifact produced by shadowJar task is the one to be uploaded to maven. You will need to configure the shadowJar task to remove the classifier (which is shadow by default)

    Here is a complete example (build.gradle) of bundling vertx-web and all its dependencies within the io.vertx group:

    plugins {
    id 'java'
    id 'maven-publish'
    id 'com.github.johnrengelman.shadow' version '5.2.0'
}

group 'org.example'
version '1.0-SNAPSHOT'

repositories {
    mavenCentral()
}

configurations {
    bundler {
        transitive = false
    }

    bundlerTansitive {
        extendsFrom bundler
        transitive = true
    }
}

dependencies {
    bundler "io.vertx:vertx-web:4.0.0"
    bundler "io.vertx:vertx-web-common:4.0.0"
    bundler "io.vertx:vertx-core:4.0.0"
    bundler "io.vertx:vertx-auth-common:4.0.0"
    bundler "io.vertx:vertx-bridge-common:4.0.0"
}

shadowJar {
    configurations = [project.configurations.bundler]
    classifier ''
}


publishing {
    publications {
        shadow(MavenPublication) { publication ->
            project.shadow.component(publication)
        }
    }
}

project.afterEvaluate {
    // this is needed because your sub-projects might have inter-dependencies
    def isBundled = { ResolvedDependency dep ->
        return configurations.bundler.dependencies.any {
            dep.moduleGroup == it.group && dep.moduleName == it.name
        }
    }

    logger.lifecycle '\nBundled artifacts and their 1st level dependencies:'

    // level one dependencies
    configurations.bundlerTansitive.resolvedConfiguration.firstLevelModuleDependencies.forEach {
        logger.lifecycle "+--- ${it.getName()}"

        // level two dependencies
        it.children.findAll({ ResolvedDependency dep -> !isBundled(dep) })
                .forEach { ResolvedDependency dep ->
                    logger.lifecycle "|    +--- ${dep.name}"
                    project.dependencies.add('shadow', [group: dep.moduleGroup, name: dep.moduleName, version: dep.moduleVersion])
                }
    }

    logger.lifecycle '\nExported Dependencies:'

    configurations.shadow.getResolvedConfiguration().getFirstLevelModuleDependencies().forEach {
        project.logger.lifecycle "+--- ${it.getName()}"
    }
}

    For javadoc if you don't care about the index (compromise, as I said), then it is just a jar task with a copy spec:

    configurations {
   javadoc {
        transitive = false
    }
}

dependencies {
    javadoc 'com.my:component-a:1.1.0:javadoc'
    javadoc 'com.my:component-b:1.1.0:javadoc'
    javadoc 'com.my:component-c:1.1.0:javadoc'
    javadoc 'com.my:component-d:1.1.0:javadoc'
}

task javadocFatJar(type: Jar) {
    archiveClassifier.set('javadoc')
    from { 
        configurations.javadoc.collect { it.isDirectory() ? it : zipTree(it) } 
    }
    with jar
}