Search code examples
build.gradleapache-flinkscoverage

How to use scoverage with Flink shadowjar custom gradle dependency configuration

Summary

I've got a Gradle Flink Scala project and am trying to add Scoverage reports, but the compileScoverageScala task can't find all the dependencies due to the custom shadow jar configuration.

build.gradle

Here's the build file, which follows Flink's Gradle example. The only difference is I've tried to add Scoverage. 

// Here are the plugins I'm using
plugins {
    id 'scala'
    id 'application'
    // shadow plugin to produce fat JARs
    id 'com.github.johnrengelman.shadow' version '5.2.0'
    id "org.scoverage" version "4.0.1"
    id "com.github.maiflai.scalatest" version "0.25"
}
...

// Here's the custom configuration used to build the shadow jar without including the main Flink libraries
configurations {
    flinkShadowJar // dependencies which go into the shadowJar

    // always exclude these (also from transitive dependencies) since they are provided by Flink
    flinkShadowJar.exclude group: 'org.apache.flink', module: 'force-shading'
    flinkShadowJar.exclude group: 'com.google.code.findbugs', module: 'jsr305'
    flinkShadowJar.exclude group: 'org.slf4j'
    flinkShadowJar.exclude group: 'log4j'
}
...

// Here's where the custom configuration is added to the classpath, Scoverage isn't picking these up.
sourceSets {
    main.compileClasspath += configurations.flinkShadowJar
    main.runtimeClasspath += configurations.flinkShadowJar

    test.compileClasspath += configurations.flinkShadowJar
    test.runtimeClasspath += configurations.flinkShadowJar

    javadoc.classpath += configurations.flinkShadowJar
}

Here's the build output:

$ ./gradlew clean build

> Task :compileScala
Pruning sources from previous analysis, due to incompatible CompileSetup.

> Task :compileScoverageScala FAILED
Pruning sources from previous analysis, due to incompatible CompileSetup.
/Users/david.perkins/dev/wffh/flink-validation-fhir/src/main/scala/com/ibm/watson/health/foundation/hri/flink/FhirValidationJob.scala:6: object core is not a member of package com.ibm.watson.health.foundation.hri.flink
import com.ibm.watson.health.foundation.hri.flink.core.BaseValidationJob
                                                  ^

The missing package is declared as a flinkShadowJar dependency. The compileScala task is able to find it, but compileScoverageScala can't.

Anyone know if there's a way to explicitly tell Scoverage to include the flinkShadowJar configuration? I'm hoping others using Flink have run into this before and know a way to fix it.

Solution

  • I got some help from the Scoverage team: https://github.com/scoverage/gradle-scoverage/issues/138

    Adding this to my sourceSets declaration fixed it.

        scoverage.compileClasspath += configurations.flinkShadowJar
    scoverage.runtimeClasspath += configurations.flinkShadowJar