plugin-under-test-metadata.properties not created by Gradle TestKit when running tests in IDEA

I am using Gradle 3.3 and trying to test a custom plugin with JUnit and Gradle TestKit. In plugin's build.gradle I have

version '0.1'

apply plugin: 'groovy'
apply plugin: 'java-gradle-plugin'

sourceCompatibility = 1.8

repositories {
    mavenCentral()
}

dependencies {
    compile project(':codegen-core')
    compile localGroovy()
    testCompile 'junit:junit:4.12'
}

The test is

package com.huawei.odmf.codegen.gradle

import org.gradle.testkit.runner.BuildResult
import org.gradle.testkit.runner.GradleRunner
import org.gradle.testkit.runner.TaskOutcome
import org.junit.Before
import org.junit.Rule
import org.junit.Test
import org.junit.rules.TemporaryFolder

import static org.junit.Assert.*

class TestOdmfCodegenPlugin {

    @Rule 
    public final TemporaryFolder testProjectDir = new TemporaryFolder()

    private File buildFile
    private File assetsDir

    @Before
    void setUp() {
        buildFile = testProjectDir.newFile("build.gradle")
        assetsDir = testProjectDir.newFolder("src", "main", "assets")
    }

    @Test
    void testPlugin() {
        buildFile << """
          plugins {
          id 'com.huawei.odmf'
        }

        apply plugin: 'com.android.application'

        odmf {
            modelFile 'odmf.xml'
        }
        """

        BuildResult result = GradleRunner.create().
                withProjectDir(testProjectDir.root).
                withArguments(OdmfCodegenPlugin.taskName).
                withPluginClasspath().
                build()

        // assertions
    }
}

Under src/main/resources/META-INF/gradle-plugins/com.huawei.odmf.properties I have

implementation-class=com.huawei.odmf.codegen.gradle.OdmfCodegenPlugin

This seems to be all that's required according to https://docs.gradle.org/current/userguide/test_kit.html#sub:test-kit-automatic-classpath-injection and the automaticClasspathInjectionQuickstart sample.

However, this test fails at withPluginClasspath() (EDIT: when run in IDEA; it works from command line) with the following stack trace (to my understanding, plugin-under-test-metadata.properties is supposed to be created by java-gradle-plugin automatically):

org.gradle.testkit.runner.InvalidPluginMetadataException: Test runtime classpath does not contain plugin metadata file 'plugin-under-test-metadata.properties'

  at org.gradle.testkit.runner.internal.PluginUnderTestMetadataReading.readImplementationClasspath(PluginUnderTestMetadataReading.java:44)
  at org.gradle.testkit.runner.internal.PluginUnderTestMetadataReading.readImplementationClasspath(PluginUnderTestMetadataReading.java:37)
  at org.gradle.testkit.runner.internal.DefaultGradleRunner.withPluginClasspath(DefaultGradleRunner.java:146)
  at org.gradle.testkit.runner.internal.DefaultGradleRunner$withPluginClasspath$0.call(Unknown Source)
  at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
  at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
  at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
  at com.huawei.odmf.codegen.gradle.TestOdmfCodegenPlugin.testPlugin(TestOdmfCodegenPlugin.groovy:40)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:497)
  at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
  at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
  at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
  at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
  at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
  at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
  at org.junit.rules.RunRules.evaluate(RunRules.java:20)
  at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
  at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
  at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
  at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
  at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
  at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
  at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
  at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
  at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
  at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
  at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
  at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
  at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
  at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

What am I missing?

Solution

plugins {
    id "org.jetbrains.gradle.plugin.idea-ext" version "0.4.2"
}

task fixIdeaPluginClasspath {
    doFirst {
        configure(tasks.pluginUnderTestMetadata) {
            def ideaClassesPath = project.buildDir.toPath().resolveSibling("out").resolve("production")
            def newClasspath = pluginClasspath as List
            newClasspath.add(0, ideaClassesPath)
            pluginClasspath.setFrom(newClasspath)
        }
    }
}
pluginUnderTestMetadata.mustRunAfter(fixIdeaPluginClasspath)

idea.project.settings {
    taskTriggers {
        beforeBuild fixIdeaPluginClasspath, pluginUnderTestMetadata
    }
}

This works with IDEA 2019.1 (and may very well work with earlier versions as well).

This utilizes JetBrains' own gradle plugin for configuring IDEA settings to execute both pluginUnderTestMetadata and the custom fixIdeaPluginClasspath before each build (the latter will only run from within the IDEA, not when running native gradle).

The first task -- pluginUnderTestMetadata -- makes sure to create the properties file, and is executed by native Gradle as well.

The second task -- fixIdeaPluginClasspath -- fixes another bug with how IDEA executes tests: The classpath generated by pluginUnderTestMetadata will only contain reference to the "$projectDir/build" directory, which is not where IDEA outputs its compiled classes; hence, you will not see changes you've made in the plugin code that were compiled by IDEA, but only those compiled by native gradle. What it does then is to prepend the IDEA classes directory to the classpath. At first I also tried removing the "$projectDir/build" reference(s), but gradle then didn't like it complaining about plugin namespace problems (too voodoo for me).

Thanks to @krzychu for pointing out pluginUnderTestMetadata (on an earlier answer's comment).