Search code examples
javamavenivytransitive-dependency

Ivy can not resolve the scope of a dependency which is a dependency of a transitive dependency


I add a dependency(let's name it as A) to ivy.xml which has a pom file in maven central. Ivy uses ibiblio for resolving the maven dependencies. The dependency(A) which is added to ivy.xml has a transitive dependency(B). So far so good till here. The dependency(C) of transitive dependency(B) can not be resolved by ivy.

I defined A in ivy.xml like this:

<dependency org="Z" name="A" rev="0.6-SNAPSHOT" conf="*->default"/>

In pom file of B, C is defined both in compile and test scopes like below:

<dependency>
      <groupId>X</groupId>
      <artifactId>C</artifactId>
    </dependency>
    <dependency>
      <groupId>X</groupId>
      <artifactId>C</artifactId>
      <type>test-jar</type>
      <scope>test</scope>
</dependency>

When I look the xml file of B which is resolved by ivy in ivy's cache file(~/.ivy2/cache/X/C/ivy-0.98.8-hadoop2.xml), it looks like this:

<dependency org="X" name="C" rev="0.98.8-hadoop2" force="true" conf="test->runtime(*),master(*)"/>
<dependency org="X" name="C" rev="0.98.8-hadoop2" force="true" conf="test->runtime(*),master(*)">
  <artifact name="C" type="test-jar" ext="jar" conf="" m:classifier="tests"/>
</dependency>

For this reason, ivy can not define C scopes correctly. For the record, I don't have permissions to modify the pom files as they are third party projects. How can I fix it ?


Solution

  • I reviewed the ivy usage of the nutch project and apologies but my conclusion is that it's overly complex for the following reasons:

    • "compile" and "test" targets are issuing separate calls to the resolve task
    • Each plugin is also calling an ivy resolve task
    • Complex logic for maintaining classpaths. Could be simplified using the cachepath task and ivy configurations.
    • Build plugins are not managed by ivy (Sonar, eclipse, rat)

    I started to refactor the build, but had to stop when I realised that I didn't understand the relationship between the main nutch artifact and the plugins... (I discovered NUTCH-1515 the hard way... big time-waster The feed plugin has missing dependencies).

    I also noticed issue NUTCH-1371 calling for the removal of ivy. This would be a tricky refactoring without significant change to the current codebase. I suspect it would have to be a multi-module build with each plugin listing its own dependencies.

    In conclusion, this work does not answer your question, but thought I needed to at least document the result of a few hours analysis :-) In light of NUTCH-1371 I don't know if your project will tolerant major ivy refactoring?

    Refactoring ivy

    Here follows what I achieved so far:

    Benefits:

    Impacts the following Nutch issues

    • NUTCH-1881 : This new approach removes resolve-test and resolve-default targets and manages the classpaths using ivy instead of the ${build.lib.dir}
    • NUTCH-1805 : Can easily setup a separate configuration for the job target with it's own dependencies.
    • NUTCH-1755 : I think this one is fixed by assigning a name to the the build.xml (see: diff)