Search code examples
mavenweb-crawlerstormcrawler

StormCrawler maven packaging error

I am trying to set up and run Storm Crawler and follow http://digitalpebble.blogspot.co.uk/2017/04/crawl-dynamic-content-with-selenium-and.html blog post.

The set of resources and configuration for StormCrawler are on my computer in /Users/deividas/git/selenium-tutorial

After running command "mvn clean package" the following error pops out:

"Could not resolve dependencies for project com.digitalpebble.crawl:selenium-tutorial:jar:1.0-SNAPSHOT: Could not find artifact ring-cors:ring-cors:jar:0.1.5 in central (https://repo.maven.apache.org/maven2)"

How can I solve this issue?

Thanks in Advance!

Solution

  • Not sure why this does not get downloaded in your case. I removed my local cache and ran the command below

        mvn clean package | grep cors

    Downloading: https://repo.maven.apache.org/maven2/ring-cors/ring-cors/0.1.5/ring-cors-0.1.5.pom
    Downloading: https://clojars.org/repo/ring-cors/ring-cors/0.1.5/ring-cors-0.1.5.pom
    Downloaded: https://clojars.org/repo/ring-cors/ring-cors/0.1.5/ring-cors-0.1.5.pom (4 KB at 5.0 KB/sec)
    Downloading: https://repo.maven.apache.org/maven2/ring-cors/ring-cors/0.1.5/ring-cors-0.1.5.jar
    Downloading: https://clojars.org/repo/ring-cors/ring-cors/0.1.5/ring-cors-0.1.5.jar
    Downloaded: https://clojars.org/repo/ring-cors/ring-cors/0.1.5/ring-cors-0.1.5.jar (6 KB at 24.4 KB/sec)

    As you can see, it gets downloaded from clojars.org

    This dependency is inherited from storm-core

     mvn dependency:tree | grep -C 20 cors

[INFO] com.digitalpebble.crawl:selenium-tutorial:jar:1.0-SNAPSHOT
[INFO] +- org.apache.storm:storm-core:jar:1.1.0:provided
[INFO] |  +- com.esotericsoftware:kryo:jar:3.0.3:provided
[INFO] |  |  +- com.esotericsoftware:reflectasm:jar:1.10.1:provided
[INFO] |  |  |  \- org.ow2.asm:asm:jar:5.0.3:provided
[INFO] |  |  +- com.esotericsoftware:minlog:jar:1.3.0:provided
[INFO] |  |  \- org.objenesis:objenesis:jar:2.1:provided
[INFO] |  +- org.clojure:clojure:jar:1.7.0:provided
[INFO] |  +- ring-cors:ring-cors:jar:0.1.5:provided

    If you look at the storm pom, you can see that the clojar repo is defined. As a workaround, you could copy the repository section to the pom of your project. Hopefully, the jar will get downloaded then. 

    <repositories>
    <repository>
        <releases>
            <enabled>true</enabled>
        </releases>
        <snapshots>
            <enabled>true</enabled>
        </snapshots>
        <id>clojars</id>
        <url>https://clojars.org/repo/</url>
    </repository>
</repositories>