Search code examples
javaeclipsemavenarchiva

How can I push a large dependency tree from maven central to a local Archiva server?


So the company I work at has firewalls on our build servers, which means they can only access our internal Archiva maven server. To my knowledge, this server is also firewalled off and cannot retrieve content from maven central directly either.

So what people do if they need a certain library is, they go get the pom.xml, jar and other dependencies from maven central, and upload them to our Archiva server through the web interface.

This is sufficient for relatively small libraries with one or two dependencies at most, but very tedious for something large, such as Eclipse Aether, which I need at the moment.

So my question is, is there a way to pull down all the stuff from maven central to my local machine and then push it back up to my company's Archiva server?

I'm sorry if this is a rather stupid question; I'm not very familiar with uploading stuff to Maven and none of my coworkers understand what I'm talking about at all (many like to commit jars directly to our repo sigh).

Thanks!


Solution

  • Sorry this will not be a real answer but it also wont fit in a comment :)

    Short: talk to the firewall guys. It makes no sense to mess around and spend hours and days working around. You will sooner or later end up with a copy of maven central. Which is also a thing they might not want.

    It is common that developers are not allowed to connect to maven central. But thats a bit paranoid. Even those companies that copy jar by jar cannot verify the jar they copied manually does not contain an atomic bomb.

    There are some options:

    Use the dependency:go-offline goal: http://maven.apache.org/plugins/maven-dependency-plugin/go-offline-mojo.html

    this will download all dependencies and plugins. Execute the goals for your build as well. The result will be that you have the required artifacts in your local repository. I don't know what archiva can do. There might be an import of a folder structure. The seem to have different storage formats (file, jackrabbit, cassandra) for repositories. Maybe file would be an option. The Sonatype Nexus repository propxy stores everything in files. So you simply could copy your local repository in one of the managed ones, refresh the cache and there you go. (or put a nexus aside archiva and mirror it that way)

    Either way you need access to maven central somewhere. Usually people seem to do that at home and bring a USB stick... which makes no sense.

    Talk to the firewall people to allow archiva to conntect to maven central and maybe ibiblio and the jboss repos. That will save lots of time.

    If they are concerned on security I've seen no-one who actually reviewed all code in a project. Be honest with yourself. Faking security by restrictions that cause a lot of work does not lead to more security. If the scanners dont detect anything in the repository proxy then there is probably nothing.

    The nexus pro version (a bit costy) contains some sort of analyse feature to see if there are known security issues inside the used jars. If that calms people down. The price tag might bring them back up.

    That would be my suggestion: every developer is only allowed to connect to your archiva (mirrorOf=*) - as some sort of agreement to the policies/restrictions. The archiva server is allowed to connect to the world (only central, jboss-repo, ibiblio). This way you have full control of what is going on and developers are free to move around. If they move into the wrong direction you will notice that in a review (which contains dependencies) or through the activity on the maven proxy.

    Earlier on one could mirror maven central with rsync. But I've stopped thinking about it years ago. Its just a waste of time.

    Don't underestimate the time required to add dependencies to a proxy. Its a simple task - yes - but it is very time consuming. If the firewall guys want to pay for that: so be it. But dont get the numbers wrong: you need someone adding files, and while that happens other wait for him. Money out of the window. So people will write scripts that copy the dependencies to local repositories. Which means: no control of what is going on. That was one of the reasons for the repository proxy as well.

    Maven Central is there to be used.

    I hope you can get to a reasonable solution :)