How to keep a local git mirror up-to-date without polling?

Question

The scenario: Our development team uses Jenkins for continuous integration, and some of our code is open source and therefore hosted at GitHub.com.

We have a local mirror of the relevant GitHub repositories, and our local GitBlit server is set to periodically poll the GitHub repositories to update the local mirror.

This "sort of works"; but the problem scenario is this:

1. A developer realizes he needs to make a change to the open-source codebase, so he pushes the change to the GitHub repository, and also updates the submodules in our closed-source Git repositories to point to the new revision.
2. The local developer then triggers an autobuild on Jenkins so he can test/verify that the changes work on all platforms
3. The Jenkins autobuild fails spectacularly because the local-mirror of the GitHub repository hasn't yet been updated to reflect the original, so when Jenkins tries to update the submodules in its various workspaces, the local-mirror doesn't recognize the revision ID that the closed-source git repositories are pointed at.

Our current work-around for this problem is to set GitBlit to poll GitHub more often, but I don't like that as a solution since it causes more periodic/unnecessary traffic across the Internet, and still doesn't entirely avoid the potential for build failures, e.g. in the case where a developer pushes changes and then triggers a build immediately afterwards.

Is there a known "best-practice" solution for this problem that would automatically give us reliable Jenkins-build-behavior and also avoid constantly polling GitHub?

Answer 1

You can use a Github webhook to notify your local infrastructure about following events:

• A repository is pushed to
• A pull request is opened
• A GitHub Pages site is built
• A new member is added to a team

Do note that this will minimize delay however in some cases e.g. network problems or Github infrastructure partial outage it can still fail your build.

Setting Jenkins autobuild to update the local mirror before the build is probably the only safe solution.