How to set up subgit to mirror an svn repo that looks like a Windows Explorer hierarchy?

Being windows users, we created one svn repo with a hierarchy of folders. The bottom nodes contain the svn standard layout:

ProjectA/
    ApplicationOne/
        ModuleX/
            trunk/
            branches/
            tags/
    ApplicationTwo/
        ModuleY/
            trunk/
            branches/
            tags/

... and so on ad infinitum. The repo now contains around 100+ real svn projects with the trunk/branches/tags structure, but almost none of them at the top level.

How would I configure subgit to handle this?

Solution

SubGit can work in two different modes: local mirror mode and remote mirror mode. Below you can find a general overview of these modes and some recommendations for your particular case.

Local Mirror Mode

In this mode both Subversion and Git repositories reside on the same host, so SubGit has local access to both SVN and Git sides.

Below I've provided basic instructions. Please find detailed documentation and common pitfalls in SubGit 'Local Mode' Book.

Configuration

subgit configure <SVN_REPO>
SubGit version <VERSION> build #<BUILD_NUMBER>

Detecting paths eligible for translation... done.
Subversion to Git mapping has been configured:
  /ProjectA/ApplicationOne/ModuleX : <SVN_REPO>/git/ProjectA/ApplicationOne/ModuleX.git
  /ProjectA/ApplicationTwo/ModuleY : <SVN_REPO>/git/ProjectA/ApplicationTwo/ModuleY.git
  ...

CONFIGURATION SUCCESSFUL
...

This command tries to auto-detect repository layout and generate configuration file at <SVN_REPO>/conf/subgit.conf. It may take a while in case of big Subversion repository like yours.

Please make sure that auto-generated configuration file looks as follows, adjust it if necessary:

...
[git "ProjectA/ApplicationOne/ModuleX"]
  translationRoot = /ProjectA/ApplicationOne/ModuleX
  repository = git//ProjectA/ApplicationOne/ModuleX.git
  pathEncoding = UTF-8
  trunk = trunk:refs/heads/master
  branches = branches/*:refs/heads/*
  shelves = shelves/*:refs/shelves/*
  tags = tags/*:refs/tags/*
...

Authors mapping

At this stage you have to create /conf/authors.txt file that maps existing SVN usernames to Git authors. Please refer to documentation for more details.

Installation

Finally you have to import your Subversion repository to Git and enable synchronization by running subgit install command:

subgit install repo
SubGit version <VERSION> build #<BUILD_NUMBER>

Subversion to Git mapping has been found:
  /ProjectA/ApplicationOne/ModuleX : <SVN_REPO>/git/ProjectA/ApplicationOne/ModuleX.git
  /ProjectA/ApplicationTwo/ModuleY : <SVN_REPO>/git/ProjectA/ApplicationTwo/ModuleY.git
  ...

Processing '/ProjectA/ApplicationOne/ModuleX'
  Translating Subversion revisions to Git commits...
Processing '/ProjectA/ApplicationTwo/ModuleY'
  Translating Subversion revisions to Git commits...
  ...

  Subversion revisions translated: <REVISIONS_NUMBER>.
  Total time: <TIME_SPENT> seconds.

INSTALLATION SUCCESSFUL

Git Server

When the installation is over and synchronization between Subversion and Git repositories is enabled, you can setup some Git server (or reuse existing Apache HTTP server). Please refer to documentation on that and see a couple of posts on this topic in our blog:
- VisualSVN Server and SubGit
- Gitolite and SubGit

Remote Mirror Mode

When using this mode one has to install SubGit into Git repository only and keep this repository synchronized with remote Subversion server hosted on a different machine.

Below you can find some basic instructions. Please refer to SubGit 'Remote Mode' Book for more details.
1. Configuration
  
  In remote mirror mode SubGit does not try to auto-detect repository layout, so you have to run subgit configure --svn-url <SVN_URL> command for every module within Subversion repository:
```
subgit configure --svn-url <SVN_ROOT_URL>/ProjectA/ApplicationOne/ModuleX <GIT_ROOT_DIR>/ProjectA/ApplicationOne/ModuleX.git
SubGit version <VERSION> build #<BUILD_NUMBER>

Configuring writable Git mirror of remote Subversion repository:
  Subversion repository URL : <SVN_ROOT_URL>/ProjectA/ApplicationOne/ModuleX
  Git repository location   : <GIT_ROOT_DIR>/ProjectA/ApplicationOne/ModuleX.git

CONFIGURATION SUCCESSFUL
...
```
  As result SubGit generates configuration file <GIT_REPO>/subgit/config for every Git repository. For your case this configuration file should look as follows:
```
...
[svn]
  url = <SVN_ROOT_URL>/ProjectA/ApplicationOne/ModuleX
  trunk = trunk:refs/heads/master
  branches = branches/*:refs/heads/*
  tags = tags/*:refs/tags/*
  shelves = shelves/*:refs/shelves/*
  fetchInterval = 60
  connectTimeout = 30
  readTimeout = 60
  keepGitCommitTime = false
  auth = default

[auth "default"]
  passwords = subgit/passwd
  useDefaultSubversionConfigurationDirectory = false
  subversionConfigurationDirectory = <SVN_CONFIG_DIR>
...
```
2. Authors mapping
  
  At this stage you have to create /subgit/authors.txt file that maps existing SVN usernames to Git authors. Please refer to documentation for more details.
3. SVN credentials
  
  In case you're not using file:// protocol you have to provide necessary credentials, so SubGit is able to authenticate against Subversion server. For more information on that please read corresponding chapter in SubGit Book.
  
  We also recommend enabling pre-revprop-change hook on Subversion side which makes further installation and maintenance a bit easier, see SubGit Book.
4. Installation
  
  Finally you have to import your Subversion repository to Git and enable synchronization by running subgit install command:
```
subgit install git
SubGit version <VERSION> build #<BUILD_NUMBER>

Translating Subversion revisions to Git commits...

  Subversion revisions translated: <REVISIONS_NUMBER>.
  Total time: <TIME_SPENT> seconds.

INSTALLATION SUCCESSFUL
```
  This command also launches background process that polls SVN server and fetches new revisions when they appear there. Basically, that means that SubGit uses dedicated process for every Git repository. Sometimes it makes sense to avoid running such processes and use some job scheduler instead.
5. Git server
  
  Those links I've provided above are relevant for remote mode as well.
  
  However, if you're going to use Atlassian Stash for Git hosting, you can use SVN Mirror Plugin which is based on SubGit engine and provides some better experience with regards to UI and maintenance.

We have the following guideline which is based on our experience:

In case of many independent Subversion repositories, it's better to use SubGit in local mirror mode as it doesn't require SVN polling and maintaining additional process(es) for that.
In case of one giant Subversion repository with many modules, it's better to use remote mirror mode with file:// protocol and also adjust basic setup slightly.

It definitely doesn't make sense to run 100+ background processes in your case, instead we recommend installing additional post-commit SVN hook that checks what particular modules were modified by a given revision and then triggers synchronization for corresponding Git repositories.

If you have any other questions, feel free to ask us here at Stack Overflow, at our issue tracker or contact us via email: [email protected].