Search code examples
gitperformance-testingbranching-and-merging

Is git suitable for A/B testing, code swapping?


I want to test multiple implementations for the same problem. For example I have a problem/code part "X" with implementation "X1" and "X2", and problem "Y" with implementation "Y1", "Y2", and "Y3". Think of them as some small, 10 line functions with slightly different codes inside them. Can I use git somehow to switch between these implementations, so replace the corresponding lines of code? I thought about branching, but I don't know how to apply it this way. I would like to switch between every implementation to a single problem, with any combination of problems. (So "X1" with "Y2", then "X2" with "Y1", etc.) Then I would like to measure the performance and choose the final implementation to use. Can I use git to achieve it?


Solution

  • This is an unorthodox use of git, and probably a simple configuration file where you choose which implementation to use would be a better approach.

    However, git is actually well suited for "code swapping", so you may for instance test different implementations like this:

    $ git init .
    $ git checkout -b  impl1
    $ cat << EOF > main.py
    > print "implementation1"
    > EOF
    $ git add -A && git commit -m "Implementation 1"
    $ git checkout -b master  ## You can probably skip this ...
    $ git checkout -b impl2
    $ rm main.py && cat << EOF > main.py
    > print "implementation2"
    > EOF
    $ git add -A && git commit -m "Implementation 2"
    

    Now you can switch between implementations like this:

    $ git checkout impl1
    $ git checkout impl2
    

    And test their performance, or whatever, against each other:

    $ git checkout impl1 && time python main.py
    Switched to branch 'impl1'
    implementation1
    python main.py  0,02s user 0,01s system 77% cpu 0,040 total
    
    $ git checkout impl2 && time python main.py
    Switched to branch 'impl2'
    implementation2
    python main.py  0,02s user 0,01s system 91% cpu 0,034 total
    

    Everything seems normal, print takes about the same time to print different strings :)