Search code examples
gittestingintegration-testingstress-testing

Stress Testing with git

I have an application that monitors git traffic of all kinds. The problem is is that I only am prepared to handle certain events like commits, branch creation/deletion, tag creation/deletion, and notes, but there are a lot of other funky situations git can get itself into.

Is there anything that anyone knows of that is like a thorough test of a large variety of git commit interactions that one could use to stress test an application meant to monitor git traffic?

Solution

  • a thorough test of a large variety of git commit interactions

    git/git/t includes a ton of tests, some of them could be used as starting point to be integrated into your stress tests.

    That 't' folder is interesting in that it does cover all git situation.

    Interestingly enough, Git 2.21 (Q1 2019) adds the feature to repeatedly run under load with the "--stress" option.

    See commit fb7d1e3, commit fa84058, commit 61f292d, commit 62c379b, commit 8cf5800, commit a9b2db3, commit 0a97e86 (05 Jan 2019), and commit d45cec4 (30 Dec 2018) by SZEDER Gábor (szeder).
    (Merged by Junio C Hamano -- gitster -- in commit 3fe47ff, 18 Jan 2019)

    test-lib: add the '--stress' option to run a test repeatedly under load

    Unfortunately, we have a few flaky tests, whose failures tend to be hard to reproduce.

    We've found that the best we can do to reproduce such a failure is to run the test script repeatedly while the machine is under load, and wait in the hope that the load creates enough variance in the timing of the test's commands that a failure is eventually triggered.
    I have a command to do that, and I noticed that two other contributors have rolled their own scripts to do the same, all choosing slightly different approaches.

    To help reproduce failures in flaky tests, introduce the '--stress' option to run a test script repeatedly in multiple parallel jobs until one of them fails, thereby using the test script itself to increase the load on the machine.

    The number of parallel jobs is determined by, in order of precedence:

    • the number specified as '--stress=<N>', or
    • the value of the GIT_TEST_STRESS_LOAD environment variable, or
    • twice the number of available processors (as reported by the 'getconf' utility), or 8.

    Make '--stress' imply '--verbose -x --immediate' to get the most information about rare failures; there is really no point in spending all the extra effort to reproduce such a failure, and then not know which command failed and why.