Junit setup with caliper

Question

I am trying to wrap the caliper code in junit so the performance tests run as part of my unit tests. It seems to work - the caliper test actually runs, but it doesn't exit successfully. What's the proper way to set this stuff up?

import static org.junit.Assert.*;

import org.junit.Test;

import com.google.caliper.Runner;
import com.google.caliper.SimpleBenchmark;

public class CaliperBenchmarkExample extends SimpleBenchmark {

        public void timeNanoTime(int reps) {
          for (int i = 0; i < reps; i++) {
            System.nanoTime();
          }
        }

    @Test
    public void testPerformance() {
        Runner.main(CaliperBenchmarkExample.class, new String[] { });
        assertTrue(true);
    }
}

Answer 1

There's no mechanism to run Caliper tests as JUnit tests. Doing so is complicated because of the way that Caliper forks a child process to isolate your benchmarks. And Caliper benchmarks tend to run for several seconds which may harm test performance.

You may want to investigate caliper-ci, an open source project that runs Caliper benchmarks continuously.