Search code examples
c++multithreadingunit-testinggoogletestnon-deterministic

What's the proper way using GTest to repeat a multi-threaded test multiple times?


Using Google Test, I want to test the behaviour of a Server.AcceptRequest method:

class Server {
public:
    // Clients can call this method, want to test that it works
    Result AcceptRequest(const Request& request) {
        queue_.Add(request);
        ... blocks waiting for result ...
        return result;
    }
private:
    // Executed by the background_thread_;
    void ProcessRequestsInQueue() {
        while (true) {
            Process(queue_.PopEarliest());
        }
    }

    MySynchronizedQueue queue_;
    std::thread background_thread_ = thread([this] {ProcessRequestsInQueue();});
};

The method accepts a client request, queues it, blocks waiting for a result, returns a result when available.

The result is available when the background thread processes the corresponding request from a queue.

I have a test which looks as follows:

TEST(ServerTest, TwoRequests) {
    Server server;
    Result r1 = server.AcceptClientRequest(request1);
    Result r2 = server.AcceptClientRequest(request2);
    ASSERT_EQ(r1, correctResultFor1);
    ASSERT_EQ(r2, correctResultFor2);
}

Since the implementation of a Server class involves multiple threads, this test might pass on one attempt but fail on another. To increase the chance of capturing a bug, I run the test multiple times:

TEST_P(ListenerTest, TwoRequests) {
  ... same as before ...
}
INSTANTIATE_TEST_CASE_P(Instantiation, ServerTest, Range(0, 100));

But now make test command treats each parameterised instantiation as a separate test, and in the logs, I see 100 tests:

Test 1: Instantiation/ServerTest.TwoRequests/1
Test 2: Instantiation/ServerTest.TwoRequests/2
...
Test 100: Instantiation/ServerTest.TwoRequests/100

Given that I do not use the parameter value, is there a way to rewrite the testing code such that the make test command would log a single test executed 100 times, rather than 100 tests?


Solution

  • Simple answer: use --gtest_repeat when executing tests would do the trick (default is 1).

    Longer answer: unit tests shouldn't be used for this kind of tests. GTest is thread-safe by design (as stated in their README), but this doesn't mean it is a good tool to perform such tests. Maybe it is a good starting point to actually begin working on real integration tests, I really recommend Python's behave framework for this purpose.