I'm trying to write simple UnitTests for basic sorting algorithms. While using gtest, I'm running into this strange problem, where tests from different suites are merging:
[==========] Running 20 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 10 tests from Suite_1/TestSortFunctions
[ RUN ] Suite_1/TestSortFunctions.TestUnsortedArray/Bubble_Sort
[ OK ] Suite_1/TestSortFunctions.TestUnsortedArray/Bubble_Sort (0 ms)
[ RUN ] Suite_1/TestSortFunctions.TestUnsortedArray/Insertion_Sort
[ OK ] Suite_1/TestSortFunctions.TestUnsortedArray/Insertion_Sort (0 ms)
[ RUN ] Suite_1/TestSortFunctions.TestUnsortedArray/Merge_Sort
[ OK ] Suite_1/TestSortFunctions.TestUnsortedArray/Merge_Sort (0 ms)
[ RUN ] Suite_1/TestSortFunctions.TestUnsortedArray/Quick_Sort
[ OK ] Suite_1/TestSortFunctions.TestUnsortedArray/Quick_Sort (0 ms)
[ RUN ] Suite_1/TestSortFunctions.TestUnsortedArray/Selection_Sort
[ OK ] Suite_1/TestSortFunctions.TestUnsortedArray/Selection_Sort (0 ms)
[ RUN ] Suite_1/TestSortFunctions.TestEmptyArray/Bubble_Sort
[ OK ] Suite_1/TestSortFunctions.TestEmptyArray/Bubble_Sort (0 ms)
[ RUN ] Suite_1/TestSortFunctions.TestEmptyArray/Insertion_Sort
[ OK ] Suite_1/TestSortFunctions.TestEmptyArray/Insertion_Sort (0 ms)
[ RUN ] Suite_1/TestSortFunctions.TestEmptyArray/Merge_Sort
[ OK ] Suite_1/TestSortFunctions.TestEmptyArray/Merge_Sort (0 ms)
[ RUN ] Suite_1/TestSortFunctions.TestEmptyArray/Quick_Sort
[ OK ] Suite_1/TestSortFunctions.TestEmptyArray/Quick_Sort (0 ms)
[ RUN ] Suite_1/TestSortFunctions.TestEmptyArray/Selection_Sort
[ OK ] Suite_1/TestSortFunctions.TestEmptyArray/Selection_Sort (0 ms)
[----------] 10 tests from Suite_1/TestSortFunctions (6 ms total)
[----------] 10 tests from Suite_2/TestSortFunctions
[ RUN ] Suite_2/TestSortFunctions.TestUnsortedArray/Bubble_Sort
[ OK ] Suite_2/TestSortFunctions.TestUnsortedArray/Bubble_Sort (0 ms)
[ RUN ] Suite_2/TestSortFunctions.TestUnsortedArray/Insertion_Sort
[ OK ] Suite_2/TestSortFunctions.TestUnsortedArray/Insertion_Sort (0 ms)
[ RUN ] Suite_2/TestSortFunctions.TestUnsortedArray/Merge_Sort
[ OK ] Suite_2/TestSortFunctions.TestUnsortedArray/Merge_Sort (0 ms)
[ RUN ] Suite_2/TestSortFunctions.TestUnsortedArray/Quick_Sort
[ OK ] Suite_2/TestSortFunctions.TestUnsortedArray/Quick_Sort (0 ms)
[ RUN ] Suite_2/TestSortFunctions.TestUnsortedArray/Selection_Sort
[ OK ] Suite_2/TestSortFunctions.TestUnsortedArray/Selection_Sort (1 ms)
[ RUN ] Suite_2/TestSortFunctions.TestEmptyArray/Bubble_Sort
[ OK ] Suite_2/TestSortFunctions.TestEmptyArray/Bubble_Sort (0 ms)
[ RUN ] Suite_2/TestSortFunctions.TestEmptyArray/Insertion_Sort
[ OK ] Suite_2/TestSortFunctions.TestEmptyArray/Insertion_Sort (1 ms)
[ RUN ] Suite_2/TestSortFunctions.TestEmptyArray/Merge_Sort
[ OK ] Suite_2/TestSortFunctions.TestEmptyArray/Merge_Sort (0 ms)
[ RUN ] Suite_2/TestSortFunctions.TestEmptyArray/Quick_Sort
[ OK ] Suite_2/TestSortFunctions.TestEmptyArray/Quick_Sort (1 ms)
[ RUN ] Suite_2/TestSortFunctions.TestEmptyArray/Selection_Sort
[ OK ] Suite_2/TestSortFunctions.TestEmptyArray/Selection_Sort (0 ms)
[----------] 10 tests from Suite_2/TestSortFunctions (4 ms total)
[----------] Global test environment tear-down
[==========] 20 tests from 2 test suites ran. (14 ms total)
[ PASSED ] 20 tests.
I intend Suite_1 tests to just test for TestUnsortedArray case for all algorithms, Suite_2 tests for TestEmptyArray case again for all algorithms. The sort functions look like simple C constructs, like: void(*algorithm)(int *array, int length). Here's code for my UnitTests file:
#include <gtest/gtest.h>
#include <vector>
#include <functional>
#include "BubbleSort.h"
#include "InsertionSort.h"
#include "MergeSort.h"
#include "QuickSort.h"
#include "SelectionSort.h"
namespace UnitTests
{
struct SortingAlgorithmTest : testing::Test
{
std::function<void(int *, int)> sorting_alg;
std::vector<int> input_array;
struct PrintToStringParamName
{
template <class ParamType>
std::string operator()(const testing::TestParamInfo<ParamType>& info) const
{
auto param = static_cast<TestSortParam>(info.param);
return param.name;
}
};
};
struct TestSortParam
{
const std::function<void(int *, int)> sorting_alg;
const std::string name;
const std::vector<int> test_array;
const std::vector<int> expected_result;
};
struct TestSortFunctions : SortingAlgorithmTest, testing::WithParamInterface<TestSortParam>
{
TestSortFunctions()
{
sorting_alg = GetParam().sorting_alg;
input_array = GetParam().test_array;
}
};
TEST_P(TestSortFunctions, TestUnsortedArray)
{
sorting_alg(input_array.data(), static_cast<int>(input_array.size()));
EXPECT_EQ(input_array, GetParam().expected_result);
}
INSTANTIATE_TEST_SUITE_P(Suite_1, TestSortFunctions,
testing::Values(
TestSortParam{ bubble_sort,
"Bubble_Sort",
{ 1, 5, 4, 0, 7, 2, 9, 3 },
{ 0, 1, 2, 3, 4, 5, 7, 9 }
},
TestSortParam{ insertion_sort,
"Insertion_Sort",
{ 1, 5, 4, 0, 7, 2, 9, 3 },
{ 0, 1, 2, 3, 4, 5, 7, 9 }
},
TestSortParam{ merge_sort,
"Merge_Sort",
{ 1, 5, 4, 0, 7, 2, 9, 3 },
{ 0, 1, 2, 3, 4, 5, 7, 9 }
},
TestSortParam{ quick_sort,
"Quick_Sort",
{ 1, 5, 4, 0, 7, 2, 9, 3 },
{ 0, 1, 2, 3, 4, 5, 7, 9 }
},
TestSortParam{ selection_sort,
"Selection_Sort",
{ 1, 5, 4, 0, 7, 2, 9, 3 },
{ 0, 1, 2, 3, 4, 5, 7, 9 }
}
), SortingAlgorithmTest::PrintToStringParamName()
);
TEST_P(TestSortFunctions, TestEmptyArray)
{
sorting_alg(input_array.data(), static_cast<int>(input_array.size()));
EXPECT_EQ(input_array, GetParam().expected_result);
}
INSTANTIATE_TEST_SUITE_P(Suite_2, TestSortFunctions,
testing::Values(
TestSortParam{ bubble_sort,
"Bubble_Sort",
{},
{}
},
TestSortParam{ insertion_sort,
"Insertion_Sort",
{},
{}
},
TestSortParam{ merge_sort,
"Merge_Sort",
{},
{}
},
TestSortParam{ quick_sort,
"Quick_Sort",
{},
{}
},
TestSortParam{ selection_sort,
"Selection_Sort",
{},
{}
}
), SortingAlgorithmTest::PrintToStringParamName()
);
}
int main(int argc, char **argv)
{
testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
Could someone help me find the error? Also, I am later thinking of moving the test arrays to one place, and I'd like to run the tests with the same respective parameters for all algorithms. What should be the right place to move these argument-sets to? Thanks!
This is functionality of GoogleTest by design - all TEST_P tests from TestSortFunctions will be run with all possible instantinations from INSTANTIATE_TEST_SUITE_P, as per documentation Please note that INSTANTIATE_TEST_SUITE_P will instantiate all tests in the given test suite, whether their definitions come before or after the INSTANTIATE_TEST_SUITE_P statement. here.
In your case just create two separate test suites: derive from TestSortFunctions and create TestSortFunctionsEmptyInputs and TestSortFunctionsNonEmpty and call INSTANTIATE_TEST_SUITE_P with these test suites if you really need to. However, I think it is perfectly OK to have one TestSortFunctions and just one TEST_P test, instantiated with Suite_1 and Suite_2. After all, all these belong to the same functions you want to test.