different behavior for different "for"s in benchmark

Question

We can write a simple benchmark using google benchmark or https://www.quick-bench.com/,

static void range_based_for(benchmark::State &state) {
    for (auto _ : state) {
        std::to_string(__LINE__);
    }
}
BENCHMARK(range_based_for);

We can also rewrite it with std::for_each,

static void std_for_each(benchmark::State &state) {
    std::for_each (std::begin(state), std::end(state), [] (auto _) {
        std::to_string(__LINE__);
    });
}
BENCHMARK(std_for_each);

Everything is good. However, when we use old school for statement, it runs without finishing.

static void old_school_for(benchmark::State &state) {
    for (auto iter = std::begin(state); iter != std::end(state); ++iter) {
        std::to_string(__LINE__);
    };
}
BENCHMARK(old_school_for);

In fact, std::for_each is implemented with this style. How can they behave different?

Answer 1

The begin/end functions are documented with a warning: says "These functions should not be called directly"

These functions should not be called directly.

REQUIRES: The benchmark has not started running yet. Neither begin nor end have been called previously.

end calls StartKeepRunning

And what does StartKeepRunning do? It resets the number of iterations to the maximum

It is clear that you're only supposed to call begin and end once. The difference in your third loop is that std::end(state) is called once per iteration, which apparently resets the iteration count back to the maximum.

I have no idea why the library is designed this way.