Search code examples
c++timebenchmarking

Benchmarking (execution time of a for loop): shouldn't we have an monotonic increasing function with the increase of the limit of the for loop


I want to benchmark a for loop. I decided to increment the variable in the for loop by 100 and measure the time accordingly.

#include <cstdio>
#include <ctime>
#include <time.h>
#include <iostream>
#include <random>
#include <iomanip>      // std::setprecision
using namespace std;

double difference(timespec start, timespec end);

int main()
{
    timespec time1, time2;

    for(int limit = 0; x < 100000; limit+= 100)
    {
      clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &time1);
      int temp = 0;
      for (int i = 0; i< limit; i++)
        temp+=temp;

      clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &time2);
      std::cout << std::fixed;
      std::cout << std::setprecision(5);
      cout<<x <<" " << difference(time1,time2)<<endl;
    }

    return 0;
}

double difference(timespec start, timespec end)
{
    timespec temp;
    if ((end.tv_nsec-start.tv_nsec)<0) {
        temp.tv_sec = end.tv_sec-start.tv_sec-1;
        temp.tv_nsec = 1000000000+end.tv_nsec-start.tv_nsec;
    } else {
        temp.tv_sec = end.tv_sec-start.tv_sec;
        temp.tv_nsec = end.tv_nsec-start.tv_nsec;
    }
    return (temp.tv_sec + temp.tv_nsec) / 1000000000.00;
}

Graph

The y-axis represents the time in seconds the x-axis represents the increasing limit of the iteration.

Hypothesis: With the increasing limit the time should increase. We should have a Strictly Increasing Function

The curvse shows otherswise. Why does it take 0.00001 seconds to loop 5300 times and 0.00002 seconds to loop 5400 times. As you can see in the graph there are plenty of times where we have this thing.

enter image description here


Solution

  • Hypothesis:

    A hypothesis is an explanation for a phenomenon. Since you're putting this forth before actually observing any phenomenon, it cannot be a hypothesis. Additionally if, as you claim, it directly contradicts the observations, it cannot work as an explanation for them, hence not a hypothesis.

    With the increasing limit the time should increase. We should have a Strictly Increasing Function

    And as far as I can tell the observation does not contradict this.

    The artifacts in the curve are easily explained by the clock's finite resolution: it clearly cannot measure time differences smaller than 1e-5; variations smaller than 1e-5 will show as flat lines or discrete 1e-5 changes.