Longer execution time of OpenCV function cv::remap() when program is put to sleep in between

Question

I am doing some image processing using OpenCV library and I discovered that the time it takes to process an image depends on the amount of time I put my thread to sleep in between image processing. I measured execution time of several parts of my program and I discovered that the function cv::remap() seems to execute two times slower if I put my thread to sleep for more then certain time period.

Below is the minimal code snippet which shows the strange behavior. I measure the time it takes to execute the cv::remap() function and then I send my threat to sleep for amount of milliseconds set in sleep_time.

#include <opencv2/imgproc.hpp>
#include <thread>
#include <iostream>

int main(int argc, char **argv) {
  cv::Mat src = ... // Init
  cv::Mat dst = ... // Init

  cv::Mat1f map_x = ... // Init;
  cv::Mat1f map_y = ... // Init;

  for (int i = 0; i < 5; i++) {
    auto t1 = std::chrono::system_clock::now();
    cv::remap(src, dst, map_x, map_y, cv::INTER_NEAREST, cv::BORDER_CONSTANT, 0);
    auto t2 = std::chrono::system_clock::now();
    std::chrono::duration<double> elapsed_time = t2 - t1;
    std::cout << "elapsed time = " << elapsed_time.count() * 1e3 << " ms" << std::endl;

    int sleep_time = 0;
    // int sleep_time = 20;
    // int sleep_time = 100;
    std::this_thread::sleep_for( std::chrono::milliseconds(sleep_time));
  }

   return 0;
}

If sleep_time is set to 0 the processing takes about 5 ms. Here is the output.

elapsed time = 5.94945 ms
elapsed time = 5.7458 ms
elapsed time = 5.69947 ms
elapsed time = 5.68581 ms
elapsed time = 5.7218 ms

But if I set the sleep_time to 100, the processing is more then two times slower.

elapsed time = 6.09076 ms
elapsed time = 13.2568 ms
elapsed time = 13.4524 ms
elapsed time = 13.3631 ms
elapsed time = 13.3581 ms

I tried out many different values for sleep_time and it seems that the execution doubles when the sleep_time is roughly three times higher then the elapsed_time (sleep_time > 3 * elapsed_time). If I increase the complexity of computation inside the function cv::remap() (e.g. increase the size of the processed image), then the sleep_time can be also set to higher values before the execution starts to double.

I am running my program on a embedded device with ARM processor iMX6 with linux operating system, but I was able to recreate the problem on my desktop running Ubuntu 16.04. I am using compiler arm-angstrom-linux-gnueabi-gcc (GCC) 7.3.0 and the Opencv version 3.3.0.

Does anybody have an idea what is going on?

Answer 1

This is probably your CPU frequency scaling kicking in.

The default frequency governor on Linux is usually "ondemand", which means clock speed is scaled down when load on CPU is low, and scaled back up when load increases. As this process takes some time, your short computation bursts fail to bring the clock speed up, and your process effectively runs on a slower CPU than you actually have.

I have tested this theory on my machine by executing

sudo cpupower frequency-set -g performance

and the effect immediately disappeared. To set the governor back, execute

sudo cpupower frequency-set -g ondemand