OCR & OpenCV: Difference between two frames on high resolution images

Question

According to this post OCR: Difference between two frames, I now know how to find pixel differences between two images with OpenCV.

I would like to improve this solution and use it with high resolution images (from a video) with rich content. The example above is not applicable with big images because the process is to slow (too much differences found, the "findCountours method" fills the tab with 250k elements which takes a huge time to process).

My application uses a RLE decoder to decode the compressed frames of the video. Once the frame is decoded, I would like to compare the current frame with the previous one in order to store the differences between the two frames in a "Mat" tab for example.

The goal of all of this is to be able to perform an analysis on the different pixels and to check if there is any latin character. This allows me to reduce the amount of pixels to analyze and to save precious time.

If anyone has other ideas instead of this one to perform such operations, feel free to propose it please.

Thank you for your help.

EDIT 1: Example of two high resolution images of a computer screen. These are for the moment the perfect example of what I'm trying to analyse. As we can see there is just a window as difference between the two big images and I would like to analyze just the new "Challenge" window for any character.

EDIT 2: I'm trying to tune the algorithm depending on the data analyzed. Typically on the two following pictures I only get the green lines as differences and no text at all (which is what is the most interesting). I'm trying to understand better how things work for this.

1st image:

2nd image:

3rd image:

As you can see I only have those green lines and never the text (at the best I can have just ONE letter when decreasing the countours[i].size())

Answer 1

In addition to the post you mentioned, you need to:

• When you binarize the mask, use a threshold higher then 0 to remove small differences.
• Remove some noise. You can find all connected components, and remove smaller ones.
• Find the area of the bigger connected components. You can use convexHull and fillConvexPoly to get the mask of the different objects on screen
• Copy the second image to a new image, with the given mask.

The result will look like:

Code:

#include <opencv2/opencv.hpp>
#include <vector>
using namespace std;
using namespace cv;

int main()
{
    Mat3b img1 = imread("path_to_image_1");
    Mat3b img2 = imread("path_to_image_2");

    Mat3b diff;
    absdiff(img1, img2, diff);

    // Split each channel
    vector<Mat1b> masks;
    split(diff, masks);

    // Create a black mask
    Mat1b mask(diff.rows, diff.cols, uchar(0));

    // OR with each channel of the N channels mask
    for (int i = 0; i < masks.size(); ++i)
    {
        mask |= masks[i];
    }

    // Binarize mask
    mask = mask > 100;

    // Results images
    vector<Mat3b> difference_images;

    // Remove small blobs
    //Mat kernel = getStructuringElement(MORPH_RECT, Size(5,5));
    //morphologyEx(mask, mask, MORPH_OPEN, kernel);

    // Find connected components
    vector<vector<Point>> contours;
    findContours(mask.clone(), contours, CV_RETR_EXTERNAL, CHAIN_APPROX_NONE);

    for (int i = 0; i < contours.size(); ++i)
    {
        if (contours[i].size() > 1000)
        {
            Mat1b mm(mask.rows, mask.cols, uchar(0));

            vector<Point> hull;
            convexHull(contours[i], hull);


            fillConvexPoly(mm, hull, Scalar(255));

            Mat3b difference_img(img2.rows, img2.cols, Vec3b(0,0,0));
            img2.copyTo(difference_img, mm);

            difference_images.push_back(difference_img.clone());
        }
    }

    return 0;
}