Search code examples
c++opencvsegmentation-faultsurfobject-detection

OpenCV features detection and matching - segfault on drawing matches


Following this example,
I'm trying to build an application to recognize objects in a video.
My program is composed of the following steps (see code sample of each step below):

  1. Read the image of the object to be recognized into a cv::Mat object.
  2. Detect keypoints in the object and compute descriptors.
  3. Read each frame of the video,
  4. Detect keypoints and compute descriptors of the frame,
  5. Match the descriptors of the frame to the descriptors of the object,
  6. Draw results.

Problem: The 6'th step causes a segmentation fault (see code below).
Question: What causes it, and how can I fix it?

Thank you!

Notes:

  1. The program runs for several frames before the segfault. The crash occurs on frame 23, which is the first frame of the video that has any content (i.e. which is not completely black).
  2. By removing the line of drawMatches(...); there is no crash.
  3. Running on Windows 7, OpenCV 2.4.2, MinGW.

Debuggind attempt:

Running the program through gdb yields the following message:

Program received signal SIGSEGV, Segmentation fault.
0x685585db in _fu156___ZNSs4_Rep20_S_empty_rep_storageE () from c:\opencv\build\install\bin\libopencv_features2d242.dll

Step 1 - reading object's image:

Mat object;
object = imread(OBJECT_FILE, CV_LOAD_IMAGE_GRAYSCALE);

Step 2 - Detecting keypoints in the object and computing descriptors:

SurfFeatureDetector detector(500);
SurfDescriptorExtractor extractor;
vector<KeyPoint> keypoints_object;
Mat descriptors_object;
detector.detect(object , keypoints_object);
extractor.compute(object, keypoints_object, descriptors_object);

Steps 3-6:

VideoCapture capture(VIDEO_FILE);
namedWindow("Output",0);
BFMatcher matcher(NORM_L2,true);
vector<KeyPoint> keypoints_frame;
vector<DMatch> matches;
Mat frame,
    output,
    descriptors_frame;

while (true)
{
    //step 3:
    capture >> frame;
    if(frame.empty())
    {
        break;
    }
    cvtColor(frame,frame,CV_RGB2GRAY);

    //step 4:
    detector.detect(frame, keypoints_frame);
    extractor.compute(frame, keypoints_frame, descriptors_frame);

    //step 5:
    matcher.match(descriptors_frame, descriptors_object, matches);

    //step 6:
    drawMatches(object, keypoints_object, frame, keypoints_frame, matches, output);
    imshow("Output", output);
    waitKey(1);
}

Screenshot just before the segfault: Screenshot

Frame 22 (completely black): Frame 22

Frame 23 (in which the segfault occurs): Frame 23


Solution

  • The problem was with the order of the parameters in drawMatches.
    The correct order is:

    drawMatches(frame, keypoints_frame, object, keypoints_object, matches, output);
    

    Explanation:

    In step 5, I'm using the match method of the matcher object:

    matcher.match(descriptors_frame, descriptors_object, matches);
    

    The signature of this method is

    void match( const Mat& queryDescriptors, const Mat& trainDescriptors,
                CV_OUT vector<DMatch>& matches, const Mat& mask=Mat() ) const;
    

    Which means that matches contains matches from trainDescriptors to queryDescriptors.
    In my case, the train descriptors are of the object and the query descriptors are of the frame, so matches contains matches from the object to the frame.

    The signature of drawMatches is

    void drawMatches( const Mat& img1, const vector<KeyPoint>& keypoints1,
                      const Mat& img2, const vector<KeyPoint>& keypoints2,
                      const vector<DMatch>& matches1to2,
                      ... );
    

    When calling drawMatches with the incorrect order of parameters:

    drawMatches(object, keypoints_object, frame, keypoints_frame, matches, output);
    

    the method looks for the coordinates of matches in the incorrect image, which might result in trying to access "out-of-bounds" pixels; Thus the segmentation fault.