Search code examples
c++opencvimage-processingkeypoint

OpenCV not generating descriptors

I am trying to use OpenCV for generating descriptors at keypoints in an image on iOS. I have done feature detection using my own algorithm and now want to extract the descriptors at these points.

I implemented the following code to do this:

cv::Mat frame = LH_ImageProcessing::extractGrayFromBGRA(Img, 480, 640);
std::vector<cv::KeyPoint> keyPoints;
cv::Mat descriptors;

for (int i = 0; i < cornersDetected; i++) {
  keyPoints.push_back(cv::KeyPoint((float) cornerArray[i*2], (float) cornerArray[i*2+1], 5));
}

cv::BriefDescriptorExtractor extractor;
extractor.compute(frame, keyPoints, descriptors);

However the descriptors Mat is always empty after I run the "compute" function. All the pointers are just NULL, although I can clearly see that the keyPoints array is a reduced size after I run it. Which means that it is removing keypoints that it can't extract descriptors for.

I thought it was my implementation so I used a built in detector (SurfDetector) and copied the implementation from the OpenCV FREAK example. But I end up with the same result.

Is anyone else having issues, or have I missed something fundamental in OpenCV?

EDIT:

So I have further tracked down the issue. It appears that the pass by reference doesn't change the original cv::Mat data structure.

The function declaration for the compute function looks as follows:

void DescriptorExtractor::compute( const Mat& image, std::vector<KeyPoint>& keypoints, Mat& descriptors ) const 
{
....
computeImpl( image, keypoints, descriptors );
}

The computeImpl is the function that actually computes the descriptors. In my case the BRIEF descriptor.

void BriefDescriptorExtractor::computeImpl(const Mat& image, std::vector<KeyPoint>& keypoints, Mat& descriptors) const

When the computeImpl function returns the descriptors variable is what I would expect. It has been initialized and contains the data I want. However, when the DescriptorExtractor::compute method returns the new descriptors structure isn't passed back to my main code, even though it is a pass by reference method.

What could be causing this?

EDIT 2:

Here are a few screenshots of my watch variables during debug.

The descriptors Mat inside the .compute function just before the function returns The descriptors Mat inside the .compute function just before the function returns

Just after the .compute function returns into my calling function Just after the .compute function returns into my calling function

Solution

  • So I finally figured this out. The trick is to use a pointer and initialize an empty cv::Mat and then pass this pointer to the compute function. OpenCV then reinitializes it in the .compute function and when everything returns the pointer is pointing to the new data structure.

    It seems that the problem came from the compiler optimizing out the creation of an empty cv::Mat, thus when compute returned the memory memory was freed.

    I changed:

    cv::Mat descriptors;
...
ext.compute(frame, keyPoints, descriptors);

    to 

    cv::Mat *descriptors = new cv::Mat;
...
ext.compute(frame, keyPoints, *descriptors);

    and all works perfectly now.