Poor results using FAST detector with BREIF extractor c++ OpenCV

Question

First, thanks for reading my question. I wrote a program that

• detect moving objects in the scene (from a stationary camera)
• track them
• try to match them with static database and recognize them

in this picture you can see the results in a sample video, the program is working fine , it detects the red car recognize it and it is tracking the car::

note:orange dots shows where extra samples are acquired for dynamic database for future use.

Problem: to put it in a nutshell my method is to

1. extract region of interests
2. local feature detection[FAST feature detector]
3. local feature extraction[SIFT descriptor extractor]
4. descriptor matching [Brute force matching]

I have 3 database that each object is compared to. I want my program to be as fast as it can be, because it's meant to run on an embedded board (like Raspberry pi 2) but it's quiet slow and not real-time at this point , I pinpoint the time consuming line and it is the SIFT descriptor extractor i tried to use other extractor that usually works well with FAST detector like BRIEF or ORB extractor and they run much faster than SIFT instead they return poor results and matching rate decrease significantly, would you please help me to understand how can I use this combination FAST detector/ BRIEF or ORB extractor/ BF or FLANN match here are mentioned functions::

void Objects::calKeypointDiscriptor(Mat inputFrame,Mat &ROI,Rect RegionArea,vector<KeyPoint> &fastKey, Mat &briefDescriptors,bool display)
{
    SurfFeatureDetector detectorSURF(300);
    SiftFeatureDetector detectorSIFT(400);
    FastFeatureDetector detectorFAST(30);
    OrbFeatureDetector  detectorORB (400);


    SurfDescriptorExtractor  extractorSURF;
    SiftDescriptorExtractor  extractorSIFT;
    OrbDescriptorExtractor   extractorORB;
    BriefDescriptorExtractor extractorBRIEF;
    FREAK                    extractorFREAK;

    Mat regionTemp;
    Mat frame=inputFrame;
    regionTemp=frame(RegionArea);
    ROI=regionTemp.clone();
    detectorFAST.detect(regionTemp, fastKey);
    extractorSIFT.compute(regionTemp, fastKey, briefDescriptors);

}

void Objects::matchDiscriptorFlann(Mat ROI,int distance,Point2i center,vector<KeyPoint>keypo,Mat descriptors,vector<Objects> objectVector,bool &matched,int vctorEnd,int &index)
{

    BFMatcher matcherBF(NORM_L2);
    FlannBasedMatcher matcherFLANN;
    Mat img_matches;


    for(int i=0; i<=vctorEnd; i++)
    {
        if (distance>0)
        {
            bool chk= euclideanDistance(objectVector[i]. center_obj,center)<distance;

        }
        else
        {
            bool chk=true;
        }


        vector< DMatch > good_matches;
        vector<DMatch> matches;


        if (descriptors.rows*descriptors.cols!=0&&objectVector[i].discriptor_obj.rows*objectVector[i].discriptor_obj.cols!=0)
        {
            matcherBF.match(descriptors,objectVector[i].discriptor_obj,matches);
            double max_dist = 0;
            double min_dist = 100;

            for( int i = 0; i < descriptors.rows; i++ )
            {
                double dist = matches[i].distance;
                if( dist < min_dist ) min_dist = dist;
                if( dist > max_dist ) max_dist = dist;
            }

            for( int i = 0; i < descriptors.rows; i++ )
            {
                if( matches[i].distance <=2*min_dist )
                {
                    good_matches.push_back( matches[i]);
                }
            }

            if(good_matches.size()>2)
            {

                matched=true;
                index=i;
            }
        }
    }
}

Answer 1

The Raspberry Pi cannot be expected to run floating point descriptors (such as SIFT or SURF) in real time. It simply does not have the processing power. Substituting faster binary detectors and descriptors is your best bet if running on the pi.

Currently, I would say your issues are:

• Using SIFT descriptor extractor and
• Using BruteForce matcher

If you wish to make your program run 'closer' to real-time on the pi, I suggest you look at the BRISK descriptor and the FLANN index kNN match.

These can both be tweaked and be quite reliable.

Also, you can use ORB (which is basically FAST but oriented) as your keypoint detector and BRISK as your desciptor. This gives decent results from my experience.

I haven't had much experience with LATCH or KAZE/AKAZE, however, I am unsure whether these will satisfy your performance requirements.

On a side note, in my experience, binary detectors such as ORB and FAST usually need to find about double the amount of points as more discriminant algorithms like SIFT or SURF. Luckily, this does not affect performance too much.