Search code examples
c++opencvrectanglesrect

Detect rectangular shapes from camera

I need some feedback for a project I'm working on. Essentially I need to be able to track rectangular shapes from a video stream coming from a camera. I am using the OpenCV library for C++ to do so. First of all I applied a colour detection (I need to track only 1 colour at a time). After that I applied the Canny edge detection in order to get the contours of the filtered image. At this point I should be able to determine whether there is a rectangle or not, it's position in the (x,y) plane (knowing the position of it's center or vertices won't be such a huge difference after all) and it's orientation with respect to such a plane. All of this should be done in "real-time" in the sense that since the signal is coming from a camera I need to be able to track these feature of the shape shown in the stream.

This is a picture of the thresholded version of the input after the colour selection (left) and its contours, detected via Canny edge detection algorithm

This is the code in case anybody needs it:

#include <sstream>
#include <string>
#include <iostream>
#include <opencv/highgui.h>
#include <opencv/cv.h>

#define HIGH_CANNY_THRESH  255
#define CANNY_KERNEL_SIZE  3
#define       FRAME_WIDTH  640
#define      FRAME_HEIGHT  480

#define    DISPLAY_IMAGES  true

using namespace std;
using namespace cv;

void createTrackbarsForHSVSel();
void morphOps(Mat &thresh);

int LOW_H  = 0;
int HIGH_H = 255;
int LOW_S  = 0;
int HIGH_S = 255;
int LOW_V  = 0;
int HIGH_V = 255;

int LOW_THRESHOLD  = 0;
int HIGH_THRESHOLD = 100;

int     CORNER_THRESH = 200;
int MAX_CORNER_THRESH = 255;

int main(int argc, char* argv[])
{

   Mat src, hsvSpace, threshold, edges;

   vector<vector<Point> > contours;   // Vectors for the contours storage
   vector<Vec4i> hierarchy;

   createTrackbarsForHSVSel();   // create trackbars for the HSV palette
   createTrackbar("Min Threshold", "Trackbars", &LOW_THRESHOLD , HIGH_THRESHOLD);
   createTrackbar("Max Threshold", "Trackbars", &HIGH_THRESHOLD, HIGH_THRESHOLD);

   VideoCapture capture;
   capture.open(0);

   printf("Starting to capture from camera0:\nisOpened = %d\n", capture.isOpened());

   capture.set(CV_CAP_PROP_FRAME_WIDTH,FRAME_WIDTH);
   capture.set(CV_CAP_PROP_FRAME_HEIGHT,FRAME_HEIGHT);

   while(1)   // loop exectues as long as the user doesn't press ESC, q or Q
   {
      capture.read(src);   // read from camera
      cvtColor(src, hsvSpace, CV_BGR2HSV);   // RGB to HSV color space transformation
      // create a binary such that 1s are between Scalar(min_, min_, min_) and Scalar(max_, max_, max_)
      inRange(hsvSpace, Scalar(LOW_H, LOW_S, LOW_V), Scalar(HIGH_H, HIGH_S, HIGH_V), threshold);
      morphOps(threshold);   // morphological operations: they allow to close the 'hole' and delete the 'dots'

      // threshold now contains the binary that only displays one colour (if the trackbars are set correctly)

      // Apply Gaussian blurring and Canny edge algorithm for the edge detection
      GaussianBlur(threshold, threshold, Size(3,3), 0, 0);   // Kernel = 3x3, Sigmas are calculated automatically (see 'getGaussianKernel()')
      Canny(threshold, edges, LOW_THRESHOLD, HIGH_THRESHOLD);

      /*
         Algorithm that approximates the edges of the figure to a rectangle.
         After that it needs to be able to calculate the rectangle position and orientation
         (will something like RotatedRect be useful?)
      */

      #if DISPLAY_IMAGES == true
         // Show images
         imshow("Camera feed", src);
         imshow("Thresholded", threshold);
         imshow("Edges", edges);
      #endif

      if((char)waitKey(30) == 'q')
         break;
   }

   return 0;
}

void createTrackbarsForHSVSel()
{
   namedWindow("Trackbars", CV_WINDOW_AUTOSIZE);

   createTrackbar("Low  hue", "Trackbars", &LOW_H , HIGH_H );
   createTrackbar("High hue", "Trackbars", &HIGH_H, HIGH_H );
   createTrackbar("Low  sat", "Trackbars", &LOW_S , HIGH_S );
   createTrackbar("High sat", "Trackbars", &HIGH_S, HIGH_S );
   createTrackbar("Low  val", "Trackbars", &LOW_V , HIGH_V );
   createTrackbar("High val", "Trackbars", &HIGH_V, HIGH_V );

   return;
}

void morphOps(Mat &thresh)
{
    // create structuring element that will be used to "dilate" and "erode" image.
    // the element chosen here is a 3px by 3px rectangle.
        // As a rule of thumb you want to dilate with larger element to make sure the object is nicely visible

    erode (thresh,thresh,getStructuringElement( MORPH_RECT, Size(3,3)));
        dilate(thresh,thresh,getStructuringElement( MORPH_RECT, Size(3,3)));

    dilate(thresh,thresh,getStructuringElement( MORPH_RECT, Size(3,3)));
    erode (thresh,thresh,getStructuringElement( MORPH_RECT, Size(3,3)));

    return ;    
}

Thanks for the help!

Solution

  • If the rectangle is as well defined after thresholding as you've shown on the sample then this is a pretty basic task.

    1. Use findContours() to get the rectangle as a set of points.
    2. Use minAreaRect() to draw the rectangle around contours (make sure the orientation stays the same.
    3. Make sure to find only rectangles in case there's some noise in the future.
    4. MinAreaRect already consists of all the information you need: (x,y), (width, height), theta. Remember to check out this before you go mad with the angle being weird.

    Should work in real-time no problem. If you code something ineffectively and it doesn't, just do the processing every 2 or 3 frames.