I have the below C++ code, which aims to detect shapes from a pre-designated image and draw around the shapes' perimeters. However, I wish to take this to the next step, and track the shapes from a camera feed rather than just an image. However, I am not to familiar with how I could make this transition.
#include <opencv2\opencv.hpp>
#include <opencv2\highgui\highgui.hpp>
int main()
{
IplImage* img = cvLoadImage("C:/Users/Ayush/Desktop/FindingContours.png");
//show the original image
cvNamedWindow("Raw");
cvShowImage("Raw", img);
//converting the original image into grayscale
IplImage* imgGrayScale = cvCreateImage(cvGetSize(img), 8, 1);
cvCvtColor(img, imgGrayScale, CV_BGR2GRAY);
//thresholding the grayscale image to get better results
cvThreshold(imgGrayScale, imgGrayScale, 128, 255, CV_THRESH_BINARY);
CvSeq* contours; //hold the pointer to a contour in the memory block
CvSeq* result; //hold sequence of points of a contour
CvMemStorage *storage = cvCreateMemStorage(0); //storage area for all contours
//finding all contours in the image
cvFindContours(imgGrayScale, storage, &contours, sizeof(CvContour), CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE, cvPoint(0, 0));
//iterating through each contour
while (contours) {
//obtain a sequence of points of contour, pointed by the variable 'contour'
result = cvApproxPoly(contours, sizeof(CvContour), storage, CV_POLY_APPROX_DP, cvContourPerimeter(contours)*0.02, 0);
//if there are 3 vertices in the contour(It should be a triangle)
if (result->total == 3) {
//iterating through each point
CvPoint *pt[3];
for (int i = 0; i < 3; i++) {
pt[i] = (CvPoint*)cvGetSeqElem(result, i);
}
//drawing lines around the triangle
cvLine(img, *pt[0], *pt[1], cvScalar(255, 0, 0), 4);
cvLine(img, *pt[1], *pt[2], cvScalar(255, 0, 0), 4);
cvLine(img, *pt[2], *pt[0], cvScalar(255, 0, 0), 4);
}
//if there are 4 vertices in the contour(It should be a quadrilateral)
else if (result->total == 4) {
//iterating through each point
CvPoint *pt[4];
for (int i = 0; i < 4; i++) {
pt[i] = (CvPoint*)cvGetSeqElem(result, i);
}
//drawing lines around the quadrilateral
cvLine(img, *pt[0], *pt[1], cvScalar(0, 255, 0), 4);
cvLine(img, *pt[1], *pt[2], cvScalar(0, 255, 0), 4);
cvLine(img, *pt[2], *pt[3], cvScalar(0, 255, 0), 4);
cvLine(img, *pt[3], *pt[0], cvScalar(0, 255, 0), 4);
}
//if there are 7 vertices in the contour(It should be a heptagon)
else if (result->total == 7) {
//iterating through each point
CvPoint *pt[7];
for (int i = 0; i < 7; i++) {
pt[i] = (CvPoint*)cvGetSeqElem(result, i);
}
//drawing lines around the heptagon
cvLine(img, *pt[0], *pt[1], cvScalar(0, 0, 255), 4);
cvLine(img, *pt[1], *pt[2], cvScalar(0, 0, 255), 4);
cvLine(img, *pt[2], *pt[3], cvScalar(0, 0, 255), 4);
cvLine(img, *pt[3], *pt[4], cvScalar(0, 0, 255), 4);
cvLine(img, *pt[4], *pt[5], cvScalar(0, 0, 255), 4);
cvLine(img, *pt[5], *pt[6], cvScalar(0, 0, 255), 4);
cvLine(img, *pt[6], *pt[0], cvScalar(0, 0, 255), 4);
}
//obtain the next contour
contours = contours->h_next;
}
//show the image in which identified shapes are marked
cvNamedWindow("Tracked");
cvShowImage("Tracked", img);
cvWaitKey(0); //wait for a key press
//cleaning up
cvDestroyAllWindows();
cvReleaseMemStorage(&storage);
cvReleaseImage(&img);
cvReleaseImage(&imgGrayScale);
return 0;
}
Any help in this matter is more than appreciated. Thank you!
If you're using C++, I'd start by rewriting this to use the C++ OpenCV API, instead of this old C one -- IMHO it's easier to use. In the process, refactor the code into smaller functions and decouple processing from I/O. Finally, consider that a video is just a sequence of images. If you can process one image, then you can process a video, one frame at a time.
So, how to achieve this. Let's start from the top, and write a main() function.
To read a video stream, we'll use cv::VideoCapture. We'll begin by initializing (and making sure that worked), and preparing some named windows to display the input and output frames.
Then we will start handling the individual frames in an infinite loop, quitting only when frame acquisition fails or the user hits an escape key. In each iteration we will:
Code:
int main()
{
cv::VideoCapture cap(0); // open the video camera no. 0
if (!cap.isOpened()) // if not success, exit program
{
std::cout << "Cannot open the video cam\n";
return -1;
}
cv::namedWindow("Original", CV_WINDOW_AUTOSIZE);
cv::namedWindow("Tracked", CV_WINDOW_AUTOSIZE);
// Process frames from the video stream...
for(;;) {
cv::Mat frame, result_frame;
// read a new frame from video
if (!cap.read(frame)) {
std::cout << "Cannot read a frame from video stream\n";
break;
}
process_frame(frame, result_frame);
cv::imshow("Original", frame);
cv::imshow("Tracked", result_frame);
if (cv::waitKey(20) == 27) { // Quit on ESC
break;
}
}
return 0;
}
NB: The use of cv::waitKey at an appropriate time is essential for the GUI to work. Read the documentation carefully.
With that done, it's time to implement our process_frame function, but first, let's make some useful global typedefs.
In the C++ API, a contour is a std::vector of cv::Point objects, and since more than one contour can be detected, we also need a std::vector of contours. Similarly, hierarchy is represented as a std::vector of cv::Vec4i objects. (the "is" is a lie-to-children, as it could be other data types too, but this is not important right now).
typedef std::vector<cv::Point> contour_t;
typedef std::vector<contour_t> contour_vector_t;
typedef std::vector<cv::Vec4i> hierarchy_t;
Let's work on the function -- it needs to take two parameters:
cv::Mat) which we don't want to modify, we just analyze it.We need to:
cv::cvtColor, so that we cancv::threshold it, binarizing the imagecv::findContours on the binary imageCode:
void process_frame(cv::Mat const& frame, cv::Mat& result_frame)
{
frame.copyTo(result_frame);
cv::Mat feedGrayScale;
cv::cvtColor(frame, feedGrayScale, cv::COLOR_BGR2GRAY);
//thresholding the grayscale image to get better results
cv::threshold(feedGrayScale, feedGrayScale, 128, 255, cv::THRESH_BINARY);
contour_vector_t contours;
hierarchy_t hierarchy;
cv::findContours(feedGrayScale, contours, hierarchy, cv::RETR_LIST, cv::CHAIN_APPROX_SIMPLE);
for (size_t k(0); k < contours.size(); ++k) {
process_contour(result_frame, contours[k]);
}
}
Last step, function to process a single contour. It needs:
cv::Mat) to draw onFirst, we want to approximate a polygon, using fraction of the perimeter length (we can use cv::arcLength to calculate that) as a parameter. We will continue by processing this approximated contour.
Next, we want to handle 3 specific cases: triangles, quadrilaterals and heptagons. We want to draw the contour of each of those using a different colour, otherwise we don't do anything. To draw the sequence of lines making up the contour, we can use cv::polylines.
Code:
void process_contour(cv::Mat& frame, contour_t const& contour)
{
contour_t approx_contour;
cv::approxPolyDP(contour, approx_contour, cv::arcLength(contour, true) * 0.02, true);
cv::Scalar TRIANGLE_COLOR(255, 0, 0);
cv::Scalar QUADRILATERAL_COLOR(0, 255, 0);
cv::Scalar HEPTAGON_COLOR(0, 0, 255);
cv::Scalar colour;
if (approx_contour.size() == 3) {
colour = TRIANGLE_COLOR;
} else if (approx_contour.size() == 4) {
colour = QUADRILATERAL_COLOR;
} else if (approx_contour.size() == 7) {
colour = HEPTAGON_COLOR;
} else {
return;
}
cv::Point const* points(&approx_contour[0]);
int n_points(static_cast<int>(approx_contour.size()));
polylines(frame, &points, &n_points, 1, true, colour, 4);
}
NB: std::vector is guaranteed to be continuous. That's why we can safely take a pointer by getting the address of the first element (&approx_contour[0]).
NB: Avoid using
using namespace std;
using namespace cv;
For more info, see Why is “using namespace std” considered bad practice?