How do I find images inside an image?

Question

How do I find images inside an image?

Right now I'm using EmguCV and I have cooked up the following code based on this Tutorial: https://www.emgu.com/wiki/index.php/Shape_(Triangle,_Rectangle,_Circle,_Line)_Detection_in_CSharp

        using (UMat gray = new UMat())
        using (UMat cannyEdges = new UMat())
        {
            CvInvoke.CvtColor(img, gray, ColorConversion.Bgr2Gray);
            CvInvoke.GaussianBlur(gray, gray, new Size(3, 3), 2);
            CvInvoke.Canny(gray, cannyEdges, 0, 10, 3);

            LineSegment2D[] lines = CvInvoke.HoughLinesP(
                cannyEdges,
                1, //Distance resolution in pixel-related units
                Math.PI / 2, //Angle resolution measured in radians.
                0, //threshold
                40, //min Line width
                1); //gap between lines


            foreach (LineSegment2D line in lines)
            {
                CvInvoke.Line(img, line.P1, line.P2, new Bgr(Color.Red).MCvScalar, 1);
            }
        }

Step 1 Step 2 Step 3

This is my best result so far, but it's not perfect since i still need some edges to be able to create all bounding boxes around the images.

Step 4

What I want, is to find all edges of every single images in the image so I can make perfect bounding boxes like so.

I'm new to EmguCV/OpenCV, but i still think my best bet is on this library for solving this problem. I just need to find the right tools and use them right, and that's what I hope someone here can help me with :)

Answer 1

Here's what you can do:

• Load the image
• Invert - img.Not()
• Convert to Grayscale - img.Convert<Gray,byte>()
• Perform binary thresholding - img.Convert<Gray, byte>().ThresholdBinary(new Gray(54), new Gray(255))

It will result to the following image, optimize the threshold value to get better results:

• Get the convex hull and bounding box. Perform filtering so you get the objects that fits your criteria. You can use the contour area, perimeter, etc.

  var Contours = new List<Contour<Point>>();
  for (Contour<Point> contours = _gray.FindContours(
     HAIN_APPROX_METHOD.CV_CHAIN_APPROX_SIMPLE, RETR_TYPE.CV_RETR_EXTERNAL); 
     contours != null; contours = contours.HNext)
  {
      Seq<Point> pts = contours.GetConvexHull(ORIENTATION.CV_CLOCKWISE);
      double diff = Math.Round(Math.Abs(pts.Area - contours.Area) / 
        pts.Area, 2);
  
      //additional constraint
      double q = contours.Area / contours.Perimeter;
  
      //bounding box of the counter
      Rectangle rect = contours.BoundingRectangle;
  
      //customize the value to suit your need
      if (contours.BoundingRectangle.Height > 5 && 
       contours.BoundingRectangle.Width > 5)
      {
          Contours.Add(contours);
      }
  } 
  

• Do what you want with Contours, (e.g. fill and use as mask to extract each image, draw bounding box, etc.)