Finding Correctly Filled Answer Bubbles with OpenCV

Question

I'm designing an answer sheet scorer and currently only have 1 major issue left to deal with and its the answer bubbles. People can fill these in all sorts of ways Filled Answer Bubbles, I have tried using Cv2.HoughCircles() but it doesn't pick up the weird circles and since you have to specific a radius if its too small or too big it wont pick them up Example of HoughCircles attempt. If I was able to at least get all the circles I could probably use Cv2.CountNonZero() after finding the range of white space to consider an answer good/bad. Does anyone have any suggestions I could give a try? Any help is appreciated.

Here's the portion that makes them.

//Gray Image
Mat GrayImage = new Mat();
Cv2.CvtColor(startingImage, GrayImage, ColorConversionCodes.BGR2GRAY);
//Making clear
Mat thresholdImage = new Mat();
Cv2.Threshold(GrayImage, thresholdImage, 100, 255, ThresholdTypes.BinaryInv);
Mat guassianBlurImage = new Mat();
Cv2.GaussianBlur(thresholdImage, guassianBlurImage, new OpenCvSharp.Size(5, 5), 0);

Mat cannyImage = new Mat();
int lower = (int)Math.Max(0, (1.0 - 0.33) * 126);
int upper = (int)Math.Min(255, (1.0 + 0.33) * 126);
            Cv2.Canny(guassianBlurImage, cannyImage, lower, upper);

//Finding the Question circles
Mat copy = guassianBlurImage.Clone();
//Image (gray), type, dp, minDist, param1, param2, minRadius, maxRadius
var circles = Cv2.HoughCircles(copy, HoughModes.Gradient, 1, 10, 1, 25, 13, 18);

//Just so we can see the circles
Foreach (var cir in circles)
{
    //Debug.Print(cir.Radius.ToString());
    Cv2.Circle(startingImage, (int)cir.Center.X, (int)cir.Center.Y, (int)cir.Radius, Scalar.Green, 4);
}

Answer 1

I cleaned up my adobe template which had the circles. They were spaced wrongly so I fixed that. This then got me better images of each singular bubble using my custom method to tile the image. Below is how I call my method and a small example of what it produces:

List<Mat> questionMats = new List<Mat>();
utils.TileImage(WarpThresholdImage, 3, 8, false, questionMats);
List<Mat> bubbleMats = new List<Mat>();
int n = 0;
foreach (var mat in questionMats)
{
utils.TileImage(mat, 8, 1, false, bubbleMats, "bubble" + n);
                    n++;
}

After this I am able to determine the min/max of white pixels using Cv2.CountNonZero() kind of jankly by changing the test image with 3 different versions which have empty bubbles, all filled, and ones that are invalid. I used the following code.

//Sample each bubble get nonzero count find min and max for normal versions of filled and unfilled. Filter based on results
int min = 20000;
int max = 0;
/* These represent the test I did to confirm ranges
 * lowerNonZeroUnFilled = 849;
 * upperNonZeroUnFilled = 1328;
 * lowerNonZeroNormalFilled = 643;
 * upperNonZeroNormalFilled = 1261;
 * lowerNonZeroBadFilled = 602;
 * upperNonZeroBadFilled = 2201; 
 */
for (int i = 0; i < bubbleMats.Count(); i++)
{
    int total = Cv2.CountNonZero(bubbleMats[i]);
    //Empty Spaces
    if (total == 0) { }
    // 600 is the lowest value a filled circle will be
    if(total > 600)
    {
        if(total < min)
        {
            min = total;
        }
        if(max < total)
        {
            max = total;
        }
    //Cv2.ImShow("Bubble" + i + "-" + total, bubbleMats[i]);
    }
}

So this gets me what I want. This probably has issues but I'm confident I can deal with it later on.