Search code examples
opencvimage-processingcomputational-geometryperspectivecamera

Finding point height on a cup using OpenCV

Suppose that I want to find the 3D position of a cup with its rotation, with image input like this (this cup can be rotated to point in any direction): enter image description here

Given that I have a bunch of 2D points specifying the top circle and bottom circle like the following image. (Let's assume that these points are given by a person drawing the lines around the cup, so it won't be very accurate. Ellipse fitting or SolvePnP might be needed to recover a good approximation. And the bottom circle is not a complete circle, it's just part of a circle. Sometimes the top part will be occluded as well so we cannot rely that there will be a complete circle)enter image description here

I also know the physical radius of the top and bottom circle, and the distance between them by using a ruler to measure them beforehand.

I want to find the complete 2 circle like following image (I think I need to find the position of the cup and its up direction before I could project the complete circles):

enter image description here

Let's say that my ultimate goal is to be able to find the closest 2D top point and closest 2D bottom point, given a 2D point on the side of the cup, like the following image:

enter image description here

A point can also be inside of the cup, like so: enter image description here

Let's define distance(a, b) as a function that find euclidean distance from point a and point b in pixel units.

From that I would be able to calculate the distance(side point, bottom point) / distance(top point, bottom point) which will be a scale number from 0 to 1, if I multiply this number to the physical height of the cup measured by the ruler, then I will know how high the point is from the bottom of the cup in metric unit.

What is the method I can use to find the corresponding top and bottom point given point on the side, so that I can finally find out the height of the point from the bottom of the cup?

I'm thinking of using PnP to solve this but my points do not have correct IDs associated with them. And I don't want to know the exact rotation of the cup, I only want to know the up direction of the cup. I also think that fitting the ellipse might help somewhat, but maybe it's not the best because the circle is not complete. If you have any suggestions, please tell me how to obtain the point height from the bottom of the cup.

Solution

  • Given the accuracy issues, I don't think it is worth performing a 3D reconstruction of the cone.

    I would perform a "standard" ellipse fit on the top outline, which is the most accurate, then a constrained one on the bottom, knowing the position of the vertical axis. After reduction of the coordinates, the bottom ellipse can be written as

    x²/a² + (y - h)²/b² = 1

    which can be solved by least-squares.

    Note that it could be advantageous to ask the user to point at the endpoints of the straight edges at the bottom, plus the lowest point, instead of the whole curve.

    Solving for the closest top and bottom points is a pure 2D problem (draw the line through the given point and the intersection of the sides, and find the intersection points with the ellipse.