I'm interested in finding the normal vector of a plane given 3 2D projection/pixel coordinates of the corners of an equilateral triangle.
I have to modify my actual use case because I'm not allowed to share the details: In my use case I have a poster of an equilateral triangle pinned to the wall. I can detect the 3 corners in pixel coordinates and want to calculate the normal vector of the triangle coming straight out of the wall.
It's my intuition that there should be an analytical solution for this and I found similar questions but fail to apply them to my use case. Here are a few of my thoughts:
I thought about bringing the 2D pixel coordinates to 3D by adding a third dimension filled with zero putting it on the XY-plane. Doing the same with a reference triangle of side length 1 gives me the source and destination for finding their homography. But because all 6 z-values are 0, I don't think this would work.
I know the distance between all 3 points is the same in 3D which I want to use as a constraint to solve for the problem.
The normal can be calculated in 3D by taking the cross product of 2 sides of the triangle
Using openCV's findHomography() function requires at least 4 3D points to solve and I don't believe I can just take the mean coordinates of my 3 points in pixel space to find find the center (aka a 4th point), correct?
openCV also comes with a getAffineTransform() function that works with 3 pairs of 2D points. I tried to use it like this:
# equilateral reference triangle
tri_ref = np.array([
[0.0,0.0],
[0.5,0.866],
[1.0,0.0]
], dtype=np.float32)
# detected corners in pixel space
tri_pixel = np.array([
[0.397,0.317],
[0.441,0.848],
[0.698,0.324]
], dtype=np.float32)
A = cv2.getAffineTransform(tri_pixel, tri_ref)
img2 = cv2.warpAffine(img1, A, (400,300))
However, the transformation looks completely wrong and I still would need to know how I can calculate the normal from the transformation matrix.
You can't really get a 100% correct 3D normal vector from three 2D points.
For example , let us consider a pinhole camera system, draw a line from a point to camera, then randomly pick another point on the line we draw, you will still have the same image with the picked point.
If we add the condition that the distance between all three points are the same, you may still have at most 4 possible answers of the vector(imaging two fixed points,the possible position of the last point which have the same distance with the two fixed points will be a circle,draw a line start from the camera to attach the circle , if you cut through the circle than there is two possible position of the last point, the only condition there is only one position is true is the line is a tangent line of the circle, but most of the time this does not happen, change the fixed points so we can have at most 4 possible position of the three points)
You can't add the 4th point by just take the mean coordinates of the 3 points because you do not know where the real position of the centroid project to the image plane.