I'm using OpenCV in C++. I have two images of faces, and I'm trying to align the faces to each other. I have features denoting the eyes, nose, and other points of interest in the faces. I use Arun's method to extract a rotation, translation, and scale that translates the centroid of the feature sets in one image to the other. Then, it rotates and scales it. And I'm trying to use this transformation to warp one image so that the face in the image aligns with the face in the other image. At least that's the idea, but I'm having problems. It seems that the rotation and translation are being performed in a different order than I expected in warpAffine. There also seems to be an issue with row/column ordering versus x/y coordinates. Also, as the title suggests, I think the warpAffine is doing the operations with respect to 0,0 in the image whereas I expect them to be done around the centroid of the face points. What should I be doing to correctly align these two sets of points? Here's my code:
// R is the rotation as computed by Arun's method
// centered_moving is the moving points with the centroid subtracted from them
// same with centered_base
for (int i = 0; i < base.cols; i++)
{
a = centered_base.col(i);
b = centered_moving.col(i).t();
H += a*b;
}
SVD sv;
Mat W, U, Vt;
sv.compute(H, W, U, Vt);
Mat R = (Vt.t())*(U.t());
// centroid_moving and centroid_base are the centroids of the two point clouds, moving is the cloud that will be translated
Mat trans = -centroid_moving + centroid_base;
Mat Afmat = Mat::zeros(2, 3, ddepth);
Mat tmpmat = Afmat(Rect(0, 0, 2, 2));
R = Mat::eye(2, 2, ddepth);
R.copyTo(tmpmat);
Afmat.at<double>(1, 2) = trans.at<double>(0);
Afmat.at<double>(0, 2) = trans.at<double>(1);
warpAffine(image_moving, affine_result, Afmat, image_moving.size());
The issue was that I was using points in row,column order whereas warpAffine wants a matrix derived from points in x,y order (aka column,row). Flipping the points and using getAffineTransform solved the issue.