I have a population matrix of 5 images with 49 extracted salience features. I want to calculate the cosine similarity in Matlab between a test image with the same extracted features 49.
1) Transform your images of size M lines X N columns in a vector M*N lines. Keep one image in a vector u and the other image in a vector v.
2) Evaluate: cosTheta = dot(u,v)/(norm(u)*norm(v)); [As far as I know there is no function in matlab that does that]
Usually people evaluate similarities among images using the projections of them on the eigenfaces. So, before doing that, people usually evaluate the eigenfaces.