I have found values of entropy, mean squared error and PSNR of edge detected images with respect to original images. I have different five edge detected images with different threshold and sigma values. To find which is the best combination of threshold and sigma, I need to find the best edge detected image.
Without visual inspection, Can I find it from the above values? I found from research paper that, if mean squared error image is low, detected edges are good. Can I use this concept? How does PSNR and Entropy affects?
Selection of the edge detection parameters is highly dependent on image quality, image content and the information that you want to extract form the image. So it is essentially application-dependent and subjective to a large extent.
No theory can help you. Probably the best you can do is to ajdust those parameters the way that pleases you on a subsample of your images, and feed this data together with the entropy, MSE and PSNR to a machine learning device.