Image similarity for same content but different weather

Question

For two images, one in a sunny weather and another in rainy weather with virtually no difference in the content and objects except the weather, Is there any metric to say that they are highly similar?

Vs... an image which is visibly not so similar..

Answer 1

I've used intermediate layers from CNNs to do that sort of robust comparison in some projects in the past. Basically, you take a CNN that has been trained for some task like image segmentation, and then try to identify layers or combinations of layers that offer a good balance of geometric/photometric features for your matching. Then at test time, you pass the images in the CNN, and compare those features with for example, a Euclidean distance. My images were similar to yours, and I needed something that was fast, so at that time Enet was a good choice for me (well, there are better choices now). I ended up using a combination of features from its 21st and 5th layers that ended up working well in practice. However, if your images are from a sequence where you can exploit temporal information, I strongly recommend that you take a look at SeqSLAM (sorry, couldn't find a non-paywall version. The interesting thing with this is that it doesn't require any CNNs, is real-time, and if memory serves, uses just very simple pyramidal intensity based comparisons for the matching, similar to SPP), as well as this paper, which improves SeqSLAM with layers from CNNs.