How do I remove rain streaks form a rainy image

Question

I am working under a professor, and I recently studied Conv-neural-networks and Generative adversarial Networks and was able to implement basic python codes on these topics using the MNIST dataset. Now I am given an assignment to try and remove rain streaks from rainy images. I read a paper where they used Auto-encoders and GAN's to do that, but I don't know what to do and how to proceed. I have 1000 clean images and 14000 rainy images 14 of each clean ones. I got this dataset from github and I also have some images of only rain streaks.I am fairly new to CNN's and GAN's and I don't even know if this is an easy task or a complex one just for me. But I am very confused. Can anyone suggest some things?

Answer 1

You can solve this problem using image to image translation methods.

• If you have paired images (clean image, and the corresponding rainy image), you can use some paired approaches like Pix2Pix (paper + github: link). The github implementation is easy to adapt to you case, just put your images in the corresponding folders, and lunch the training.
• If you don't have paired images, but images in the two domains (clear images, and rainy images), use an unsupervised image to image translation. CycleGan is a good architecture and it works very well (link). The implementation is easy to use also.

NB: In all the cases, you will need a GPU for training the models!