Swapping face parts relative to facial landmarks

Question

I got an image of an individual with a beard:

Using a mask, I was able to extract the beard:

I want to move the beard on another person's face, such as this one:

I want to do this by getting the nose location of the first person, the nose location of the 2nd person and position the beard accordingly.

What are some ways to accomplish this goal? How can I do this by getting the facial landmarks? Are there any non-Deep Learning methods of doing this?

Answer 1

You could accomplish the goal by training a conv-net that takes as input images of the person's face, and returns the co-ordinates of the tip of the nose for positioning the person's beard. To train a conv-net, you will need data. You can potentially create your own data using an image annotation tool such as this, or find a relevant dataset online such as this one, to train your model. Here is a keypoint detection tutorial to get you started.

There may be non-deep learning methods of doing this. Maybe you could create SIFT descriptors for a person's nose, find the closest matching descriptors in the images you want to detect the tip of the nose in, and use their location to determine the position of the tip of the nose in your image of interest. However, I would recommend you go with the deep learning approach. Keypoint detection is a popular task in deep learning and you are likely to find many resources online for help.