Virtual Hair on Detected face using Emgu CV - C#
My current project has required me to learn face detection/tracking and image processing, given my experience in c#, I chose Emgu CV as my choice library for face detection and tracking. From what I've learned so far, I can do face detection and tracking, and basic image processing.
My goal is to be able to place virtual hair on the detected face. What I want to achieve is similar to [this video]: http://www.youtube.com/watch?v=BdPmECfUFcI.
What I would like to know is the technique(s) to use in handling hair placement for different kind of hairstyles on the detected face. In what image format do I store the the hair?
After watching the video I noticed it considers the head as a flat rectangle and not as a rectangular prism (the 3D object), so it doesn't consider the use of perspective transformations and I will not consider it too. This is a limitation but serves as a decent first step in doing such placements. Note that it is not a simply matter of taking perspective into consideration, your face tracking algorithm also needs to be able to handle more complicated configurations (the eyes might not be fully visible, for example).
So, the first thing you want is a bounding rectangle aligned according to the angle the eyes make with the x axis, illustrated in the following right figure (the red segment indicates the connection between the eyes). The left figure shows a typical bounding box aligned to the axis, which doesn't serve for this problem.
The problem is also simplified after you consider the head is symmetric, so you know the top middle point in the above figure is the middle of the top of your head. Also, considering that a typical head will likely be larger at top than at bottom, then you have something like in the following figure where the width of the rectangle is close to the width of the forehead. You could also consider a bounding rectangle on only upper half of the head, for example.
Now all that is left is positioning some object in this rectangle. For that, you need to augment the description of this object to be positioned so it is not purely pixels. We can define "entrance width" (EW) and "entrance middle point" (EM). This EW establishes the width needed in the other rectangle (the head one) to position it. So, if EW is smaller than the needed value, you upscale this object, respectively for when EW is larger. Note that the full width of the head's rectangle is usually an overestimation to position this object, so you can experiment with percentages of the width. The EM value is useful to know how you will position this object over the head. In the following figure, EW is the horizontal blue dashed horizontal, and EM is the middle point on it. The vertical blue line indicates how much over the EM you want to move this object inside the top segment of head's rectangle.
The only other special thing this object needs is a value that is considered as background. So when painting this object it is easy to know whether to make a point fully transparent (the background value) or fully opaque (anything else). This was the sketch I had in mind of what needs to be basically done.