opencv overlaying one image over another with masking and in-painting

Question

I'm using OpenCV to do in-painting on a few polygons (drawn on paper, see examples).

Some legend specs:

• The green frame is there to draw the scene "borders", just there for reference.
• The blue ball is floating on the scene and when the ball hits a polygon, the scene is re-rendered with the proper masking, as if the ball is crushing the objects.

Here is a code for reference, assume that inpaintedScene, transformedScene and outputFrame are cv::Mat:

 cv::Mat mask_image(outputFrame.size(), CV_8U, BLACK_COLOR);
 std::vector<cv::Point*> destroyedPolygons = //some sub-polygons that has been hit by the ball
 std::vector<int> destroyedPolygonsPointCount = //points count for each destroyed polygon
 for (int i = 0; i < destroyedPolygons.size(); i++)
 {
    const cv::Point* ppt[1] = { destroyedPolygons[i] };
    int npt[] = { destroyedPolygonsPointCount[i] };
    cv::fillPoly(mask_image, ppt, npt, 1, WHITE_COLOR);
 }

 // now that the mask is prepared, copy the points from the inpainted to the scene
 inpaintedScene.copyTo(transformedScene, mask_image);

 // here I have two options:
 //option 1
 outputFrame += transformedScene;

 //option 2
 transformedScene.copyTo(outputFrame, transformedScene);

These are the results which none of them is good for me:

Results of option no.1 (+=):

This is not good for me since I get transparency over the destroyed polygons.

Results of option no.2 (copyTo):

This is also not good because as you can see, the destroyed part of the polygon is kind a "bordered" or "framed" with black color (even though the polygon is in another color) - what can solve this?

Answer 1

Found it!

I've added warpPerspective to transformedScene with "nearest-neighbour" interpolation:

cv::warpPerspective(transformedScene, transformedScene, warpImageMat, outputFrame.size(), CV_INTER_NN);

where warpImageMat is type of cv::Mat

Read here more about warpPerspective function of OpenCV

Cheers!