Search code examples
pythonopencvimage-processingbackground-subtractionransac

OpenCV how to replace cut out object with background

I have two images, one image which contains a box and one without. There is a small vertical disparity between the two pictures since the camera was not at the same spot and was translated a bit. I want to cut out the box and replace the hole with the information from the other picture.

I want to achieve something like this (a slide from a computer vision course)

enter image description here

I thought about using the cv2.createBackgroundSubtractorMOG2() method, but it does not seem to work with only 2 pictures.

Simply subtracting the picture from another does not work either because of the disparity. The course suggests using RANSAC to compute the most likely relationship between two pictures and subtract the area thaht changed a lot. But how do I actually fill in the holes?

Many thanks in advance!!

Solution

  • If you plant ot use only a pair of images (or only a few images), image stitching methods are better than background subtraction.

    The steps are:

    1. Calculate homography between the two images.
    2. Warp the second image to overlap the second.
    3. Replace the region with the human with pixels from the warped image.

    This link shows a basic example of image stitching. You will need extra work if both images have humans in different places, but otherwise it should not be hard to tweak this code.