Search code examples
pythonopencvmaskbitmask

opencv - replacing part of image with another using a binary mask but aliased line prevents good result

I'm working with binary masks in openCV in Python and have created one with fillConvexPoly. First result can be seen in figure 1. As you can see the edge is not smooth, but rather looks like stairs.

So I use the anti-aliasing flag in fillconvexpoly, which is cv2.LINE_AA. The result looks promising, as you can see in figure 2. Problem is that using this mask in bitwise operations results in errors. Why?

Well, upon closer inspection and looking at the actual pixel values, I can see that the line is represented by a decrease in pixel values rather than pixel(x,y)=255, pixel(x+1,y)=0 and pixel(x+1,y+1)=255. In figure 3 (large image) you can see the decrease of value in pixels in one row. In figure 4 you can see what I would like as a result. I do understand that that is not realistically possible since the line doesn't have that angle, but there should be a better representation that in has in figure 1 right?. The reason I need it like that is because I'm replacing part of an image with another image, and the mask represents the replaced area. I use bitwise operations to get my result. Using bitwise operations with values other than 0 or 255 result in errors and different colors etc. So I need it to be 0 or 255. Result for now in figure 5, where the blue area is replaced (used to be red). As you can see, the result is not very attractive. If that would be a 'straight' line the result would look way better. (applying blur on straight edges afterwards makes it look good btw, but the stairwise motion is still visible after the blur).

What I have tried:

  1. Thresholding the pixelvalues. This results in the fillconvexpoly without LINE_AA, which is logical because the horizontal pixel values decrease for f.e. 15 pixels and then reach 0, where the decrease now starts from the next row, which again results in 15 pixels decreasing in value in the same row, ... So thresholding any value results in the same result with a different offset.
  2. Morphological Operations, looks like opening/closing/dilation/erosion etc .don't work on these types of lines. Also kind off logical when I read how they work and what they're used for.
  3. Blurred the mask and thresholding afterwards, which results in the same jagged/stairwise edge.
  4. Scanned stackoverflow for a few days for a solution.

If you have another suggestion for this problem, I would love to hear it. Thanks in advance.

EDIT: Using a blur on the final result works well for lines with a lot of steps in it, f.e. every 2 or 3 pixels. From: original. Applied blur on the edge: result But applying this technique on a very long line with few steps: original: see figure 5, and result. As you can see, the result is not bad, but still contains this stepwise pattern. Really trying to get rid of it. (The glow coming from underneath is something I'm handling in the future)

Solution

  • You can solve this with an equivalent of the addWeighted function for array-based weights. Assuming you have 8-bit arrays image, overlay, and mask, and since your mask seems to apply to the overlay rather than the base image, you could do something like

    import cv2
import numpy as np

... # get/read image and overlay, create mask

alpha = mask / 255 # convert to 0-1 range, could also use cv2.normalize
combined = np.uint8(overlay * alpha + image * (1 - alpha))