Why is it that when I try to overlay a PIL image onto another, both of which have the same size, they end up not aligning properly?

Question

This is my test frame, I am trying to find which map it is by pasting every single map on top and checking if the images match. It is cut perfectly and the image I paste on top is also cut perfectly but I am getting this result:

Why am I getting this behavior and how to fix it? This is my code:

bottom_image = Image.open("test_frames/test_frame_20.png").convert("RGBA")
top_image = Image.open("../map_images/split.png").convert("RGBA")
lower_hsv = np.array([0, 0, 122])
upper_hsv = np.array([174, 53, 130])

top_image = top_image.resize((bottom_image.size[0], bottom_image.size[1]), Image.ANTIALIAS)

img_array = np.array(top_image)
img_array[..., :3] = [255, 0, 0]

best_result = 9**999
rotation_degrees = 0
red_image = Image.fromarray(img_array)

for i in range(1, 5):
    red_image_copy = red_image.copy().rotate(i*90)
    result_image = Image.alpha_composite(bottom_image.convert("RGBA"), red_image_copy)
    numpy_array = np.array(result_image)

    opencv_image = cv2.cvtColor(numpy_array, cv2.COLOR_RGB2BGR)
    hsv_image = cv2.cvtColor(opencv_image, cv2.COLOR_BGR2HSV)

    mask = cv2.inRange(hsv_image, lower_hsv, upper_hsv)
    result = np.sum(mask)

    if result < best_result:
        best_result = result
        rotation_degrees = i*90

print(rotation_degrees)
result_image = Image.alpha_composite(bottom_image, red_image.rotate(rotation_degrees))
result_image.save("result_image.png")

Just want to repeat that both images are cut perfectly and theres no space to the left and right on the image I paste on top.

Links to download both images i use:

map: https://ibb.co/bH8K5Ty test_frame: https://ibb.co/fMRqN3s

Answer 1

Your input pictures aren't the same size. One is significantly larger.

They are almost square, but not really. One is 894x952 (tall), the other is 354x334 (wide).

One of your input pictures is rotated by 90 degrees relative to the other.

You swapped, or rather failed to swap, the width and height before the resize operation.

The resize operation then squashed the picture in one direction and stretched it in the other.

Please step through your code with a debugger and also inspect the results of every operation.