Bad matches between images when performing image registration

Question

Original image 1

Original image 2

I am trying to match two microscopy images (please see the attached file). However, the matches are horrible and the homography matrix produces an unacceptable result. Is there a way to improve this registration?

import cv2 # Imports the Open CV2 module for image manipulation.
import numpy as np # Imports the numpy module for numerical manipulation.
from tkinter import Tk # Imports tkinter for the creation of a graphic user interface.
from tkinter.filedialog import askopenfilename # Imports the filedialog window from tkinter

Tk().withdraw()
filename1 = askopenfilename(title='Select the skewed file')

Tk().withdraw()
filename2 = askopenfilename(title='Select the original file')


img1 = cv2.imread(filename1)
img2 = cv2.imread(filename2)

img1 = cv2.cvtColor(img1, cv2.COLOR_BGR2GRAY)
img2 = cv2.cvtColor(img2, cv2.COLOR_BGR2GRAY)

orb = cv2.ORB_create(nfeatures=10000)

kp1, des1 = orb.detectAndCompute(img1, None)
kp2, des2 = orb.detectAndCompute(img2, None)

matcher = cv2.DescriptorMatcher_create(cv2.DESCRIPTOR_MATCHER_BRUTEFORCE_HAMMING)
matches = matcher.match(des1, des2, None)
matches = sorted(matches, key = lambda x:x.distance)

points1 = np.zeros((len(matches), 2), dtype=np.float32)
points2 = np.zeros((len(matches), 2), dtype=np.float32)

for i, match in enumerate(matches):
    points1[i, :] = kp1[match.queryIdx].pt
    points2[i, :] = kp2[match.trainIdx].pt

h, mask = cv2.findHomography(points1, points2, cv2.RANSAC)

height, width = img2.shape

im1Reg = cv2.warpPerspective(img1, h, (width, height))

img3 = cv2.drawKeypoints(img1, kp1, None, flags=0)
img4 = cv2.drawKeypoints(img2, kp2, None, flags=0)
img5 = cv2.drawMatches(img1, kp1, img2, kp2, matches[:10], None)

img = np.dstack((im1Reg, img2, im1Reg))

cv2.imshow("Shifted", img3)
cv2.imshow("Original", img4)
cv2.imshow("Matches", img5)
cv2.imshow("Registered", im1Reg)
cv2.imshow("Merged", img)
cv2.waitKey(0)

Image showing the matches I get

Answer 1

(I may be wrong, since haven't dealt with microscopy image processing and there must exist commonly spread ways to solve typical problems in the area, you should investigate this area if it's not a toy project).

In my opinion you should try another decision to solve your problem instead of using any kind of point-feature-based image descriptors (ORB, SURF etc.). First of all, not all of them provide subpixel accuracy you may need in processing microscopy images. But the main reason is the math behind that descriptors. Refer to any CV book or paper. Here is the link to ORB-descriptor paper. Notice images authors use for matching detected points. Good points are one on edges and corners of the image so it's can be used to match objects of sharp and outstanding shape.

Well-known example:

Matched points are on letters (unique shape) and textured drawing. Try to detect plain green textbook (without any letters and anything on its cover) with this tool and you will fail.

So I think that your images are not one can be processed this way (since objects are not sharp in shape, not textured and very close to each other). It would be hard even for man's eye to match similar circles (in case of less obvious example, e.g. shift one view left/right a little).

But what I notice at glance, your image can be greatly described by circles on it. Hough-based circle detection is much easier (both for understanding and computing) and what is really important, it can give almost 100% accuracy on such images. You can easily operate with circles number, size, position etc.

Though, microscopy CV is a separate area with its own common tools to use and there might be lots of pros and cons to use Hough or something else. But at first glance it seemed to be much more accurate choose than point features description.