Generating bird eye view of image in OpenCV(Python) without knowing exact positions of reference points and camera properties

Question

I want to do some particle image velocimetry on dancing people but I was not able to record movies from the top (a bird's eye view). Instead, I managed to shot them from an upper position and my goal is to transform this series of images into a top/down bird's eye view using warpPerspective from openCV in Python. The issues that I have are the following:

• I do not have reference points on the image;
• I do not know the distortion of the camera (the movie was shot with a Nikon D7000 and a 18-140 mm Nikon varifocal lens).

Here is the image I want to transform

Original image

P1, P2, P3 and P4 are reference points that I chose to transform the perspective. I chose them because I know, from the geometry of the white pillars in the image, they form a rectangle (approximately). Their position in pixel are (1248, 2160), (2730, 1764), (3336, 2994) and (4728, 2196) respectively.

Based on an earlier similar question, I tried to follow this answer but the results were poor. Here is my code

import cv2 as cv
import numpy as np
import matplotlib.pyplot as plt


im = cv.imread("images/_DSC3318.JPG")
plt.imshow(im)
plt.show()
(height, width) = im.shape[:2]
assert (width, height) == (6000, 4000), "or whatever else"

K = np.eye(3)
K[0,0] = K[1,1] = 1000 # 500-5000 is common
K[0:2, 2] = (width-1)/2, (height-1)/2

dc = np.float32([-0.32,  0.24,  0.  ,  0.  ,  0.  ]) # k1, k2, p1, p2, k3
im_undistorted = cv.undistort(im, K, dc)

modelpts = np.float32([
    [0.,  6.],
    [8.,  0.],
    [8., 6.],
    [0., 0.]])*20

pts1 = np.float32([(1248, 2160),
                  (2730, 1764),
                  (3336, 2994),
                  (4728, 2196)])

pts2 = cv.undistortImagePoints(pts1, K, dc)
impts_undist = pts2.reshape((-1, 1, 2))
H = cv.getPerspectiveTransform(impts_undist, modelpts)
topdown = cv.warpPerspective(im_undistorted, H, dsize=(90*15, 60*15))

K and cd are some parameters (that I don't really understand) used to obtained an undistorted image;pts1 are the coordinate in pixel of P1,...,P4 mentioned above; pts2 are supposed to be the coordinates of pts1 in the undistorted image, modelpts are the coordinates of P1,...,P4 in the bird's eye image (my ultimate goal) which does not work.

To illustrate my point here are the resulting undistorted image im_undistort

Undistorted image

And here is the final top/down view

Top/down view

My guesses to why it does not work are:

• I do not have a precise location of modelpts corresponding to pts1;
• the order of the points in modelpts do not correspond to the order of points in pts1, therefore generating a completely wrong top/down view;
• parameters K and dc used in cv.undistort are not precise enough to correct for image distortion (though I doubt it affect much since the undistorted image is not too "wrong").

Answer 1

Based on the Original Image you attached, I mocked up a quick very simple wrapPerspective. As you only provided a screenshot of it instead of the raw image, my points won't match yours but you'll get the idea.

import cv2
import os
import numpy as np

imagepath = r'.\originalImage.jpg'
img = cv2.imread(imagepath)

# Ratio 8m width to 6m height (4:3)
width = 200
height = 150

#Input points are read from the corners drawn in the reduced size screenshot provided
inputpts = np.float32([[268, 175], [470, 220], [329, 298], [120, 215]])
outputpts = np.float32([[0,0], [width-1, 0], [width-1, height-1], [0, height-1]])

m = cv2.getPerspectiveTransform(inputpts, outputpts)
outimg = cv2.warpPerspective(img, m, (width, height), cv2.INTER_LINEAR)

cv2.imshow('Result', outimg)
k = cv2.waitKey(0)
cv2.destroyAllWindows()

As a comment, the angle of the camera is very strong to extract a top down view of the desired area, your image will be stretched a lot.