I am trying to get the 2D coordinate on image plane given the 3D world coordinate in Unreal engine 4.
The world coordinate system is x(increase from right to left), y(increase from outside to inside screen), z(increase from bottom to top)
The image coordinate system I want is origin on bottom left and x(increase from left to right), y(increase from bottom to top)
I set up my camera intrinsic and extrinsic as below while the camera is located on
(-474.9739990234375, -67.140998840332031, 128.10400390625)
and its rotation is
(0, 0.80071002244949341, 99.797645568847656)
related to the origin of world coordinate.
import numpy as np
fov = 69.4
width = 1920.0
height = 1080.0
aspect_ratio = width / height
fx = width / (2 * np.tan(fov * np.pi / 2 / 180))
fy = height / (2 * np.tan(fov / aspect_ratio / 2 / 180 * np.pi))
intrinsic = [[-fx, 0, width / 2], [0, fy, height / 2], [0 , 0, 1]]
pitch = 0.80071002244949341 * np.pi / 180
yaw = 99.797645568847656 * np.pi / 180
roll = 0 * np.pi / 180
ry = [[np.cos(pitch), 0, np.sin(pitch)], [0, 1, 0], [-np.sin(pitch), 0, np.cos(pitch)]]
rx = [[1, 0, 0], [0, np.cos(roll), -np.sin(roll)], [0, np.sin(roll), np.cos(roll)]]
rz = [[np.cos(yaw), -np.sin(yaw), 0], [np.sin(yaw), np.cos(yaw), 0], [0, 0, 1]]
rotation_matrix = np.matmul(np.matmul(rx, ry), rz)
translation_matrix = np.array([[-474.9739990234375, -67.140998840332031, 128.10400390625]])
extrinsic = np.concatenate([rotation_matrix, translation_matrix.T], axis=1)
extrinsic = np.concatenate([extrinsic, np.array([[0, 0, 0, 1]])], axis=0)
extrinsic = np.linalg.inv(extrinsic)
extrinsic = extrinsic[:3]
print(extrinsic)
Let's say I have an object in the scene which are also in the camera view. Its coordinate is
[-582.88897705078125, 143.63600158691406, 88.338996887207031]
Using the code below I should get the projection of the 3D point on image plane.
world_coordinate = np.array([[-582.88897705078125, 143.63600158691406, 88.338996887207031, 1]])
camera_coordinate = np.dot(extrinsic, world_coordinate.T)
print(camera_coordinate)
camera_coordinate[0] *= 1
tmp = camera_coordinate[1].copy()
camera_coordinate[1] = camera_coordinate[2]
camera_coordinate[2] = tmp
print(camera_coordinate)
image_coordinate = np.dot(intrinsic, camera_coordinate)
image_coordinate /= image_coordinate[2]
print(image_coordinate)
But I got
[[-3.52097326e+03]
[-3.59180874e+02]
[ 1.00000000e+00]]
Which does not lie on image plane. How do I solve this?
Turns out it was a mistake. The actual world coordinate system is x(increase from outside to inside screen), y(increase from left to right), z(increase from bottom to top).
Just map the project the yz plane to image space then it'll be done.