Project 3D points to 2D points in python

Question

I'm trying to project 3D body keypoints to 2D keypoints, My 3D points are:

points = np.array([[-7.55801499e-02, -3.69511306e-01, -2.63576955e-01],
   [ 0.00000000e+00,  0.00000000e+00,  0.00000000e+00],
   [ 3.08661222e-01, -2.93346141e-02,  3.72593999e-02],
   [ 5.96781611e-01, -2.82074720e-01,  4.71359938e-01],
   [ 5.38534284e-01, -8.05779934e-01,  4.68694866e-01],
   [-3.67936224e-01, -1.09069087e-01,  9.90774706e-02],
   [-5.24732828e-01, -2.87176669e-01,  6.09635711e-01],
   [-4.37022656e-01, -7.87327409e-01,  4.43706572e-01],
   [ 1.33009470e-09, -5.10657072e-09,  1.00000000e+00],
   [ 1.13241628e-01,  3.25177647e-02,  1.24026799e+00],
   [ 3.43442023e-01, -2.51034945e-01,  1.90472209e+00],
   [ 2.57550180e-01, -2.86886752e-01,  2.75528717e+00],
   [-1.37361348e-01, -2.60521360e-02,  1.19951272e+00],
   [-3.26779515e-01, -5.59706092e-01,  1.75905156e+00],
   [-4.65996087e-01, -7.69565761e-01,  2.56634569e+00],
   [-1.89841837e-02, -3.19088846e-01, -3.69913191e-01],
   [-1.61812544e-01, -3.10732543e-01, -3.47061515e-01],
   [ 7.68100023e-02, -1.19293019e-01, -3.72248143e-01],
   [-2.24317372e-01, -1.02143347e-01, -3.32051814e-01],
   [-3.77829641e-01, -1.19915462e+00,  2.56900430e+00],
   [-5.45104921e-01, -1.13393784e+00,  2.57149625e+00],
   [-5.66698492e-01, -6.89325571e-01,  2.67840290e+00],
   [ 4.65222150e-01, -6.44857705e-01,  2.83186650e+00],
   [ 5.27995050e-01, -4.69421804e-01,  2.87518311e+00],
   [ 1.77749291e-01, -1.74753308e-01,  2.88810611e+00]])

I plotted them using:

fig = plt.figure()
ax = plt.axes(projection='3d')
ax.set_xlim3d(1, -1)
ax.set_ylim3d(1, -1)
ax.set_zlim3d(1, -1)
ax.scatter3D(points[:, 0], points[:, 1],
      points[:, 2], cmap='Greens')

The result is:

I want an array of 2D points with the same camera view, so my desired result a 2D array:

I have tried so far:

import cv2
ans = []
for k in range(25):
  tmp = np.array(s[0, k, :]).reshape(1,3)
  revc = np.array([0, 0, 0], np.float)  # rotation vector
  tvec = np.array([0, 0, 0], np.float)  # translation vector
  fx = fy = 1.0
  cx = cy = 0.0
  cameraMatrix = np.array([[fx, 0, cx], [0, fy, cy], [0, 0, 1]])
  result = cv2.projectPoints(tmp, revc, tvec, cameraMatrix, None)
  ans.append(result[0])

ans = np.array(ans).squeeze()

But the result I'm getting is:

plt.scatter(ans[:,0], ans[:, 1])

I can't figure out why the information is lost during projection, kindly help me in this. Also its not necessary for me to use OpenCV so you can suggest other methods like using numpy too.

Thanks

Answer 1

Here's a way to do this from "scratch". I have the following import statements:

import numpy as np
import matplotlib.pyplot as plt
from numpy import sin,cos,pi
from scipy.linalg import norm

After your 3d plotting code, I added the following:

azim = ax.azim*pi/180
elev = ax.elev*pi/180
elev *= 1.2          # this seems to improve the outcome

a_vec = np.array([cos(azim),sin(azim),0])
normal = cos(elev)*a_vec + np.array([0,0,sin(elev)])

z_vec = np.array([0,0,1])
y_comp = z_vec - (z_vec@normal)*normal
y_comp = y_comp/norm(y_comp)
x_comp = np.cross(y_comp,normal)

proj_mat = np.vstack([x_comp,y_comp]) # build projection matrix
proj_mat = -proj_mat                  # account for flipped axes
points_2D = points @ proj_mat.T       # apply projection

plt.figure()
plt.scatter(*points_2D.T)
plt.gca().set_aspect('equal', adjustable='box')
plt.axis('off')
plt.show()

The resulting points: