covert rgb png and depth txt to point cloud

I have a series of rgb files in png format, as well as the corresponding depth file in txt format, which can be loaded with np.loadtxt. How could I merge these two files to point cloud using open3d?

I followed the procedure as obtain point cloud from depth numpy array using open3d - python, but the result is not readable for human.

The examples is listed here:

the source png:
the pcd result:

You can get the source file from this link ![google drive] to reproduce my result. By the way, the depth and rgb are not registerd.

Thanks.

Solution

I had to play a bit with the settings and data and used mainly the answer of your SO link.

import cv2
import numpy as np
import open3d as o3d

color = o3d.io.read_image("a542c.png")
depth = np.loadtxt("a542d.txt")

vertices = []
for x in range(depth.shape[0]):
    for y in range(depth.shape[1]):
        vertices.append((float(x), float(y), depth[x][y]))
pcd = o3d.geometry.PointCloud()
point_cloud = np.asarray(np.array(vertices))
pcd.points = o3d.utility.Vector3dVector(point_cloud)
pcd.estimate_normals()
pcd = pcd.normalize_normals()
o3d.visualization.draw_geometries([pcd])

However, if you keep the code as provided, the whole scene looks very weird and unfamiliar. That is because your depth file contains data between 0 and almost 2.5 m. I introduced a cut-off at 500 or 1000 mm plus removed all 0s as suggested in the other answer. Additionally I flipped the x-axis (float(-x) instead of float(x)) to resemble your photo.

# ...
vertices = []
for x in range(depth.shape[0]):
    for y in range(depth.shape[1]):
        if 0< depth[x][y]<500:
            vertices.append((float(-x), float(y), depth[x][y]))

For a good perspective I had to rotate the images manually. Probably open3d provides methods to do it automatically (I quickly tried pcd.transform() from your SO link above, it can help you if needed). Results

500 mm cut-off: and 1000 mm cut-off: .