How to stream depth image from a basic ToF camera module with Point Cloud Library(PCL)

Question

Inforamtion:

1. I have a simple ToF(Time of Flight) camera module provided by a vendor that only contains a Depth Node.

2. I've already setup the PCL environment and can compile and execute the sample code it provides.

3. The ToF camera module comes with a source code shows how to get depth raw data(the x, y, z value) from the hard device, but doesn't tell how to stream it as both point cloud image and depth image.

4. Win 7 64bit, Visual Studio 2008, PCL all-in-one 32bit.

As a result, I plan to use PCL to show the Point cloud image and depth image with the x, y, z data I can get from that camera module, further more, if streaming is possible.

However, as far as I know right now is that PCL tends to store all the point cloud data as a .pcd file, and then reads it thus output a point cloud image and a depth image.

It is obviously too slow to do the streaming in such way if I have to savePCD() and readPCD() every time in each frame. So I studied the "openni grabber" and "openni range image visualization" sample code and tried execute them, sadly "No device connected." is all I got.

I have a few ideas to ask for advises before I try:

1. Is there a way to use Openni on a device except Kinect, Xtion and PrimeSense? Even if it's just a device with no name and only has a depth node?

2. Can PCL show point cloud image and depth image without accessing a .pcd file? In other words, can I just assign the value of each vertex and construct a image?

3. Can I just normalize all the vertices and construct a image with barely Opencv?

4. Is there any other method to stream that ToF camera module in point cloud image and depth image?

Answer 1

1) Changing the OpenNI grabber to use your own ToF camera will be much more work than to just use the example from the camera in a loop shown below.

2) Yes PCL can show point cloud image and depth without accessing a .pcd file. What the .pcd loader does is to parse the pcd-file and place the values in the cloud format. You can do this directly from your camera data as shown below.

3) No idea what you mean here. I propose you try to use the pcl visualizer or cloud viewer as proposed below.

You can do something like:

pcl::PointCloud<pcl::PointXYZ>::Ptr cloud (new pcl::PointCloud<pcl::PointXYZ>);
cloud->isDense = true;
cloud->width = widthOfTOFsensor;
cloud->height = heightOfTOFsensor;
cloud->size = cloud->width * cloud->height;

//Create some loop

    //grabNewFrame from TOFsensor

    for(int pointIndex=0;pointIndex<cloud->size();pointIndex++)
    {
        cloud->points[pointIndex].x = tofSensorData[pointIndex].x; //Don't know the tofData format, so I just guessed something.
        cloud->points[pointIndex].y = tofSensorData[pointIndex].y;
        cloud->points[pointIndex].z = tofSensorData[pointIndex].z;
    }

    // Plot the data using pcl visualizer or cloud viewer, see:

http://pointclouds.org/documentation/tutorials/cloud_viewer.php#cloud-viewer

http://pointclouds.org/documentation/tutorials/pcl_visualizer.php#pcl-visualizer