Search code examples
opencvpoint-cloud-librarykinect-sdkopenkinectofxkinect

3D Object tracking detection using Kinect

I am working on identifying an object by using Kinect sensor so to get x,y,z coordinates of the object.

I am trying to find the related information for this but could not able to find much. I have seen the videos as well but nobody is sharing the information or any sample code? This is what I want to achieve https://www.youtube.com/watch?v=nw3yix3XomY

Proabably, few people may asked same question but as I am new to the Kinect and these libraries due to which I need little more guidance.

I read somewhere that object detection is not possible using Kinect v1. We need to use 3rd party libraries like open CV or point-clouds (pcl).

Can somebody help me that even by using third party libraries how exactly can I identify object via a Kinect sensor?

It will be really helpful.

Thank you.

Solution

  • As the author of the video you linked stated in the comment, following this PCL tutorial will help you. As you found out already, realizing this may not be possible using the standalone SDK. Relying on PCL will help you not reinvent the wheel.

    The idea there is to:

    1. Downsample the cloud to have less data to deal with in the next steps (this also reduces noise a bit).
    2. Identify keypoints/features (i.e. points, areas, textures that remain somehow invariant to some transformations).
    3. Compute the keypoint descriptors, mathematical representations of these features.
    4. For each scene keypoint descriptor, find nearest neighbor into the model keypoints descriptor cloud and add it to the correspondences vector.
    5. Perform clustering on the keypoints and detect the model in the scene.

    The software in the tutorial needs the user to manually feed in the model and scene files. It doesn't do that on live feed, as the video you linked.

    The process should be pretty similar though. I'm not sure how cpu-intensive the detection is, so it might require additional performance tweaking.

    Once you have frame-by-frame detection in place, you could start thinking about actually tracking an object across the frames. But that's another topic.