How to calibrate kinect skeleton data to video feed?

Question

How do I calibrate an xna camera projection so that kinect's skeleton data points line up with the video feed when overlayed?

Answer 1

I've done this in WPF, I assume it will be the same with XNA..

This is the code I used, which I believe was taken from one of the sample Kinect projects in the SDK's documentation. Your results may vary. Note that the 320x240 part seems to have nothing to do with the actual resolution you're using.. I had the depth stream set to 640x480 and the RGB stream set to 1280x1024.

nui is my Runtime object.

private Point getDisplayPosition(Joint joint)
{
    float depthX, depthY;
    nui.SkeletonEngine.SkeletonToDepthImage(joint.Position, out depthX, out depthY);
    depthX = Math.Max(0, Math.Min(depthX * 320, 320));  //convert to 320, 240 space
    depthY = Math.Max(0, Math.Min(depthY * 240, 240));  //convert to 320, 240 space

    int colorX, colorY;
    ImageViewArea iv = new ImageViewArea();
    // only ImageResolution.Resolution640x480 is supported at this point
    nui.NuiCamera.GetColorPixelCoordinatesFromDepthPixel(ImageResolution.Resolution640x480, iv, (int)depthX, (int)depthY, (short)0, out colorX, out colorY);

    return new Point(colorX, colorY);
}