Kinect Depth FPS slows Significantly when Human Detected

Question

I have a application where I create my own Depth Frame (using the Kinect SDK). The problem is when a human is detected the FPS of the depth (and then the color too) slows down significantly. Here is a movie of when the frame slows down. The code I am using:

        using (DepthImageFrame DepthFrame = e.OpenDepthImageFrame())
        {
            depthFrame = DepthFrame;
            pixels1 = GenerateColoredBytes(DepthFrame);

            depthImage = BitmapSource.Create(
                depthFrame.Width, depthFrame.Height, 96, 96, PixelFormats.Bgr32, null, pixels1,
                depthFrame.Width * 4);

            depth.Source = depthImage;
        }

...

    private byte[] GenerateColoredBytes(DepthImageFrame depthFrame2)
    {
        short[] rawDepthData = new short[depthFrame2.PixelDataLength];
        depthFrame.CopyPixelDataTo(rawDepthData);

        byte[] pixels = new byte[depthFrame2.Height * depthFrame2.Width * 4];

        const int BlueIndex = 0;
        const int GreenIndex = 1;
        const int RedIndex = 2;


        for (int depthIndex = 0, colorIndex = 0;
            depthIndex < rawDepthData.Length && colorIndex < pixels.Length;
            depthIndex++, colorIndex += 4)
        {
            int player = rawDepthData[depthIndex] & DepthImageFrame.PlayerIndexBitmask;

            int depth = rawDepthData[depthIndex] >> DepthImageFrame.PlayerIndexBitmaskWidth;

            byte intensity = CalculateIntensityFromDepth(depth);
            pixels[colorIndex + BlueIndex] = intensity;
            pixels[colorIndex + GreenIndex] = intensity;
            pixels[colorIndex + RedIndex] = intensity;

            if (player > 0)
            {
                pixels[colorIndex + BlueIndex] = Colors.Gold.B;
                pixels[colorIndex + GreenIndex] = Colors.Gold.G;
                pixels[colorIndex + RedIndex] = Colors.Gold.R;
            }
        }

        return pixels;
    }

FPS is quite crucial to me since I am making an app that saves pictures of people when they are detected. How can I maintain a faster FPS? Why is my application doing this?

Answer 1

G.Y is correct that you're not disposing properly. You should refactor your code so the DepthImageFrame is disposed of ASAP.

...
private short[] rawDepthData = new short[640*480]; // assuming your resolution is 640*480

using (DepthImageFrame depthFrame = e.OpenDepthImageFrame())
{
    depthFrame.CopyPixelDataTo(rawDepthData);
}

pixels1 = GenerateColoredBytes(rawDepthData);    
...

private byte[] GenerateColoredBytes(short[] rawDepthData){...}

You said that you're using the depth frame elsewhere in the application. This is bad. If you need some specific data from the depth frame, save it separately.

dowhilefor is also correct that you should look at using a WriteableBitmap, it's super simple.

private WriteableBitmap wBitmap;

//somewhere in your initialization
wBitmap = new WriteableBitmap(...);
depth.Source = wBitmap;

//Then to update the image:
wBitmap.WritePixels(...);

Also, you're creating new arrays to store pixel data again and again on every frame. You should create these arrays as global variables, create them a single time, and then just overwrite them on every frame.

Finally, although this shouldn't make a huge difference, I'm curious about your CalculateIntensityFromDepth method. If the compiler isn't inlining that method, that's a lot of extraneous method calls. Try to remove that method and just write the code where the method call is right now.