UWP Kinect V2 keep frame rate constant (30fps)

Question

I am processing frames received from Kinect v2 (Color and IR) in UWP. The program runs on remote machine (XBOX One S). The main goal is to get frames and write them to the disk with 30 fps for Color and IR to later process them further.

I am using the following code to check the frame rate:

public MainPage()
{
    this.InitialiseFrameReader(); // initialises MediaCapture for IR and Color
}
    const int COLOR_SOURCE = 0;
    const int IR_SOURCE = 1;

    private async void InitialiseFrameReader()
    {
        await CleanupMediaCaptureAsync();

        var allGroups = await MediaFrameSourceGroup.FindAllAsync();

        if (allGroups.Count == 0)
        {
            return;
        }

        _groupSelectionIndex = (_groupSelectionIndex + 1) % allGroups.Count;
        var selectedGroup = allGroups[_groupSelectionIndex];
        var kinectGroup = selectedGroup;

        try
        {
            await InitializeMediaCaptureAsync(kinectGroup);
        }
        catch (Exception exception)
        {
            _logger.Log($"MediaCapture initialization error: {exception.Message}");
            await CleanupMediaCaptureAsync();
            return;
        }

        // Set up frame readers, register event handlers and start streaming.
        var startedKinds = new HashSet<MediaFrameSourceKind>();
        foreach (MediaFrameSource source in _mediaCapture.FrameSources.Values.Where(x => x.Info.SourceKind == MediaFrameSourceKind.Color || x.Info.SourceKind == MediaFrameSourceKind.Infrared)) //
        {
            MediaFrameSourceKind kind = source.Info.SourceKind;
            MediaFrameSource frameSource = null;

            int frameindex = COLOR_SOURCE;
            if (kind == MediaFrameSourceKind.Infrared)
            {
                frameindex = IR_SOURCE;
            }
            // Ignore this source if we already have a source of this kind.
            if (startedKinds.Contains(kind))
            {
                continue;
            }
            MediaFrameSourceInfo frameInfo = kinectGroup.SourceInfos[frameindex];
            if (_mediaCapture.FrameSources.TryGetValue(frameInfo.Id, out frameSource))
            {
                // Create a frameReader based on the source stream
                MediaFrameReader frameReader = await _mediaCapture.CreateFrameReaderAsync(frameSource);

                frameReader.FrameArrived += FrameReader_FrameArrived;
                _sourceReaders.Add(frameReader);

                MediaFrameReaderStartStatus status = await frameReader.StartAsync();
                if (status == MediaFrameReaderStartStatus.Success)
                {
                    startedKinds.Add(kind);
                }
                
            }
        }
    }

    private async Task InitializeMediaCaptureAsync(MediaFrameSourceGroup sourceGroup)
    {
        if (_mediaCapture != null)
        {
            return;
        }

        // Initialize mediacapture with the source group.
        _mediaCapture = new MediaCapture();
        var settings = new MediaCaptureInitializationSettings
        {
            SourceGroup = sourceGroup,

            SharingMode = MediaCaptureSharingMode.SharedReadOnly,

            StreamingCaptureMode = StreamingCaptureMode.Video,

            MemoryPreference = MediaCaptureMemoryPreference.Cpu
        };
        await _mediaCapture.InitializeAsync(settings);
    }


private void FrameReader_FrameArrived(MediaFrameReader sender, MediaFrameArrivedEventArgs args)
        {          
            using (var frame = sender.TryAcquireLatestFrame())
            {
                if (frame != null)
                {    


                //Settings.cameraframeQueue.Enqueue(null, frame.SourceKind.ToString(), frame.SystemRelativeTime.Value); //Add to Queue to process frame 
                Debug.WriteLine(frame.SourceKind.ToString() + " : " + frame.SystemRelativeTime.ToString());         
            }
        }
    }

I am trying to debug the application to check the frame rate so I have removed further processing.

I am not sure if I am not calculating it properly or something else is wrong.

For example, System Relative Time from 04:37:06 to 04:37:48 gives :

IR: Fps(Occurrence)

31(1) 30(36) 29(18) 28(4)

Color: Fps(Occurrence)

30(38) 29(18) 28(3)

I want this frame rate to be constant (30 fps) and aligned so IR and Color and same number of frames for that time.

This does not include any additional code. As soon as I have a process queue or any sort of code, the fps decreases and ranges from 15 to 30.

Can anyone please help me with this?

Thank you.

UPDATE:

After some testing and working around, it has come to my notice that PC produces 30fps but XBOX One (remote device) on debug mode produces very low fps. This does however improve when running it on release mode but the memory allocated for UWP apps is quite low. https://learn.microsoft.com/en-us/windows/uwp/xbox-apps/system-resource-allocation

Answer 1

XBOX One has maximum available memory of 1 GB for Apps and 5 for Games. https://learn.microsoft.com/en-us/windows/uwp/xbox-apps/system-resource-allocation

While in PC the fps is 30 (as the memory has no such restrictions).

This causes the frame rate to drop. However, the fps did improve when running it on release mode or published to MS Store.