windows phone 8.1 dispaly PreviewFrame and calculate mean value of red pixels for every frame

Question

Currently I'm using Lumia.Imaging to get preview frame and display it.

I create new method "GetPreview()" to go though pixels, find red pixel and than I would like to calculate mean value of red pixels for every frame.

My problem is that when I'm going through pixel there are lags in app :(

• What is the proper solution to calculate mean of red pixels for every frame without performance loss?

• Additional how to turn on Flash light when preview starts ?

      private async Task startCameraPreview()
  {
      // Create a camera preview image source (from the Lumia Imaging SDK)
      _cameraPreviewImageSource = new CameraPreviewImageSource();
  
      // Checking id of back camera 
      DeviceInformationCollection devices = await Windows.Devices.Enumeration.DeviceInformation.FindAllAsync(Windows.Devices.Enumeration.DeviceClass.VideoCapture);
      String backCameraId = devices.FirstOrDefault(x => x.EnclosureLocation != null && x.EnclosureLocation.Panel == Windows.Devices.Enumeration.Panel.Back).Id;
      await _cameraPreviewImageSource.InitializeAsync(backCameraId); // use the back camera
      var previewProperties = await _cameraPreviewImageSource.StartPreviewAsync();
      fps = previewProperties.FrameRate.Numerator/previewProperties.FrameRate.Denominator;
  
      _cameraPreviewImageSource.PreviewFrameAvailable += drawPreview; // call the drawPreview method every time a new frame is available
  
      // Create a preview bitmap with the correct aspect ratio using the properties object returned when the preview started.
      var width = 640.0;
      var height = (width / previewProperties.Width) * previewProperties.Height;
      var bitmap = new WriteableBitmap((int)width, (int)height);
      _writeableBitmap = bitmap;
  
      // Create a BitmapRenderer to turn the preview Image Source into a bitmap we hold in the PreviewBitmap object
      _effect = new FilterEffect(_cameraPreviewImageSource);
      _effect.Filters = new IFilter[0]; // null filter for now
      _writeableBitmapRenderer = new WriteableBitmapRenderer(_effect, _writeableBitmap);
  }
  
  
  private async void drawPreview(IImageSize args)
  {
      // Prevent multiple rendering attempts at once
      if (_isRendering == false)
      {
          _isRendering = true;
          await _writeableBitmapRenderer.RenderAsync(); // Render the image (with no filter)
          // Draw the image onto the previewImage XAML element
          await Windows.ApplicationModel.Core.CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(CoreDispatcherPriority.High,
              () =>
              {
                  getPreview();
                  previewImage.Source = _writeableBitmap; // previewImage is an image element in MainPage.xaml
                  _writeableBitmap.Invalidate(); // force the PreviewBitmap to redraw
              });
  
  
          _isRendering = false;
      }
  }
  
  private void getPreview()
  {
      var pixelBuffer = _writeableBitmap.PixelBuffer;
  
      for (uint i = 0; i + 4 < pixelBuffer.Length; i += 4)
      {
          var red = pixelBuffer.GetByte(i + 2);
      }
  }
  

Answer 1

Instead of inspecting all pixels after the Lumia Imaging SDK has processed the image, but before you invalidate the bitmap you could:

• Invalidate the writeable bitmap imidiatelly then do your analysis step in a seperate async Task. That means that the content will be displayed right away, and your analysis will be done seperatelly. Some pseudocode based on your sample would be:

    await Windows.ApplicationModel.Core.CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(CoreDispatcherPriority.High, () => {                
    var analysisTask = Task.Run(() => getPreview());

    previewImage.Source = _writeableBitmap; // previewImage is an image element in MainPage.xaml
    _writeableBitmap.Invalidate(); // force the PreviewBitmap to redraw

    await analysisTask;
    });

This way the task of analysing the image doesn't block the update on the screen. Of course this option might not be viable if you need the result of the analysis in the rendering chain itself.

• Create a custom filter for the analysis, this way you will be able to take advantage of the optimized Lumia Imaging SDK processing.

  To get started on writing custom filters look at the documentation.