Search code examples
c#.netmultithreadingimage-processingvideo-processing

How to process video frames in different thread (C#)


Background

I am working on an application that requires some image-processing on video stream and display the original video and processed video side-by-side.

Situtation

Below is the event handler when a new frame is received from camera. pictureBox1is where the original video is shown. GetInputImage() function will steal a from pictureBox1,so that some image processing can be performed on that frame.

private void camera_NewFrame(object sender, ref Bitmap image)
{
    if (!isReadingPictureBox)
    {
       if (pictureBox1.Image != null)
           pictureBox1.Image.Dispose();
       pictureBox1.Image = (Bitmap)image.Clone();
    }
}

private void GetInputImage()
{
    if (inputImage != null)
        inputImage.Dispose();

    isReadingPictureBox = true;
    if (pictureBox1.Image != null)
        inputImage = new Bitmap(pictureBox1.Image);
    isReadingPictureBox = false;
}

The image processing is heavy and takes time to process a single image. Thus the frame rate of output video is expected to be very less that the original.

Application must show the original video without getting affected by the image processing. So I want to perform the image processing in a different thread.

private void ProcessThread(some args)
{
    GetInputImage();
    if (inputImage != null) {
        // Do Image Processing on "inputImage"
        // Show result in pictureBox2
    }
}

Questions

[1] Is the method of grabbing a frame (above), OK ? Or the one below is better ?

private void camera_NewFrame(object sender, ref Bitmap image)
{
    pictureBox1.Image = image; // picturBox1's image will not be read for processing

    if(!isReadingInputImage) {
        if (inputImage != null)
            inputImage.Dispose();
        inputImage = (Bitmap)image.Clone(); // GetInputImage() is not required now.
    }
}

[2] How to make the ProcessMyThread(), run for every frame available at the moment when previous frame is finished processing ? Is this(below) approach OK ?

private void ProcessMyThread(some args)
{
    do {
        GetInputImage();
        if (inputImage != null) {
            // Do Image Processing on inputImage;
            // Show result in pictureBox2
        }
    }while(someCondition);
}

Or should I trigger a processing event for each frame within camera_NewFrame() func ?


Solution

  • I solved the problem myself using background worker.

    private void camera_NewFrame(object sender, ref Bitmap image)
    {
        pictureBox1.Image = image;
    
        if (backgroundWorker1.IsBusy != true)
        {
            lock (locker)
            {
                if (inputImage != null)
                    inputImage.Dispose();
                inputImage = (Bitmap)image.Clone();
            }
            backgroundWorker1.RunWorkerAsync();
        }
    }
    
    private void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
    {
        lock (locker)
        {
             baseImage = (Bitmap)inputImage.Clone();
        }
        // Do Image processing here on "baseImage"
        // show result in pictureBox2
    }