I am using TPL Dataflow to load a video (I am using Emgu.CV library for loading) from a path and via TPL Dataflow first plot it in a Windows Form application (after that there will be a communication step between with a board). I had another post that helped me very much with TPL Dataflow here: Asynchronous Task, video buffering
But after setting up the TPL Dataflow the first image is only loaded in the GUI and after that (while the Block seems to running because prints are displayed in cmd) the image does not refresh... I cannot understand what is wrong? Does it have to do with scheduler or with the TPL Dataflow? Below is the code:
public async void CreateVideoProcessingNetwork()
{
string video_path = @"C:\.......\video_640x360_360p.mp4";
/* displayVideo Block*/
var display_video = new ActionBlock<Bitmap>(async received_bitmap =>
{
Console.WriteLine("Inside display_video");
PicturePlot2.Refresh();
PicturePlot2.Image = received_bitmap;
Console.WriteLine("Image size = " + received_bitmap.Size);
Console.WriteLine("Image width = " + received_bitmap.Width);
await Task.Delay(30);
});
var loadVideo = new ActionBlock<string>(async path =>
{
capture = new VideoCapture(path);
Mat matrix = new Mat();
capture.Read(matrix);
var mem_stream = new MemoryStream();
Bitmap HWimage;
while (matrix.Rows != 0 && matrix.Width != 0)
{
Console.WriteLine("Inside LoadVideo");
matrix = new Mat();
capture.Read(matrix);
Bitmap bitmap = new Bitmap(matrix.Width, matrix.Rows);
bitmap = matrix.ToBitmap();
bitmap.Save(mem_stream, System.Drawing.Imaging.ImageFormat.Jpeg);
byte[] image_array = mem_stream.ToArray();
Console.WriteLine("image_array = " + image_array.Length);
using (var mem_stream_hw = new MemoryStream(image_array)) HWimage = new Bitmap(mem_stream_hw);
var accepted = await display_video.SendAsync(HWimage);
if (!accepted) break;
await Task.Delay(25);
}
});
PropagateCompletion(loadVideo, display_video);
loadVideo.Post(video_path);
loadVideo.Complete();
await display_video.Completion;
}
Am I understanding something wrong? I would like to do some kind of pipelining in the video display via TPL Dataflow.
UPDATE:
After I did the changed mentioned the video plays. But there is one more problem. The data creation rate of the "producer" is faster (Actually it depends from the Thread.Sleep time in the TransformManyBlock ) and after some seconds of video playing the app crashes with the following error:
Unhandled Exception: System.Runtime.InteropServices.ExternalException: A generic error occurred in GDI+.
at System.Drawing.Graphics.MeasureString(String text, Font font, SizeF layoutArea, StringFormat stringFormat)
at System.Drawing.Graphics.MeasureString(String text, Font font, Int32 width)
at System.Windows.Forms.ThreadExceptionDialog..ctor(Exception t)
at System.Windows.Forms.Application.ThreadContext.OnThreadException(Exception t)
at System.Windows.Forms.Control.WndProcException(Exception e)
at System.Windows.Forms.Control.ControlNativeWindow.OnThreadException(Exception e)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
at System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG& msg)
at System.Windows.Forms.Application.ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(IntPtr dwComponentID, Int32 reason, Int32 pvLoopData)
at System.Windows.Forms.Application.ThreadContext.RunMessageLoopInner(Int32 reason, ApplicationContext context)
at System.Windows.Forms.Application.ThreadContext.RunMessageLoop(Int32 reason, ApplicationContext context)
at System.Windows.Forms.Application.Run(Form mainForm)
at ntComlabGUI.Program.Main()
If for example the Thread.Sleep is removed the error occurs almost immediately (in the first 1-2 secs). How can someone do flow control? In the Asynchronous Task, video buffering post it was mentionted the BoundedCapacity method. but i've tried it and the error still exists. Below is the code:
public async void CreateVideoProcessingNetwork()
{
//string video_path = @"C:\Projects_Repo\ComlabDMA_Ethernet_video\ntComlabGUI_Ultrascale_ethernet\ntComlabGUI\video_640x360_360p.mp4";
string video_path = @"C:\Projects_Repo\ComlabDMA_Ethernet_video\ntComlabGUI_Ultrascale_ethernet\ntComlabGUI\video_640x360_360p.mp4";
/* Video Loading TPL Block */
var video_loader = new TransformManyBlock<string, Bitmap>(load_video,
new ExecutionDataflowBlockOptions { BoundedCapacity = 128 });
IEnumerable<Bitmap> load_video(string path)
{
capture = new VideoCapture(path);
Mat matrix = new Mat();
capture.Read(matrix);
var mem_stream = new MemoryStream();
while (matrix.Rows != 0 && matrix.Width != 0)
{
capture.Read(matrix);
Bitmap bitmap = new Bitmap(matrix.Width, matrix.Rows);
bitmap = matrix.ToBitmap();
yield return bitmap;
//Thread.Sleep(1);
}
yield break;
}
/* Video Loading TPL Block */
var display_video = new ActionBlock<Bitmap>(async received_image =>
{
PicturePlot2.Image = received_image;
await Task.Delay(33);
},
new ExecutionDataflowBlockOptions()
{
TaskScheduler = TaskScheduler.FromCurrentSynchronizationContext(),
BoundedCapacity = 128
});
var linkOptions = new DataflowLinkOptions { PropagateCompletion = true };
video_loader.LinkTo(display_video, linkOptions);
video_loader.Post(video_path);
video_loader.Complete();
await display_video.Completion;
}
Perhaps is the following link the solution? : How do I arrange flow control in TPL Dataflows?
Thanks in advance for helping, and also for fast response, really appreciated!
Most probably the problem is that the PicturePlot2 component doesn't like being manipulated by non-UI threads. To ensure that the ActionBlock's delegate will be invoked on the UI thread, you can configure the TaskScheduler option of the block like this:
var display_video = new ActionBlock<Bitmap>(async received_bitmap =>
{
Console.WriteLine("Inside display_video");
PicturePlot2.Refresh();
PicturePlot2.Image = received_bitmap;
Console.WriteLine("Image size = " + received_bitmap.Size);
Console.WriteLine("Image width = " + received_bitmap.Width);
await Task.Delay(30);
}, new ExecutionDataflowBlockOptions()
{
TaskScheduler = TaskScheduler.FromCurrentSynchronizationContext()
});
For this to work, it is required that the block will be instantiated on the UI thread. That's because the Windows Form applications install a special SynchronizationContext on the UI thread when they are launched, and we want the TaskScheduler.FromCurrentSynchronizationContext method to capture this context.