I have a program accessing database and downloading images. I was using BlockingCollection for that purpose. However, to access some UI elements I decided to use combination of Backgroundworker and BlockingCollection. It reduced speed of processing considerably as compared to speed when only Blockingcollection was used. What can be the reason? Or as I am now accessing UI elements, there is reduction in speed?
Here is the code I am working on:
private void button_Start_Click(object sender, System.EventArgs e)
{
BackgroundWorker bgWorker = new BackgroundWorker();
bgWorker.DoWork += bw_DoWork;
bgWorker.RunWorkerCompleted += bw_RunWorkerCompleted;
bgWorker.ProgressChanged += bw_ProgressChanged;
bgWorker.WorkerSupportsCancellation = true;
bgWorker.WorkerReportsProgress = true;
Button btnSender = (Button)sender;
btnSender.Enabled = false;
bgWorker.RunWorkerAsync();
}
and Do_Work() is as follows:
{
HttpWebRequest request = null;
using (BlockingCollection<ImageFileName> bc = new BlockingCollection<ImageFileName>(30))
{
using (Task task1 = Task.Factory.StartNew(() =>
{
foreach (var fileName in fileNames)
{
string baseUrl = "http://some url";
string url = string.Format(baseUrl, fileName);
request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
request.ContentType = "application/x-www-form-urlencoded";
var response = (HttpWebResponse)request.GetResponse();
Stream stream = response.GetResponseStream();
img = Image.FromStream(stream);
FileNameImage = new ImageFileName(fileName.ToString(), img);
bc.Add(FileNameImage);
Thread.Sleep(100);
Console.WriteLine("Size of BlockingCollection: {0}", bc.Count);
}
}))
{
using (Task task2 = Task.Factory.StartNew(() =>
{
foreach (ImageFileName imgfilename2 in bc.GetConsumingEnumerable())
{
if (bw.CancellationPending == true)
{
e.Cancel = true;
break;
}
else
{
int numIterations = 4;
Image img2 = imgfilename2.Image;
for (int i = 0; i < numIterations; i++)
{
img2.Save("C:\\path" + imgfilename2.ImageName);
ZoomThumbnail = img2;
ZoomSmall = img2;
ZoomLarge = img2;
ZoomThumbnail = GenerateThumbnail(ZoomThumbnail, 86, false);
ZoomThumbnail.Save("C:\\path" + imgfilename2.ImageName + "_Thumb.jpg");
ZoomThumbnail.Dispose();
ZoomSmall = GenerateThumbnail(ZoomSmall, 400, false);
ZoomSmall.Save("C:\\path" + imgfilename2.ImageName + "_Small.jpg");
ZoomSmall.Dispose();
ZoomLarge = GenerateThumbnail(ZoomLarge, 1200, false);
ZoomLarge.Save("C:\\path" + imgfilename2.ImageName + "_Large.jpg");
ZoomLarge.Dispose();
// progressBar1.BeginInvoke(ProgressBarChange);
int percentComplete = (int)(((i + 1.0) / (double)numIterations) * 100.0);
//if (progressBar1.InvokeRequired)
//{
// BeginInvoke(new MethodInvoker(delegate{bw.ReportProgress(percentComplete)};))
//}
}
Console.WriteLine("This is Take part and size is: {0}", bc.Count);
}
}
}))
Task.WaitAll(task1, task2);
}
}
}
A better option might be to make retrieving the data and writing it to disk run synchronously, and instead use Parallel.ForEach() to allow multiple requests to be in-flight at the same time. That should reduce the amount of waiting in a couple spots:
So perhaps something more like this:
Parallel.ForEach(fileNames,
(name) =>
{
string baseUrl = "http://some url";
string url = string.Format(baseUrl, fileName);
var request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
request.ContentType = "application/x-www-form-urlencoded";
var response = (HttpWebResponse)request.GetResponse();
Stream stream = response.GetResponseStream();
var img = Image.FromStream(stream);
// Cutting out a lot of steps from the 2nd Task to simplify the example
img.Save(Path.Combine("C:\\path", fileName.ToString()));
});
One possible problem you could run into with this approach is that it will start generating too many requests at once. That might cause resource contention issues, or perhaps the webserver will interpret it as malicious behavior and stop responding to you. You can limit the number of requests that happen at the same time by setting the MaxDegreeOfParallelism. The following example shows how you could limit the operation to process no more than 4 files at the same time.
var options = new ParallelOptions { MaxDegreeOfParallelism = 4 };
Parallel.ForEach(fileNames, (name) => { /* do stuff */ }, options);