ConcurrentQueue Operation is giving OutofMemory Error.

Question

I understand this question may be too general. But I tried many things and I am not able to figure out how to resolve this.

I am using ConcurrentQueue for multithreading operation. One thread is downloading Images from server and saving it to queue. Here is the code for that:

 public static void DownloadImage()
    {
        string baseUrl = "http://someurl";
        //int numIterations = 5;

        HttpWebRequest request = null;
        foreach (var fileName in fileNames)
        {
                string url = string.Format(baseUrl, fileName);
                request = (HttpWebRequest)WebRequest.Create(url);
                request.Method = "GET";
                request.ContentType = "application/x-www-form-urlencoded";
                var response = (HttpWebResponse)request.GetResponse();
                Stream stream = response.GetResponseStream();
                img = Image.FromStream(stream);
                ImageFileName FileNameImage = new ImageFileName(fileName, img);
                ImageQueue.Enqueue(FileNameImage);
                Console.WriteLine("Count after Enqueue: {0}", ImageQueue.Count);

         }

And another thread takes images from Queue and saves them on destination folder. Here is the code for that:

public static void SaveImage()
    {
        while (true)
        {
            if (!ImageQueue.IsEmpty)
            {
                foreach (var newobject2 in ImageQueue)
                {

                    Image img2 = newobject2.Image;
                    img2.Save("C:\\path" + newobject2.ImageName);
                    ZoomThumbnail = img2;
                    ZoomSmall = img2;
                    ZoomLarge = img2;

                    ZoomThumbnail = GenerateThumbnail(ZoomThumbnail, 86, false);
                    ZoomSmall = GenerateThumbnail(ZoomSmall, 400, false);
                    ZoomLarge = GenerateThumbnail(ZoomLarge, 1200, false);

                    ZoomThumbnail.Save("C:\\path" + newobject2.ImageName + "_Thumb.jpg");
                    ZoomSmall.Save("C:\\path" + newobject2.ImageName + "_ZoomSmall.jpg");
                    ZoomLarge.Save("C:\\path" + newobject2.ImageName + "_ZoomLarge.jpg");
                    ImageFileName imgobject3 = new ImageFileName();
                    ImageQueue.TryDequeue(out imgobject3);
                    Console.WriteLine("Count after Deque: {0}", ImageQueue.Count);

                }


            }


        }

    }

I am calling these two threads from Button_Click() like this:

Thread DownloadThread = new Thread(DownloadImage);
  DownloadThread.Start();
  Thread SaveThread = new Thread(SaveImage);
  SaveThread.Start();

I am getting MemoryFull error whenever queue reaches count of 68. I am not sure how I can avoid that. I have tried using Thread.Sleep to avoid this. For Example, I tried: Thread.Sleep(500) after foreach loop. Whenever I try it inside foreach it works totally fine as at any given point ImageQueue.Count = 1. Where I am getting wrong?

Answer 1

You never actually remove your images from the queue. You iterate through them, but you never dequeue them. You also have the problem that when enumerating the queue the enumerable will stop whenever there are currently no more items in the queue. You don't want to do this, you may not be done yet.

What you want to do is use a BlockingCollection instead of a ConcurrentQueue, so that if there are currently no more items you wait for more, rather than stopping. Once you do this, you can use foreach(var item in queue.GetConsumingEnumerable()). This makes all of the needed changes. It removes the items from the queue as you iterate it, and it waits for more items if there are currently none.

In addition to making this change, as is mentioned in Guffa's answer, you need to dispose of your image objects. You can dispose of them at the end of your loop body, once you're done with the image.