Multithreaded Single Producer Multiple Consumer Implementation

Question

I would like to implement a Multiple file downloading with pattern of single producer and multiple consumer.

What I have: - Code which finds new links to be downloaded in a loop - When a new link is found - it calls download function - Download function accepts source file path and destination file path and downloads the file.

What I want to do - I want to download X number of files simultaneously (I dont know total number of files) - At any times I should be able to download X files simultaneously - as soon as 1 of the X file finish downloading - the calling function should be able to add new download right away - which in turn downloading right away

• So I have a producer function which keeps adding new download to queue (at any time maximum X downloads)
• Multiple X thread which consumes the downloads and start downloading individually. Once it finishes download - the producer should be able to add new download - which will spawn new thread.

EXAMPLE would be really appreciated

Answer 1

ReaderWriterLockSlim class is designed to do that.

Also, check this brilliant website about threading:

http://www.albahari.com/threading/part4.aspx#_Reader_Writer_Locks

The example comes from the website above.

class SlimDemo
{
  static ReaderWriterLockSlim _rw = new ReaderWriterLockSlim();
  static List<int> _items = new List<int>();
  static Random _rand = new Random();

  static void Main()
  {
    new Thread (Read).Start();
    new Thread (Read).Start();
    new Thread (Read).Start();

    new Thread (Write).Start ("A");
    new Thread (Write).Start ("B");
  }

  static void Read()
  {
    while (true)
    {
      _rw.EnterReadLock();
      foreach (int i in _items) Thread.Sleep (10);
      _rw.ExitReadLock();
    }
  }

  static void Write (object threadID)
  {
    while (true)
    {
      int newNumber = GetRandNum (100);
      _rw.EnterWriteLock();
      _items.Add (newNumber);
      _rw.ExitWriteLock();
      Console.WriteLine ("Thread " + threadID + " added " + newNumber);
      Thread.Sleep (100);
    }
  }

  static int GetRandNum (int max) { lock (_rand) return _rand.Next(max); }
}