Passing lots of variables between threads for logging purposes efficiently

Question

I am using VS 2005

I have about 10-12 variables of different types such as double, int, string, long and bool that I want to log on another thread with min. overhead.

Currently, each time when I have to log, I "new" an arraylist, add all variables in it and then pass it as an argument to LoggerClass.Log(). In LoggerClass it sets/resets the event and another thread reads the value of arraylist.

Since the function gets called a lot, doing "new" is not very efficient and I am looking for a way to pass all the values to another thread without dynamic memory allocation.

One alternate in my mind is to not "new" the arraylist but just keep adding to it but the problem with that is the size of arraylist can become enoromous.

Any suggestions?

Heres the code

//This function gets called a thousand times per second

private void MyLogic(MyClass LogObj)
{

  ArrayList MyArrList = new ArrayList(15);//This line causes periodic performance outliers
                            MyArrList.Add("Q");
                            MyArrList.Add(LogObj.valString);
                            MyArrList.Add(LogObj.ValDouble);
                            MyArrList.Add(LogObj.ValInt);
                            MyArrList.Add(LogObj.valString2);
                            MyArrList.Add(LogObj.ValDouble2);
                            MyArrList.Add(LogObj.ValInt2);
                            MyArrList.Add(LogObj.valString3);
                            MyArrList.Add(LogObj.ValDouble3);
                            MyArrList.Add(LogObj.ValInt3);
                            MyArrList.Add(LogObj.valString4);
                            MyArrList.Add(LogObj.ValDouble4);
                            MyArrList.Add(LogObj.ValInt4);

  MyLogger.Log(MyArrList);
}


Class MyLogger
{
  private Queue         m_logQueue;

  public void Log(ArrayList LogArr) 
  {
    lock (m_LogQueue)
    {
      m_LogQueue.Enqueue(LogArr);
    }

    m_Event.Set();
  }
}

private void LogThread()
{           
  ArrayList ValuesToLog = new ArrayList(); 
  StringBuilder StringBuf = new StringBuilder(); 
  string csFileName = "MyLog";

  using (StreamWriter sw = new StreamWriter(new FileStream(csFileName, FileMode.Append, FileAccess.Write, FileShare.ReadWrite)))
  {
    while (m_bRun)
    {
      m_Event.WaitOne();
      try 
      {
        while (true)
        {
          lock (m_logQueue)
          {
            if (m_logQueue.Count > 0)
            {
              ValuesToLog = (ArrayList)m_logQueue.Dequeue();                                     
            }
            else
              break;
          }

          StringBuf = new StringBuilder();
          StringBuf.Append(DateTime.Now);
          StringBuf.Append(",");

          for (int i = 0; i < (ValuesToLog.Count - 1); i++)
          {
            StringBuf.Append(",");
            StringBuf.Append(ValuesToLog[i]);
          }

          sw.WriteLine(StringBuf.ToString());

          if (m_bAbort) break;
          if (m_EventStop.WaitOne(0, true))
            break;
        }
        sw.Flush();
      }
      catch(Exception e)
      {

      }
    }
  }
}

Answer 1

If you're really sure that the problem is allocating new list every time, you can recycle them.

Have a collection of unused lists, that starts empty. When there is a list in the collection, take that (and remove it from the collection). Otherwise, create a new one. When you finish logging, Clear() the list and return it to the collection.

Of course, you access the collection from multiple threads, so you have make sure you're accessing it in a thread-safe manner.