Handling realtime data

Question

My program will be requesting realtime data from a website. This data can change at any time, so I need to request it repeatedly and frequently to monitor changes. The data is to be displayed on a windows form chart as a histogram along with some moving averages. Based on this information, the user needs to be able to interact with the form to set parameters for a part of the program to take action based on the incoming data. How should I be handling this data? My current plan is to have a separate thread collecting the data and writing it to the main form, but I'm unsure how to monitor that data for changes without A) making the interface unresponsive and B) starting another thread. A is unacceptable for obvious reasons, and if I were to do B I feel like I might as well just throw that code into the thread that's collecting the data.

Answer 1

What you should do in this case is have a worker thread do the polling of the website and queue all of the data it finds into a ConcurrentQueue. Then have your UI thread periodically poll this queue for new data. You do not want to have that worker thread interacting with the UI thread at all. Do not use Control.Invoke or other marshaling techniques for situations like this.

public class YourForm : Form
{
  private CancellationTokenSource cts = new CancellationTokenSource();
  private ConcurrentQueue<YourData> queue = new ConcurrentQueue<YourData>();

  private void YourForm_Load(object sender, EventArgs args)
  {
    Task.Factory.StartNew(Worker, TaskCreationOptions.LongRunning);
  }

  private void UpdateTimer_Tick(object sender, EventArgs args)
  {
    YourData item;
    while (queue.TryDequeue(out item))
    {
      // Update the chart here.
    }
  }

  private void Worker()
  {
    CancellationToken cancellation = cts.Token;
    while (!cancellation.WaitHandle.WaitOne(YOUR_POLLING_INTERVAL))
    {
      YourData item = GetData();
      queue.Enqueue(item);
    }
  }   
}

The example above is based on WinForms, but the same principals would carry over to WPF as well. The important points in the example are.

• Use a System.Windows.Timer.Timer (or equivalent if using WPF) to pull the data items from the queue using TryDequeue. Set the tick frequency to something that provides a good balance between refreshing screen quickly, but not so quickly that it dominates the UI thread's processing time.
• Use a task/thread to acquire the data from the website and load it into the queue using Enqueue.
• Use the CancellationTokenSource to cancel the operation via Cancel (which I did not include in the example) and to also drive the polling interval by calling WaitOne on the WaitHandle provided by the CancellationToken.

While using Control.Invoke or other marshaling techniques is not necessarily a bad thing it is not the panacea that it is often made out to be either. Here are but a few disadvantages to using this technique.

• It tightly couples the UI and worker threads.
• The worker thread dictates how often the UI should update.
• It is an expensive operation.
• The worker thread has to wait for the UI thread to process the message so throughput is decreased.

The advantages of having the UI thread poll for updates are as follows.

• The UI and worker threads are more loosely coupled. In fact, neither really knows anything about the other.
• The UI thread gets to decide on its own how often the updates are applied. This is really how it should be anyway.
• There are no expensive marshaling operations.
• Neither the UI nor the worker threads have their executions impeded by the other. You get more throughput on both threads.