Smooth chart update from multithreaded data source

We have several graphs that show filtered data from surface EMG sensors. This data is received over TCP and propagated using an event. The datapackets are of type DataPackets and are as such filtered out. I'm using a buffer to let packets pass through at 30FPS. I'm listening to this event using Reactive Extensions as follows:

Observable.FromEvent<Packet>(h => this.DataService.PacketReceived += h, h=> this.DataService.PacketReceived -= h)
          .OfType<DataPacket>()
          .Buffer(TimeSpan.FromSeconds(1.0 / 30))
          .ObserveOnDispatcher()
          .Subscribe(
              packet => this.ReceiveDataPackets(packet.ToList()),
              err => this.Log.Error("Error subscribing to data packets", err),
              () => this.Log.Info("Finished listening to data packets"));

To display the EMG data we use the Telerik ChartView. The problem I'm having is that data update is not smooth, the graph is choppy.

There are probably several reasons for this:

Telerik chart is not fast enough for 1000 data points per second
DispatcherTimer does not fire at a constant rate
data is not received at a constant rate

Point 1 is solved by sampling the input data so that only 1000 points are visible in the graph.

Point 2 cannot be solved unfortunately. I tried raising the priority to Render but this does not help at all. http://social.msdn.microsoft.com/Forums/en-US/5eea6700-1c79-4da6-9b68-efa480ed3a36/simplify-wpf-dispatcher-calls?forum=rx

Point 3 is related to point 2. I try to solve both with a timed queue using System.Debug.Stopwatch. DataPackets contain a timestamp and this is used to let them through at a constant rate on the Dispatcher thread. I suspect that this will not help much since the DispatcherRate is not linked to refresh rate of the renderthread.

What can I do to reduce choppiness? I tried LightningChart Ultimate which is supposed to be much much faster. It does indeed have much better performance and there is no need to do any sampling, it can render each and every datapoint. The samples provided with LightningChart run butter smooth, but they read their data in the main thread. When I implement their chart in our multi-threaded program it still suffers from a combination of point 2 and 3 (and the fact that it is much more expensive than the Telerik chartview.)

[Update]

Classic mistake. My datasource was using a DispatcherTimer to gather data. Changing this to an Observable.Interval massively increased performance.

Solution

I have had similar troubles with chart control a while ago. I too found that most charting controls just could not handle 1000+ datapoints. I also found that trying to get it re-render more than 5-15 times per second was pretty tough on the dispatcher.

Things I would suggest to you are:

Aim for a much lower frame rate. You are hoping to get updates to your chart every 33ms?
Look to reduce your data set to 1000 points. Again play with the numbers to see what works for your target spec PC and your controls/datatemplating. This old post may help you. It provides a way to pick the X most useful points to render from a collection
Dial back any fancy animations or extra UI that is rendered per point on the chart. if you are adding more items that each have tool tips, brushes, animations and excess layout panels, you will pay the cost. Also note that you will not only pay the cost as they are created, but also later when the GC tries to clear out these thousands of UI objects.
Do less work on the dispatcher. I know it is not much, but you could move that ToList() and do it on the other thread. Also, dont send empty lists to be processed (if applicable to your events)

Consider using the D3/DDD charts. While they were not great when I reviewed them, colleagues have had success with them since then.

Observable.FromEvent<Packet>(h => this.DataService.PacketReceived += h, h=> this.DataService.PacketReceived -= h)
  .OfType<DataPacket>()
  .Buffer(TimeSpan.FromSeconds(1.0 / 8))  //Reduce the FPS
  .Select(packet=>packet.ToList())        //Reduce work done on dispatcher
  .Where(packet=>packet.Count>0)          //Dont send empty sets to dispatcher
  .ObserveOnDispatcher()
  .Subscribe(
      packet => this.ReceiveDataPackets(packet),
      err => this.Log.Error("Error subscribing to data packets", err),
      () => this.Log.Info("Finished listening to data packets"));

I am not sure if it helps, but here is another link to some code from a presentation. Part of the presentation was about how to stream data via Rx to a WPF Chart. You could potentially rip the whole thing.