Search code examples
c#.netcomplex-event-processingstreaminsight

StreamInsight and long running queries


I have an application built on top of StreamInsight 1.2 that has several standing queries running some calculations in User Defined Operators. Occasionally one of the calculations will take a relatively long time to complete (a few seconds), and I'm seeing that it blocks the other queries from executing. I'm not very familiar with how StreamInsight works so from this I'm speculating that all queries are run on a single thread. Is there some way to have the queries run in parallel?

Aside from somehow putting the queries on separate threads, are there any other things I could do to prevent one sporadically long running query from blocking other queries?


Solution

  • If the calculation will occasionally take a few seconds and that's the nature of it and you are running Standard Edition, the best thing that you can do is to not block the thread while the calculation is taking place. StreamInsight doesn't have anything built-in for this ... it's your responsibility. From StreamInsight's perspective, your UDO is taking some time. It doesn't know why that is. And if the thread is blocked, it can't run anything else until your UDO returns. So I would a) validate that you are running standard edition and then b) if you are, change the UDO so that it's non-blocking/async.