Search code examples
c#.netstreaminsight

StreamInsight: CleanseInput drops events

I got an issue and I don't know why it appears. I have a InputAdapter which receives randomly generated events for testing purposes.

The following code creates a query which drops a lot of events. which I actually want to be kept.

var atgs = new AdvanceTimeGenerationSettings(config.Input.EventCount, 
                    TimeSpan.FromSeconds(config.Input.Delay), true);
                var ats = new AdvanceTimeSettings(atgs, null, AdvanceTimePolicy.Adjust);

                var dstream = CepStream<Dataclass>.Create("Data Input Stream", typeof (InAdapterFactory),
                    config.Input, EventShape.Point, ats);



                Query output = dstream.ToQuery(myApplication,
                    "Sample unbounded query",
                    "Query with no output adapter",
                    EventShape.Point,
                    StreamEventOrder.FullyOrdered);
                output.Start();

                var instream = output.ToStream<Dataclass>();

Does anyone have an idea why between the DataInputStream and DatainputStream_CleanseInput only a fifth of all events get passed through? See the following pic. Is there a way to avoid that?

https://dl.dropboxusercontent.com/u/15482726/CleanseIssue.jpg

I am very grateful for any help.

Best regards, Joe

Solution

  • Those are going to be events dropped due to CTI violations. While you have your AdvanceTimePolicy set to Adjust, this only applies to Interval events that cross the CTI span in question. That is, if the Interval event has a start time before the last-issued CTI and an end time after the last-issued CTI, the start time (only) of the Interval event will be adjusted. Point events are always dropped.