Search code examples
azureazure-stream-analytics

Basic query with TIMESTAMP by not producing output


I have a very basic setup, in which I never get any output if I use the TIMESTAMP BY statement.

I have a stream analytics job which is reading from Event Hub and writing to the table storage.

The query is the following:

SELECT
    * 
INTO
    MyOutput
FROM
    MyInput TIMESTAMP BY myDateTime;

If the query uses timestamp statement, I never get any output events. I do see incoming events in the monitoring, there are no errors neither in monitoring nor in the maintenance logs. I am pretty sure that the source data has the right column in the right format.

If I remove the timestamp statement, then everything is working fine. The reason why I need the timestamp statement in the first place is because I need to write a number of queries in the same job, writing various aggregations to different outputs. And if I use timestamp in one query, I am required to use it in all other queries itself.

Am I doing something wrong? Perhaps SELECT * does not play well with TIMESTAMP BY? I just did not find any documentation explaining that...


Solution

  • {"myDateTime":"2015-08-02T10:59:02.0000000Z", "EventEnqueuedUtcTime":"2015-08-07T10:59:07.6980000Z"}

    Late tolerance window: 00.00:00:05

    All of your events are considered late arriving because myDateTime is 5 days before EventEnqueuedUtcTime. Can you try sending new events where myDateTime is in UTC and is "now" so it matches within a couple of seconds?

    Also, when you started the job, what did you pick as the job start date time? Can you make sure you pick a date before the myDateTime values? You might try this first.