Search code examples
azureazure-stream-analytics

Azure Stream Analytics - output an event only when it is different to the previous one


i have a special situation that stream anlytics does not send an output event when using LAG in combination with input data in an array (json).

That is the input data from an datalogger:

20.04.2019 16:05:04> Device: [WISE-4012_00D0C9E43D10], Data:[{"PE": 1,"UID":"WISE-4012_00D0C9E43D10","TIM":"2019-04-20T16:04:55+01:00","Record":[[0, 0, 1, 0],[0, 1, 1, 0],[0, 2, 1, 0],[0, 3, 1, 0]]}]

I am using this query in azure stream analytics:

SELECT
    PE, UID, TIM,
    UDF.DI1(InputIoThub) As DI1
FROM
    [IoTHub]
WHERE
    LAG(DI1, 1) OVER (LIMIT DURATION(minute, 1)) <> DI1

according tho this instructions: https://learn.microsoft.com/en-us/azure/stream-analytics/stream-analytics-stream-analytics-query-patterns

It looks like LAG can not deal with the UDF.

I replaced the UDF with following:

SELECT
   PE, UID, TIM,
   GetArrayElement(GetArrayElement(Record,0), 3) AS DI_1
INTO
   [toPowerBI]
FROM
   [IoTHub]
WHERE
   LAG(DI_1, 1) OVER (LIMIT DURATION(minute, 1)) <> DI_1

but it does not work too.

Without the WHERE clouse, the stream generates an correct output, so I tried to use HAVING instead without success.

thank you very much for your help!


Solution

  • I hope my question was not confusing, I was searching for a solution to send the digital input (DI_1) from an datalogger only when the value has changed. Reason therefore is to calculate a time difference in PowerBI.

    In the meantime I found a solution:

    The LAG function works fine, also with the WHERE

    SELECT
       PE, UID, TIM,
       GetArrayElement(GetArrayElement(Record,0), 3) AS DI_1
    INTO
       [toPowerBI]
    FROM
       [IoTHub]
    WHERE
       LAG(GetArrayElement(GetArrayElement(Record,0), 3), 1) 
       OVER (LIMIT DURATION(minute, 10)) <>    
       GetArrayElement(GetArrayElement(Record,0), 3)