Search code examples
jsonazureazure-stream-analytics

Reference data join on stream analytics input not giving output


I'm trying to set a rule in Azure Stream Analytics job with the use of reference data and input stream which is coming from an event hub.

This is my reference data JSON packet in BLOB storage:

{
    "ruleId": 1234, 
    "Tag" : "TAG1", 
    "metricName": "velocity", 
    "alertName": "velocity over 500",
    "operator" : "AVGGREATEROREQUAL",
    "value": 500
}

And here is the transformation query in the stream analytics job:

WITH
transformedInput AS
(
    SELECT
        metric = GetArrayElement(DeviceInputStream.data,0),
        masterTag = rules.Tag, 
        ruleId = rules.ruleId, 
        alertName = rules.alertName,
        ruleOperator = rules.operator, 
        ruleValue = rules.value
    FROM 
        DeviceInputStream
        timestamp by EventProcessedUtcTime
    JOIN 
        rules
        ON DeviceInputStream.masterTag = rules.Tag
)
--rule output--
SELECT
    System.Timestamp as time,
    transformedInput.Tag as Tag,
    transformedInput.ruleId as ruleId,
    transformedInput.alertName as alert,
    AVG(metric.velocity) as avg
    INTO
    alertruleblob
FROM
    transformedInput
GROUP BY
    transformedInput.masterTag,
    transformedInput.ruleId,
    transformedInput.alertName,
    ruleOperator, 
    ruleValue, 
    TumblingWindow(second, 6)
HAVING
    ruleOperator = 'AVGGREATEROREQUAL' AND avg(metric.velocity) >= ruleValue

This is not yielding any results. However, when I do a test with sample input and reference data I get the expected results. But this doens't seem to be working with the streaming data. My use case is if the average velocity is greater than 500 for a 6 second window, store that result in another blob storage. The value of velocity has been greater than 500 for sometime, but I'm not getting any results.

What am I doing wrong?


Solution

  • This was working all along. I just had to specify the input path of the reference blob in the reference input path of stream analytics including the file name. I was basically referencing only the blob container without the actual file. So when I changed the path pattern to "filename.json", I got the results. It was a stupid mistake.