Search code examples
azureazure-eventhubazure-stream-analytics

Does azure stream analytics read data coming from all partitions


Azure event hub has partition feature for scalability. While reading data using app service, one eventprocessorHost can be tied to one partition only. There is no way to act collectively on data coming from multiple partitions. But while using Stream analytics, we can aggregate data based on time. So, does it take care of all the partitions while aggregating the data? Means, if reading are passed to 8 partitions, aggregate should includes all these readings in calculation. Thanks


Solution

  • Yes. Based on the documentation there are a couple of scenario's.

    When the output does support partitioning as well, like another Event Hub, you can use the Partition By:

    you must make sure that your query is partitioned. This requires you to use Partition By in all the steps. Multiple steps are allowed, but they all must be partitioned by the same key. Currently, the partitioning key must be set to PartitionId in order for the job to be fully parallel.

    When the output does not have support for partitioning (like Power BI) data is read without taking in the origin partition data (and so it will read from all partitions).