We have an IoT service running on Azure which produces a lot of events. We need to build a new feature which allows our end-user to configure alerts based on system events. It allows the user to pick an event and configure an action (e-mail, webhook, etc) to be executed when such an event occurs. We're evaluating Azure Eventhub and possibly Azure Stream Analtyics as candidates for the job.
The problem we face is: We think we will get a lot of stream analytics jobs running. When for example we have 3000 customers each configuring 3 alerts we need to run 9000 stream analtyics jobs which select specific events from the eventhub pushing it in a queue which does the alert processing. This will not only be a tough maintenance job, but I think it not a really cost effective solution.
Any thoughts on this or better solutions?
I am assuming events from multiple customers go to a fixed small set of event hubs and actions go to a fixed small set of queues. You can design it such that a single Azure stream analytics job handles processing for multiple customers. Reference Data (lets call it CustomerToAlertLookup) can be used to decide which customer has configured which alert, and event stream can be joined with CustomerToAlertLookup to decide if there should be an alert or not.
Number of Azure stream analytics jobs required would be a factor of number of outputs and manageability preferences.
I can attempt to write a sample query if you add a hypothetical but more concrete scenario.