Search code examples
azureazure-cosmosdbazure-eventhubazure-stream-analytics

Using Stream Analytics to insert data into CosmosDB

I have problems that my CosmosDB is busy and therefore i am losing data since the Stream Analyzer isn't buffering data.

My current Azure setup is as follows:

WebAPI -> EventHub -> Stream Analytics job -> CosmosDB

My issue is that if i start my stream back in time and it then has alot of records, the CosmosDB is busy and returns HTTP 429 (Exceeded limit), this will then cause me to have dataloss.

Is there a way to throttle the Stream Analytics job or should i make a completely different design to avoid having dataloss?

Solution

  • According to the following MSDN ASA does in fact retry on 429, i just expected another behavior. https://blogs.msdn.microsoft.com/streamanalytics/2016/04/18/troubleshooting-azure-stream-analytics-jobs-on-new-portal-2/