I am trying to upgrade our app to listen for blob uploads via EventGrid instead of using the old blob polling process in Azure functions. However when I run my Azure function to write to the text to the blob it emits the same message multiple times (somewhere between 2 and 6 times). I would like the event to only emit once as only one blob is being created.
my code to write to the function is as follows:
[FunctionName("CreateRequests")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = "request")] HttpRequest req,
IBinder binder,
ILogger log)
{
try
{
string data = await new StreamReader(req.Body).ReadToEndAsync();
string schema = req.GetQueryParameterDictionary()["schema"];
string processType = req.GetQueryParameterDictionary()["processType"];
var arrayOfThings = JsonConvert.DeserializeObject<List<object>>(data);
int countOfContracts = contracts.Count();
foreach (var item in arrayOfThings)
{
var blobId = Guid.NewGuid();
BlobAttribute dynamicBlobAttribute = new BlobAttribute($"{GetAzureBlobFolder(processType)}/{schema}/instance-{Guid.NewGuid()}/{blobId}");
dynamicBlobAttribute.Connection = "RequestStorage";
using (var writer = binder.Bind<TextWriter>(dynamicBlobAttribute))
{
writer.Write(_contract);
}
}
return new OkResult();
}
catch (Exception ex)
{
HttpError err = new HttpError(ex, false);
return new BadRequestErrorMessageResult(err.ExceptionMessage);
}
}
Here is the related listener:
[FunctionName("DidAddReevalRequestToBlob")]
public async Task Run([BlobTrigger("reevaluationrequests/{name}", Source = BlobTriggerSource.EventGrid,Connection = "RequestStorage")] Stream myBlob,
string name, ILogger log)
{
string contents = string.Empty;
using (var reader = new StreamReader(myBlob))
{
contents = reader.ReadToEnd();
}
// await "do other stuff"
}
This issue does not occur when I upload the file through the azure portal directly but only when I write the blob in code. this leads me to believe that I am writing the blob incorrectly.
I have also unsuccessfully tried to rewrite the way I upload the file however I am sure there are things I missed there.
related logs:
[2022-11-15T16:50:01.738Z] Executing 'DidAddReevalRequestToBlob' (Reason='New blob detected(EventGrid): azure-webjobs-hosts/timers/eth000893dvale-2057002712/FIPA.Functions.AutomatedRequest.TimerProcessNextAutomatedRequests.Run/status', Id=6c1f6544-e682-4ea2-b6eb-9bdf55ff8662)
[2022-11-15T16:50:01.740Z] Trigger Details: MessageId: 578038d0-42b8-4194-9f43-b63715e63384, DequeueCount: 1, InsertedOn: 2022-11-15T16:50:01.000+00:00, BlobCreated: 2022-11-14T20:43:09.000+00:00, BlobLastModified: 2022-11-15T16:50:00.000+00:00
[2022-11-15T16:50:01.935Z] Executed 'DidAddReevalRequestToBlob' (Failed, Id=6c1f6544-e682-4ea2-b6eb-9bdf55ff8662, Duration=362ms)
[2022-11-15T16:50:02.742Z] Executing 'DidAddReevalRequestToBlob' (Reason='New blob detected(EventGrid): azure-webjobs-hosts/timers/eth000893dvale-2057002712/FIPA.Functions.AutomatedRequest.TimerProcessNextAutomatedRequests.Run/status', Id=9320d805-eb39-414b-a752-39ef13b52faa)
[2022-11-15T16:50:02.744Z] Trigger Details: MessageId: 578038d0-42b8-4194-9f43-b63715e63384, DequeueCount: 2, InsertedOn: 2022-11-15T16:50:01.000+00:00, BlobCreated: 2022-11-14T20:43:09.000+00:00, BlobLastModified: 2022-11-15T16:50:00.000+00:00
[2022-11-15T16:50:02.911Z] Executed 'DidAddReevalRequestToBlob' (Failed, Id=9320d805-eb39-414b-a752-39ef13b52faa, Duration=330ms)
[2022-11-15T16:50:03.354Z] Executing 'DidAddReevalRequestToBlob' (Reason='New blob detected(EventGrid): azure-webjobs-hosts/timers/eth000893dvale-2057002712/FIPA.Functions.AutomatedRequest.TimerProcessNextAutomatedRequests.Run/status', Id=0b3b082c-ef47-4d3a-9b29-f3a405b7369e)
[2022-11-15T16:50:03.355Z] Trigger Details: MessageId: 578038d0-42b8-4194-9f43-b63715e63384, DequeueCount: 3, InsertedOn: 2022-11-15T16:50:01.000+00:00, BlobCreated: 2022-11-14T20:43:09.000+00:00, BlobLastModified: 2022-11-15T16:50:00.000+00:00
[2022-11-15T16:50:03.529Z] Executed 'DidAddReevalRequestToBlob' (Failed, Id=0b3b082c-ef47-4d3a-9b29-f3a405b7369e, Duration=330ms)
[2022-11-15T16:50:03.966Z] Executing 'DidAddReevalRequestToBlob' (Reason='New blob detected(EventGrid): azure-webjobs-hosts/timers/eth000893dvale-2057002712/FIPA.Functions.AutomatedRequest.TimerProcessNextAutomatedRequests.Run/status', Id=347c98c0-89e9-4a5d-b895-f63448bddfa0)
[2022-11-15T16:50:03.967Z] Trigger Details: MessageId: 578038d0-42b8-4194-9f43-b63715e63384, DequeueCount: 4, InsertedOn: 2022-11-15T16:50:01.000+00:00, BlobCreated: 2022-11-14T20:43:09.000+00:00, BlobLastModified: 2022-11-15T16:50:00.000+00:00
[2022-11-15T16:50:04.132Z] Executed 'DidAddReevalRequestToBlob' (Failed, Id=347c98c0-89e9-4a5d-b895-f63448bddfa0, Duration=340ms)
[2022-11-15T16:50:04.592Z] Executing 'DidAddReevalRequestToBlob' (Reason='New blob detected(EventGrid): azure-webjobs-hosts/timers/eth000893dvale-2057002712/FIPA.Functions.AutomatedRequest.TimerProcessNextAutomatedRequests.Run/status', Id=92086bb8-8723-42c3-abe8-ef68f551b72e)
[2022-11-15T16:50:04.594Z] Trigger Details: MessageId: 578038d0-42b8-4194-9f43-b63715e63384, DequeueCount: 5, InsertedOn: 2022-11-15T16:50:01.000+00:00, BlobCreated: 2022-11-14T20:43:09.000+00:00, BlobLastModified: 2022-11-15T16:50:00.000+00:00
[2022-11-15T16:50:04.757Z] Executed 'DidAddReevalRequestToBlob' (Failed, Id=92086bb8-8723-42c3-abe8-ef68f551b72e, Duration=311ms)
[2022-11-15T16:50:04.761Z] Message has reached MaxDequeueCount of 5. Moving message to queue 'webjobs-blobtrigger-poison'.
It turned out to in fact be a problem with the way I was uploading the file. the refactored code in "CreateRequests" now looks like this:
string Connection = Environment.GetEnvironmentVariable("RequestStorage");
var json = JsonConvert.SerializeObject(_contract);
byte[] byteArray = Encoding.ASCII.GetBytes(json);
Stream myBlob = new MemoryStream(byteArray);
var blobClient = new BlobContainerClient(Connection, blobRequest.ProcessType);
var blob = blobClient.GetBlobClient($"{blobRequest.Schema}/instance-{Guid.NewGuid()}/{blobId}");
await blob.UploadAsync(myBlob);
One thing to note is that, in my situation, if an exception was thrown my BlobTrigger would immedeatly try the request again. I suspect this is a behavior of EventGrid but I don't really know. so I needed to make sure that what I was providing to the function wasn't throwing errors or it would continue to call multiple times.