Intermittent Trigger Issues with Event Grid for Blob Upload and Deletion in Azure Function

Question

I am using an Event Grid subscription to trigger an Azure Function for handling blob uploads and deletions in Azure Blob Storage. However, I'm encountering intermittent issues with the triggers not firing consistently.I mean that the trigger is not firing when it should. For example, if I upload a file and delete it at 10 am, the triggers work correctly, and the corresponding actions are executed. However, if I perform the same upload and delete operations at 11 am, the triggers do not fire, and no actions are executed.

Setup:

Storage Account (Event Grid)

Event Grid Subscription subscribed to Microsoft.Storage.BlobCreated and Microsoft.Storage.BlobDeleted events.

Azure Function Event Grid Trigger

Issue: The problem is that the Event Grid trigger is not firing every time a blob is uploaded or deleted. When the trigger does fire, both the upload and delete actions work correctly. However, the trigger itself is inconsistent.

Steps Taken:

Subscribed to both Microsoft.Storage.BlobCreated and Microsoft.Storage.BlobDeleted events.

Implemented function code to handle different event types and execute the necessary business logic.

Despite this setup, the triggers do not consistently fire for every blob upload or deletion event.

Request for Help: Could someone help me troubleshoot and resolve these intermittent issues with the Event Grid triggers for blob uploads and deletions? Any insights or guidance would be greatly appreciated.

Thank you! enter image description here enter image description here

const { app } = require('@azure/functions');

const { BlobServiceClient, StorageSharedKeyCredential } = require('@azure/storage-blob');

const { SearchClient, AzureKeyCredential } = require('@azure/search-documents');

const pdf = require('pdf-parse');

const path = require('path');

// Azure Blob Storage configuration

const accountName = "";

const accountKey = "";

const containerName = 'content';

// Azure Cognitive Search configuration

const searchServiceName = "";

const indexName = "demoindex2";

const apiKey = "";

const blobServiceClient = new BlobServiceClient(

    `https://${accountName}.blob.core.windows.net`,

    new StorageSharedKeyCredential(accountName, accountKey)

);

const containerClient = blobServiceClient.getContainerClient(containerName);

const searchClient = new SearchClient(

    `https://${searchServiceName}.search.windows.net/`,

    indexName,

    new AzureKeyCredential(apiKey)

);

async function indexBlob(blobName) {

    try {

        const blobClient = containerClient.getBlobClient(blobName);

        const downloadResponse = await blobClient.download();

        const encodedName = Buffer.from(blobName).toString('base64');

        const properties = await blobClient.getProperties();

        const pdfBuffer = await streamToBuffer(downloadResponse.readableStreamBody);

        const pdfText = await pdf(pdfBuffer);

        const blobContent = pdfText.text;

        const document = {

            id: encodedName,

            content: blobContent,

            metadata_storage_content_type: properties.contentType || null,

            metadata_storage_size: properties.contentLength || null,

            metadata_storage_last_modified: properties.lastModified ? new Date(properties.lastModified).toISOString() : null,

            metadata_storage_content_md5: properties.contentMD5 ? Buffer.from(properties.contentMD5).toString('base64') : null,

            metadata_storage_name: blobName,

            metadata_storage_path: blobClient.url,

            metadata_storage_file_extension: path.extname(blobName),

            metadata_content_type: properties.contentType || null,

            metadata_language: null,

            metadata_author: null,

            metadata_creation_date: properties.creationTime ? new Date(properties.creationTime).toISOString() : null,

        };

        await searchClient.uploadDocuments([document]);

        console.log(`Document "${document.id}" has been indexed`);

    } catch (error) {

        console.error(`Error indexing document: ${error}`);

    }

}

async function deleteDocument(blobName) {

    try {

        const encodedName = Buffer.from(blobName).toString('base64');

        await searchClient.deleteDocuments([{ id: encodedName }]);

        console.log(`Document "${encodedName}" has been deleted from the index`);

    } catch (error) {

        console.error(`Error deleting document from the index: ${error}`);

    }

}

app.eventGrid('process-event-grid', {

    handler: async (context, eventGridEvent) => {

        try {

            const event = eventGridEvent.eventType;

            const blobUrl = eventGridEvent.data?.url;

            

            if (!blobUrl) {

                console.error("Event data does not contain 'url':", eventGridEvent);

                return;

            }

            

            const blobName = blobUrl.substring(blobUrl.lastIndexOf('/') + 1);

            if (event === 'Microsoft.Storage.BlobCreated') {

                console.log(`Blob created: ${blobName}`);

                await indexBlob(blobName);

            } else if (event === 'Microsoft.Storage.BlobDeleted') {

                console.log(`Blob deleted: ${blobName}`);

                await deleteDocument(blobName);

            } else {

                console.log(`Unhandled event type: ${event}`);

            }

        } catch (error) {

            console.error(`Error processing event: ${error}`, eventGridEvent);

        }

    }

});

async function streamToBuffer(readableStream) {

    return new Promise((resolve, reject) => {

        const chunks = [];

        readableStream.on("data", (data) => {

            chunks.push(data instanceof Buffer ? data : Buffer.from(data));

        });

        readableStream.on("end", () => {

            resolve(Buffer.concat(chunks));

        });

        readableStream.on("error", reject);

    });

}

Answer 1

The problem is that the Event Grid trigger is not firing every time a blob is uploaded or deleted.

• Function goes in Idle state after some time, if no process is running or no trigger is triggered. Make sure there is no time gap in between trigger event.
• In consumption and premium plan there is no option to keep function app always on. But, In dedicated app service plan there is an option in configuration>>general settings>>Always On which can be enabled.

I am using a simple event rigger code which triggers when blob is uploaded or deleted.

const { app } = require('@azure/functions');

app.eventGrid('eventGridTrigger1', {
    handler: (event, context) => {
        context.log('Event grid function processed event:', event);
    }
});

Host.Json

{
  "version": "2.0",
  "logging": {
    "applicationInsights": {
      "samplingSettings": {
        "isEnabled": true,
        "excludedTypes": "Request"
      }
    }
  },
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[4.*, 5.0.0)"
  }
}

I have configured the function and event grid endpoint in azure storage events like below:

When i uploaded and deleted the documents in storage container function got triggered successfully. check below:

Output: