Store data in a Azure WebJob

Question

I have created a gRPC client that receives data from a server. The client runs as a Azure WebJob.

The received data looks like that there is a replay id for each message. I currently store this replay id locally as a file. The received replay id is appended to the file. The Replay Id is needed so that if there is a problem it can continue from that ID.

If I run the client locally this is not a problem, but I think that as a WebJob is not ideal. My problem currently is that the files that are created/modified at runtime are stored here:

C:\local\Temp\jobs\continuous\SalesforceIngestor.App\ev4ygzfw.ezh

If the WebJob restarts for any reason, the data in the replayIdStorage.txt file will be lost.

Is there any way to safely store data in a WebJob so that it can be retrieved after a restart?

Or would Azure Blob Storage be a better option?

Answer 1

I tried below .Net Web Job code and can able to store the Web Jobs data of the web app in my storage account container.

Code:

Program.cs:

using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Configuration;
using System;
using System.IO;

namespace WebJob1
{
    internal class Program
    {
        static void Main()
        {
            var config = new JobHostConfiguration();

            if (config.IsDevelopment)
            {
                config.UseDevelopmentSettings();
            }

            var host = new JobHost(config);
            host.RunAndBlock();
        }
    }
}

Function.cs:

using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;
using System;
using System.IO;

namespace WebJob1
{
    public class Functions
    {
        private const string ConnectionStringName = "AzureWebJobsStorage"; 

        public static void ProcessQueueMessage([QueueTrigger("queue")] string message, TextWriter log)
        {
            log.WriteLine(message);
            string replayId = GenerateReplayIdFromMessage(message);
            StoreReplayIdInBlobStorage(replayId);
        }

        private static string GenerateReplayIdFromMessage(string message)
        {
            return Guid.NewGuid().ToString();
        }

        private static void StoreReplayIdInBlobStorage(string replayId)
        {
            var storageAccount = CloudStorageAccount.Parse(Environment.GetEnvironmentVariable(ConnectionStringName));
            var blobClient = storageAccount.CreateCloudBlobClient();
            var containerName = "replayids"; 
            var container = blobClient.GetContainerReference(containerName);
            container.CreateIfNotExists();

            var blobName = "replayIdStorage.txt"; 
            var blob = container.GetBlockBlobReference(blobName);
            blob.UploadText(replayId);
        }
    }
}

app.config:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <connectionStrings>
        <!-- The format of the connection string is "DefaultEndpointsProtocol=https;AccountName=NAME;AccountKey=KEY" -->
        <!-- For local execution, the value can be set either in this config file or through environment variables -->
        <add name="AzureWebJobsDashboard" connectionString="<storage_connec_string>" />
        <add name="AzureWebJobsStorage" connectionString="<storage_connec_string>" />
    </connectionStrings>
    <startup>
        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.8" />
    </startup>
    <runtime>
        <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
            <dependentAssembly>
                <assemblyIdentity name="Microsoft.WindowsAzure.Storage" publicKeyToken="31bf3856ad364e35" culture="neutral" />
                <bindingRedirect oldVersion="0.0.0.0-7.0.0.0" newVersion="7.0.0.0" />
            </dependentAssembly>
            <dependentAssembly>
                <assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" culture="neutral" />
                <bindingRedirect oldVersion="0.0.0.0-12.0.0.0" newVersion="12.0.0.0" />
            </dependentAssembly>
        </assemblyBinding>
    </runtime>
</configuration>

Output:

It runs successfully as below,

I got two containers created in my storage account at Azure portal as below,

I got Id in azure-webjobs-hosts container as below,

And got below in azure-jobs-host-output container,

Then, I published my code to web app as below,

It successfully published to web jobs in web app at Azure Portal. Then, I click on Run to start running the web job as below,

Web Job running started successfully as below,

Then, I got web app details stored im my storage account azure-jobs-host-output container as below,