What causes azure-hosted hangfire jobs to behave like this?

Question

I have been having trouble for a while with two recurring jobs (the ones at the top of this list) that don't get run even though they are scheduled.

I can trigger them just fine, and they get rescheduled, but when the schedule time comes around they don't run, and the "Next Execution" time just slips into the past.

Now there are a bunch of other jobs that are having the same problem. These are supposed to run hourly, but if they get past the schedule time they just don't run.

Visiting the dashboard makes no difference. The web app is always on. Hangfire will never run these jobs unless I trigger them manually. Jobs that AREN'T in this state still run just fine as scheduled every day or every hour.

What would cause this?

My hangfire instance (version 1.7.6) is in an Azure WebApp that is set to be always running. It uses an Azure-sql database for its data store.

Here's my Bootstrapper.cs code:

using System.Configuration;
using System.Web.Hosting;
using Hangfire;

namespace MyApi
{
    public class HangfireBootstrapper : IRegisteredObject
    {
        public static readonly HangfireBootstrapper Instance = new HangfireBootstrapper();

        private readonly object _lockObject = new object();
        private bool _started;

        private BackgroundJobServer _backgroundJobServer;

        private HangfireBootstrapper()
        {
        }

        public void Start()
        {
            lock (_lockObject)
            {
                if (_started) return;
                _started = true;

                HostingEnvironment.RegisterObject(this);
                var jobOptions = new BackgroundJobServerOptions();
                jobOptions.ServerName = ConfigurationManager.AppSettings.Get("hangfire:servername");
                jobOptions.Queues = new[] {"k1"};

                GlobalConfiguration.Configuration
                    .UseSqlServerStorage("Kdb");

                _backgroundJobServer = new BackgroundJobServer(jobOptions);
            }
        }

        public void Stop()
        {
            lock (_lockObject)
            {
                if (_backgroundJobServer != null)
                {
                    _backgroundJobServer.Dispose();
                }

                HostingEnvironment.UnregisterObject(this);
            }
        }

        void IRegisteredObject.Stop(bool immediate)
        {
            Stop();
        }
    }
}

Here's code that is used to queue the majority of the jobs:

jobMgr.AddOrUpdate($"Script.{i1}.{name}", Job.FromExpression(() => HangfireJobs.ReplayQueue.EnqueueScript(scriptId, i1, null)),cronExpression);

Here's how my EnqueueScript method is defined:

    [Queue("k1")]
    public static void EnqueueScript(Guid scriptId, int env, PerformContext context)
    {
        try
        {
           ...

Answer 1

This issue was resolved by upgrading Hangfire to version 1.7.8.

The Hangfire bug report is viewable at https://github.com/HangfireIO/Hangfire/issues/1459

It appears to have been introduced around version 1.7.4 (maybe earlier, but certainly by then) and fixed with version 1.7.8.