Long running webjobs in Azure

I was recently having some issues with a webjob in Azure. It was calling a Web API 2 endpoint (also running on Azure). But the process took to long and server shuts down the connection before it finished. Well it actually didn’t take that much time; approximately 4 minutes. After some digging it seemed that Azure has a “hard coded” request limit of 230 seconds that you´re not able to change. After that, the server will send a 500 internal server error back to the client.

The first thought was to move logic into the webjob. But that wasn’t a great solution at the moment. So next step was to try a background task. There´s a lot of different methods for this like Hangfire, Quartz.NET or FluentScheduler. But the one I´ve come up with was actually the less sophisticated and build in QueueBackgroundWorkItem (QBWI). It was added in .NET Framework 4.5.2 and in the release notes summarized as follows:

The HostingEnvironment.QueueBackgroundWorkItem method lets you schedule small background work items. ASP.NET tracks these items and prevents IIS from abruptly terminating the worker process until all background work items have completed. This method can’t be called outside an ASP.NET managed app domain.

The QueueBackgroundWorkItem can delay an AppDomain up to 90 seconds before shutdown. This was good enough to get the work done.

Example implementation in Web API 2:

/// <summary>
/// My public API method
/// <summary>
public async Task MyLongRunningJob(CancellationToken cancellationToken)
{
    HostingEnvironment.QueueBackgroundWorkItem(async ct => await MyLongRunningJobAsync(ct));
}

Which asynchronously calls:

/// <summary>
/// My private long running job.
/// <summary>
private async Task MyLongRunningJobAsync(CancellationToken cancellationToken)
{
    try
    {
        cancellationToken.ThrowIfCancellationRequested();

        // do your async work here ...
    }
    catch (TaskCanceledException tce)
    {
        // Caught TaskCanceledException - signaled cancellation
    }
    catch (Exception ex)
    {
        // An exception was thrown ...
    }
}

Conclusion: For small scheduled background work items this works just fine. But if you´re planning for long running jobs in Azure, do consider letting the webjob doing the actual job. But make sure you´re not moving all the code logic into the webjob itself, but into a separate layer for better separation of concerns. Another way is to use a task runner solution like Hangfire. It depends on your current situation.