Sitecore Jobs and IIS Pool Recycles

Question

From what I understand a Sitecore job simply runs as a background thread within the web application, i.e. a IIS worker process. How does this work with the periodic IIS pool recycling?

I mean if you have a long-running job and a recycle happens, won't the job simply be killed along with the process? What's their strategy to prevent this from happening? As is often the case with Sitecore I couldn't find any official documentation that would explain how they deal with this.

Answer 1

Yes, this is correct, Sitecore Jobs and Scheduled Tasks run as background threads, and any App Pool restart or unexpected shutdown of the server will cause the job to be killed. If you have a long-running job then you must defensively code for this possible outcome, and allow the code to both gracefully fail and pick up where it left off the next time the job is run, as well as any code consuming the output of the task to defensively code around possible missing data.

Remember, the app pool may restart due to a number of reasons, such as a deployment or periodic recycling due to IIS settings, or due to other unexpected reasons just as server errors, server restarts or operating system updates.

It may be be possible to wrap your code in a ShutdownGuard() block, but it will not get around all the possible reasons a Sitecore Job and background thread is ended unexpectedly. You could also use a Service Bus to act as a queue to allow you to pick up the processing where you left off. This would be the same as well even for standard non-Sitecore applications.

Answer 2

Sitecore included its own keep-alive mechanism because IIS does recycle apps by default, and Sitecore wanted to override that behavior out of the box.

Sitecore comes with two built-in features which in tandem prevent application pool timeouts: the KeepAlive page and the UrlAgent.

The UrlAgent (Sitecore.Tasks.UrlAgent) is a simple Sitecore agent which accesses a predefined URL at regular intervals.

The keep alive page is basically an empty ASP.NET form which is located in “[webroot]/sitecore/service/keepalive.aspx”.

Out of the box Sitecores Web.config only contains a “naive” UrlAgent configuration: it assumes an application pool timeout of more than one hour and a website which is mapped to localhost, as shown below.

<agent type="Sitecore.Tasks.UrlAgent" method="Run" interval="01:00:00">
  <param desc="url">/sitecore/service/keepalive.aspx</param>
  <LogActivity>true</LogActivity>
</agent>

To configure the UrlAgent for actual use, copy the XML shown below into a configuration file (e.g. “KeepAliveAgent.config”), change the interval attribute and URL parameter as required, and place it in “[webroot]/App_Config/Include”. Sitecore will merge these settings into the main Web.config file when your site starts up.

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <scheduling>
      <!-- The time format is "HH:MM:SS" -->
      <agent type="Sitecore.Tasks.UrlAgent" method="Run" interval="00:15:00">
        <!-- Replace [WEBSITE] with the appropriate domain -->
        <param desc="url">http://[WEBSITE]/sitecore/service/keepalive.aspx</param>
        <LogActivity>true</LogActivity>
      </agent>
    </scheduling>
  </sitecore>
</configuration>

When configuring a UrlAgent to prevent application pool timeouts keep the following in mind:

The “interval” attribute value must be less than the application pool timeout.

Practically the lower limit of the interval is equal to the interval set in the node “/configuration/sitecore/scheduling/frequency” in Web.config (expressed as XPath), which is 5 minutes by default.

When not set to an absolute URL, the UrlAgent assumes that the specified partial URL is located on localhost. In many scenarios this is not the case (e.g. multiple sites per server), so it’s usually better to provide a publicly reachable absolute URL.

The “keep alive page” can be any page on the website, but the default page is well suited since it doesn’t contain any functionality or visitor tracking scripts.

Reference

What is the purpose of keepalive.aspx?

https://reasoncodeexample.com/2012/03/19/keeping-sitecore-alive/