How/Why would the scheduler website be getting redirected to the 404 page?

Question

Looking at the log file on our CM server, I noticed that there are a lot of warnings getting logged as follows:

6128 00:01:09 WARN Request is redirected to document not found page. Requested url: /Error/404?item=%2f&user=sitecore%5cAnonymous&site=scheduler, User: sitecore\Anonymous, Website: scheduler

Several of these log entries show up every minute.

We do have several scheduled tasks running but nothing that is set to trigger off that often.

I have verified the the scheduling site is listed below the main website as per this How/Why would the scheduler website log attempts to access it's home page? question.

ShowConfig.aspx sites section:

    <sites>
<site name="shell" virtualFolder="/sitecore/shell" physicalFolder="/sitecore/shell" rootPath="/sitecore/content" startItem="/home" language="en" database="core" domain="sitecore" loginPage="/sitecore/login" content="master" contentStartItem="/Home" enableWorkflow="true" enableTracking="false" analyticsDefinitions="content" xmlControlPage="/sitecore/shell/default.aspx" browserTitle="Sitecore" htmlCacheSize="10MB" registryCacheSize="15MB" viewStateCacheSize="1MB" xslCacheSize="25MB" disableBrowserCaching="true" contentLanguage="en" enableItemLanguageFallback="false" enableFieldLanguageFallback="false" itemwebapi.mode="StandardSecurity" itemwebapi.access="ReadWrite" itemwebapi.allowanonymousaccess="false"/>
<site name="login" virtualFolder="/sitecore/login" physicalFolder="/sitecore/login" enableTracking="false" database="core" domain="sitecore" disableXmlControls="true" language="en"/>
<site name="admin" virtualFolder="/sitecore/admin" physicalFolder="/sitecore/admin" enableTracking="false" enableWorkflow="true" domain="sitecore" loginPage="/sitecore/admin/login.aspx"/>
<site name="service" enableTracking="true" virtualFolder="/sitecore/service" physicalFolder="/sitecore/service"/>
<site name="modules_shell" virtualFolder="/sitecore modules/shell" physicalFolder="/sitecore modules/shell" rootPath="/sitecore/content" startItem="/home" language="en" database="core" domain="sitecore" content="master" enableTracking="false" enableWorkflow="true"/>
<site name="modules_website" virtualFolder="/sitecore modules/web" physicalFolder="/sitecore modules/web" rootPath="/sitecore/content" startItem="/home" language="en" database="web" domain="extranet" allowDebug="true" cacheHtml="true"/>
<!--  PROD CM  -->
<site name="clientname" hostName="wcm.clientname.com.au" targetHostName="www.clientname.com.au" database="master" virtualFolder="/" physicalFolder="/" rootPath="/sitecore/content/clientname" startItem="/Home" dictionaryPath="/sitecore/content/clientname/Content Modules/Dictionary" dictionaryAutoCreate="true" domain="clientname" allowDebug="true" cacheHtml="false" htmlCacheSize="50MB" registryCacheSize="0" viewStateCacheSize="0" xslCacheSize="5MB" filteredItemsCacheSize="10MB" enablePreview="true" enableWebEdit="true" enableDebugger="true" disableClientData="false" cacheRenderingParameters="true" renderingParametersCacheSize="10MB" formsRoot="/sitecore/content/clientname/Content Modules/Forms/WebForms" loginPage="/login" enableItemLanguageFallback="true" patch:source="clientname.Website.config"/>

<site name="scheduler" enableTracking="false" domain="sitecore"/>
<site name="system" enableTracking="false" domain="sitecore"/>
<site name="publisher" domain="sitecore" enableTracking="false" enableWorkflow="true"/>
</sites>

Answer 1

The removal of the website site, while perfectly fine to remove, also removed the catch all site for all requests that don't explicitly match the hostname configured on your client site config.

This is generally needed if you have a default binding setup in your IIS site. Default bindings allow any request, regardless of the hostname, to be allowed to proceed to the Sitecore site.

So, without seeing any other part of your setup, and seeing your log file get hit, I will assume that IIS has either a default binding setup, or other hostnames not represented here.

For example, with a default binding, maybe a server health monitor is hitting the site by IP address or server hostname. Sitecore can't resolve the site, so then hits the scheduler site because that is the least restrictive site definition.

To solve for this, choose one of these options:

1. Use a wildcard in your client site hostname to capture more or equate to the bindings available in IIS.
2. Create another site definition after your client site (you could call it website) that doesn't have a hostname configured so that any other requests will still work.
3. Turn off all other bindings in IIS except the one your need, including the default binding, if you do not wish to use it.

I generally lean towards option 2. With this option, you can have that site point to a start item that has a redirect layout on it that will 301 redirect the caller to the primary website. This also provides you an oppotunity to log the request made and figure out who is calling and what url.