Sitecore 9.1 SXA 1.8
When trying to access the XML sitemap on our website, a 404 page is returned about 25% of the time. It works 75% of the time. We have a multisite configuration, with an XML sitemap for each site. In the 'Settings' item we have 'Sitemap Mode' set to 'Stored in cache'. We shouldn't need to set the 'Target Hostname' property on the 'Site Grouping' site, as it only prefills the hostname for a URL if it's unknown (and you get the /sitecore/content/etc... paths).
Our logs show the following error:
ERROR Application error. Exception: System.Web.HttpException Message: File does not exist. Source: System.Web at System.Web.StaticFileHandler.GetFileInfo(String virtualPathWithPathInfo, String physicalPath, HttpResponse response)
at System.Web.StaticFileHandler.ProcessRequestInternal(HttpContext context, String overrideVirtualPath) at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)17348 14:58:22 WARN Could not find proper static error page for site: SITENAME. Please generate it. 17348 14:58:22 INFO SitemapHandler (sitemap.xml) : cannot resolve site or url (https://www.xyz.com/error-404?404;https://APPSERVICENAME__9650:80/VIRTUALFOLDERNAMEOFOURSITE/sitemap.xml) 17348 14:58:22 WARN WebUtil.QueryStringCache cache is cleared by Sitecore.Caching.Generics.Cache`1+DefaultScavengeStrategy[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] strategy. Cache running size was 17.5 KB.
We can see where the error is coming from:
Sitecore.XA.Feature.SiteMetadata.Pipelines.HttpRequestBegin SitemapHandler process:
public override void Process(HttpRequestArgs args)
{
Uri url = HttpContext.Current.Request.Url;
if (!url.PathAndQuery.EndsWith("/sitemap.xml", StringComparison.OrdinalIgnoreCase))
return;
if (this.CurrentSite == null || !UrlUtils.IsUrlValidForFile(url, this.CurrentSite, "/sitemap.xml"))
{
Log.Info("SitemapHandler (sitemap.xml) : " + string.Format("cannot resolve site or url ({0})", (object) url), (object) this);
}
else
{
Item settingsItem = this.GetSettingsItem();
SitemapStatus sitemapStatus = settingsItem != null ? settingsItem.Fields[Sitecore.XA.Feature.SiteMetadata.Templates.Sitemap._SitemapSettings.Fields.SitemapMode].ToEnum<SitemapStatus>() : SitemapStatus.Inactive;
string sitemap;
switch (sitemapStatus)
{
case SitemapStatus.Inactive:
Log.Info("SitemapHandler (sitemap.xml) : " + string.Format("sitemap is off (status : {0})", (object) sitemapStatus), (object) this);
return;
case SitemapStatus.StoredInCache:
sitemap = this.GetSitemapFromCache();
...
So it fails before it tries to retrieve the sitemap from the cache. By the time the Log.Info is hit in the above code, the URL is also a 404 URL (see logs above).
I can see the below from using a dll search (using Agent Ransack) on Sitecore.Kernel:
...Cannot resolve the IP address of the web server with the specified hostname: {0}.
rawUrl+{0}_sitecore_username{0}_{1}placeholderTypeflagNameparameterName?404;
- Could it be that the 'GetHostIPAddress' method below (where the same warning is present) has anything to do with this, given the proximity above?
In Sitecore.Kernel - Sitecore.Web - WebUtil - GetHostIPAddress, we have the following method:
public static string GetHostIPAddress()
{
HttpContextWrapper httpContext = WebUtil.GetHttpContext();
try
{
return WebUtil.GetHostIPAddress((HttpContextBase) httpContext);
}
catch (SocketException ex)
{
Log.SingleWarn(string.Format("Cannot resolve the IP address of the web server with the specified hostname: {0}.", (object) httpContext.Request.Url.Host), (object) typeof (WebUtil));
}
return string.Empty;
}
So we have two questions:
- Where does the 404 URL in the above error message get generated?
- Why is our URL rewritten to https://APPSERVICENAME__9650:80/VIRTUALFOLDERNAMEOFOURSITE/sitemap.xml from https://www.xyz.com/VIRTUALFOLDERNAMEOFOURSITE/sitemap.xml
