1

I need to set noindex attribute for each generated sitemap XML. I create sitemap.xml using Sitemap Controller

  [Route("sitemap_index.xml")]
        public ActionResult SitemapIndexXml()
        {
            return this.TryInvoke<ActionResult>(() =>
            {
                var siteName = SitecoreConstants.DefaultSiteName;
                var uri = Request.Url;
                if (uri != null)
                {
                    var currentSite = SiteFacade.GetSiteNameByHostName(uri.Host);
                    if (!string.IsNullOrEmpty(currentSite))
                    {
                        siteName = currentSite;
                    }
                }

                var expirationDate = DateTime.Today.AddDays(1);
                var xml = this.MemoryCacheService.GetOrSet(
                    string.Format(SitemapCacheKey, siteName, string.Empty, "sitemap_index"),
                    expirationDate,
                    () => this.SitemapSerializationService.SerializeSitemapIndex(siteName));

                return this.Content(xml, "xml", Encoding.UTF8);
            }, "SitemapController.SitemapIndexXml");
        }

1 Answer 1

1

This can be achieved in several ways, I will describe 2 ways below:

  1. Robots.txt: If you want to exclude all the pages below a particular route, you can add an entry in the Robots.txt like below:

Disallow: /Account/*

  1. A Sitemap settings template: To have a more granular control at the page level, you should create a base template and include a setting for enable/disable in the crawling index. This template should be referenced in all the page items. In the header component, you can then include a check to add meta tag <META NAME="robots" CONTENT="noindex,nofollow"> based on the setting opted for the page.
2
  • What about HttpContext.Response.AppendHeader("X-Robots-Tag", "noindex"); For 1) didn't Google Webmaster tool disabled options via Robots.txt in 2019. Commented May 16, 2022 at 10:22
  • yes you may use that as well, as I said there are several ways to achieve what you want. Commented May 16, 2022 at 11:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.