Unwanted Sitecore URL

Question

I am able to access a page using the path:

https://<<My Domain>>/Sitecore/content/page_path

This has been crawled by Google, I don't want these URL's to be accessible.

I would like to know what is the correct way to handle these? Should this be resolved and URL be changed? or should we be blocking the requests for anything containing Sitecore/Content in request URL?

My Site definition is:

database="web" domain="extranet" disableClientData="false" cacheHtml="true" htmlCacheSize="250MB" cacheRenderingParameters="true" renderingParametersCacheSize="100MB" allowDebug="true" enablePreview="true" enableWebEdit="true" enableDebugger="true" enableAnalytics="true" notFoundItem="/sitecore/content/ZZZ/ZZZX/Home/Error"/>

Thanks,

Samridhi

Answer 1

You need to block them. Ideally, Google should never have been told about these URLs in the first place, it looks like you might have some mismanaged Site Url Generation going on elsewhere.

But for purposes of getting these links OFF Google again, you need to block them. Easiest way to do it, is to use a Rewrite rule. Something like:

<rule name="sitecore predixes" stopProcessing="true">
    <match url="^sitecore/?(.*)" />
    <action type="Redirect" url="http://www.hostname.com/{R:1}" />
</rule>

(you need to create a proper match url, to filter out /sitecore/content)

Then enable this rewrite rule on IIS and tell Google to reindex.

Secondly; you need to ensure that all the URLs you generate for your site are correct. Here is a good resource to help you get started: Safely Defining Multiple Sites in Sitecore

Answer 2

Following the best SEO practises would be the best.

1. Implement canonical URL on all pages
2. Consider using Robots.txt
3. Make sure your XML Sitemap is correct
4. Maybe using a 301 Redirect to strip out the /sitecore/Content/ part of the URL