Techniques and Rules to Eliminating Template fields from Indexing

Question

I am investigating my site's indexing and looking at working around the Azure Search index 1000 field limit (see here Rebuilding Sitecore Core Index on Azure search throwing exceeding index limitation exception). This is affecting our sitecore-master-index and sitecore-web-index but not core.

FYI using Sitecore 8.2u7 with SXA, SPE, WFFM.

Some obvious steps I can think of are to exclude the following templates from indexing:

1. WFFM templates
2. SPE

These alone are not enough, however.

Running a little SPE script I see 350+ templates under /sitecore/templates/System. What would be the consequences of excluding all of those? Can anyone provide a list of standard templates to remove, or is this simply a case of

exclude the things you can live without and leave the rest?

Answer 1

Try this if you are on a version earlier than SXA 1.7.1.

I maintain a solution using Sitecore 8.2.7 and SXA 1.7.0. I learned from @adamnaj that SXA 1.7.1 includes a patch to remove tons of unnecessary fields. Although I'm not using Azure, this was a good opportunity to try and speed the indexing up.

If you have a look at some configs included with SXA 1.7.1 you can get the complete list of fields to exclude.

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <contentSearch>
      <indexConfigurations>
        <defaultSolrIndexConfiguration type="Sitecore.ContentSearch.SolrProvider.SolrIndexConfiguration, Sitecore.ContentSearch.SolrProvider">
          <!-- Remove these after upgrading to 1.7.1 later -->
          <exclude hint="list:ExcludeField">
            <LearnMoreTarget_A4270003 tag="{A4270003-480F-467A-BBEA-0D1B05B28656}">{A4270003-480F-467A-BBEA-0D1B05B28656}</LearnMoreTarget_A4270003>
            <PrivacyWarningButtonText_6AF8DFD6 tag="{6AF8DFD6-F541-400A-83C0-2A90A1FE8E6F}">{6AF8DFD6-F541-400A-83C0-2A90A1FE8E6F}</PrivacyWarningButtonText_6AF8DFD6>
            <PrivacyWarningContent_F2AD4C49 tag="{F2AD4C49-ECC4-4D42-A967-2756182705AC}">{F2AD4C49-ECC4-4D42-A967-2756182705AC}</PrivacyWarningContent_F2AD4C49>
            <PrivacyWarningType_73501930 tag="{73501930-26ED-493F-ADC9-3AC77FC5ED66}">{73501930-26ED-493F-ADC9-3AC77FC5ED66}</PrivacyWarningType_73501930>
            <!-- Removed for brevity -->
          </exclude>
        </defaultSolrIndexConfiguration>
      </indexConfigurations>
    </contentSearch>
  </sitecore>
</configuration>

Answer 2

I wrote this simple SPE script to loop through all the templates in Sitecore and output them in table format, which can be useful in deciding what to index and what to exclude:

Get-Item -Path master: -Query "/sitecore/templates//*[@@templateid='{455A3E98-A627-4B40-8035-E683A0331AC7}']" | 
    Show-ListView -Property @{Label="Path"; Expression={ $_.Parent.Parent.Paths.FullPath } },
        @{Label="Template"; Expression={ $_.Parent.Parent.Name } },
        @{Label="TemplateID"; Expression={ $_.Parent.Parent.ID } },
        @{Label="FieldName"; Expression={ $_.Name } },
        @{Label="FieldID"; Expression={ $_.ID } }

Answer 3

I expanded slightly on Matthew Dresser's script to generate the XML for adding the fields to your config. For example I wanted to exclude Data Exchange Framework templates:

gi -Path master: -Query "/sitecore/templates/Data Exchange//*[@@templateid='{455A3E98-A627-4B40-8035-E683A0331AC7}']"  | foreach  {
    $name = $_.Name
    $id = $_.ID
    $shortid =  $_.ID.tostring().substring(1, 6)
    write-host "<$name$shortid>$id</$name$shortid>"
}

Output:

<Value549FB9>{549FB9D5-A47E-45B0-A71D-4D66D88F7A2D}</Value549FB9>
<ShouldNotRetry26BC0B>{26BC0B33-609C-44CF-B74D-73A29383037B}</ShouldNotRetry26BC0B>
...

You can patch into your SXA config like this:

  <configuration>
    <indexes hint="list:AddIndex">
      <index id="sitecore_sxa_master_index" role:require="Standalone or Reporting or ContentManagement or Processing">
        <configuration ref="contentSearch/indexConfigurations/defaultCloudIndexConfiguration">
          <documentOptions ref="contentSearch/indexConfigurations/defaultCloudIndexConfiguration/documentOptions">
            <exclude hint="list:AddExcludedField">
              <Value549FB9>{549FB9D5-A47E-45B0-A71D-4D66D88F7A2D}</Value549FB9>
        ...