Sitecore 9.1.1 Azure Search custom index not updating when running rebuild

Question

I'm trying to port an application to Azure Search from Solr. I've got the sitecore indexes working and rebuilding. The documents are being added to the azure search index. However for any custom index I am setting up its not working. The index appears in the index manager and I can trigger and index rebuild. The rebuild process runs but no units are processed. There are no obvious log errors in the main log file or the crawler log. This is my example patch file config

<?xml version="1.0" encoding="UTF-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/" xmlns:search="http://www.sitecore.net/xmlconfig/search/">
   <sitecore role:require="Standalone or ContentManagement" search:require="Azure">
      <contentSearch>
         <indexConfigurations>
            <projectIndexConfiguration type="Sitecore.ContentSearch.Azure.CloudIndexConfiguration, Sitecore.ContentSearch.Azure">
               <initializeOnAdd>true</initializeOnAdd>
               <fieldMap type="Sitecore.ContentSearch.Azure.FieldMaps.CloudFieldMap, Sitecore.ContentSearch.Azure">
                  <fieldNames hint="raw:AddFieldByFieldName">
                     <field fieldName="name" type="System.String" cloudFieldName="name" searchable="YES" retrievable="YES" facetable="YES" filterable="YES" sortable="YES" boost="1f" settingType="Sitecore.ContentSearch.Azure.CloudSearchFieldConfiguration, Sitecore.ContentSearch.Azure" />
                  </fieldNames>
               </fieldMap>
               <documentOptions ref="contentSearch/indexConfigurations/defaultCloudIndexConfiguration/documentOptions">
                  <indexAllFields>false</indexAllFields>
                  <include hint="list:AddIncludedTemplate">
                     <Project>{AD72B630-C956-4C9E-8A88-623AF5931C0E}</Project>
                  </include>
                  <include hint="list:AddIncludedField">
                     <!--Article Content-->
                     <name>{E380368B-9ACB-4083-B1BE-36172FAD9A23}</name>
                  </include>
               </documentOptions>
            </projectIndexConfiguration>
          </indexConfigurations>
          <configuration type="Sitecore.ContentSearch.ContentSearchConfiguration, Sitecore.ContentSearch">
            <indexes hint="list:AddIndex">
               <index id="project_master_index" type="Sitecore.ContentSearch.Azure.CloudSearchProviderIndex, Sitecore.ContentSearch.Azure">
                  <param desc="name">$(id)</param>
                  <param desc="connectionStringName">cloud.search</param>
                  <param desc="totalParallelServices">5</param>
                  <param desc="propertyStore" ref="contentSearch/indexConfigurations/databasePropertyStore" param1="$(id)" />
                  <configuration ref="contentSearch/indexConfigurations/projectIndexConfiguration" />
                  <schemaBuilder ref="contentSearch/searchServiceSchemaBuilder" />
                  <searchService type="Sitecore.ContentSearch.Azure.Http.CompositeSearchService, Sitecore.ContentSearch.Azure" />
                  <strategies hint="list:AddStrategy">
                     <strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/syncMaster" />
                  </strategies>
                  <commitPolicyExecutor type="Sitecore.ContentSearch.CommitPolicyExecutor, Sitecore.ContentSearch">
                     <policies hint="list:AddCommitPolicy">
                        <policy type="Sitecore.ContentSearch.ModificationCountCommitPolicy, Sitecore.ContentSearch">
                           <Limit>1000</Limit>
                        </policy>
                     </policies>
                  </commitPolicyExecutor>
                  <locations hint="list:AddCrawler">
                     <crawler type="Sitecore.ContentSearch.SitecoreItemCrawler, Sitecore.ContentSearch">
                        <Database>master</Database>
                        <Root>/sitecore/Content/Site/home/projects</Root>
                     </crawler>
                  </locations>
                  <enableItemLanguageFallback>false</enableItemLanguageFallback>
                  <enableFieldLanguageFallback>false</enableFieldLanguageFallback>
                </index>
            </indexes>
         </configuration>
      </contentSearch>
   </sitecore>
</configuration>

Answer 1

We faced a similar issue recently on Sitecore 9.2 version where the rebuild was successful, but the custom index had 0 or few documents on Azure Search. After opening a case with Sitecore Support, they told us that the issue was on switching between the created temporary index (to avoid any downtime) and the index itself. We manage to solve it by:

1. Remove the 'index-catalog' index from the azure search service;
2. Restart the CM instance;
3. Rebuild all ContentSearch indexes from the CM instance.

Answer 2

Try defining your index like this:

<index id="project_master_index" type="Sitecore.ContentSearch.Azure.CloudSearchProviderIndex, Sitecore.ContentSearch.Azure">
    <param desc="name">$(id)</param>
    <param desc="connectionStringName">cloud.search</param>
    <param desc="totalParallelServices">5</param>
    <param desc="propertyStore" ref="contentSearch/indexConfigurations/databasePropertyStore" param1="$(id)" />
    <configuration ref="contentSearch/indexConfigurations/defaultCloudIndexConfiguration">
        <documentOptions ref="contentSearch/indexConfigurations/defaultCloudIndexConfiguration/documentOptions">
            <indexAllFields>true</indexAllFields>
            <exclude hint="list:AddExcludedField">
                <!-- Exclude fields which you don't want to have in the index -->
            </exclude>
        </documentOptions>
    </configuration>
    <strategies hint="list:AddStrategy">
        <strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/syncMaster" />
    </strategies>
    <commitPolicyExecutor type="Sitecore.ContentSearch.CommitPolicyExecutor, Sitecore.ContentSearch">
        <policies hint="list:AddCommitPolicy">
            <policy type="Sitecore.ContentSearch.ModificationCountCommitPolicy, Sitecore.ContentSearch">
                <Limit>1000</Limit>
            </policy>
        </policies>
    </commitPolicyExecutor>
    <locations hint="list:AddCrawler">
        <crawler type="Sitecore.ContentSearch.SitecoreItemCrawler, Sitecore.ContentSearch">
            <Database>master</Database>
            <Root>/sitecore/Content/Site/home/projects</Root>
        </crawler>
    </locations>
    <enableItemLanguageFallback>false</enableItemLanguageFallback>
    <enableFieldLanguageFallback>false</enableFieldLanguageFallback>
</index>

I can confirm that the above configuration works fine for me. The <indexAllFields>true</indexAllFields> might do the trick for you. And just exclude what you don't need.

Answer 3

After defining the custom index as well as creating the corresponding index in Solr, make sure to Populate Managed Schema from Sitecore Control Panel. Try rebuilding the index after this.