Remote CD Indexing Not Working

Question

1. Republish All Items (Completed with no errors)
2. Rebuild Indexes Using Control Panel (Completed with no errors)
3. Inspect Data/indexes directory. (Date modified is from over a month ago)
4. Verified correct Data Folder.
5. Cleaned out the History, PublishQueue and EventQueue tables.

Indexing is not working on my QA environment. It is using a backup from PROD which works fine.

We have 1 CM server and 2 CD servers. The indexes are working on the 1 CM server.

Here is the relevent XML from sitecore/contentSearch/configuration/indexes:

<index id="ac_products_master_index" type="Sitecore.ContentSearch.LuceneProvider.LuceneIndex, Sitecore.ContentSearch.LuceneProvider" patch:source="xActiveCommerce.ContentSearch.config">
        <param desc="name">$(id)</param>
        <param desc="folder">$(id)</param>
        <!-- This initializes index property store. Id has to be set to the index id -->
        <param desc="propertyStore" ref="contentSearch/indexConfigurations/databasePropertyStore" param1="$(id)" />
        <configuration ref="contentSearch/indexConfigurations/defaultLuceneIndexConfiguration" />
        <strategies hint="list:AddStrategy">
          <!-- NOTE: order of these is controls the execution order -->
          <strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/syncMaster" />
          <productCategoryStrategy type="ActiveCommerce.ContentSearch.Maintenance.Strategies.ProductCategorySynchronousStrategy, ActiveCommerce.ContentSearch">
            <param desc="contentDatabase">master</param>
            <listening hint="list:AddPublishDatabase">
              <web>web</web>
            </listening>
          </productCategoryStrategy>
          <variableProductStrategy type="ActiveCommerce.ContentSearch.Maintenance.Strategies.VariableProductSynchronousStrategy, ActiveCommerce.ContentSearch">
            <param desc="contentDatabase">master</param>
          </variableProductStrategy>
        </strategies>
        <commitPolicyExecutor type="Sitecore.ContentSearch.CommitPolicyExecutor, Sitecore.ContentSearch">
          <policies hint="list:AddCommitPolicy">
            <policy type="Sitecore.ContentSearch.TimeIntervalCommitPolicy, Sitecore.ContentSearch" />
          </policies>
        </commitPolicyExecutor>
        <locations hint="list:AddCrawler">
          <crawler type="ActiveCommerce.ContentSearch.Crawlers.ProductCrawler, ActiveCommerce.ContentSearch">
            <Database>master</Database>
            <Root>/sitecore/commerce/products</Root>
          </crawler>
        </locations>
      </index>
      <index id="ac_products_web_index" patch:source="xActiveCommerce.ContentSearch.config" type="Sitecore.ContentSearch.LuceneProvider.SwitchOnRebuildLuceneIndex,Sitecore.ContentSearch.LuceneProvider">
        <param desc="name">$(id)</param>
        <param desc="folder">$(id)</param>
        <!-- This initializes index property store. Id has to be set to the index id -->
        <param desc="propertyStore" ref="contentSearch/indexConfigurations/databasePropertyStore" param1="$(id)" />
        <configuration ref="contentSearch/indexConfigurations/defaultLuceneIndexConfiguration" />
        <strategies hint="list:AddStrategy">
          <!-- NOTE: order of these is controls the execution order -->
          <strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/onPublishEndAsync" />
          <prodCatOnPublishEndAsync type="ActiveCommerce.ContentSearch.Maintenance.Strategies.ProductCategoryOnPublishEndStrategy, ActiveCommerce.ContentSearch">
            <param desc="database">web</param>
          </prodCatOnPublishEndAsync>
          <variableProdOnPublishEndAsync type="ActiveCommerce.ContentSearch.Maintenance.Strategies.VariableProductOnPublishEndStrategy, ActiveCommerce.ContentSearch">
            <param desc="database">web</param>
          </variableProdOnPublishEndAsync>
        </strategies>
        <commitPolicyExecutor type="Sitecore.ContentSearch.CommitPolicyExecutor, Sitecore.ContentSearch">
          <policies hint="list:AddCommitPolicy">
            <policy type="Sitecore.ContentSearch.TimeIntervalCommitPolicy, Sitecore.ContentSearch" />
          </policies>
        </commitPolicyExecutor>
        <locations hint="list:AddCrawler">
          <crawler type="ActiveCommerce.ContentSearch.Crawlers.ProductCrawler, ActiveCommerce.ContentSearch">
            <Database>web</Database>
            <Root>/sitecore/commerce/products</Root>
          </crawler>
        </locations>
      </index>

Here is the remoteRebuild key:

<remoteRebuild type="Sitecore.ContentSearch.Maintenance.Strategies.RemoteRebuildStrategy, Sitecore.ContentSearch" />

EDIT:

After adding XML keys for a remoteRebuild strategy. It seems like CD2 is getting indexed, but CD1 is not. The following error is showing in the CD1 logs:

9796 2016:11:17 20:03:45 FATAL [Index=ac_products_web_index] RemoteRebuildStrategy skipped. Full Rebuild was not detected.

The next thing I tried was rebuilding the index from the developer tab. And, CD1 was successfully indexed. (http://www.bv02.com/rebuilding-the-search-index-on-remote-servers-with-sitecore-7/) It seems that a full rebuild is not initiated from the control panel, but it is initiated from the developer tab.

Answer 1

You need to use

<strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/remoteRebuild" />

strategy after your

<strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/onPublishEndAsync" />

It is needed if you have separated CM and CD servers. Because you do the pulbish on CM and it does not know about these by default. You need to also add this strategy to your CD servers.

Answer 2

You question is more focused on the indexing occurring rather than the strategy used. I believe this issue is best addressed by the configuration outlined in Content delivery setup - Lucene indexes.