OnPublishEndAsync not triggering on CD Server

Question

I am setting up an existing site to use separate authoring and delivery servers. We have configured the onPublishEndAsync event to use the web database, and check for threshold is set to true. When I publish the site, and look at the logs: My authoring server has rebuilt the indexes appropriately, however, the indexes on the CD server are out of date.

Is there a configuration that will force my CD environment to detect when it's source database has been published to and initiate at least an incremental build?

Sitecore 8.0.150812 (also patched with https://kb.sitecore.net/articles/992608)

The Sitecore.ContentSearch.DefaultConfigurations.config has the following relevant setting:

<onPublishEndAsync type="Sitecore.Support.ContentSearch.Maintenance.Strategies.OnPublishEndAsynchronousStrategy, Sitecore.Support.440383.449997"> 
    <param desc="database">web</param>
    <!-- Whether or not a full index rebuild should be triggered when the number of items in the EventQueue exceeds the number specified 
             in ContentSearch.FullRebuildItemCountThreshold. -->
    <CheckForThreshold>true</CheckForThreshold>
</onPublishEndAsync>

Here is a portion of one of the custom indexes:

    <index id="custom_index"
             type="Sitecore.ContentSearch.LuceneProvider.LuceneIndex, Sitecore.ContentSearch.LuceneProvider">
        <param desc="name">$(id)</param>
        <param desc="folder">$(id)</param>
        <!-- This initializes index property store. Id has to be set to the index id -->
        <param desc="propertyStore" ref="contentSearch/indexConfigurations/databasePropertyStore" param1="$(id)" />
        <configuration ref="contentSearch/indexConfigurations/eventIndexConfiguration" />
        <strategies hint="list:AddStrategy">
          <!-- NOTE: order of these is controls the execution order -->
          <strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/onPublishEndAsync" />
          <strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/intervalAsyncMaster" />
        </strategies>
        <commitPolicyExecutor type="Sitecore.ContentSearch.CommitPolicyExecutor, Sitecore.ContentSearch">
          <policies hint="list:AddCommitPolicy">
            <policy type="Sitecore.ContentSearch.TimeIntervalCommitPolicy, Sitecore.ContentSearch" />
          </policies>
        </commitPolicyExecutor>
        <locations hint="list:AddCrawler">
          <crawler type="Sitecore.ContentSearch.SitecoreItemCrawler, Sitecore.ContentSearch">
            <Database>master</Database>
            <Root>/sitecore/content/mysites/Home</Root>
          </crawler>
        </locations>
      </index>

Answer 1

The most likely issue here is the scalability settings for the instance names. Scalability settings allow your different instances to listen to the shared event queue and know which ones to respond to.

Delivery Scalability Settings

Your delivery instances should have a config patch with something like the snippet below. The instance name is not as important here, but the name of the publishing instance tells the delivery instance which events to listen for from the publisher:

<?xml version="1.0"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
   <sitecore>
      <settings>
         <!-- INSTANCE NAME
            Unique name for Sitecore instance.
            Default value: (machine name and IIS site name)
         -->
         <setting name="InstanceName">
            <patch:attribute name="value"></patch:attribute>
         </setting>
         <!-- PUBLISHING INSTANCE
            Assigns the instance name of dedicated Sitecore installation for publishing operations.
            When empty, all publishing operations are performed on the local installation of Sitecore.
            Default vaue: (empty)
         -->
         <setting name="Publishing.PublishingInstance">
           <patch:attribute name="value">CM</patch:attribute>
         </setting>
      </settings>
   </sitecore>
</configuration>

Authoring Scalability Settings

The scalability settings on the authoring need to match for both publishing and instance name. This makes sure the right instance name goes into the logs, and also lets the authoring system know that it is the publisher. Your authoring instance should have a config patch with something like this:

<?xml version="1.0"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
   <sitecore>
      <settings>
         <!-- INSTANCE NAME
            Unique name for Sitecore instance.
            Default value: (machine name and IIS site name)
         -->
         <setting name="InstanceName">
            <patch:attribute name="value">CM</patch:attribute>
         </setting>
         <!-- PUBLISHING INSTANCE
            Assigns the instance name of dedicated Sitecore installation for publishing operations.
            When empty, all publishing operations are performed on the local installation of Sitecore.
            Default vaue: (empty)
         -->
         <setting name="Publishing.PublishingInstance">
           <patch:attribute name="value">CM</patch:attribute>
         </setting>
      </settings>
   </sitecore>
</configuration>

NOTE: Based on details from https://theagilecoder.wordpress.com/2013/09/30/sitecore-publishing-stuck-on-initializing-after-applying-scalability-settings/

Answer 2

Events are delivered from one instance to another as records in EventQueue data table. By default Sitecore uses Core DB for storing these events, but that can be changed in config.

<eventing defaultProvider="sitecore">
  <providers>
    <clear/>
    <add name="sitecore" type="Sitecore.Eventing.EventProvider, Sitecore.Kernel" systemDatabaseName="core"/>
  </providers>
</eventing>

Check that both these instances use shared Core DB. If that is not possible for any reason, set up SQL replication for Core database from CM to CD to synchronize events.

You can find more information regarding SQL replication in Sitecore documentation - SQL Server Replication Guide

Answer 3

Other answers here very relevant, but also worth mentioning that all servers, database included, need their time synchronized.

Answer 4

There are different things that need to be verified. I have several blog post concerning this. Below is a brief overview of the several steps that you need to verified:

1. Scalability Settings Supposed you have different CM and CD servers. You will need to define one of your CM server as a Job Server. You need to set the scalability settings as shown below:

CM Server

<setting name="InstanceName">
  <patch:attribute name="value"></patch:attribute>
</setting>

<setting name="Publishing.PublishingInstance">
  <patch:attribute name="value">CM1</patch:attribute>
</setting>

On the Job Server

<setting name="InstanceName">
  <patch:attribute name="value">CM1</patch:attribute>
</setting>

<setting name="Publishing.PublishingInstance">
  <patch:attribute name="value"></patch:attribute>
</setting>

More info at Scalability Setting. Note that the instance name in other CM Scalability setting needs to be blank because sitecore will replace it with the machine name when inserting the record in the Database.

2. Event Queue You need to check if the event queue is activated. This allows for different servers to execute their events when there is a new entry in the event queue table.

<eventQueue>
  <!-- Time between checking the queue for newly queued events. If new events are found they will be raised. -->
  <processingInterval>00:00:02</processingInterval>
</eventQueue>

More info at Event Queue

3. Server Synchronization Based on the number of items that have been published, your server needs to get in sync with the job server before being able to trigger the events. You can follow this link to know how to check if all your servers are in sync.

EDIT 1

Change the following

configuration ref="contentSearch/indexConfigurations/eventIndexConfiguration" 

to

configuration ref="contentSearch/indexConfigurations/defaultLuceneIndexConfiguration"

Change the index type from LuceneIndex to SwitchOnRebuildLuceneIndex

Answer 5

While a lot of the suggestions were helpful, my issue ended up being related to migrating to different servers. Prior to inheriting this website, it appears that the database was consistently backed up and migrated across servers; from local, to staging, to production, and now a new production. The properties table in the core database was storing all instances (CD and CM server names). I ended up deleting all the records in that table. Once that happened, indexing improved. After that I was seeing a few intermittent issues potentially being caused by index locking - so I implemented the Switch on Rebuild strategy. Since making those two tweaks I am no longer experiencing issues.