I am setting up an existing site to use separate authoring and delivery servers. We have configured the onPublishEndAsync event to use the web database, and check for threshold is set to true. When I publish the site, and look at the logs: My authoring server has rebuilt the indexes appropriately, however, the indexes on the CD server are out of date.

Is there a configuration that will force my CD environment to detect when it's source database has been published to and initiate at least an incremental build?

Sitecore 8.0.150812 (also patched with https://kb.sitecore.net/articles/992608)

The Sitecore.ContentSearch.DefaultConfigurations.config has the following relevant setting:

<onPublishEndAsync type="Sitecore.Support.ContentSearch.Maintenance.Strategies.OnPublishEndAsynchronousStrategy, Sitecore.Support.440383.449997"> 
    <param desc="database">web</param>
    <!-- Whether or not a full index rebuild should be triggered when the number of items in the EventQueue exceeds the number specified 
             in ContentSearch.FullRebuildItemCountThreshold. -->
    <CheckForThreshold>true</CheckForThreshold>
</onPublishEndAsync>

Here is a portion of one of the custom indexes:

    <index id="custom_index"
             type="Sitecore.ContentSearch.LuceneProvider.LuceneIndex, Sitecore.ContentSearch.LuceneProvider">
        <param desc="name">$(id)</param>
        <param desc="folder">$(id)</param>
        <!-- This initializes index property store. Id has to be set to the index id -->
        <param desc="propertyStore" ref="contentSearch/indexConfigurations/databasePropertyStore" param1="$(id)" />
        <configuration ref="contentSearch/indexConfigurations/eventIndexConfiguration" />
        <strategies hint="list:AddStrategy">
          <!-- NOTE: order of these is controls the execution order -->
          <strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/onPublishEndAsync" />
          <strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/intervalAsyncMaster" />
        </strategies>
        <commitPolicyExecutor type="Sitecore.ContentSearch.CommitPolicyExecutor, Sitecore.ContentSearch">
          <policies hint="list:AddCommitPolicy">
            <policy type="Sitecore.ContentSearch.TimeIntervalCommitPolicy, Sitecore.ContentSearch" />
          </policies>
        </commitPolicyExecutor>
        <locations hint="list:AddCrawler">
          <crawler type="Sitecore.ContentSearch.SitecoreItemCrawler, Sitecore.ContentSearch">
            <Database>master</Database>
            <Root>/sitecore/content/mysites/Home</Root>
          </crawler>
        </locations>
      </index>
  • Can you post the relevant config snippets please, on how you've set this up. Makes it easier to spot where the problem may be. – Mark Cassidy Oct 11 '16 at 1:02

The most likely issue here is the scalability settings for the instance names. Scalability settings allow your different instances to listen to the shared event queue and know which ones to respond to.

Delivery Scalability Settings

Your delivery instances should have a config patch with something like the snippet below. The instance name is not as important here, but the name of the publishing instance tells the delivery instance which events to listen for from the publisher:

<?xml version="1.0"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
   <sitecore>
      <settings>
         <!-- INSTANCE NAME
            Unique name for Sitecore instance.
            Default value: (machine name and IIS site name)
         -->
         <setting name="InstanceName">
            <patch:attribute name="value"></patch:attribute>
         </setting>
         <!-- PUBLISHING INSTANCE
            Assigns the instance name of dedicated Sitecore installation for publishing operations.
            When empty, all publishing operations are performed on the local installation of Sitecore.
            Default vaue: (empty)
         -->
         <setting name="Publishing.PublishingInstance">
           <patch:attribute name="value">CM</patch:attribute>
         </setting>
      </settings>
   </sitecore>
</configuration>

Authoring Scalability Settings

The scalability settings on the authoring need to match for both publishing and instance name. This makes sure the right instance name goes into the logs, and also lets the authoring system know that it is the publisher. Your authoring instance should have a config patch with something like this:

<?xml version="1.0"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
   <sitecore>
      <settings>
         <!-- INSTANCE NAME
            Unique name for Sitecore instance.
            Default value: (machine name and IIS site name)
         -->
         <setting name="InstanceName">
            <patch:attribute name="value">CM</patch:attribute>
         </setting>
         <!-- PUBLISHING INSTANCE
            Assigns the instance name of dedicated Sitecore installation for publishing operations.
            When empty, all publishing operations are performed on the local installation of Sitecore.
            Default vaue: (empty)
         -->
         <setting name="Publishing.PublishingInstance">
           <patch:attribute name="value">CM</patch:attribute>
         </setting>
      </settings>
   </sitecore>
</configuration>

NOTE: Based on details from https://theagilecoder.wordpress.com/2013/09/30/sitecore-publishing-stuck-on-initializing-after-applying-scalability-settings/

  • Thanks Jay. I added the patch files, and my show.config display: Authoring Config: <setting name="InstanceName" value=""/> <setting name="Publishing.PublishingInstance" value="CM"/> CD: <setting name="InstanceName" value="CM" patch:source="SwitchMasterToWeb.config"/> <setting name="Publishing.PublishingInstance" value="CM"/> – MikeD Oct 11 '16 at 2:39
  • Do the values of InstanceName and PublishingInstance have to correlate to the IIS host name or sites node in any way? – MikeD Oct 11 '16 at 2:47
  • I noticed in the log files, when the instance starts it logs the instance name as Server name + IIS container. So I am using that as the values. Still no dice, but I'll try rebuilding the index and full republish. – MikeD Oct 11 '16 at 3:33
  • @MikeD the instance and publish instance name in the scalability setting can be different from the IIS. If you check in the event queue table you will see the entry CM. Since you will have many other servers, sitecore will automatically insert the instance name. The instance name acts as a reference in order for other servers to know the event has been triggered – Hishaam Namooya Oct 11 '16 at 5:46
  • Some useful links for my comment above: hishaamn.wordpress.com/2016/02/24/sitecore-event-queues and hishaamn.wordpress.com/2016/04/05/… – Hishaam Namooya Oct 11 '16 at 5:48

Events are delivered from one instance to another as records in EventQueue data table. By default Sitecore uses Core DB for storing these events, but that can be changed in config.

<eventing defaultProvider="sitecore">
  <providers>
    <clear/>
    <add name="sitecore" type="Sitecore.Eventing.EventProvider, Sitecore.Kernel" systemDatabaseName="core"/>
  </providers>
</eventing>

Check that both these instances use shared Core DB. If that is not possible for any reason, set up SQL replication for Core database from CM to CD to synchronize events.

You can find more information regarding SQL replication in Sitecore documentation - SQL Server Replication Guide

  • Thanks Dmitry - Both instances do share the same core DB – MikeD Oct 11 '16 at 3:35

Other answers here very relevant, but also worth mentioning that all servers, database included, need their time synchronized.

  • Thanks @techphoria. I have confirmed, all three participating servers (CM, CD, DB) have the same time. I'm not sure aside from remoting in and looking and the clock, how synchronized you recommend - but they appear to be the same. – MikeD Oct 11 '16 at 14:11

There are different things that need to be verified. I have several blog post concerning this. Below is a brief overview of the several steps that you need to verified:

1. Scalability Settings Supposed you have different CM and CD servers. You will need to define one of your CM server as a Job Server. You need to set the scalability settings as shown below:

CM Server

<setting name="InstanceName">
  <patch:attribute name="value"></patch:attribute>
</setting>

<setting name="Publishing.PublishingInstance">
  <patch:attribute name="value">CM1</patch:attribute>
</setting>

On the Job Server

<setting name="InstanceName">
  <patch:attribute name="value">CM1</patch:attribute>
</setting>

<setting name="Publishing.PublishingInstance">
  <patch:attribute name="value"></patch:attribute>
</setting>

More info at Scalability Setting. Note that the instance name in other CM Scalability setting needs to be blank because sitecore will replace it with the machine name when inserting the record in the Database.

2. Event Queue You need to check if the event queue is activated. This allows for different servers to execute their events when there is a new entry in the event queue table.

<eventQueue>
  <!-- Time between checking the queue for newly queued events. If new events are found they will be raised. -->
  <processingInterval>00:00:02</processingInterval>
</eventQueue>

More info at Event Queue

3. Server Synchronization Based on the number of items that have been published, your server needs to get in sync with the job server before being able to trigger the events. You can follow this link to know how to check if all your servers are in sync.

EDIT 1

Change the following

configuration ref="contentSearch/indexConfigurations/eventIndexConfiguration" 

to

configuration ref="contentSearch/indexConfigurations/defaultLuceneIndexConfiguration"

Change the index type from LuceneIndex to SwitchOnRebuildLuceneIndex

  • Good stuff Hishaam. I have one CM and one CD. Running your EQSTamp query shows me my CM has EQStamp_sc8cm1=751578 and my CD has EQStamp_sc8cd1=751578. Both the CD and CM have 00:00:02 for processingInterval value. My CM has 'sc8cm1' set for both instanceName and Publishing Instance. My CD has 'sc8cd1' for instanceName and 'sc8cm1' for PublishingInstance. – MikeD Oct 11 '16 at 14:07
  • Yes. This means that your server is in sync and the rebuild index should be triggered on your CD server. – Hishaam Namooya Oct 11 '16 at 14:11
  • Well shucks, those indexes just want to be stubborn. I made a content change, republished and verified the servers remained in sync. The indexes on the CD server have remained unchanged since last night when I manually re-indexed directly on the CD box. Hopefully its not a easter egg left by the previous dev team. – MikeD Oct 11 '16 at 14:34
  • What index strategy are you using? – Hishaam Namooya Oct 11 '16 at 14:35
  • onPublishEndAsync. I noticed in my log files on the CD server, when I publish, it does trigger a cache clear. Periodically, I also see: Starting update of index for the database 'comment' (1 pending). This is interesting, I would think 'web' should be the database. I think the comment database is in place for analytics (I'm still absorbing what I've inherited) – MikeD Oct 11 '16 at 14:50
up vote 2 down vote accepted

While a lot of the suggestions were helpful, my issue ended up being related to migrating to different servers. Prior to inheriting this website, it appears that the database was consistently backed up and migrated across servers; from local, to staging, to production, and now a new production. The properties table in the core database was storing all instances (CD and CM server names). I ended up deleting all the records in that table. Once that happened, indexing improved. After that I was seeing a few intermittent issues potentially being caused by index locking - so I implemented the Switch on Rebuild strategy. Since making those two tweaks I am no longer experiencing issues.

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.