Two CD instances, one is missing content, the second is working as expected

Question

When we deploy our site to our staging slot using the Sitecore Azure module we're running into odd behavior. The second web role instance, SitecoreWebRole_IN_1 is displaying all of the expected content, but the first web role, SitecoreWebRole_IN_0, is missing some content.

We've tried:

• Clearing the sitecore cache
• Forcing an Azure Warmup
• Republishing the missing items in question
• Verified that the Azure.ServiceRuntime.config file is using:
  <agent type="Sitecore.Azure.Managers.Publishing.PublishTargetRemoteManager, Sitecore.Azure" method="Synchronize" interval="00:05:00" />.
• Re-imaging the instance
• Scaling to only one instance
• Pointing the instance to different databases with the same content and doing a full republish
• Completely deleting the slot and redeploying to a new staging slot
  

The staging slot is sharing databases with the production slot. Publishing is happening from a PaaS Content Editor deployment. Deployment is from an IaaS VM using the Stiecore Azure Module. Sitecore is running v8.1 rev 151207. The Azure module is v8.1.5344.

Answer 1

In most cases where there are issues with publishing to content delivery servers, caches not clearing, or events not firing, it is generally due to Instance Name issues.

The Instance Name

In the ScalabilitySettings.config, when enabled for a distributed environemnt, there is a Sitecore Setting called InstanceName.

  <sitecore>
    <settings>
      <!--  INSTANCE NAME
            Unique name for Sitecore instance.
            Default value: (machine name and IIS site name)
      -->
      <setting name="InstanceName">
        <patch:attribute name="value"></patch:attribute>
      </setting>
    </settings>
  </sitecore>

The default value, uses the Machine Name and IIS Site Name. However in rare cases where servers have the same name, this can result in issues.

In the Core database, the EventQueue table uses the instancename of the Sitecore server to determine whether or not it has acted upon events that have been registered. If two servers, inadvertantly have the same InstanceName then when the first server acts on any events in the table, it will set a bit saying that it has acted on it. When the second server checks the table, using it's InstanceName as a filter, if it sees the event has been dealt with, it does nothing.

Typically, ensuring that the instance name is unique for each Sitecore server will prevent any issues with publishing or events not clearing.