Sitecore core index is rebuilding continuously

Question

We are seeing continuous logs are logged for sitecore_core indexing with zero item processed in log file. What is the solution to fix the issue?

Logs:

ManagedPoolThread #2 04:09:29 INFO  Job started: Index_Update_IndexName=sitecore_core_index
ManagedPoolThread #2 04:09:29 INFO  Job ended: Index_Update_IndexName=sitecore_core_index (units processed: )
ManagedPoolThread #7 04:09:29 INFO  Job started: Index_Update_IndexName=sitecore_core_index
ManagedPoolThread #7 04:09:29 INFO  Job ended: Index_Update_IndexName=sitecore_core_index (units processed: )
ManagedPoolThread #10 04:09:29 INFO  Job started: Index_Update_IndexName=sitecore_core_index
ManagedPoolThread #10 04:09:29 INFO  Job ended: Index_Update_IndexName=sitecore_core_index (units processed: )
ManagedPoolThread #9 04:09:29 INFO  Job started: Index_Update_IndexName=sitecore_core_index
ManagedPoolThread #9 04:09:29 INFO  Job ended: Index_Update_IndexName=sitecore_core_index (units processed: )
ManagedPoolThread #9 04:09:29 INFO  Job started: Index_Update_IndexName=sitecore_core_index
ManagedPoolThread #9 04:09:30 INFO  Job ended: Index_Update_IndexName=sitecore_core_index (units processed: )
ManagedPoolThread #19 04:09:30 INFO  Job started: Index_Update_IndexName=sitecore_core_index
ManagedPoolThread #19 04:09:30 INFO  Job ended: Index_Update_IndexName=sitecore_core_index (units processed: )
ManagedPoolThread #4 04:09:30 INFO  Job started: Index_Update_IndexName=sitecore_core_index
ManagedPoolThread #4 04:09:30 INFO  Job ended: Index_Update_IndexName=sitecore_core_index (units processed: )
ManagedPoolThread #1 04:09:30 INFO  Job started: Index_Update_IndexName=sitecore_core_index
ManagedPoolThread #1 04:09:30 INFO  Job ended: Index_Update_IndexName=sitecore_core_index (units processed: )
ManagedPoolThread #1 04:09:30 INFO  Job started: Index_Update_IndexName=sitecore_core_index
ManagedPoolThread #1 04:09:30 INFO  Job ended: Index_Update_IndexName=sitecore_core_index (units processed: )
ManagedPoolThread #1 04:09:30 INFO  Job started: Index_Update_IndexName=sitecore_core_index
ManagedPoolThread #1 04:09:30 INFO  Job ended: Index_Update_IndexName=sitecore_core_index (units processed: )
ManagedPoolThread #6 04:09:30 INFO  Job started: Index_Update_IndexName=sitecore_core_index
ManagedPoolThread #6 04:09:30 INFO  Job ended: Index_Update_IndexName=sitecore_core_index (units processed: )
ManagedPoolThread #17 04:09:30 INFO  Job started: Index_Update_IndexName=sitecore_core_index
ManagedPoolThread #17 04:09:30 INFO  Job ended: Index_Update_IndexName=sitecore_core_index (units processed: )
ManagedPoolThread #15 04:09:30 INFO  Job started: Index_Update_IndexName=sitecore_core_index
ManagedPoolThread #15 04:09:30 INFO  Job ended: Index_Update_IndexName=sitecore_core_index (units processed: )
ManagedPoolThread #16 04:09:30 INFO  Job started: Index_Update_IndexName=sitecore_core_index
ManagedPoolThread #16 04:09:30 INFO  Job ended: Index_Update_IndexName=sitecore_core_index (units processed: )

Answer 1

There are a few steps involved in order to troubleshoot and fix this issue.

Step 1: The first step would be to check what index update strategy is used for sitecore_core_index. You can search for <index id="sitecore_core_index" on /sitecore/admin/showconfig.aspx page, <strategies node. By default, IntervalAsynchronousStrategy index update strategy is configured for it.

When this strategy is used, an index will be updated based on a time interval rather than OnPublishEnd event. The interval is configured in Sitecore.ContentSearch.DefaultConfigurations.config config file. By default, the index update will be triggered every minute.

  <intervalAsyncCore type="Sitecore.ContentSearch.Maintenance.Strategies.IntervalAsynchronousStrategy, Sitecore.ContentSearch">
    <param desc="database">core</param>
    <param desc="interval">00:01:00</param>
    <CheckForThreshold>true</CheckForThreshold>
  </intervalAsyncCore>

Step 2:

• Verify that the interval parameter is set to a value that will not cause unnecessarily frequent updates, and adjust it to fit your needs. sitecore_core_index is considered to be a less critical index that you do not need to updat frequently.
• Verify that this index update strategy is not combined with other strategies that it is not compatible with (e.g. SynchronousStrategy). For more information about this and other index update strategies, please refer to this Sitecore documentation page.

Step 3: The next step can be to change log level for Crawling Log to DEBUG (only for a short period of time while troubleshooting). The values is stored in Sitecore.ContentSearch.config config file.

  <logger name="Sitecore.Diagnostics.Crawling" additivity="false">
    <level value="DEBUG"/>
    <appender-ref ref="CrawlingLogFileAppender"/>
  </logger>

And verify Crawling.log.txt log files for more information on the issue why the index is updated so frequently. If there is an issue, it will be logged. (e.g. EventQueue, Solr connectivity, etc.)

Step 4: The strategy uses EventQueue table in core database. It is possible that EventQueue table gets flooded with lots of events. Two admin pages can be verified to see details on running, queued and finished jobs and EventQueue statistics:

• /sitecore/admin/Jobs.aspx
• /sitecore/admin/EventQueueStats.aspx

If the aforemntioned two pages will not provide enough info, you can run this SQL query on your EventQueue table in Core database for more details, it will show events for updating sitecore_core_index:

SELECT TOP (10000) [Id]
      ,[EventType]
      ,[InstanceType]
      ,[InstanceData]
      ,[InstanceName]
      ,[RaiseLocally]
      ,[RaiseGlobally]
      ,[UserName]
      ,[Stamp]
      ,[Created]
  FROM [db_Core].[dbo].[EventQueue]
  WHERE InstanceData = '{"FullRebuild":false,"IndexName":"sitecore_core_index"}'

On Jobs.aspx page, verify if Index_Update_IndexName=sitecore_core_index job is queued many times in a short period of time.

Additionally, please refer to this KnowledgeBase article, it provides similar details provided above.

Update: The reason why the log file shows that zero units were processed during index update job is because it is a bug. INFO-level log records for index update job will only be shown in log.txt files when an index was actually updated. If the update strategy was executed and the EvenQueue is empty, it will only add DEBUG-level log record to Crawling.log.txt file, without adding an INFO-level log records (the one you have provided above):

33056 21:42:02 DEBUG [Index=sitecore_core_index] IntervalAsynchronousStrategy executing.
33056 21:42:02 DEBUG [Index=sitecore_core_index] Event Queue is empty. Incremental update returns

That means that by default an INFO-level log record for index update job should always contain a certain number of processed units, greater than zero. Therefore, in the log file chunk you have provided, instead of showing zero processed units, it should have displayed some other number greater than zero.

Sitecore Support has registered this as a bug. Unfortunately, there is no easy way to fix this and display the correct number of items. To track the future status of this bug report, please use the reference number 95080.