3

Our Sitecore 9.3 platform has a custom index called "OurProject_crawledcontent" which is used by the search functionality on the main website. This index is rebuilt by a Sitecore task every morning, which starts a crawler that crawls the site and builds an index based on the HTML.

Since this is a custom index which doesn't ever have to be updated outside of the daily rebuild, it's configured to use the "manual" index update strategy.

We are using SolrCloud.

To make sure that the index won't be empty during the daily rebuild, we are using the SwitchOnRebuildSolrCloudSearchIndex type for this index (on CM). We are following the rules that are document here: https://doc.sitecore.com/xp/en/developers/91/platform-administration-and-architecture/switch-solr-indexes.html

This is not working correctly. While the daily rebuild task is busy, our search index is empty.

This is probaby caused by the fact that we're having trouble with this specific rule:

Only use the SwitchOnRebuildSolrCloudSearchIndex index type in combination with an active index update strategy (index has at least one index update strategy other than manual).

We are using a manual update strategy, so to comply with this rule, we also added a second update strategy as a dummy that should not do anything. It solely exists to comply wit the rule mentioned above. This second update strategy is of type IntervalAsynchronousStrategy with a very large interval and listening to the "core" database (because this database doesn't change often anyway).

These are the relevant parts of our configuration on the CM server:

<fakeIntervalAsync type="Sitecore.ContentSearch.Maintenance.Strategies.IntervalAsynchronousStrategy, Sitecore.ContentSearch">
<param desc="database">core</param>
<param desc="interval">99.00:00:00</param>
<CheckForThreshold>false</CheckForThreshold>
</fakeIntervalAsync>
<index id="OurProject_crawledcontent" type="Sitecore.ContentSearch.SolrProvider.SwitchOnRebuildSolrCloudSearchIndex, Sitecore.ContentSearch.SolrProvider">
<param desc="name">$(id)</param>
<param desc="mainalias">OurProject_crawledcontent_main_alias</param>
<param desc="collection">OurProject_crawledcontent</param>
<param desc="rebuildalias">OurProject_crawledcontent_rebuild_alias</param>
<param desc="rebuildcollection">OurProject_crawledcontent_rebuild</param>
<param ref="contentSearch/indexConfigurations/databasePropertyStore" desc="propertyStore" param1="$(id)"/>
<configuration ref="contentSearch/indexConfigurations/defaultSolrIndexConfiguration">
<documentOptions type="Sitecore.ContentSearch.SolrProvider.SolrDocumentBuilderOptions, Sitecore.ContentSearch.SolrProvider">
<indexAllFields>true</indexAllFields>
<include hint="list:AddIncludedTemplate"/>
<include hint="list:AddIncludedField"/>
<fields hint="raw:AddExcludedSpecialField"/>
<fields hint="raw:AddComputedIndexField"/>
</documentOptions>
</configuration>
<strategies hint="list:AddStrategy">
<strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/manual"/>
<strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/fakeIntervalAsync"/>
</strategies>
<locations hint="list:AddCrawler">
<crawler type="OurProject.WebSearch.Crawlers.WebCrawler, OurProject.WebSearch">
</crawler>
</locations>
</index>

This is the code executed in our daily Sitecore task:

var index = Sitecore.ContentSearch.ContentSearchManager.GetIndex(indexName);
Sitecore.ContentSearch.Maintenance.IndexCustodian.FullRebuild(index);

This is a part of the code that we use on our website's search page, which yields 0 results during the rebuild process (and works perfectly otherwise):

using (var context = Sitecore.ContentSearch.ContentSearchManager.GetIndex(indexName).CreateSearchContext())
{
  // var searchQuery = context.GetQueryable . . .
}

It's not working. While the index is being rebuilt by the daily task, the index seems to be empty, so the search page gives 0 results.

I have three questions:

  1. Why does this rule even exist, that the SwitchOnRebuildSolrCloudSearchIndex type requires at least one non-manual index update strategy? What does an UPDATE strategy have to do with the REBUILD mechanism?

  2. Given that this rule exists, am I correct in assuming that adding a dummy/fake update index strategy should work, simply because it makes us follow the rules, even though this update strategy itself doesn't do anything?

  3. If the answer to question 2 is "yes", does anyone have an idea why it's not working in our case?

6
  • 1
    Some questions - do you have more than one CM? if so do secondary ones use 'SolrSearchIndex' class instead of rebuild? and have you specified InstanceName? Do you notice a change in the collection being referenced by an alias before/after a rebuild? Nov 20 at 19:56
  • @DeanOBrien, thanks for your reply. We have only one CM. There are two CDs, which are using SolrSearchIndex. Yes, we have specified an InstanceName which is unique each CM or CD. I can't answer the last question about the collection right now. I'll have to look that up. (Not very experienced with Solr, but definitely a good suggestion to look into this!)
    – Peter M.
    Nov 21 at 8:29
  • @DeanOBrien, my SOLR knowledge is limited, so I hope my answer makes sense: I've opened the Solr admin UI, then checked the contents of aliases.json. When I trigger an index rebuild in Sitecore and then re-open aliases.json in the Solr admin UI, I don't see any changes.
    – Peter M.
    Nov 21 at 12:51
  • 1
    I am more familiar with normal SOLR not in cloud mode, so cant say for definite. That said, I think LISTALIAS command should help. Please run this command before and after, then share the sections relating to your custom index. solr.apache.org/guide/8_2/collection-aliasing.html#listaliases If no change is shown, this should confirm that the switch is not being performed and that as I suspect you are always pointing to the same index Nov 21 at 14:51
  • 1
    Also, please check <yourinstance>/sitecore/admin/showconfig.aspx to confirm that your patch is being applied as you expect. Please also check search.log to see if anything useful is being listed there during the rebuild Nov 21 at 14:54

1 Answer 1

1

Going to add some suggestions here rather than continuing in comments.

Based on the comments, it sounds like the SwitchOnRebuildSolrCloudSearchIndex class has been correctly assigned and is working to some degree, as its switching the aliases over. As desribed in the link you provide in the question:

When a rebuild operation finishes, the CREATEALIAS command swaps the collections the aliases reference.

As a guess, I would say it sounds like your Sitecore instance is somehow pointing at the non active collection to begin with. Then when a rebuild is triggered, that index is reset (while being pointed at) and a rebuild of that index begins. After its complete, it then switches and points to the old one.

To test this, please try making a minor change to the name of an item. Then trigger a rebuild. I think there is a chance that the change will not be shown after the rebuild completes. But perhaps will then show if you were to trigger a subsequent rebuild (because the old one from #1 is pointed to after #2 rebuild completes).

If the above is true, then you should be able to manually trigger the CREATEALIAS command. Which I think would flip it back into sync.

/admin/collections?action=CREATEALIAS&name=name&collections=collectionlist

If the above does not work, then please do the following. Look at the search.log immediately after you call the search component. The last entry in the log should show the query that is being sent to SOLR from your application. You can call this directly against solr by pasting that query into the URL immediately after this: http://localhost:8983/solr/collection1/select?q=

Once you are able to confirm the same results show direct in SOLR, then use this method to call the two different collections during a rebuild. See if you can confirm if the primary active collection ever shows 0 results, or if it is the secondary one prior to it being rebuilt (as it should be).

Also, earlier in the comments I suggested looking at the search.log for errors. But in fact exceptions relating to the rebuilding process would actually be shown in the crawling.log. Please review this also to see if anything shown.

https://solr.apache.org/guide/8_11/solrcloud-query-routing-and-read-tolerance.html#:~:text=By%20default%2C%20SolrCloud%20will%20run,you%20want%20to%20search%20against.

2
  • 1
    Thanks a lot for all your helpful suggestions! I've upvoted your answer and all your comments. We still haven't pinpointed our specific problem, but I think our Solr (aliases) configuration might be off... I'll continue the investigation there, but that probably gets out of scope of the original question, so better not to continue it here.
    – Peter M.
    Nov 24 at 10:26
  • no problem - googd lusk with finding the solution Nov 24 at 19:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.