11

Sitecore keeps a local copy of the lucene index in the file system of each instance and doesn't support sharing in the indexes between instances. How can I achieve this?

Improve this question

6 Answers 6

Reset to default
20

You cannot. Lucene is a "local filesystem" technology.

If you want something that is shared between multiple Content Delivery server instances; you need to switch from Lucene to SOLR.

If you're using the recommended Sitecore ContentSearch APIs this is (mostly) a relatively simple matter of switching your configuration files and setting up the SOLR index properly.

Benefits of using Solr:

  • It is heavily optimized for search performance with a powerful query cache.
  • It can handle a large number of documents.
  • There is detailed configuration support for language indexing.
  • You can shard indexes across multiple servers.
  • With Solr Cloud you can also split an index across multiple physical locations.

From Sitecore Search Scaling Guide

Improve this answer
2
  • I've to use only lucene. Is it possible to xcopy Lucene indexes between CM and CD?
    – Sandeep
    Commented Oct 17, 2016 at 17:54
  • 4
    It is, but you would need to shut down your Sitecore instance while doing so - files will be locked otherwise. If that is acceptable to you, then this is indeed possible.
    – Mark Cassidy ♦
    Commented Oct 17, 2016 at 17:55
6

another point: if you are leveraging xDB in Sitecore 8.x - and you have a scaled environment, you NEED a Solr index to make sure you have the analytics and list indexes updated on all servers

Improve this answer
3
  • Could you expand on "leveraging xDB" - does this mean using it at all?
    – Kasaku
    Commented Oct 18, 2016 at 11:21
  • yes it does mean using it at all - indexes are core of some of the xDB operations and the indexes will not be updated correctly on all servers
    – Klaus Petersen
    Commented Oct 25, 2016 at 18:24
  • To be clear, on scaled CD servers with a single CM?
    – Kasaku
    Commented Oct 25, 2016 at 18:27
5

I second Mark's answer. In a "huge" content delivery farm you really want to consider a centralized search provider like Solr.

Coveo is another good option as an alternative to Solr. The starter edition is free, and if your indexes are small enough it may do the job for you with minimal fuss. http://www.coveo.com/en/solutions/coveo-for-sitecore/download

Improve this answer
2
  • It's seems nice option. What about learning curve using Coveo?
    – Sandeep
    Commented Oct 17, 2016 at 18:37
  • 1
    Coveo will probably tell you it's super easy, one click install, zero code changes, etc. In reality, it will require you to learn the system and plan a small project to deploy it to your various environments. Solr comes with the same caveats, and is probably a bit harder to learn with no prior knowledge. Coveo is written in .NET so if you're familiar with that stack you may find it easier than Solr. I can't endorse one over the other :) Each has its strengths and weaknesses.
    – Chris Sulham
    Commented Oct 17, 2016 at 18:46
3

You are asking about managing indexes in "a huge content delivery web farm".

It could be a huge in two dimensions - vertically and horizontally.

Horizontally means that you have lots of servers. For that case, you got great answer - move index to out-of-the-process service like Solr or Coveo.

Vertically means that your issue is an index size.

That is very typical situation, and it originates from the fact that out-of-the-box Sitecore is configured for usability, not for performance.

sitecore_master_index and sitecore_web_index are designed for Content Management. These indexes include all versions of all Sitecore items and they used in Item Buckets, Media Selection dialog, Fields with search and etc.

CD requirements are different. You do not need all the items. You need only items that user should see as search results. It could be pages, products, articles, sometimes media-items and similar business objects.

The easiest way to bust performance of index in CD is to limit number of items in index. In typical case, Index rebuild speed will be at least ten times faster.

Update these config sections:

  • Filter Items by Root Item – sitecore/contentSearch/configuration/indexes/index/locations/crawler/Root

  • Filter items by Template Type – sitecore/contentSearch/indexConfigurations/yourIndexConfiguration/documentOptions/include

If that is not enough, try Domain Indexes. With that pattern you build custom indexe for every business domain instead of using standard one. You can find more details in my blog post - Indexing Patterns in Sitecore

Improve this answer
3

If you decide to stick with Lucene, this post provides information to help you out.

Maintaining Lucene Indexes Uptime in Sitecore Production Environments

It describes the following:

1) Use SwitchOnRebuildLuceneIndex to keep a working index while the rebuild is executed in a secondary folder.

2) Update your configuration files so that publishing and index rebuilds on a Content Management server trigger rebuilds on Content Delivery servers.

Improve this answer
0

We use SOLR in our production environment. It removes a lot of load from all the servers.

Few learnings from our setup: The configuration is quite complex (availability, scalability, index config, which fields, which templates, what to exclude, async or sync update, etc.). The out-of-the-box configuration for storing ALL fields in the index for all languages is just not working as it bloats the space and memory needed on the indexing server a lot.

We are applying the Domain Indexes pattern as well and a huge Master Index with special tweaks and computed fields for the authoring search.

Not an easy topic, so don't underestimate this...

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.