20

I'm trying to find a recommendation from Sitecore in terms of configuring Solr for high availability and how to use it properly with Sitecore. Does anybody know where I can find it? If not, I'm considering these options, any preference or alternative?

Master-Slave:

Is it still "valid" or is it deprecated since SolrCloud? In this case, How should I configure Sitecore on each role to use one server for indexing, and another (load balanced servers) for querying?

Zookeeper:

How many instances do I really need? I understand (based on this) that I need 3 Zookeepers minimum to keep the cluster running if one goes down, but apparently they should not be on the same machine as the actual Solr instances meaning I need 5 instances (3 ZK and 2 Solr). And again, Or should I configure Sitecore to use the cluster properly?

1 Answer 1

20

We are using Zookeeper with Sitecore 8.1 Update 2, and as per the documentation here: https://zookeeper.apache.org/doc/r3.1.2/zookeeperAdmin.html we are using 3 instances. This is the minimum required.

Three ZooKeeper servers is the minimum recommended size for an ensemble, and we also recommend that they run on separate machines.

This works really well for us and ensures no downtime when indexes are being rebuilt.

This is an excellent blog post that we followed to set this up:

http://www.chrissulham.com/sitecore-on-solr-cloud-part-2/

A note of caution though as pointed out by Grant Killian in the comments below:

Note: SolrCloud setups are under experimental support starting from Sitecore XP 8.2, meaning some issues are possible. In case of related issues, Sitecore Support will help with troubleshooting, while there is no guarantee that a fix will be available. In earlier versions of Sitecore CMS/XP only standalone Solr setups are supported.

more info here: kb.sitecore.net/articles/227897

We carried out the following process to install this on our 3 Solr Search Servers in Production:

  1. Download Solr 5.2.1 (i'd recommend Solr 5.1 though as Sitecore support will help with issues for Sc 8.2: kb.sitecore.net/articles/227897)
  2. Prepare Solr configuration for Sitecore Indexes - Steps 1, 2, 3 and 4.
  3. Configuring ZooKeeper ensemble - Steps 1-5
  4. Configuring SolrCloud with ZooKeeper ensemble - All Steps
  5. Upload index configuration into ZooKeeper - All Steps
  6. Create collections

Here are some commands you might find useful:

Registering Config:

zkcli -zkhost localhost:2181 -cmd upconfig -confdir D:\Solr\server\solr\configsets\sitecore_configs\conf -confname scbasic

Creating a Collection:

solr create -c sitecore -d D:\Solr\server\solr\configsets\sitecore_configs -n scbasic -shards 1 -replicationFactor 1 -p 8983

Add replica to collection:

http://localhost:8983/solr/admin/collections?action=ADDREPLICA&collection=sitecore&shard=shard1&node=172.16.128.32:8983_solr

Remove replica from collection:

http://localhost:8983/solr/admin/collections?action=DELETEREPLICA&collection=collection1&shard=shard1&replica=core_node1

I wrote some PowerShell scripts to do most of the above for me so I could run them on Staging and Production and I found this easier than calling the API manually.

The script loops round an array of the sitecore index names. The Core is created on one of the Solr Servers and then 2 replica cores are created on the other two Solr servers.

$SitecoreIndexNames = @('sitecore_core_index', 'sitecore_master_index', 'sitecore_web_index', 'sitecore_analytics_index', 'sitecore_marketing_asset_index_master', 'sitecore_marketing_asset_index_web', 'sitecore_marketingdefinitions_master', 'sitecore_marketingdefinitions_web', 'sitecore_testing_index', 'sitecore_suggested_test_index', 'sitecore_fxm_master_index', 'sitecore_fxm_web_index', 'sitecore_list_index', 'social_messages_master', 'social_messages_web')

Here are the key functions:

function Create-SolrCore($solrBaseUrl, $coreName, $configSet, $solrCollectionName, $solrConfigSetName) {
        $restUrl = "{0}/admin/collections?action=CREATE&name={1}&property.name={1}&collection={1}&collection.configName={3}&numShards=1&shard=shard1" -f $solrBaseUrl, $coreName, $solrCollectionName, $solrConfigSetName

        Write-Host "adding core: $restUrl"
        $result1 = Invoke-WebRequest $restUrl -Method GET -UseBasicParsing
        Write-Host "Created Solr core $coreName."
    }

function Create-Replica($solrBaseUrl, $coreName, $configSet, $solrCollectionName, $solrConfigSetName, $replicataIP) {
    $restReplicaUrl = "{0}/admin/collections?action=ADDREPLICA&collection={1}&property.name={1}&numShards=1&shard=shard1&node={2}" -f $solrBaseUrl, $coreName, $replicataIP
    Write-Host "adding replica: $restReplicaUrl"
    $result2 = Invoke-WebRequest $restReplicaUrl -Method GET -UseBasicParsing
    Write-Host "Created replica for $SolrCollectionName."
}

function Create-Replicas($indexNames, $nonSwitchOnRebuildIndexNames, $coreNamePrefix, $baseUrl) {

    $SitecoreIndexNames | % {
        $index = Convert-SitecoreIndexNameToCoreName($_)

        try {
            Create-Replica $SolrBaseUrl $index $SolrConfigSetName $SolrCollectionName $SolrConfigSetName $ReplicataIP

        } catch {
            Write-Warning "Unable to add replica for core $index. $_"
        }
    }
}


function Create-SolrCores($indexNames, $nonSwitchOnRebuildIndexNames, $coreNamePrefix, $baseUrl) {

    $SitecoreIndexNames | % {
        $index = Convert-SitecoreIndexNameToCoreName($_)
        try {
            Create-SolrCore $SolrBaseUrl $index $SolrConfigSetName $SolrCollectionName $SolrConfigSetName

        } catch {
            Write-Warning "Unable to create core for $index index. This probably means it already exists. $_"
        }
    }
}

I based these on Kam's Solr Cannon Scripts: http://kamsar.net/index.php/2016/03/The-Solr-Cannon/

Once you have done this you can point Sitecore at the Solr ensemble like so:

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
    <sitecore>
        <settings>
            <setting name="ContentSearch.Solr.ServiceBaseAddress">
                <patch:attribute name="value">http://{ipofinstance}:8983/solr</patch:attribute>
            </setting>
        </settings>
    </sitecore>
</configuration>
8
  • Really good post series: So the short answer is: Minimum 3 servers behind a load balancer, each of them with Solr and Zookeeper. Configure Sitecore to use load balancer's end point. Right? Commented Oct 20, 2016 at 8:56
  • 1
    Yeah, I've added more info above to help you but thats the crux of it. Works well for us so far. Commented Oct 20, 2016 at 8:58
  • I've added some of my Powershell scripts for you above too Vincent. Commented Oct 20, 2016 at 10:55
  • 1
    Are you concerned with what Sitecore will officially support, or comfortable using "experimental" approaches? Refer to kb.sitecore.net/articles/227897 to see what I'm getting at. There are, of course, options running "in the wild" that function fine but are not formally supported. That may or may not matter in your case, but I wanted to point this out.
    – G Killian
    Commented Oct 20, 2016 at 21:31
  • Killian - that is certainly something to consider. Especially for those on older versions of Sitecore. We are planning on upgrading to 8.2 in the next week or two so should be covered for any support issues regarding search once we are. We may need to downgrade slightly to Solr 5.1 though. I'll update my answer to include some info on this. Commented Oct 21, 2016 at 13:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.