We see performance issue on CM server. Content editor is very slow periodically and resource monitor shows that HDD is busy - reading and writing sitecore_analytics_index files nearly 50% of the time. We have around 1 million contacts in xDB. Can Lucene handle it or we have to use Solr?
Sitecore 8.0u3
The sitecore_analytics_index stores information about interactions, contacts, and some other xDB entities. The more data you collect, the bigger the index will grow. The more visits you have in a given time period, the more frequent index reads and writes will be.
Whether or not you should switch from Lucene to Solr depends on whether the high hard disk activity is a problem for you. If you believe that the performance of your CM is slowed down because of this Lucene index, then you should definitely try to migrate the index to a Solr instance hosted on a separate server. This will certainly take the burden off the CM.
Switching to Solr is not necessarily the only solution. Sure, the official documentation says you should upgrade if you have more than 50k documents in the index. But you can also try to switch from a spinning disc HDD to an SSD—this will make disk reads and writes much faster.
The general reasons for using Solr instead of Lucene are:
When you need to index large numbers of items (50,000 and up), Solr performs better, so in your case you have to use SOLR
Solr is more robust. If your site depends on search as the primary interface, consider using Solr.
If you use multiple content delivery servers (or plan to do so later), use Solr. Solr works automatically in such an environment. You could use Lucene, but you have to make sure that indexes are synchronized across servers yourself.
You should therefore use Solr if you plan to scale your site (have a distributed setup with multiple servers).
More informations you find here: https://doc.sitecore.net/sitecore_experience_platform/setting_up__maintaining/search_and_indexing/indexing/using_solr_or_lucene
The Analytics index can cripple Solr too, I've seen this in production environments. You want to pay special attention to the memory settings for Solr and the JVM and make sure you have enough disk capacity and RAM to handle it as it grows. As merges take place, the analytics index of ~1 million documents can grow up to 20 GB in size.
It may not be enough to just switch to Solr, you may need to consider Solr Cloud and sharding the analytics index.
As an alternate solution, if you can upgrade to Sitecore 8.2, consider the ContentSearch.Analytics.IndexAnonymousContacts setting. Turning this to false will greatly reduce the size of your analytics index, as well as the resource load it puts on your server. That may be cheaper/faster than switching to Solr. This blog post has more details: http://sitecoreart.martinrayenglish.com/2016/10/taming-your-sitecore-analytics-index-by.html
