We have our analytics_index over 150 million documents now (with Solr on a standalone windows server). How can we stop Sitecore from optimizing the index? Solr says optimizing should only be done on fairly static indexes. (Also, it places a copy of the index in disk cache as it optimizes it, building a new index and the I/Ops on the disk is always through the roof.) Any way to tell Sitecore not to optimize the analytics index when it adds more documents to it? We noticed that Sitecore.ContentSearch.Solr.Index.Analytics.config has a TimedIndexRefreshStrategy of one minute for this index. Should we set that to nightly as well?
3
-
Are you seeing a warning in Solr about the optimisation? As far as I am aware, Sitecore only optimise the index when it is rebuilt, not when new documents are added, or in the case of the sitecore_master_index it is optimized every 12 hours by default, thanks to the agent "Sitecore.ContentSearch.Tasks.Optimize". I think the TimedIndexRefreshStrategy simply checks the queue of elementes pending to index and process them, without a Optimize at the end.– Vicent GalianaCommented Jul 13, 2017 at 16:38
-
We're seeing the index double the disk size then shrink back down to original size. We thought that was what optimizing does-and we see that happen all day long. We assume updating the index triggers an optimize from Sitecore's side. Something must be telling it to optimize. Do you know of a way to turn off optimizing capability on that index?– Darren WalkerCommented Jul 13, 2017 at 16:52
-
I couldn't find any other call to Optimize apart from the index rebuild. There is a setting to force a rebuild when there are too many items to index (FullRebuildItemCountThreshold), but I don-t think it is used in your strategy. In Solr cloud 6.0 I can see the warning in the logging panel when the indexes are optimized. Checking this would be my first step.– Vicent GalianaCommented Jul 14, 2017 at 7:07
Add a comment
|
1 Answer
There is an agent, Sitecore.ContentSearch.Tasks.Optimize, that controls which indexes are regularly optimized and how frequently. Open up your <your_site>/sitecore/admin/showconfig.aspx and look for this agent. It should look something like the below:
<agent type="Sitecore.ContentSearch.Tasks.Optimize" method="Run" interval="12:00:00">
<indexes hint="list">
<index>sitecore_master_index</index>
...
</indexes>
</agent>
Take a look and see if there is an entry in the <indexes> list for the sitecore_analytics_index. If so, remove it from the list and the index should no longer be regularly optimized. If not, then scheduled optimization is likely not your issue, but rather on-demand (i.e. at the end of a rebuild) optimization.
If on-demand indexing is your issue, share with us the <strategies> that you have configured for your sitecore_analytics_index.
-
We don't have the analytics index setting by the scheduled agent-that's good to know! We are curious what methods are there for on-demand optimization. Is the timedindexrefreshstrategy one of those methods? You mention the end of a rebuild triggers an optimize: Analytics index is not rebuilt by the Sitecore rebuild index option. How are analytics optimizations triggered? Commented Jul 19, 2017 at 22:56