4

I have a Sitecore 8 instance with a bucket with aprox. 150K items, and like 200K total.

Publishing is working very slow. I mean, really slow, sometimes 5 items per second. I've stripped all code we had on the pipeline and the issue persists.

I believe this may be related with the insertion of items during publish into the Lucene index. I've tested a manual insertion of an item into the index and I can see it takes a long time, up to two or three seconds sometimes:

var contentSearchIndex = ContentSearchManager.GetIndex("sitecore_master_index");
contentSearchIndex.Refresh((SitecoreIndexableItem)scItem);

I've tried performance tuning the index with this article, but no luck:

https://community.sitecore.net/technical_blogs/b/sitecore_7_development_team/posts/sitecore-7-performance-tuning-part-2

Is there a way of having a fast indexing with a relatively large amount of items? Is there something else that comes to mind that I could try?

Here's a copy of my index config file:

https://drive.google.com/file/d/0B2fC8NS-H2h4TG1lcG9CRUlfN3M/view?usp=sharing

  • What sort of server equipment are the Web and DB roles running on? Indexing is heavy on Disc IO when writing on the Web instance and tends to be very taxing on Network and all facets of the DB server (Disk, Mem, CPU). – Laver Oct 12 '16 at 13:32
  • @Laver it is not a production server. Both the Web server and DB server are Virtual Machines with 16GB Ram and 4 2Ghz cores assigned. I would need to ask to IT about the disk spec, but lets assume it is a HD@7200RPM. They run on the same HyperV host. – Mat-Tap Oct 12 '16 at 13:40
  • Are media files being published also? – Hishaam Namooya Oct 12 '16 at 13:49
  • @HishaamNamooya We are not using the Media Library right now. The items being published do not have links to items on the Media Library. – Mat-Tap Oct 12 '16 at 13:52
  • Can you include links to your master index config file and your web index config file? I want to see your update strategies. – Sergeant Sitecore Oct 12 '16 at 14:31
5

I think your Lucene index needs a trim. Especially if it hasn't been this slow always, but just gradually started degrading little by little.

Since it's not a production server, my first recommendation would be to issue a full Index Rebuild and let it run for however long it needs. If you see a really slow rebuild speed, your problem lies elsewhere unfortunately.

Also: <setting name="ContentSearch.MaxMergeMB" value="512" /> this setting seems very low to me. I've seen Lucene happily use up 6-8 GB of memory for around 125.000 items. Increasing this to 4 or even 8 GB would be my next visiting point.

1

You could try deleting the Indexes directory and rebuilding all (in case there is some corruption).

I know some who disable Lucene indexing entirely during publishing and then do a re-index once all the dust has settled. Lucene has some cascading actions (Optimize and segment merges) that can be slow, esp if resources are limited. You can disable indexing by changing the config settings, or programmatically via:

Sitecore.ContentSearch.Maintenance.IndexCustodian.PauseIndexing();
IndexCustodian.ResumeIndexing();

You can check the status of indexing with the following:

IndexCustodian.IsIndexingPaused( ISearchIndex index )

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.