1

In Sitecore 9.1.1

The Sitecore_master_index is huge on CM server. It goes up to like 450-500k documents than IIS resets. (think this is because memory dump?)

On Test-environment, which has similar items in master-db, including media library, the result of rebuild sitecore_master_index is only about 60k documents.

Why and what can cause the sitecore_master_index to get so massive?

(I've emptied the recycle bin)

Update

I set the DisableCacheSizeLimits to false and now it completed the rebuild. I also did a Cleanup Database, cleared cache in admin/cache.aspx and cleared the Queues in db.

But i'm not sure if only changing DisableCacheSizeLimits was enough. Before that change, during index rebuild the server was on 99% memory usage for a long time, probably causing the IIS reset.

Improve this question

3 Answers 3

Reset to default
2

I assume that the master index configuration is the same in both environments. Have you tried to do a cleanup of the master database before rebuilding the master index in Control Panel -> Cleanup database?

You can try to see if removing the media library items being indexed fixes the IIS restart issue by adjusting the Root index path in Sitecore.ContentSearch.Azure.Index.Master.config

Improve this answer
1

When it comes to sitecore_master_index, by default Sitecore creates 1 document for every language and every version of every item in Sitecore content tree.

If you have and item which has 3 languages and there are 4 versions of that item in all 3 languages, it will generate 12 documents in sitecore_master_index.

You're saying that on Test environment you have "similar items in master-db". If on Test you only have around 60k documents while on the other environment you have around 500k documents, there are multiple possible reasons for that, e.g.:

  • Not all versions are indexed on Test. To check that one, go to Test and start full rebuild of sitecore_master_index from Indexing Manager application in Sitecore Control Panel.

  • You have more languages on the environment with 500k documents.

  • And the one which I think is the most probable - you have much more versions of your items on the environment with 500k documents. Probably this environment is used more and there are new versions created frequently. If you don't need them, you can think of removing older versions ( One time delete of old versions ).

You may also think of clearing Event Queue table from master database and rebuilding indexes manually later, if there are too many changes queued in Event Queue and you have issues with starting your Sitecore instance.

Improve this answer
1

With the DisableCacheSizeLimits setting enabled you're giving full memory to Sitecore, so rebuilding the master index will increase some of the caches as it's accessing each and every item that is adding to the index. I'd suggest you to keep this setting disabled and fine tune your caches, even though it can be a bit tedious work, the results are way better. You can check this behavior by monitoring the DB caches while indexing.

You can also disable those DB caches during the index rebuild, by using this setting: ContentSearch.Indexing.DisableDatabaseCaches

That being said, have a look also at the following settings:

  • ContentSearch.IndexUpdate.BatchSize try to reduce the batch size, by defult is set to 300.
  • Also check ContentSearch.ParallelIndexing.Enabled

I hope it helps you!

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.