Coveo cloud indexing too slow

Question

Using Coveo cloud (Coveo for Sitecore 4.0.450.0), the rebuilding of just coveo_web and coveo_master indexes takes too long. Once all items are added, the Indexing Manager dialog is stuck for over an hour spinning.

This is the status I see on the index in coveo cloud:

Performing rebuild
 Started an hour ago - No items processed 

This is what is see in Indexing manager:

Rebuilding...
Job running: Index_Update_IndexName=Coveo_master_index
[Items Added to Index: 6278]

Job running: Index_Update_IndexName=Coveo_web_index
[Items Added to Index: 6277]

I've already opened up ports as described here: https://onlinehelp.coveo.com/en/cloud/granting_the_coveo_cloud_platform_access_to_your_on-premises_ces_index.htm

Answer 1

The Committed Documents step in Coveo for Sitecore (referred to as Waiting for documents to be searchable... in the documentation) is simply querying the index to notify you when your documents are really accessible from a search page. 10/sec with 7k documents looks OK to me, but it looks like this step is not the one that is taking all of that "over an hour" time.

I suspect the Deleted Documents step to be the culprit since it is usually the longest part in the indexing validation process. It validates that the old documents are deleted from the index.

If you don't mind waiting for that visual feedback, you can simply close the dialog and your documents will be available at a later time.

This slowest part could not be improved before because of technical issues, though the next Coveo for Sitecore version will contain such improvement to the process and should handle better the case you are currently experiencing.

You could also try to rebuild one index at a time instead, or re-index small parts of the tree, it can only help :)