4

Coveo Rabbit MQ on the Master CES server are getting loaded with over a million items and its causing CES to consume all the harddrive space on the CES Server.

Soon after that the indexes go into the Read only mode.

We tried deleting the million plus queue items but they keep reloading.

Has anyone else faced this?

2
  • does your CES logs show the indexing calls? anything useful there?
    – Diego
    Oct 7, 2016 at 3:23
  • CES logs and sitecore logs do not show any errors.
    – ASura
    Oct 7, 2016 at 14:22

2 Answers 2

6

What we discovered after hours of troubleshooting is that there was a corrupted message in the queue. When we removed that one message, the rest of the messages were processed properly.

I had to re-implement https://www.akshaysura.com/2015/05/01/add-max-message-limit-for-rabbitmq-for-coveo/ on the newer version of the CES and RabbitMQ in order to limit the damage in the future.

2
  • You consumed the corrupted message in the Queues >> [The queue stuck] >> Get Message section correct? Just for the records. Oct 13, 2016 at 18:23
  • Yes that is correct.
    – ASura
    Oct 13, 2016 at 18:26
3

When RabbitMQ queues start to pile up messages, it means there are no more consumers connected to the queues. Your Coveo Enterprise Search sources might have encountered errors and disconnected. You can manually reconnect them by disabling and re-enabling live indexing on the Queue Sitecore sources. They will resume their message consumption and the situation will resolve.

Unfortunately, if the index is already in read-only mode due to a lack of disk space, you will have to free up space, and put the index in read-write mode manually beforehand.

1
  • CES was running fine, we tried recreating the queues and even the index.
    – ASura
    Oct 11, 2016 at 15:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.