9

I'm running Sitecore 9.0.1 and trying to rebuild a reporting database. The problem is that the process is ridiculously slow. How slow? Here's a screenshot:

enter image description here

After 26 hours 127 items were processed of 178.839 in total. I went over the logs of the processing and reporting server, they show no error but this:

ManagedPoolThread #8 17:12:01 WARN  Authentication on CES Discovery service failed.
Exception: System.Net.WebException
Message: The remote server returned an error: (403) Forbidden.
Source: System
   at System.Net.HttpWebRequest.GetResponse()
   at Sitecore.CES.Client.WebClient.<>c__DisplayClass6_0.<ExecuteRequest>b__0()
   at Sitecore.CES.Client.WebClient.Execute[T](Func`1 action, String requestUri)
   at Sitecore.CES.Client.ResourceConnector`1.Request(String endpoint, Object[] parameters)
   at Sitecore.CES.Discovery.EndpointSource.GetEndpoint(String serviceName)

ManagedPoolThread #8 17:12:01 ERROR Could not update device detection database
Exception: System.ArgumentNullException
Message: Null ids are not allowed.
Parameter name: endpointUri
Source: Sitecore.Kernel
   at Sitecore.Diagnostics.Assert.ArgumentNotNullOrEmpty(String argument, String argumentName)
   at Sitecore.CES.Client.WebClient.DownloadBinaryContent(String endpointUri, String resourcePath, DownladBinaryContentParameters parameters)
   at Sitecore.CES.DeviceDetection.Providers.FiftyOneDegrees.FiftyOneDeviceDetectionClient.GetNewerVersion()
   at Sitecore.CES.DeviceDetection.Providers.FiftyOneDegrees.FiftyOneDeviceDetectionClient.DoUpdate(String serviceName, String& newDatabasePath, String& newDatabaseVersion)
   at Sitecore.CES.DeviceDetection.Providers.DeviceDetectionClient.Update()

Could this cause this kind of behavior? This problem happens across all Sitecore servers we run for this client, CM, CD, PROC, REP, all. Or is there anything else there?

Does anyone has any tip on what could cause such slowliness?

thank you

  • Is this a PaaS solution? If so an in azure, have you looked at DTU for your databases? Are you using Elastic pools? – Justin Bradley Dec 18 '18 at 23:54
  • Hey Justin, it's on premise. I've requested the DBA responsible for a DB analysis but all databases sit on the same server and master for example is not presenting the same kind of slowliness. – Anderson Fortaleza Dec 19 '18 at 0:29
  • 1
    That CES error looks like an issue that is resolved in a hotfix. Do you have this hotfix applied already? kb.sitecore.net/articles/828414 That said, I don't think that would cause that extreme level of slowness, but worth ruling out. Looking at the processing role metrics and database metrics may give you some insight into where the bottleneck lies. – George Tucker Dec 19 '18 at 1:11
  • 1
    I dont think the GeoIP (CES) and Device errors are you issue. You should see those all the time if you have not subscribed to those services. I once ran a rebuild for 10 days. – Chris Auer Dec 19 '18 at 1:46
  • 1
    I opened a ticket with Sitecore a few months ago over this same issue. There response was to add more agents to process the rebuild. They referred to this article. doc.sitecore.net/sitecore_experience_platform/82/… – Chris Auer Dec 19 '18 at 1:51
4

I have applied the hotfix as instructed by George Tucker and increased the number of history workers as instructed by Chris Auer, in my case I have raised from 1 to 500. I'm getting much better speed now, around 1.500 items per minute! I have no idea if the hotfix had any role on this because I did both things before running the rebuild again.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.