2

We upgraded from Sitecore 8.1 to 9.0 some weeks ago, and we moved to Azure PaaS. Since then we are suffering from application recycles in some of our instances.

Looking at event viewer, we saw tens of "recycles" of a concrete page due to "Global configuration changed" like the following in some of the instances

<Event>
<System>
<Provider Name="HttpPlatformHandler"/>
<EventID>2000</EventID>
<Level>4</Level>
<Task>0</Task>
<Keywords>Keywords</Keywords>
<TimeCreated SystemTime="2018-12-12T06:32:29Z"/>
<EventRecordID>798817468</EventRecordID>
<Channel>Application</Channel>
<Computer>............</Computer>
<Security/>
</System>
<EventData>
<Data>Global configuration changed. Change path: MACHINE/WEBROOT/APPHOST/....WEBAPP__89A7/...../MyPage, ApplicationManager: 0000013AD53DFBB0 [TID 2516] [PID 7800]</Data>
</EventData>
</Event>
<Event>
<System>
<EventData>
<Data>Recycling application MACHINE/WEBROOT/APPHOST/....WEBAPP__89A7/...../MyPage [TID 2516] [PID 7800]</Data>
</EventData>

And sometimes we saw a global configuration change on a concrete instance that cause recycles on all the web apps and their not being able to server requests for some minutes.

<Event>
<System>
<Provider Name="HttpPlatformHandler"/>
<EventID>2000</EventID>
<Level>4</Level>
<Task>0</Task>
<Keywords>Keywords</Keywords>
<TimeCreated SystemTime="2018-12-14T08:04:54Z"/>
<EventRecordID>1396347156</EventRecordID>
<Channel>Application</Channel>
<Computer>.........</Computer>
<Security/>
</System>
<EventData>
<Data>Change in global configuration [TID 3296] [PID 13784]</Data>
</EventData>
</Event>
<Event>
<System>
<Provider Name="HttpPlatformHandler"/>
<EventID>2000</EventID>
<Level>4</Level>
<Task>0</Task>
<Keywords>Keywords</Keywords>
<TimeCreated SystemTime="2018-12-14T08:04:54Z"/>
<EventRecordID>1396347218</EventRecordID>
<Channel>Application</Channel>
<Computer>......</Computer>
<Security/>
</System>
<EventData>
<Data>Global configuration changed. Change path: MACHINE/WEBROOT/APPHOST/SITECORE-CD-WEBAPP__89A7, ApplicationManager: 00000216AA53FE90 [TID 3296] [PID 13784]</Data>
</EventData>
</Event>
<Event>
<System>
<Provider Name="HttpPlatformHandler"/>
<EventID>2000</EventID>
<Level>4</Level>
<Task>0</Task>
<Keywords>Keywords</Keywords>
<TimeCreated SystemTime="2018-12-14T08:04:54Z"/>
<EventRecordID>1396347265</EventRecordID>
<Channel>Application</Channel>
<Computer>......</Computer>
<Security/>
</System>
<EventData>
<Data>Recycling application MACHINE/WEBROOT/APPHOST/SITECORE-CD-WEBAPP__89A7 [TID 3296] [PID 13784]</Data>
</EventData>
</Event>

Is anyone suffering a similar problem? Any idea what can cause a configuration change in a specific instance, not affecting the other ones?

Investigating the issue we found people experiencing similar problems with Sitecore 9 on Windows Server 2012, that can be fixed with a Windows update:

https://www.clearpeople.com/insights/blog/2018/november/sitecore-9-restarts-multiple-times-on-windows-2012.

But I wouldn't expect that Microsoft hadn't applied this hotfix to its Azure web apps.

Thanks!

  • Can you look in the Sitecore logs to see if it tells you specifically what files are being modified? – Dan Sinclair Dec 17 '18 at 15:53
  • I checked, not clue in logs. – Marc Cals Dec 17 '18 at 15:54
  • Are you getting any errors on the front end when this happens? Had an issue very similar and was receiving 503 Gateway timeout error. Sitecore provided a patch for it. – Justin Bradley Dec 17 '18 at 17:31
  • @justin It restart suddenly sometimes with errors 500 just before others not. We haven't find yet any pattern. Can you provide me the kb sitecore of the patch to investigate? – Marc Cals Dec 17 '18 at 17:45
  • @MarcCals - here is the KB - kb.sitecore.net/articles/910936 – Justin Bradley Dec 17 '18 at 18:18
1

Your app might be initiating the recycle due to a Proactive Auto Heal. Usually this means that memory is getting dangerously high or it has detected a memory leak that will cause OutOfMemoryExceptions.

This feature can be disabled, however I recommend you leave it enabled and try fix the root cause of the issue.

I've found that scaling up the hosting plan to one with more memory and cores also helps, but again it's only a band-aid fix.

  • Hi @MarkGibbons. We checked and It´s not the proactive auto heal neither a memory leak. The memory stills always around 5Gb. Thanks! – Marc Cals Feb 15 '19 at 17:55
0

Azure Dynamic cache can cause such issues, you can try disabling it in App settings WEBSITE_DYNAMIC_CACHE=0.

See https://blogs.msdn.microsoft.com/waws/2017/08/02/asp-net-and-asp-net-core-application-restarts-on-azure-app-service/

  • We had this setting enabled from month ago and make not difference. Thanks! – Marc Cals Feb 15 '19 at 17:55
0

Finally we find the problem, it´s due to an Azure problem with slots, unrelated with Sitecore.

We were able to recreate the issue to Azure with a blank web app, just doing a manual swap, any scale out done after the swap cause the web app to restart.

The issue description to Azure was: "After customer does a swap as mentioned, on the next change notification, the application host config will get updated binding information to reflect the swapped hostnames of the slots. However, this isn’t triggered by the swap, but rather happens as a result of the swap. It can happen that the next change notification doesn’t come for a few hours or in fact even a few days when austocale is done."

Resuming the swap it´s doing a change in a config file, that it´s not noticed at the moment of swapping, it´s noticed when the autoscale it´s done, causing a restart of the wep app due to a change in a config file.

To avoid this problem it´s just so easy as add the following setting to the web app

WEBSITE_ADD_SITENAME_BINDINGS_IN_APPHOST_CONFIG=1

More info in https://github.com/projectkudu/kudu/wiki/Configurable-settings

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.