We are currently using Azure DevOps to deploy our Sitecore 9.1.1 and 9.2 solutions to Azure. In Azure DevOps you have the option of "leaving the App_Data" folder alone during deployment, i.e. not overwrite it. We currently use this option.

However, this also means that you will accumulate a large amount of log-files and diagnostics files over time, which as far as I understand, can slow down deployments.

Is there any Sitecore best practice in this area - should you leave the App_Data folder intact or clean it up on every deploy (assuming you include your license file and other must-have stuff in the App_Data folder, in every deploy)?

  • 1
    I ensure all my log files are using application insights instead. Currently looking at ways to make sure all files generated will not stay inside app_data if possible. Aug 19 '19 at 20:24
  • @VincentLui I guess you increased the max for how much data Application Insights is allowed to create then? I find that it is already difficult to use Application Insights, because there is too much data in there...
    – Hos
    Aug 20 '19 at 3:27
  • Have to use application insights if an application needs to take advantage of automatic alerts. Aug 21 '19 at 5:04

There are automatic cleanup tasks in place to deal with that.

Basically if you look under <scheduling> in a default Sitecore.config, you will find these.

<agent type="Sitecore.Tasks.UrlAgent" method="Run" interval="00:15:00">
  <param desc="url">/sitecore/service/keepalive.aspx</param>
<!-- Agent to process tasks from the task database (TaskDatabase) -->
<agent type="Sitecore.Tasks.TaskDatabaseAgent" method="Run" interval="00:10:00" />
<!-- Agent to clean up client data -->
<agent type="Sitecore.Tasks.CompactClientDataAgent" method="Run" interval="04:00:00" />
<!-- Agent to clean up publishing queue -->
<agent type="Sitecore.Tasks.CleanupPublishQueue, Sitecore.Kernel" method="Run" interval="04:00:00">
<!-- Agent that cleans up the event queue.
      The length of time that the EventQueue information should be kept.
      If both options are specified, IntervalToKeep is used.
      Supported options:
      DaysToKeep = The number of days that the EventQueue information should be kept. Example:<DaysToKeep>1</DaysToKeep>
      IntervalToKeep = The time in hh:mm:ss that the EventQueue information should be kept. Example: <IntervalToKeep>04:00:00</IntervalToKeep>
<agent type="Sitecore.Tasks.CleanupEventQueue, Sitecore.Kernel" method="Run" interval="04:00:00">
<!-- Agent to clear the HTML cache periodically -->
<agent type="Sitecore.Tasks.HtmlCacheClearAgent" method="Run" interval="00:00:00" />
<agent type="Sitecore.Tasks.HtmlCacheClearAgent" role:require="ContentManagement">
  <patch:attribute name="interval">00:05:00</patch:attribute>
<!-- Agent to remove expired Client authentication tickets  -->
<agent type="Sitecore.Tasks.CleanupAuthenticationTicketsAgent" method="Run" interval="04:00:00" />
<!-- Agent to publish database periodically -->
<agent type="Sitecore.Tasks.PublishAgent" method="Run" interval="00:00:00" role:require="Standalone or Reporting or ContentManagement">
  <param desc="source database">master</param>
  <param desc="target database">web</param>
  <param desc="mode (full or smart or incremental)">incremental</param>
  <param desc="languages">en, da</param>
<!-- Agent to clean up work files -->
<agent type="Sitecore.Tasks.CleanupAgent" method="Run" interval="06:00:00">
  <!-- Specifies files to be cleaned up.
          If rolling="true", [minCount] and [maxCount] will be ignored.
          [minAge] and [maxAge] must be specified as [days.]hh:mm:ss. The default value
          of [minAge] is 30 minutes.
          [strategy]: number of files within hour, day, week, month, year
          [recursive=true|false]: descend folders?
  <files hint="raw:AddCommand">
    <remove folder="$(dataFolder)/logs" pattern="*log.*.txt" maxAge="30.00:00:00" />
    <remove folder="$(dataFolder)/diagnostics" pattern="*.*" maxAge="30.00:00:00" recursive="true" />
    <remove folder="$(dataFolder)/viewstate" pattern="*.txt" maxAge="2.00:00:00" recursive="true" />
    <remove folder="$(tempFolder)/diagnostics" pattern="*.*" maxAge="00:10:00" recursive="true" />
    <remove folder="/App_Data/MediaCache" pattern="*.*" maxAge="90.00:00:00" recursive="true" />
<!-- Agent to dump performance counter data -->
<agent type="Sitecore.Tasks.CounterDumpAgent" method="Run" interval="01:00:00">
<agent type="Sitecore.Tasks.CloneNotificationsCleanupAgent" method="Run" interval="1.00:00:00">

As you can see, App_Data is taken care of (look for the CleanupAgent near the bottom). You may want to adjust some of the retention numbers, but overall the sanitising of App_Data is automatic.

Be aware that the ~/App_Data/DeviceDetection is not being cleaned up by default. You may want to add that.

Source: Cleanup Device Detection Databases

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.