Caches not evicting until application fails despite high memory usage

Question

I have a solution that I have been running gradually incrementing load-tests on (i.e. I am using jmeter to create some load but I'm incrementing the load slowly, rather than all at once) until CPU and memory usage max out. When my application finally gets up to peak memory consumption, rather than seeing the caches evicted (observing using the cache admin page and performance counters in perfmon) I am seeing the site simply fall over and the app pool recycle (which does clear the cache, but obviously not in the way that I am looking for).

Note that I did not set a default cache priority (via the Caching.Priority setting; confirmed in the ShowConfig), so my cache entries should all have the Sitecore.Caching.CachePriority.Lowest caching priority, though I don't think this is related (posting a separate question about the purpose of this enum).

Expectations After Reviewing the Sitecore.Caching Namespace

From analyzing the types in the Sitecore.Caching namespace of the Sitecore.Kernel assembly, it looks like Sitecore's caching leverages .NET's System.Runtime.Caching assembly (specifically, the MemoryCache type) in order to support features like cache expiration and cache eviction when system resources are running low (see the Sitecore.Caching.MemoryCacheAdapter class). However, if this is true then the application should be evicting objects from the caches when resources are running low.

Expected Behavior vs Cache Size Limits

I know that Sitecore allows developers to set various cache size limits and will automatically purge caches of entries when those size limits are reached. This is not the behavior that I'm looking for. I am looking to see if there is a way to get Sitecore to proactively respond to high server memory consumption by evicting items from its caches, which should be native functionality of the MemoryCache class (given that all cache entries in Sitecore are given the System.Runtime.Caching.CacheItemPriority.Default priority). For this reason, I have intentionally disabled cache size limits (via the Caching.DisableCacheSizeLimits setting).

Question

I'm not sure if this is a bug, or if this is expected native behavior. Can anyone help to clarify the actual expected behavior and, if different from what I described, perhaps a way to customize Sitecore's caches so that they do react to system resource consumption?

Answer 1

I have seen the behavior you describe when using MemoryCache before and so have others. Unfortunately, it seems to be treated as a feature rather than a bug. However, when I looked into this previously on 8.2 update 2, I found that Sitecore does not use MemoryCache by default. It should only use MemoryCache if you configure it to in the <cacheContainerConfiguration> section of the configuration. A clean instance will contain a CacheContainers.config.example file with the following contents:

<!--
Purpose: This include file configures the Cache containers for various cache instances
To enable this file, rename it so that it has a ".config" extension.
If no containers are specified for a cache, the default Sitecore.Caching.Generics.Cache<TKey> instance is used. 
-->
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <!-- CACHE CONTAINER CONFIGURATION
      Defines the configuration of the cache containers.
      Supported attributes:
          name:       the name of the cache instance.
          type:       the cache container that is instantiated for the cache instance.
      Supported child parameters:
          name:       the name of the cache container.
          maxSize     the maximum size of the cache container.
    -->
    <cacheContainerConfiguration>
      <!-- Example -->
      <!--<cacheContainer name="website[html]" type="Sitecore.Caching.MemoryCacheAdapter, Sitecore.Kernel">
        <param desc="name">$(0)</param>
        <param desc="maxSize">$(1)</param>
        <cacheSizeCalculationStrategy type="Sitecore.Caching.DefaultCacheSizeCalculationStrategy, Sitecore.Kernel" />
      </cacheContainer>-->
    </cacheContainerConfiguration>
  </sitecore>
</configuration>

Note the comment about the default type being Sitecore.Caching.Generics.Cache<TKey>. This class will normally check the size of a new cache entry as it is being added. If the new entry would put the cache over it's configured size limit, it will evict expired and, if necessary, random entries to make room for the new entry. However, if the DisableCacheSizeLimits setting is true, it will not check the size or evict any entries. You should only set this to true if you are very confident that your server has more than enough memory for you worst case scenario.