What is the best practice approach to populating a navigation component

Question

I have been spending a bit of time looking at the Sitecore Debugger. Using this you can see the 'hotspots', the components on the page that take the longest time to load and make the most calls to the database.

One of these hotspots in our implementation is the primary navigation component, which as you can imagine has to read lots of items in order to determine what to include in the menu. This got me thinking, about how we might improve our implementation and also to wondering what the best practice approach was.

As a brief intro to our approach, we do the following:

1. Settings node has MultiList field for "PrimaryNavigation" Items (these form top level)
2. Each PrimaryNavigation Item also has a MultiList field for "Sections" (these form sections within a dropdown)
3. Each section has a MultiList field for "MenuItems" (which is effectively the final link)

Much of the data retrieval utilizes a method called GetLinkedItem, which cycles through the various MultiList fields grabbing the items for the menu.

public static List<Item> GetLinkedItems(Database database, string fieldValue)
{
    var items = new List<Item>();
    if (!string.IsNullOrEmpty(fieldValue))
    {
        var ids = fieldValue.Split(new char[] { '|' });
        foreach (var id in ids)
        {
            var linkedItem = database.GetItem(new Sitecore.Data.ID(ParseId(id)));

            if (linkedItem != null && !linkedItem.Publishing.NeverPublish)
            {
                items.Add(linkedItem);
            }
        }
    }
    return items;
}

With this approach, the debugger is showing some 4500 item reads to populate a menu that contains approximately 150 items.

I think one of the reasons for the discrepancy is that when an item is loaded, it also reads its child items. Either way, this seems to me to be a bit inefficient.

My first thought was to try getting the data from the web index, with code similar to the one below:

public static List<ContentSearchModelBase> GetLinkedItemsFromIndex(Database database, string fieldValue)
{
    var items = new List<ContentSearchModelBase>();

    if (!string.IsNullOrEmpty(fieldValue))
    {
        var ids = fieldValue.Split(new char[] { '|' }).Select(x => new ID(x));

        var index = database.Name == "master" ? "sitecore_master_index" : "sitecore_web_index";

        using (var context = ContentSearchManager.GetIndex(index).CreateSearchContext())
        {
            var q = context.GetQueryable<ContentSearchModelBase>()
                .Where(x => !x.NeverPublish && ids.Contains(x.ItemId)).GetResults();

            var result = q.Hits.Select(h => h.Document).ToList();

            if (result != null)
                return result;
        }
    }
    return items;
}

However, whilst this reduced the number of Item reads, it was drastically less efficient!

I appreciate in the grand scheme of things - when the component gets cached the overhead becomes less of an issue. But I was wondering if anyone can recommend a better more efficient approach to building a multi-layer navigation menu component?

Answer 1

At the risk of taking this post into subjective space (although your post leans in this direction already), in my view you're actually working a problem in your general approach and trying to solve it with technology.

Use the APIs, don't re-create them

First of all; you're not giving the Sitecore APIs a fighting chance of implementing any layer of caching, since you're essentially bypassing it entirely by taking it straight down to the basics of .GetItem() based off a field value.

Multilist field actually has methods to draw out either the TargetIDs and a .GetItems() method - both of which you should be using as opposed to string splitting and individual .GetItem calls. Why? Because this is where Sitecore would have a chance to utilize some level of caching. Not that it does, to my knowledge, but that's not really the point. You shouldn't bypass native API functionality and cook your own.

If you're going to build your own methods, make your abstraction on a higher level and implement the caching that would help yourself, as in get the TargetIDs from your field, return items from your own cache and only do .GetItem on the ones you don't have in the cache. Your cache here can (almost) be completely static - menu navigation lists don't change all that often in real life. Drop the cache on a publish event.

Why the search index didn't help you

It didn't help because you're esentially doing the same thing. But instead of going to Sitecore to get the list of TargetIDs you instead take a roundtrip to Solr to do the same thing. And THEN you still go back to doing individual .GetItem() calls to get to your items. You're not saving anything - using Solr to do a string split for you isn't helping anyone.

For the index approach to give you any kind of improvement, you need to think about it way differently. What you need to arrive at, is a situation where you can query solr for "all of the items that this multilist field has selected" and return the Navigation Title for each of them. And then return a string array as opposed to Sitecore Items. You don't NEED a full Item to build your navigation - presumably - you only need the text that goes into the menu.

The index approach can absolutely be made to be your most effecient approach to all of this - but it requires you to actually use the index for its strengths. Use it to return your end result, don't use it to return an intermediate result that you then anyway end up sending into Sitecore low level API calls.