Content Item batch retrieval in .Net code?

Question

Is there a way to retrieve a set of content items from Sitecore's APIs in batches, besides using the REST API? I'm looking at improving performance for a client's Sitecore site. A lot of the legacy code consists of this common pattern for retrieving multiple content items:

 foreach(string itemId in itemIds)
 {
      var content = Sitecore.GetContentItem(itemId);
      list.Add(content);
 }

Seems kind of inefficient to make all those calls back and forth from the database one a time per ID. Profiling with Visual Studio confirmed my suspicion, with it sometimes take a few seconds to resolve all the content items. I know good caching can somewhat mitigate the cost of this, but I'm wondering if there's a better way of doing this type of batch retrieval of content items.

If not, could parallelizing this loop possibly help? I'd imagine it would, since using something like TPL would allow you to dedicate cores for each content item retrieval, with something similar happening on the database side as well so there'll be parallelized queries as well.

As far as the nature of data interaction; the Sitecore data retrieval is for retrieving content only. The client code is meant to have read-only access to Sitecore data; it will not be updating the Sitecore data itself. As far as data location, scenarios of data retrieval can be in some cases just pulling the children of a content item; or pulling multiple content items from various locations in the content tree in the same content retrieval code block.

Thoughts?

Answer 1

When you need to retrieve a lot of content, your first port of call should be the ContentSearch API as it is orders of magnitude faster than just about any other way of querying content.

By default the sitecore_master_index and sitecore_web_index will index all fields, just make sure the crawler root location is set correctly in config. In your case you may need to add a new field that you can use to more easily filter out items you don't need.

Don't be too quick to create a new index if you don't need to as custom indexes are difficult to maintain - especially if you need to support both Solr and Azure search.