Solr - Recommended way to search by filters

Question

I'm trying to build a generic Search method GetSearchResults that would fetch results from Solr.

Sample use Case: Get a list of all products where the searchTerm can be in the Title OR Description field AND other facet filters like Color, brand etc.

What would be the recommended approach among these:

1. First get all products from Solr and then filter by criteria in the caller method.
2. Build the query predicate in the caller method & pass it as an argument to the Solr method.

If there is any other better approach, please let me know.

Approach 1: Get all products from Solr and then filter by criteria in the caller method.

public List<T> GetSearchResults<T>(string searchIndexName, List<string> templateNames, string languageIsoCode) where T : SearchResultItem
        {
            ISearchIndex searchIndex = ContentSearchManager.GetIndex(searchIndexName);

            using (IProviderSearchContext context = searchIndex.CreateSearchContext())
            {
                var query = PredicateBuilder.True<T>();
                
                if (templateNames != null && templateNames.Count > 0)
                {
                  foreach(string templateName in templateNames)
                  {
                    query = query.Or(x => x.TemplateName.Equals(templateName, StringComparison.InvariantCultureIgnoreCase));
                  }                        
                }

                if (!string.IsNullOrWhiteSpace(languageIsoCode))
                {
                  query = query.And(x => x.Language.Equals(languageIsoCode, StringComparison.InvariantCultureIgnoreCase)); 
                }

               IQueryable<T> results = context.GetQueryable<T>().Where(query);
               return results != null ? results.ToList() : null;               
            }
        }

public void CallerMethod(string searchTerm, List<KeyValuePair<string, string>> facets)
{
   List<ProductSearchRelatedItem> results = GetSearchResults<ProductSearchRelatedItem >("custom_index", null, "en")
   foreach(ProductSearchRelatedItem psri in results)
   {
      /// process the filtering logic here
   }
}

Approach 2: Build the query predicate in the caller method & pass it as an argument to the Solr method.

public List<T> GetSearchResults<T>(string searchIndexName, Expression<Func<T, bool>> query) where T : SearchResultItem
        {
            ISearchIndex searchIndex = ContentSearchManager.GetIndex(searchIndexName);

            using (IProviderSearchContext context = searchIndex.CreateSearchContext())
            {
               IQueryable<T> results = context.GetQueryable<T>().Where(query);
               return results != null ? results.ToList() : null;               
            }
        }

public void CallerMethod(string searchTerm, List<KeyValuePair<string, string>> facets)
{
   var query = PredicateBuilder.True<T>();
   /// build all the query AND/OR logic here
   ......
   List<ProductSearchRelatedItem> results = GetSearchResults<ProductSearchRelatedItem >("custom_index", query)       
}

Answer 1

In the scenario you've described, the recommended approach would be to build the query predicate in the caller method and pass it as an argument to the Solr method (Approach 2). This approach allows you to build a more targeted and efficient query by only fetching the results that match the search criteria, rather than fetching all products and then filtering them in the caller method.

Building the query predicate in the caller method also makes it easier to add or modify search criteria and filters, as you can simply update the query logic without having to change the implementation of the Solr method.

With Approach 2, you can use a library like PredicateBuilder to construct complex query predicates that combine multiple search criteria and filters using logical operators such as AND and OR. This can help make the query code more readable and maintainable.

In addition, you can also consider using Solr's built-in search features, such as faceting, to further refine the search results based on facets such as color, brand, or price range. This can help users to filter and navigate the search results more easily.

Overall, Approach 2 provides a more flexible and scalable approach to implementing a search functionality in your application, while minimizing the amount of data that needs to be fetched and processed.

Answer 2

The actual Solr query is more than 10x less than the total searching time we got in our application. There is something more than the searching itself that costs much more processing time during the search execution. In fact, most of our search time is spent on the result data being transferred, parsed and deserialized. That means that if we want to improve the search performance we need to carefully control what is transferred, parsed and deserialized during our search requests.

Here, it’s extremely important to limit your search result set during the querying but not getting all of the results and manipulate them after that using Linq to Object.

The predicate is the filter part of your expression tree that tells the system if a specific object has matched your predicate criteria.

So the best approach is to build the query predicates first and pass it to your solr query.

Here is a useful link that you may refer to know about the Solr search performance guide.

https://www.adapty.com/blog/solr-search-implementation-and-performance-improvement-guide-on-sitecore-based-website/

Hope this make sense.

Answer 3

When you need to build up more complex queries, or even dynamic queries, that require more work, like in your case, a list of all products where the searchTerm can be in the Title OR Description field AND other facet filters like Color, brand, etc. I would say only go with it Approach 2

You can also try out Filter() method. using the Filter() method instead of Where() on the IQueryable instance. You could use Where(), but the benefit of using the Filter() is that it doesn't calculate relevancy, so if you are doing any other queries, the scoring will not be affected. Furthermore, filters can be cached to optimize search performance.

Credit goes to Soren. For more details here: https://soen.ghost.io/a-re-introduction-to-the-contentsearch-api-in-sitecore-part-2/

Hope it helps!