Multiple partial word search not working

Question

I have implemented a global search on our website which gets the data result from _content field in solr indexes.

Single word search works perfectly fine but I am currently facing an issue if I have two or more words but the second word is partial. example

If I search "thrill" - search works fine.

If I search "thri" - search works fine and gives me the result of content which contains thril.

If I search "thrill rides" - search works fine

but if I search "thrill ri" or "thrill rid"- it does not return any result where I am expecting the result pages which contains "thrill rides"

It is the second partial string which does not work

I tried searching the same query inside the Sitecore content manager and it gives me the same result as on my website

So I am thinking that it is not something related in the code (below) but in the configuration.

      var startItem = sitecoreContext.GetHomeItem<Item>();
    using (var searchContext = ContentSearchManager.GetIndex(new SitecoreIndexableItem(startItem)).CreateSearchContext())
    {
        var filterPredicate = PredicateBuilder.True<SearchResultItem>();                    
        filterPredicate = filterPredicate.And(x => x.Content.Contains(queryItem)).Or(x => x.Name.Contains(queryItem));
        var query = searchContext.GetQueryable<SearchResultItem>().Filter(filterPredicate);
                    }

may be I need to add some configuration in

<analyzer type="index"> in schema.xml

Currently it has default settings

 <fieldType name="text_general" class="solr.TextField" positionIncrementGap="100">
      <analyzer type="index">
        <tokenizer class="solr.StandardTokenizerFactory" />
        <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" />       
        <filter class="solr.LowerCaseFilterFactory" />
      </analyzer>
      <analyzer type="query">
        <tokenizer class="solr.StandardTokenizerFactory" />
        <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" />
        <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true" />
        <filter class="solr.LowerCaseFilterFactory" />
      </analyzer>
    </fieldType>              

Can anyone please help me. Thanks

Answer 1

As it is mentioned in documentation, Solr does not support the wildcard query (this is a type of query to which .contains("phrase") will be converted) for search phrases:

Solr’s standard query parser supports single and multiple character wildcard searches within single terms. Wildcard characters can be applied to single terms, but not to search phrases. https://lucene.apache.org/solr/guide/6_6/the-standard-query-parser.html

When you try to search "thri", your query is parsed as <str name="parsed query">content:thri</str> and Solr searches for "thri" as a part of word or an entire word.

But when you try to search "thrill rid", your query is parsed as <str name="parsedquery">PhraseQuery(_content:"thrill rid")</str>. In this case, Solr searches of containing the "thrill" and "rid" just as entire words in the document and you don't get any results, because you don't have a document with "rid".

The easiest way to supporting multi word search with search by the part of word is to apply the code like below:

var querySplitted = queryItem.Split(' '); //Split the queryItem by the white space (or any other symbols if you need)

var predicate = PredicateBuilder.False<TIndexModel>();

var namePredicate = PredicateBuilder.True<TIndexModel>();
var contentPredicate = PredicateBuilder.True<TIndexModel>();

foreach (var query in querySplitted)
{

     namePredicate = namePredicate.And(item => item.Name.Contains(query).Boost(1.5f));
     contentPredicate = contentPredicate.And(item => item.Content.Contains(query));
}

predicate = predicate.Or(namePredicate).Or(contentPredicate);

var query = searchContext.GetQueryable<SearchResultItem>().Filter(predicate);

The code above works as you expect.

Another way is to change the tokenizer.

Currently, you use the StandardTokenizerFactory for both "index" and "query" analyzers. This tokenizer splits your "thrill rides" into two words: "thrill", "rides".

If you want to search by parts of a word, you need apply, for example, N-Gram Tokenizer, <tokenizer class="solr.NGramTokenizerFactory" minGramSize="3" maxGramSize="5"/>, which will devide your "thrill rides" phrase to the foolowing parts: thri thril hril hrill rill rill ill ill rid rid ride ride rides ides. In this case you will be able to search by the part of word. You need apply the NGramTokenizerFactory for both analyzers:

<fieldType name="text_general" class="solr.TextField" positionIncrementGap="100" multiValued="false">
    <analyzer type="index">
      <tokenizer class="solr.NGramTokenizerFactory" minGramSize="3" maxGramSize="5"/>
      <filter class="solr.StopFilterFactory" words="stopwords.txt" ignoreCase="true"/>
      <filter class="solr.LowerCaseFilterFactory"/>
    </analyzer>
    <analyzer type="query">
      <tokenizer class="solr.NGramTokenizerFactory" minGramSize="3" maxGramSize="5"/>
      <filter class="solr.StopFilterFactory" words="stopwords.txt" ignoreCase="true"/>
      <filter class="solr.SynonymFilterFactory" expand="true" ignoreCase="true" synonyms="synonyms.txt"/>
      <filter class="solr.LowerCaseFilterFactory"/>
    </analyzer>
  </fieldType>

In case of using NGramTokenizerFactory, you need to use == operator instead of .Contains(). For example: predicate.Or(item => item.Name == query).

In case you change the analyzer to NGramTokenizerFactory be careful as the size of index will increase and potentially can cause a performance issue. I recommend you think twice before changing of analyzer.