8

We have a client site , with SOLR for searching products . The Code base for returning the search results is shown below. The products searched are based on search term entered by user

Code

var query = context.GetQueryable<SearchResultItem>();
                    query = query.AddContentSearchQuery(this._searchTerm);
                    query = query.Filter(predicate);

AddContentSearchQuery Matches the search text

public static IQueryable<SearchResultItem> AddContentSearchQuery(this IQueryable<SearchResultItem> query, string searchTerms)
        {
            if (string.IsNullOrEmpty(searchTerms))
                return query;
            searchTerms = searchTerms.Trim(' ').Replace(' ', '@');

            return query.Where(i => i.Content == searchTerms);
        }

here i.Content is from Sitecore.ContentSearch.SearchTypes.SearchResultItem

[DataMember]
**[IndexField("_content")]**
public virtual string Content { get; set; }

Can you please help me in understanding the analyzer used here ( Does it tokenize the search content)

This is the entry from Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config

<field fieldName="_content"                       returnType="string"     type="Sitecore.ContentSearch.ComputedFields.MediaItemContentExtractor,Sitecore.ContentSearch">
              <mediaIndexing ref="contentSearch/indexConfigurations/defaultSolrIndexConfiguration/mediaIndexing"/>
            </field>

What I understand is the _content field indexes the full content of the website. but is it also Tokenized ?

  • Is there a specific issue you're facing? – Paul George Feb 17 '17 at 12:57
  • Well I basiclly wanted to understand the SOLR code .. the code which does this -- I cant seem to wrap my head around the content field query.Where(i => i.Content == searchTerms); – Abhay Dhar Feb 17 '17 at 16:19
6

If you look at the _content field definition in the Solr schema.xml file you see this:

<field name="_content" type="text_general" indexed="true" stored="false" />

The field is indexed and has a type of text_general

Here is the definition for the text_general field type:

<fieldType name="text_general" class="solr.TextField" positionIncrementGap="100">
  <analyzer type="index">
    <tokenizer class="solr.StandardTokenizerFactory" />
    <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" />
    <!-- in this example, we will only use synonyms at query time
    <filter class="solr.SynonymFilterFactory" synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
    -->
    <filter class="solr.LowerCaseFilterFactory" />
  </analyzer>
  <analyzer type="query">
    <tokenizer class="solr.StandardTokenizerFactory" />
    <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" />
    <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true" />
    <filter class="solr.LowerCaseFilterFactory" />
  </analyzer>
</fieldType>

So it is tokenized using the solr.StandardTokenizerFactory

Hope that helps

  • Yes thanks .. What I could understand was the _content field.. Sorry I am a beginner in SOLR search – Abhay Dhar Feb 17 '17 at 16:20
  • 1
    Don't ever be sorry for being a beginner at something - we all are beginners at some point! – Richard Seal Feb 17 '17 at 16:23
3

If you're searching against 'products' you would perhaps be better off search against specific fields rather than the generic 'content' field?

I realise I'm answering a question that hasn't explicitly been asked, but it may be helpful anyway.

If searching multiple words in an 'AND' fashion that allows the user to reduce the number of results by adding words, you may want to filter out 'stop words', otherwise you will always receive zero results if the search contains one because your query says it must be present and it is ignored by the tokenizer even when present. You mentioned you're using SOLR but for Lucene these are found at Lucene.Net.Analysis.Standard.StandardAnalyzer.STOP_WORDS_SET. Stop words would be removed form your terms before querying.

The sample below does this and also uses LIKE instead of equals to allow for minor typos and additionally boosts results where the term is the same as the item name.

var terms = searchTerms.Split(new[] { ' ' }, StringSplitOptions.RemoveEmptyEntries).ToList();
terms = terms.Except(Lucene.Net.Analysis.Standard.StandardAnalyzer.STOP_WORDS_SET, StringComparer.OrdinalIgnoreCase).ToList();

var termsPredicate = PredicateBuilder.True<SearchResultItem>();

foreach (var term in terms)
{
    // AND = must contain every word
    termsPredicate = termsPredicate.And(  
    PredicateBuilder.False<_Base_Article>() // OR = in either of these two fields
       .Or(p => p.Name.Like(word, 0.8f).Boost(2.0f)) // minimal likeness 0.8, boost direct name match
       .Or(p => p.Content.Like(word, 0.8f))
    );
}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.