1

I'm creating a search where users can filter/facet based on terms that may contain a space (e.g Temporary work) on Sitecore 8.1 with Lucene.

My first thought was to store the fields as untokenized; when I do this the field is stored in the index as expected (with uppercase letters e.g. Temporary work), but when sitecore creates the lucene query from the Linq provider it lowers the search term which means I never get a match.

To get around this casing issue I figured the field should be analyzed, but with an analyzer that doesn't tokenize based on whitespace. I read that "KeywordAnalyzer - 'Tokenizes' the entire stream as a single token. This is useful for data like zip codes, ids, and some product names" so I created this in my index configuration:

<field fieldName="Job Type" type="System.String" storageType="YES" indexType="TOKENIZED" boost="1f" settingType="Sitecore.ContentSearch.LuceneProvider.LuceneSearchFieldConfiguration, Sitecore.ContentSearch.LuceneProvider">
   <analyzer type="Lucene.Net.Analysis.KeywordAnalyzer, Lucene.Net" />
</field>

I expected to see the term 'Temporary work' in the index but when I look in Luke I get the below where temporary and work have been split:

enter image description here

What am I misunderstanding here.

3
  • Fundamentally, this is the same problem - stackoverflow.com/questions/9431114/… - Sitecore rewrites your query to be all lowercase. I believe it to be a bug, albeit a very old one (that post if from 2012). Your best bet is to a) write your own analyzer that lowercases content or b) rewrite the query so it does not pass through Sitecore's LowerCaseRewriteStrategy. I will post a better answer, when I have a bit more time on my hands.
    – Mark Cassidy
    Apr 11 '18 at 8:34
  • Did you try with the Sitecore.ContentSearch.LuceneProvider.Analyzers.LowerCaseKeywordAnalyzer? This one should convert to lowercase and keep a single token - used this one already and it worked..
    – Gatogordo
    Apr 11 '18 at 9:27
  • @Gatogordo Using that analyzer does seem to make the query work (it returns only temporary work items), but when I use FacetOn(r => r.Fields["job_type"]) it returns 'temporary' and 'work' as separate facet values rather than one 'temporary work' facet value.
    – SamMullins
    Apr 11 '18 at 9:46
0

First of all: change your analyzer to Sitecore.ContentSearch.LuceneProvider.Analyzers.LowerCaseKeywordAnalyzer. You can find a lot of information an Sitecore analyzers in this blog from Adam Conn.

The LowerCaseKeywordAnalyzer has the KeywordTokenizer and LowerCaseFilter. The KeywordTokenizer emits the entire input as a single token and the LowerCaseFilter normalizes token text to lower case. This should fix the querying.

But.. you do have a second issue as well. The name of your field has a space in it. As you can read on this kb article the Lucene search provider has issues with Sitecore fields that have spaces in the names.

The following issues may occur when using the Lucene search provider and dealing with Sitecore fields that have whitespaces in their names:

  1. The value of the field may not be stored in the index even though the storage type of the field is set to YES in the index configuration.
  2. The analyzer associated with the field may be ignored both during indexing and searching. The default analyzer is used instead.

So, rename your field or replace whitespaces with the underscores in the field name:

<field fieldName="job_type" storageType="YES" indexType="TOKENIZED" vectorType="NO" boost="1f" type="System.String" settingType="Sitecore.ContentSearch.LuceneProvider.LuceneSearchFieldConfiguration, Sitecore.ContentSearch.LuceneProvider">
  <analyzer type="Sitecore.ContentSearch.LuceneProvider.Analyzers.LowerCaseKeywordAnalyzer, Sitecore.ContentSearch.LuceneProvider" />
</field>

Do not forget to reindex after making changes to the index configuration.

1
  • Just to clarify, you actually need two entries in the configuration if keeping the space in the field name. You need one field definition with fieldName="Name Of Field" and then you need to duplicate the field definition and set fieldName="name_of_field" on the duplicated one.
    – SamMullins
    Apr 11 '18 at 13:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.