5

In our Sitecore site we have a few 'English' languages:

  • en
  • en-GB
  • en-US

We wanted to do a search using the Lucene index (a custom one, but that doesn't actually matter) and included the language in it to get only results for the current language. But when we search on "en" we also get results for the other "en" based languages. When we search on "en-gb" we get nothing.

We tried Luke and got the same results. So I assumed the field would be tokenized and indeed, in the default Lucene config I found:

<field fieldName="_language" storageType="YES" indexType="TOKENIZED" vectorType="NO" boost="1f" type="System.String" settingType="Sitecore.ContentSearch.LuceneProvider.LuceneSearchFieldConfiguration, Sitecore.ContentSearch.LuceneProvider">
  <analyzer type="Sitecore.ContentSearch.LuceneProvider.Analyzers.LowerCaseKeywordAnalyzer, Sitecore.ContentSearch.LuceneProvider" />
</field>

This explained the behavior, but now I am wondering if we can search on language (when having multiple 'English' languages) without adding a custom field for this?

Improve this question
2
  • Why o why would they make this field tokenized? I would rebel and change it to UNTOKENIZED if I were you. See if anything breaks.
    – Mark Cassidy
    Oct 3, 2016 at 8:59
  • That was the first idea, but indeed wondering if that would break stuff as this is the standard config... I was hoping someone would have seen and fixed this before.
    – Gatogordo
    Oct 3, 2016 at 9:02

2 Answers 2

Reset to default
3

Apparently the fact that we were using a custom index was an important factor after all. Sitecore Support gave us a solution:

It seems that if you have a custom index with its own configuration the language is stored in the index in a short format. We had to include the _language field as defined in the standard config in the fieldmap of our custom index. So we add this to the custom fieldmap:

<field fieldName="_language" storageType="YES" indexType="TOKENIZED" vectorType="NO" boost="1f" type="System.String" settingType="Sitecore.ContentSearch.LuceneProvider.LuceneSearchFieldConfiguration, Sitecore.ContentSearch.LuceneProvider">
  <analyzer type="Sitecore.ContentSearch.LuceneProvider.Analyzers.LowerCaseKeywordAnalyzer, Sitecore.ContentSearch.LuceneProvider" />
</field>

It seemed weird that this fixes our issue, but it does. Somehow I expected the standard Sitecore fields in the custom index to come from the default configuration but apparently they do not.

I also checked the search log files (should have done that sooner) and now the queries show full languages instead of short ones as before. And we get the correct results.

Improve this answer
2
  • 2
    I sometimes suspect; someone somewhere does not understand Lucene :P Short ID; short format language names... Seems to me like jumping through hoops, as opposed to just not tokenizing content that should not be tokenized. /rant :P
    – Mark Cassidy
    Oct 3, 2016 at 15:40
  • Just a note: While the field is TOKENIZED it uses Sitecore's LowerCaseKeywordAnalyzer which just like untokenized but case-insensitive (while true untokenized is case-sensitive).
    – Søren Kruse
    Feb 3, 2017 at 8:33
2

Well the obvious non-breaking alternative would be, to just add a sister field definition.

<field fieldName="_searchlanguage" storageType="YES" indexType="UNTOKENIZED" vectorType="NO" boost="1f" type="System.String" settingType="Sitecore.ContentSearch.LuceneProvider.LuceneSearchFieldConfiguration, Sitecore.ContentSearch.LuceneProvider">
  <analyzer type="Sitecore.ContentSearch.LuceneProvider.Analyzers.LowerCaseKeywordAnalyzer, Sitecore.ContentSearch.LuceneProvider" />
</field>

But as mentioned in my comment above; I don't see what use a TOKENIZED language field could be. Not to Sitecore, or anyone for that matter.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.