OrderBy not being respected for item field when using content search

Question

Using Lucene, and content search I cannot sort items by their name nor custom title field.

The items are imported from a read-only data provider and are successfully indexed (I've inspected it manually)

The results I get back are semi-in-order but not but incoherent. Without the orderby clause, the items are more in-order than without it which is unusual.

Here's the simplified code:

ProviderSearchContext
  .GetQueryable<MyCustomSearchResultItem>()
  .Filter(m => m.TemplateId == "{SomeGuid}") 
  .OrderBy(m => m.Title) 
  .Skip(20 * (Page - 1))
  .Take(20)   
  .GetResults() 

public class MyCustomSearchResultItem: SearchResultItem {
        [IndexField("title")]
        public string Title { get; set; }
}

Answer 1

Lucene uses tokenization for the fields. Since most string fields have spaces so it will not work with OrderBy without proper configuration.

In the Lucene configuration file's fieldNames block section, you will need to make sure the field you are using for "ordering by" is specified. In addition, an analyzer is needed to ensure it is added to the index documents correctly. You can specify a custom analyzer as a child of the field element in the fieldMap section. it will look like this -

<fieldNames hint="raw:AddFieldByFieldName">
   <field fieldName="sortingField" storageType="YES" indexType="UN_TOKENIZED"    vectorType="NO" boost="1f" type="System.String" settingType="Sitecore.ContentSearch.LuceneProvider.LuceneSearchFieldConfiguration, Sitecore.ContentSearch.LuceneProvider">
       <analyzer type="Sitecore.ContentSearch.LuceneProvider.Analyzers.LowerCaseKeywordAnalyzer, Sitecore.ContentSearch.LuceneProvider" />
   </field>
   ...
</fields>

the indexType value should be “UN_TOKENIZED”, to make sure OrderBy clause to use the entire string for ordering and not the tokens.

Answer 2

It appears that Sitecore automatically tokenizes the field I'm interested in.

Found inside Sitecore.ContentSearch.Lucene.DefaultIndexConfiguration.config:

<field fieldName="title" storageType="NO" indexType="TOKENIZED" vectorType="NO" boost="1f" type="System.String" settingType="Sitecore.ContentSearch.LuceneProvider.LuceneSearchFieldConfiguration, Sitecore.ContentSearch.LuceneProvider" />

This in tern causes the field to be broken up into multidimensional commponents rather than a single field to be sorted on.

More about this can be found here: https://community.sitecore.net/technical_blogs/b/sitecore_7_development_team/posts/sorting-and-ordering-results-in-sitecore-7