3

How do I go about configuring fields within a Solr index to be tokenized based on a semicolon ";" delimiter?

Initially, I had fields configured as such:

<field
   fieldName="example"
   storageType="YES"
   indexType="UN_TOKENIZED"
   vectorType="NO"
   boost="1f"
   returnType="text"
   settingType="Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration, Sitecore.ContentSearch.SolrProvider"/>

which causes the field to be tokenised on individual words.

Then I changed

returnType="string"

which causes the whole field to be treated as a single value.

I want it to tokenize using a semicolon delimiter. How do I configure the field to do this?

7

You can use solr.PatternTokenizerFactory tokenizer to achieve this

Define a new fieldtype in solr schema

<fieldType name="semicolonDelimited" class="solr.TextField">
  <analyzer>      
    <tokenizer class="solr.PatternTokenizerFactory" pattern=";" />
  </analyzer>
</fieldType> 

and then define the field which uses semicolonDelimited fieldtype

<field name="content_custom" type="semicolonDelimited"/>

Test Scenerio

  1. Create a sample solr core
  2. Define FieldType and Field as shown below
<fieldType name="semicolonDelimited" class="solr.TextField">
  <analyzer>      
    <tokenizer class="solr.PatternTokenizerFactory" pattern=";" />
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
</fieldType>  
<field name="content_custom" type="semicolonDelimited"/>
  1. Restart Solr Service
  2. Import test data to sample core

Sample JSON

{
  "id": 1,
  "content_custom": "Ten;10"
},
{
  "id": 2,
  "content_custom": "Eleven;11"
},
{
  "id": 3,
  "content_custom": "Twelve;12"
},
{
  "id": 4,
  "content_custom": "Thirteen;13"
}

Copy JSON and upload it to Solr enter image description here

  1. Now Search for content_custom:t*n and then it will show two results

enter image description here

| improve this answer | |
  • This is great. I didn't know you could do this. – Chris Auer Sep 23 at 21:09
  • @Surya Thanks for answering! Can that config live in a Sitecore config patch file, or does it need to go into one of the Solr XML configs? – Matthew Dresser Sep 23 at 21:36
  • 1
    @MatthewDresser The example I gave is for solr schema XML only. Once these updates are done in solr, you can define a field in sitecore configuration to point it to newly define datatype. – Surya Sep 24 at 9:21
  • @Surya where is the schema.xml file stored? I have Solr installed in 'C:\solr' but cannot see this file anywhere? Also there seems to be solr configs for each index, do I do this at the global solr level, or per index? – Matthew Dresser Sep 25 at 11:09
  • 1
    For old solr versions, file name used to be schema.xml but now file name is managed-schema (without extension) and you can find under C:\Solr\server\solr\<CORE Folder>\conf\ folder and this is not a global change but at core level. – Surya Sep 25 at 11:24
1

If you were to do it with a ComputedIndex field. it would be like this. But maybe you can do it with just solr configs.

using System.Collections.Generic;

namespace Sitecore.Foundation.Indexing.Infrastructure.Fields
{
    using Sitecore.ContentSearch;
    using Sitecore.ContentSearch.ComputedFields;
    using Sitecore.Foundation.SitecoreExtensions.Extensions;

    public class SemicolonContentComputedField : IComputedIndexField
    {
        public string FieldName { get; set; }
        public string ReturnType { get; set; }

        public object ComputeFieldValue(IIndexable indexable)
        {
            if (!(indexable is SitecoreIndexableItem indexItem))
            {
                return null;
            }

            var item = indexItem.Item;
            if (item == null) return null;

            if (item.IsDerived(Constants.Templates.PhpContent))
            {
                var results = new List<string>();
                var splits = item[Templates.PhpContent.Fields.SemiColonField].Split(';');
                foreach (var split in splits)
                {
                    results.Add(split);
                }

                return results;
            }

            return null;
        }
    }
}

Then the config would be stringcollection

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:set="http://www.sitecore.net/xmlconfig/set/">
    <sitecore>
        <contentSearch>
            <indexConfigurations>
                <defaultSolrIndexConfiguration type="Sitecore.ContentSearch.SolrProvider.SolrIndexConfiguration, Sitecore.ContentSearch.SolrProvider">
                    <fieldMap type="Sitecore.ContentSearch.SolrProvider.SolrFieldMap, Sitecore.ContentSearch.SolrProvider">
                        <fieldNames hint="raw:AddFieldByFieldName">
                            <field fieldName="semicolon" returnType="stringCollection" />
                        </fieldNames>
                    </fieldMap>
                    <documentOptions type="Sitecore.ContentSearch.SolrProvider.SolrDocumentBuilderOptions, Sitecore.ContentSearch.SolrProvider">
                        <fields hint="raw:AddComputedIndexField">
                            <field fieldName="semicolon" storageType="no" indexType="untokenized">Sitecore.Foundation.Indexing.Infrastructure.Fields.SemicolonContentComputedField, Sitecore.Foundation.Indexing</field>
                        </fields>
                    </documentOptions>
                </defaultSolrIndexConfiguration>
            </indexConfigurations>
        </contentSearch>
    </sitecore>
</configuration>
| improve this answer | |
  • I have set up my indexer almost identically to how you describe, but when I examine the solr documents I see "fieldName": ["Rim mounted;Exposed"] where I would expect to get ["Rim mounted", "Exposed"]. I attached the debugger and my code returns a List<string> with two elements from the ComputeFieldValue method. – Matthew Dresser Oct 2 at 15:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.