9

We are using SOLR 5.4 and Sitecore 8.2 and we have implemented Full Content Search using _Content field which is displaying search result in Company Pages and various datasources linked to these company pages.

I need to display the Company Page in Search result, which is referring to the datasources item.

Now the question is, how can we write/configure that kind of computed index field?

  • When you say "which is referring the datasources item" which datasource item you are referring,? do you need to check any specific component/rendering datasource? or something else? – Ankit Joshi Dec 13 '17 at 12:05
  • I need to display company page which is using datasource item since search result shows datasource item by default but I need to show company page which is using this datasource . I need to get link between datasource and company page which I am not able to get – user2332873 Dec 13 '17 at 12:15
  • As far as I understood, currently you have datasource items in search results and render an datasource item name. But you want show the company page item name which is referring the datasource item instead of show the name of datasource item. Right? – Artsem Prashkovich Dec 13 '17 at 12:31
  • Yes this is exactly what I want – user2332873 Dec 13 '17 at 13:22
13

From what I understand, you are looking for a way to get the datasources that your current page is referring to to be indexed along with said page. Several times now we have used the approach defined here:

https://kamsar.net/index.php/2014/05/indexing-subcontent/

Create a ComputedField Class

This chunk of code will trigger anytime an item is indexed and should push to your index whatever content is referred to as a datasource within your Renderings :

using System.Collections.Generic;
using System.Linq;
using Blade.Utility;
using Sitecore;
using Sitecore.ContentSearch;
using Sitecore.ContentSearch.ComputedFields;
using Sitecore.Data.Fields;
using Sitecore.Data.Items;
using Sitecore.Layouts;

namespace Foo.ContentSearch.ComputedFields
{
    /// <summary>
    /// Computed field that contains all textual content of items that are rendering data sources on the current item's layout details
    /// </summary>
    public class SubcontentField : IComputedIndexField
    {
        public object ComputeFieldValue(IIndexable indexable)
        {
            var sitecoreIndexable = indexable as SitecoreIndexableItem;

            if (sitecoreIndexable == null) return null;

            // find renderings with datasources set
            var customDataSources = ExtractRenderingDataSourceItems(sitecoreIndexable.Item);

            // extract text from data sources
            var contentToAdd = customDataSources.SelectMany(GetItemContent).ToList();

            if (contentToAdd.Count == 0) return null;

            return string.Join(" ", contentToAdd);
        }

        /// <summary>
        /// Finds all renderings on an item's layout details with valid custom data sources set and returns the data source items.
        /// </summary>
        protected virtual IEnumerable<Item> ExtractRenderingDataSourceItems(Item baseItem)
        {
            string currentLayoutXml = LayoutField.GetFieldValue(baseItem.Fields[FieldIDs.LayoutField]);
            if (string.IsNullOrEmpty(currentLayoutXml)) yield break;

            LayoutDefinition layout = LayoutDefinition.Parse(currentLayoutXml);

            // loop over devices in the rendering
            for (int deviceIndex = layout.Devices.Count - 1; deviceIndex >= 0; deviceIndex--)
            {
                var device = layout.Devices[deviceIndex] as DeviceDefinition;

                if (device == null) continue;

                // loop over renderings within the device
                for (int renderingIndex = device.Renderings.Count - 1; renderingIndex >= 0; renderingIndex--)
                {
                    var rendering = device.Renderings[renderingIndex] as RenderingDefinition;

                    if (rendering == null) continue;

                    // if the rendering has a custom data source, we resolve the data source item and place its text fields into the content to add
                    if (!string.IsNullOrWhiteSpace(rendering.Datasource))
                    {
                        // DataSourceHelper is a component of Blade
                        var dataSource = DataSourceHelper.ResolveDataSource(rendering.Datasource, baseItem);

                        if (dataSource != baseItem)
                        {
                            yield return dataSource;
                        }
                    }
                }
            }

        } 

        /// <summary>
        /// Extracts textual content from an item's fields
        /// </summary>
        protected virtual IEnumerable<string> GetItemContent(Item dataSource)
        {
            foreach (Field field in dataSource.Fields)
            {
                // this check is what Sitecore uses to determine if a field belongs in _content (see LuceneDocumentBuilder.AddField())
                if (!IndexOperationsHelper.IsTextField(new SitecoreItemDataField(field))) continue;

                string fieldValue = (field.Value ?? string.Empty).StripHtml();

                if (!string.IsNullOrWhiteSpace(fieldValue)) yield return fieldValue;
            }
        }

        public string FieldName { get; set; }
        public string ReturnType { get; set; }
    }
}

Update your configs

Add the reference to the ComputedFields class in your index config (please note the example below is referencing Lucene, you will likely have a similar structure with a different node indexConfiguration/defaultSolrIndexConfiguration would be my guess):

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
    <sitecore>
        <contentSearch>
            <indexConfigurations>
                <defaultLuceneIndexConfiguration>
                    <fields hint="raw:AddComputedIndexField">
                        <!-- indexes subcontent contents into parent's _content field in the index (for better site search) -->
                        <field fieldName="_content" type="Foo.ContentSearch.ComputedFields.SubcontentField, Foo" />
                    </fields>
                </defaultLuceneIndexConfiguration>
            </indexConfigurations>
        </contentSearch>
    </sitecore>
</configuration>

A Step further

On top of this approach, often times we have a Site/Data/SearchSettings item located in a global place with regards to the site itself (and in a multi-site environment, we would have several of these items). On this item we have a multi-list field that would point at all renderings that would be considered "modules" or "components" to a Page item. From there you can pick and choose which datasource/subcontent items get indexed and which don't (some items may be data centric and some may be html content). It's a pretty small addition (a settings item and some code in the computed field class) but it helps give content editors granularity into what is indexed:

ComputedField(IIndexable indexable) method:

...
        var sitecoreIndexable = indexable as SitecoreIndexableItem;

        if (sitecoreIndexable == null) return null;

        var sc = new SitecoreContext(Sitecore.Context.ContentDatabase);

        var searchSettings = sc.GetItem<Search_Settings>(searchSettingsItem.ID.ToString().ToUpper());
        if (searchSettings == null)
            return null;
        indexedRenderings = searchSettings.Indexed_Renderings.Select(i => new ID(i).ToString()).ToList();
        var customDataSources = ExtractRenderingDataSourceItems(item, indexedRenderings);
...

ExtractRendringDatasourceItems(Item baseItem, List<string> indexedRenderings) method (added parameter):

...
        if (rendering == null) continue;

        // only index valid renderings
        if (!indexedRenderings.Contains(rendering.RenderingID.ToString().ToUpper())) continue;

        // if the rendering has a custom data source, we resolve the data source item and place its text fields into the content to add
        if (!string.IsNullOrWhiteSpace(rendering.Datasource))

...

Dependencies

Lastly, since your datasources will likely change, it won't necessarily get the change in the index since the parent doesn't change. You will want to make the subcontent/datasource change trigger a bubble-up index (http://www.techphoria414.com/Blog/2013/November/Sitecore-7-Computed-Fields-All-Templates-and-Datasource-Content).

Create GetDatasourceDependencies pipeline

This will help gather the referrers and flag them as dependencies:

public class GetDatasourceDependencies : BaseProcessor
{
    public override void Process(GetDependenciesArgs context)
    {
        Func<ItemUri, bool> func = null;
        Assert.IsNotNull(context.IndexedItem, "indexed item");
        Assert.IsNotNull(context.Dependencies, "dependencies");
        Item item = (Item)(context.IndexedItem as SitecoreIndexableItem);
        if (item != null)
        {
            if (func == null)
            {
                func = uri => (bool)((uri != null) && ((bool)(uri != item.Uri)));
            }
            System.Collections.Generic.IEnumerable<ItemUri> source = Enumerable.Where<ItemUri>(from l in Globals.LinkDatabase.GetReferrers(item, FieldIDs.LayoutField) select l.GetSourceItem().Uri, func).Distinct<ItemUri>();
            context.Dependencies.AddRange(source.Select(x => (SitecoreItemUniqueId)x));
        }
    }
}

Add the pipeline to your config:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <indexing.getDependencies help="Processors should derive from Sitecore.ContentSearch.Pipelines.GetDependencies.BaseProcessor">
        <!-- When indexing an item, make sure any items for whom it is a datasource get re-indexed as well -->
        <processor type="Foo.ContentSearch.GetDatasourceDependencies, Foo"/>
      </indexing.getDependencies>
    </pipelines>
  </sitecore>
</configuration>

Hopefully this helps!

4

Similar to the answer provided by @vandsh, but with less dependencies I created this gist, which is outlined below.

Note: This method assumes all datasources are referenced by ID. It relies on the links database to find linked items, which would fail if relying on a query.

Define Which Renderings are "indexable"

This avoids indexing renderings such as Navigation or Related Articles, which really aren't pertinent to the actual page content.

You'll want to update this list as you build your module library with relevant text-heavy renderings.

Config Patch

<configuration>
    <sitecore>
        <indexedRenderings>
            <indexedRendering name="RichText" id="{4F8942FF-C4E4-408A-B3D5-4657EE5050CA}" />
            <indexedRendering name="EmployeeBio" id="{53F8166B-6627-4C1B-BED5-FBD0B36EC527}" />
            <indexedRendering name="HighlightStatic" id="{75100A87-B7B0-4D1F-BE85-1A8FC6342514}" />
            <indexedRendering name="IntroText" id="{89D7EE42-A5E1-4F24-B247-BFD382E501D4}" />
        </indexedRenderings>
    </sitecore>
</configuration>

Class To Read Config

using System.Collections.Generic;
using System.Linq;
using System.Xml;
using Sitecore.Configuration;
using Sitecore.Data;
using Sitecore.Xml;

namespace Common.Sitecore.ConfigElements
{
    public class IndexedRenderings
    {
        public string Name { get; set; }
        public ID Id { get; set; }

        public static List<IndexedRenderings> IndexableRenderings()
        {
            return (from XmlNode node in Factory.GetConfigNodes("indexedRenderings/indexedRendering")
                select new IndexedRenderings
                {
                    Name = XmlUtil.GetAttribute("name", node),
                    Id = new ID(XmlUtil.GetAttribute("id", node))
                }).ToList();
        }
    }
}

Define Custom RenderedContent Field

This field will contain the filtered text of all datasources (allowed by config section) added to the page item. The contents is stored in the index as renderedContent. It checks both the Layout and FinalLayout of the item for renderings that are marked as allowed, then iterates on the fields of the datasource, ultimately piping everything to the same renderedContent string.

Note: You probably don't want to use this field for display on a search results page, for example, but can still be leveraged to provide relevant results.

Config Patch

<configuration>
    <sitecore>
        <contentSearch>
            <indexConfigurations>
                <defaultLuceneIndexConfiguration> <!-- swap for your configuration, e.g. SOLR!!! -->
                    <documentOptions>
                        <fields hint="raw:AddComputedIndexField">
                            <field fieldName="renderedContent" storageType="no" indexType="tokenized">Common.Sitecore.Search.ComputedFields.RenderedContent,Common</field>
                        </fields>
                    </documentOptions>
                </defaultLuceneIndexConfiguration>
            </indexConfigurations>
        </contentSearch>
    </sitecore>
</configuration>

Custom Field Class

using System.Collections.Generic;
using System.Linq;
using System.Text.RegularExpressions;
using Sitecore;
using Sitecore.ContentSearch;
using Sitecore.ContentSearch.ComputedFields;
using Sitecore.Data;
using Sitecore.Data.Fields;
using Sitecore.Data.Items;
using Sitecore.WordOCX.HtmlDocument;
using Common.Sitecore.ConfigElements;

namespace Common.Sitecore.Search.ComputedFields
{
    public class RenderedContent : AbstractComputedIndexField
    {
        private IEnumerable<ID> _indexableRenderings;

        public override object ComputeFieldValue(IIndexable indexable)
        {
            var item = (Item)(indexable as SitecoreIndexableItem);

            if (item == null)
                return null;

            _indexableRenderings = IndexedRenderings.IndexableRenderings().Select(i => i.Id);

            if (!_indexableRenderings.Any())
                return null;

            var deviceItem = item.Database.Resources.Devices.GetAll().FirstOrDefault(d => d.IsDefault);

            if (deviceItem == null)
                return null;

            var itemDatasources = ExtractRenderingDataSourceItems(item, _indexableRenderings, item.Fields[FieldIDs.FinalLayoutField], deviceItem).ToList(); // for later `Distinct()`
            itemDatasources.AddRange(ExtractRenderingDataSourceItems(item, _indexableRenderings, item.Fields[FieldIDs.LayoutField], deviceItem));

            itemDatasources = itemDatasources.Distinct().ToList();

            if (!itemDatasources.Any())
                return null;

            var content = new List<string>();

            foreach (var itemDatasource in itemDatasources)
                GetItemContent(itemDatasource, content);

            if (!content.Any())
                return null;

            var aggregateContent = content.Aggregate((a, b) => $"{a} {b}");

            aggregateContent = StripLineBreaksAndWhiteSpace(aggregateContent);

            return aggregateContent;
        }

        private static IEnumerable<Item> ExtractRenderingDataSourceItems(Item item, IEnumerable<ID> indexableRenderings, Field layoutField, DeviceItem deviceItem)
        {
            if (layoutField == null)
                return Enumerable.Empty<Item>();

            var layoutXml = LayoutField.GetFieldValue(layoutField);

            if (string.IsNullOrEmpty(layoutXml))
                return Enumerable.Empty<Item>();

            var renderings = item.Visualization.GetRenderings(deviceItem, false);

            if (renderings == null || renderings.Length == 0)
                return Enumerable.Empty<Item>();

            var datasourceItems = new List<Item>();

            for (var renderingIndex = renderings.Length - 1; renderingIndex >= 0; renderingIndex--)
            {
                var rendering = renderings[renderingIndex];

                if (rendering == null || rendering.Database != item.Database)
                    continue;

                if (!indexableRenderings.Contains(rendering.RenderingID))
                    continue;

                var datasourceId = rendering.Settings.DataSource;

                if (!ID.IsID(datasourceId))
                    continue;

                var datasourceItem = item.Database.GetItem(new ID(datasourceId));

                if(datasourceItem == null)
                    continue;

                datasourceItems.Add(datasourceItem);
            }

            return datasourceItems;
        }

        /// <summary>
        /// Extracts textual content from an item's fields
        /// </summary>
        private static void GetItemContent(Item dataSource, ICollection<string> content)
        {
            //.Where(field => !field.Name.StartsWith("__")
            foreach (Field field in dataSource.Fields)
            {
                if (!IndexOperationsHelper.IsTextField(new SitecoreItemDataField(field)))
                    continue;

                var fieldValue = StripHtmlTags(field.Value ?? string.Empty);

                if (!string.IsNullOrWhiteSpace(fieldValue))
                    content.Add(fieldValue);
            }

            foreach (Item child in dataSource.Children)
            {
                GetItemContent(child, content);
            }
        }

        private static string StripHtmlTags(string source)
        {
            if (source == null)
                return null;

            var doc = new HtmlDocument();
            doc.LoadHtml(source);
            return doc.DocumentNode.InnerText;
        }

        private static string StripLineBreaksAndWhiteSpace(string source)
        {
            if(string.IsNullOrWhiteSpace(source))
                return null;

            source = source.Replace(System.Environment.NewLine, "");

            source = Regex.Replace(source, @"\s+", " ");

            return source.Trim(' ');
        }
    }
}

Tell Sitecore to Update Indexed Page Content if a Datasource is Changed

Since a content author will (not might) update a datasource and not update the associated page, the below changes instruct Sitecore to find any pages that may be relying on the datasource and to reindex them. If relying on a shared datasource, all pages relying on that datasource will update.

Config Patch

<configuration>
    <sitecore>
        <pipelines>
            <indexing.getDependencies>
                <processor type="Common.Sitecore.Search.GetDependencies.GetDatasourceDependencies, Common"/>
            </indexing.getDependencies>
        </pipelines>
    </sitecore>
</configuration>

GetDependencies Class

using System.Linq;
using Sitecore;
using Sitecore.ContentSearch;
using Sitecore.ContentSearch.Pipelines.GetDependencies;
using Sitecore.Data;
using Sitecore.Data.Items;
using Sitecore.Diagnostics;

namespace Common.Sitecore.Search.GetDependencies
{
    public class GetDatasourceDependencies : BaseProcessor
    {
        public override void Process(GetDependenciesArgs context)
        {
            Assert.IsNotNull(context.IndexedItem, "indexed item");
            Assert.IsNotNull(context.Dependencies, "dependencies");

            var item = (Item)(context.IndexedItem as SitecoreIndexableItem);

            if (item == null)
                return;

            bool Func(ItemUri uri) => (uri != null) && uri != item.Uri;

            var source = (
                from l 
                    in Globals.LinkDatabase.GetReferrers(item, FieldIDs.LayoutField)
                select l.GetSourceItem().Uri)
                    .Where(Func)
                    .ToList();

            source.AddRange(
            (
                from l
                    in Globals.LinkDatabase.GetReferrers(item, FieldIDs.FinalLayoutField)
                select l.GetSourceItem().Uri)
                    .Where(Func)
            );

            source = source.Distinct().ToList();

            context.Dependencies.AddRange(source.Select(x => (SitecoreItemUniqueId)x));
        }
    }
}

At this point you can write queries against this field as you typically would. Example of a custom search object with the field implemented:

using System.Runtime.Serialization;
using Sitecore.ContentSearch;
using Sitecore.ContentSearch.SearchTypes;

namespace Common.Sitecore.Seach
{
    public class CustomSearchResultItem : SearchResultItem
    {
        [DataMember]
        [IndexField("renderedContent")]
        public string RenderedContent { get; set; }
    }
}
  • Storing the template IDs in a single config node, will that assume the same renderings will be indexed for all sites in the event that this was in a multi site environment? You could probably make this a property on the site config to make it site specific and eliminate the need for the Class to read config all together. – vandsh Jan 13 '18 at 0:23
  • 1
    Absolutely! Refactor away! – jrap Jan 13 '18 at 0:42
  • I am getting issue in GetDatasourceDependencies issue is that this works fine for English language but when ever i publish content for non English version it's still taking dependency in English version but dependency is actually in non-English version – user2332873 Feb 23 '18 at 11:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.