Sitecore HTML Indexing

Question

We have custom indexer which indexes all the RTF Data in present in content tree and this indexed RTF Data can have Div's ,breaker Tags and anchor tags.

When we get search result out from Solr we need to show only 150 Characters from this custom RTF field and this caused HTML to break at front end since Solr is returning 150 Characters inclusive of tags which is causing broken HTML at front end

We can not strip HTML while indexing because we need anchor tags to be shown in result

Any Solution to this guys

Answer 1

I wrote this extension method a few years back for almost this same use case. It truncates a string containing HTML to a given number of characters, not counting the HTML markup. It assumes that there will be a partial word at the end and removes it. It also optionally adds some trailing text, e.g. ellipsis or a read more link. It's not perfect, but it does the job pretty well and is much lighter weight than something like HtmlAgilityPack.

public static class StringHtmlExtensions
{
    /// <summary>
    ///     Truncates a string containing HTML to a number of text characters, keeping whole words.
    ///     The result contains HTML and any tags left open are closed.
    /// </summary>
    public static string TruncateHtml(this string html, int maxCharacters, string trailingText = null)
    {
        if (string.IsNullOrEmpty(html))
            return html;

        // find the spot to truncate
        // count the text characters and ignore tags
        var textCount = 0;
        var charCount = 0;
        var ignore = false;
        foreach (var c in html)
        {
            charCount++;
            if (c == '<')
                ignore = true;
            else if (!ignore)
                textCount++;

            if (c == '>')
                ignore = false;

            // stop once we hit the limit
            if (textCount >= maxCharacters)
                break;
        }

        // Truncate the html and keep whole words only
        var trunc = new StringBuilder(html.TruncateWords(charCount));

        // keep track of open tags and close any tags left open
        var tags = new Stack<string>();
        var matches = Regex.Matches(trunc.ToString(),
            @"<((?<tag>[^\s/>]+)|/(?<closeTag>[^\s>]+)).*?(?<selfClose>/)?\s*>",
            RegexOptions.IgnoreCase | RegexOptions.Compiled | RegexOptions.Multiline);

        foreach (Match match in matches)
        {
            if (match.Success)
            {
                var tag = match.Groups["tag"].Value;
                var closeTag = match.Groups["closeTag"].Value;

                // push to stack if open tag and ignore it if it is self-closing, i.e. <br />
                if (!string.IsNullOrEmpty(tag) && string.IsNullOrEmpty(match.Groups["selfClose"].Value))
                    tags.Push(tag);

                // pop from stack if close tag
                else if (!string.IsNullOrEmpty(closeTag))
                {
                    // pop the tag to close it.. find the matching opening tag
                    // ignore any unclosed tags
                    while (tags.Pop() != closeTag && tags.Count > 0)
                    {
                    }
                }
            }
        }

        if (html.Length > charCount)
            // add the trailing text
            trunc.Append(trailingText);

        // pop the rest off the stack to close remainder of tags
        while (tags.Count > 0)
        {
            trunc.Append("</");
            trunc.Append(tags.Pop());
            trunc.Append('>');
        }

        return trunc.ToString();
    }

    /// <summary>
    ///     Truncates text to a number of characters and adds trailing text, i.e. elipses, to the end
    /// </summary>
    public static string Truncate(this string text, int maxCharacters, string trailingText = null)
    {
        if (string.IsNullOrEmpty(text) || maxCharacters <= 0 || text.Length <= maxCharacters)
            return text;
        return text.Substring(0, maxCharacters) + trailingText;
    }

    /// <summary>
    ///     Truncates text and discars any partial words left at the end
    /// </summary>
    public static string TruncateWords(this string text, int maxCharacters, string trailingText = null)
    {
        if (string.IsNullOrEmpty(text) || maxCharacters <= 0 || text.Length <= maxCharacters)
            return text;

        // trunctate the text, then remove the partial word at the end
        return Regex.Replace(text.Truncate(maxCharacters),
            @"\s+[^\s]+$", string.Empty, RegexOptions.IgnoreCase | RegexOptions.Compiled) + trailingText;
    }
}

Answer 2

I would index the rtf content in two different fields. One field which is index and stored. Add the HTMLStripCharFilterFactory charfilter on this field and do you linguistic manipulation on that field. Do you fragment out ofnthis field.

The second is for fragemnt dataretrieval only when you need the full content of the field - not a fragment. It should be a simple string field with no manipulation