19

In only our Staging/Production environment, items are being duplicated in our web index after a publish is performed. The only distinction between the duplicate items (via Luke) is that any indexed images are prefixed with "/sitecore/shell"

Duplicate content

If I perform a manual index rebuild from Indexing Manager, the results are as expected.

The index uses a copy of the default IndexConfiguration with some slight modifications (edited), renamed to be newsIndexConfiguration and it is referenced properly on the index implementation

<index id="$(News.Index.Name.Web)" type="Sitecore.ContentSearch.LuceneProvider.LuceneIndex, Sitecore.ContentSearch.LuceneProvider">
    <param desc="name">$(id)</param>
    <param desc="folder">$(id)</param>
    <!-- This initializes index property store. Id has to be set to the index id -->
    <param desc="propertyStore" ref="contentSearch/indexConfigurations/databasePropertyStore" param1="$(id)" />
    <configuration ref="contentSearch/indexConfigurations/newsIndexConfiguration" />
    <strategies hint="list:AddStrategy">
        <strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/onPublishEndAsync" />
    </strategies>
    <commitPolicyExecutor type="Sitecore.ContentSearch.CommitPolicyExecutor, Sitecore.ContentSearch">
        <policies hint="list:AddCommitPolicy">
            <policy type="Sitecore.ContentSearch.TimeIntervalCommitPolicy, Sitecore.ContentSearch" />
        </policies>
    </commitPolicyExecutor>
    <locations hint="list:AddCrawler">
        <crawler type="Sitecore.ContentSearch.SitecoreItemCrawler, Sitecore.ContentSearch">
            <Database>web</Database>
            <Root>/sitecore/content/Global/News/Articles</Root>
        </crawler>
    </locations>
</index>

It goes without saying that there is custom code all over this implementation at this point. However, this is a new feature that started with the default configuration. I cannot figure out what might be causing the duplicates to be added on publish.

It was erring without the <indexAllFields> element set to true, which I can live with, but certainly isn't ideal.

I should note, when I perform a Republish (including all children) at the newsroom root, it adds these multiple values, not just 1 additional duplicate, but 5 or even more sometimes, it feels quite arbitrary.

In the past, in this instance, I've gotten around this by filtering out results that contain "/sitecore/shell" in the image field, but this is merely a Band-Aid. Any help to solve this would be greatly appreciated.

  • Version 8.0 150621 – jrap Oct 11 '16 at 19:57
12

This looks like a bug with the OnPublishEndAsync strategy.

It becomes apparent when authors start creating new versions of content each time they lock and edit an item.

It has been fixed in 8.1 but still resides in the version you are using.

See this knowledge base article

https://kb.sitecore.net/articles/992608

  • 1
    As the KB article from Ian mentions, this is also corrected in 8.0 Update 6, so you don't need to do an upgrade to 8.1 to get the fix. – Jason St-Cyr Oct 12 '16 at 2:40
  • I was extremely hopeful that this was going to solve my particular issue, but unfortunately it did not. I will look into it further and report back. Thanks for the answer though Ian! – jrap Oct 12 '16 at 14:43
  • 1
    I was able to reproduce my exact error with a fresh install, with the patch included. I have reached out to Sitecore Support for further troubleshooting help. Will post solution once we get it. – jrap Oct 12 '16 at 19:10
10

The issue was that in my efforts to trim the index of waste, i.e. remove fields that I'm not accessing, I removed a certain special field (though it is not marked in any way as being important, though by name it makes sense in hindsight). I removed the "_uniqueid" field from the index configuration. Specifically here:

<fieldMap>
...
  <field fieldName="_uniqueid"            storageType="YES" indexType="TOKENIZED"    vectorType="NO" boost="1f" type="System.String" settingType="Sitecore.ContentSearch.LuceneProvider.LuceneSearchFieldConfiguration, Sitecore.ContentSearch.LuceneProvider">
    <analyzer type="Sitecore.ContentSearch.LuceneProvider.Analyzers.LowerCaseKeywordAnalyzer, Sitecore.ContentSearch.LuceneProvider" />
  </field>
...

I had created a new IndexConfiguration.config file unique to my index. I tested and I was able to remove all other fields from the index. Toggling this field on or off reproduced the issue.

EDIT:

With the _uniqueid field omitted from the index configuration, the field will still appear in the index (via Luke or other means). Including it in the index configuration will solve this problem.

3

It may depend on what version of Sitecore you are running, from this SO post (https://stackoverflow.com/questions/27272832/urllink-in-sitecore-indexs-returns-media-url-with-sitecore-shell-media) it looks like there may be a known bug.

Accepted answer from link above:

Just to give an update for this. Confirmed by Sitecore it's a bug. They are finding a workaround, I'll update here when they do.

Thanks.

-------------updated on 15/12/2014----------------------

Ticket's closed now. Sitecore's solution is to remove this "urlLink" field from the index in the future(They said they will request this).

Reason being that url should be generated by LinkManager based on current site context. However site context doesn't exist while indexing(For content item you can check the path, but for media item you cannot).

Of course there's another solution is to create your own computeredField to override the logic for media item. However I agree with Sitecore, it feels more right to remove this field from the index.

On top of all that, the initial reason for using the urlLink field was to drop the need for querying Sitecore completely while doing a Search. All the content are coming from Index's stored fields value. But on the other hand, pagination is normally being used for search results, so even there're requests to Sitecore, it shouldn't be a lot.

That'd be all for now, happy to see more opinions and to discuss, thanks!

  • 1
    stackoverflow.com/help/how-to-answer I would recommend reading this. It's worth providing further context from links provided if you can. Rather then just linking to them. In this case using the quote option, quote the accepted answer. – Anicho Oct 11 '16 at 19:59
  • I don't think the answer you linked to is the same problem or the right fix. – Anicho Oct 11 '16 at 20:02
  • 2
    Thanks for editing @Anicho, should have provided context/quote – vandsh Oct 11 '16 at 20:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.