ReadSitecoreItemsStepProcessor and Buckets

Question

Using DEF 2.0.1 and Sitecore 9, when reading Sitecore items using ReadSitecoreItemsStepProcessor, it doesn't find items from the source specified by Item Root when the folder is a bucket e.g. in folder sitecore/content/Parks/Imported Data/Experiences:

But if the folder is converted to normal items, ReadSitecoreItemsStepProcessor finds these items. Has anyone tried this with buckets or have any suggestions?

Answer 1

The best option would be to create your own Sitecore.DataExchange.Providers.Sc.Processors.PipelineSteps.ResolveSitecoreItemStepProcessor class by decompiling (use DotPeek or ILSpy tools) Sitecore.DataExchange.Providers.Sc.dll and using this instead of OOTB Processor class.

There is a DoSearch method there.

You also need to create your own Repository class and your own repository.Search() method. Don't forget to override also repository class in tenant settings so your class is hit not OOTB one.

Buckets are searched differently so you need to use similar code to this one:

var bucketItem = Sitecore.Context.Database.GetItem(bucketPath);
  if (bucketItem != null && BucketManager.IsBucket(bucketItem))
  {     
     using (var searchContext = ContentSearchManager.GetIndex(bucketItem as IIndexable).CreateSearchContext())
    {
        var result = searchContext.GetQueryable<SearchResultItem().Where(x => x.Name == itemName).FirstOrDefault();
        if(result != null)
            Context.Item = result.GetItem();
    }
  }

This is also the reason why it is not working for you.

Answer 2

I encountered this issue and ended up creating my own custom step (inheriting from the original) to solve it:

Create a new pipeline step template

I called it Resolve Sitecore Item with Bucket Support Pipeline Step.

1. Add the Resolve Sitecore Item Pipeline Step template as a base template
   • Path: /sitecore/templates/Data Exchange/Providers/Sitecore/Pipeline Steps/Resolve Sitecore Item Pipeline Step
   • ID: {D44AE29C-D708-4B3F-B04C-99E7F9288720}

Create a custom step processor

using System.Linq;
using Sitecore.Buckets.Managers;
using Sitecore.ContentSearch;
using Sitecore.ContentSearch.SearchTypes;
using Sitecore.Data;
using Sitecore.Data.Items;
using Sitecore.DataExchange.Contexts;
using Sitecore.DataExchange.Local.Extensions;
using Sitecore.DataExchange.Providers.Sc.DataAccess.Readers;
using Sitecore.DataExchange.Providers.Sc.Plugins;
using Sitecore.DataExchange.Providers.Sc.Processors.PipelineSteps;
using Sitecore.DataExchange.Repositories;
using Sitecore.Services.Core.Diagnostics;
using Sitecore.Services.Core.Model;

namespace Custom.Sc.DataExchange.Providers.Sc.PipelineSteps
{
    public class ResolveSitecoreItemWithBucketSupportStepProcessor : ResolveSitecoreItemStepProcessor
    {
        protected override ItemModel DoSearch(object value, ResolveSitecoreItemSettings resolveItemSettings, IItemModelRepository repository, PipelineContext pipelineContext, ILogger logger)
        {
            ItemModel result = base.DoSearch(value, resolveItemSettings, repository, pipelineContext, logger);
            if (result == null)
            {
                var valueReader = resolveItemSettings.MatchingFieldValueAccessor?.ValueReader as SitecoreItemFieldReader;
                if (valueReader == null)
                {
                    return null;
                }
                Database database = Sitecore.Configuration.Factory.GetDatabase(repository.DatabaseName);
                Item parentItem = database?.GetItem(ID.Parse(resolveItemSettings.ParentItemIdItem));
                if (parentItem != null && BucketManager.IsBucket(parentItem))
                {
                    IProviderSearchContext searchContext = ContentSearchManager.GetIndex($"sitecore_{repository.DatabaseName}_index").CreateSearchContext();

                    string fieldName = valueReader.FieldName;
                    string convertedValue = this.ConvertValueForSearch(value);
                    SearchResultItem searchResult = searchContext.GetQueryable<SearchResultItem>().FirstOrDefault(x => x[fieldName] == convertedValue);
                    return searchResult?.GetItem()?.GetItemModel();
                }

            }
            return null;
        }
    }
}

Create a custom step converter

This is necessary because the default step processor for the Resolve Sitecore Item Pipeline Step has a SupportedIds attribute which does not include your custom template's ID.

Make sure you replace the [YOUR-NEW-TEMPLATE-ID] with the template ID of your new pipeline step template you created.

using Sitecore.DataExchange.Attributes;
using Sitecore.DataExchange.Providers.Sc.Converters.PipelineSteps;
using Sitecore.DataExchange.Repositories;

namespace Custom.Sc.DataExchange.Providers.Sc.PipelineSteps
{
    [SupportedIds([YOUR-NEW-TEMPLATE-ID])]
    public class ResolveSitecoreItemWithBucketSupportStepConverter : ResolveSitecoreItemStepConverter
    {
        public ResolveSitecoreItemWithBucketSupportStepConverter(IItemModelRepository repository) : base(repository) { }
    }
}

Set the custom step converter and step processor on your template

1. In the standard values for your new step template, set two fields:
   1. Set Converter Type to your custom converter (e.g. Custom.Sc.DataExchange.Providers.Sc.PipelineSteps.ResolveSitecoreItemWithBucketSupportStepConverter,Custom.Sc.DataExchange)
   2. Set Processor Type to your custom step processor (e.g. Custom.Sc.DataExchange.Providers.Sc.PipelineSteps.ResolveSitecoreItemWithBucketSupportStepProcessor,Custom.Sc.DataExchange)

Answer 3

There is a patch provided by Sitecore Support that follows the approach described in Peter's answer.

Please note that this implementation works only with bucket folders and creates duplicate items if they are stored in the parent folder root.