7

I've utilized Data Exchange Framework and built pipeline batches.

Now it's unclear how the created/updated items can be published automatically. I did not find any publishing pipeline steps or even mention of publishing stuff in Sitecore.DataExchange. Publishing seems to be an essential step within pipeline after pulling data but looks like DEF is not responsible for that.

So, what is the right way of doing automatic publish when DEF is used?

20

You can add a custom pipeline step to perform the publishing. Below is an example of one that can publish either a specific item (e.g. the root item below which all your DEF items were imported) or an item in the pipeline context (e.g. each item as it's imported).

As with all custom pipeline steps, you'll need to perform the following steps:

  1. Add a custom pipeline step template
  2. Add a plugin for settings
  3. Add a pipeline step converter
  4. Add a pipeline step processor
  5. Update the standard values of your template to reference your step converter and processor

1. Create your template

  1. Create a new template called Publish Item Pipeline Step
    • I created mine at /sitecore/templates/Data Exchange/Providers/Sitecore/Pipeline Steps/Publish Item Pipeline Step
  2. Add /sitecore/templates/Data Exchange/Framework/Pipeline Steps/Base Pipeline Steps/Base Pipeline Step as a base template
  3. Add fields
    • EndpointFrom
      • Type: Droptree
      • Source: query:./ancestor-or-self::*[@@templateid='{327A381B-59F8-4E88-B331-BEBC7BD87E4E}']//descendant-or-self::*[@@templateid='{CAADA2F1-1D71-452A-87BF-A3AE335CA169}']
    • Item
      • Type: Droptree
      • Short Description: The exact item to publish. A value in this field will override the Item Location field.
    • ItemLocation
      • Type: Droplink
      • Source: {DD61BBDB-CF41-40F6-9923-3031C7AB47C8}
      • Short Description: Location within the pipeline's context of the item to publish
    • Publishing Targets
      • Type: Checklist
      • Source: /sitecore/system/Publishing targets
    • Publish Subitems
      • Type: Checkbox

2. Add plugin

using Sitecore.DataExchange;
using Sitecore.Services.Core.Model;

public class PublishItemSettings : IPlugin
{
    public ItemModel ItemModel { get; set; }
    public string[] TargetDatabaseNames { get; set; }
    public bool PublishSubitems { get; set; }
}

3. Add pipeline step converter

using System;
using System.Collections.Generic;
using System.Linq;
using Sitecore.DataExchange;
using Sitecore.DataExchange.Attributes;
using Sitecore.DataExchange.Converters.PipelineSteps;
using Sitecore.DataExchange.Extensions;
using Sitecore.DataExchange.Models;
using Sitecore.DataExchange.Plugins;
using Sitecore.DataExchange.Repositories;
using Sitecore.Services.Core.Model;

[SupportedIds("YOUR-TEMPLATE-ID")]
public class PublishItemPipelineStepConverter : BasePipelineStepConverter
{
    public PublishItemPipelineStepConverter(IItemModelRepository repository) : base(repository) { }

    private void AddPublishItemSettings(ItemModel source, PipelineStep pipelineStep)
    {
        if (pipelineStep == null)
        {
            throw new ArgumentNullException(nameof(pipelineStep));
        }

        IEnumerable<ItemModel> publishingTargets = GetReferencesAsModels(source, "Publishing Targets");
        if (publishingTargets == null || !publishingTargets.Any())
        {
            Context.Logger.Error($"No publishing targets were selected for the pipeline step. (pipeline step: {pipelineStep.Name}, pipeline step item: {source.GetItemId()}, field: Publishing Targets)");
            return;
        }

        var settings = new PublishItemSettings
                       {
                           ItemModel = GetReferenceAsModel(source, "Item"),
                           TargetDatabaseNames = publishingTargets.Select(model => GetStringValue(model, "Target database")).ToArray(),
                           PublishSubitems = GetBoolValue(source, "Publish Subitems")
                       };

        pipelineStep.AddPlugin(settings);
    }

    private void AddDataLocationSettings(ItemModel source, PipelineStep pipelineStep)
    {
        if (pipelineStep == null)
        {
            throw new ArgumentNullException(nameof(pipelineStep));
        }

        var settings = new DataLocationSettings
                       {
                           DataLocation = GetGuidValue(source, "ItemLocation")
                       };
        pipelineStep.AddPlugin(settings);
    }

    private void AddEndpointSettings(ItemModel source, PipelineStep pipelineStep)
    {
        var settings = new EndpointSettings();
        var endpoint = ConvertReferenceToModel<Endpoint>(source, "EndpointFrom");
        if (endpoint != null)
        {
            settings.EndpointFrom = endpoint;
        }

        pipelineStep.AddPlugin(settings);
    }

    protected override void AddPlugins(ItemModel source, PipelineStep pipelineStep)
    {
        this.AddPublishItemSettings(source, pipelineStep);
        this.AddDataLocationSettings(source, pipelineStep);
        this.AddEndpointSettings(source, pipelineStep);
    }
}

4. Add pipeline step processor

using System;
using System.Collections.Generic;
using System.Linq;
using Sitecore.Data;
using Sitecore.Data.Items;
using Sitecore.DataExchange.Attributes;
using Sitecore.DataExchange.Contexts;
using Sitecore.DataExchange.Extensions;
using Sitecore.DataExchange.Models;
using Sitecore.DataExchange.Plugins;
using Sitecore.DataExchange.Providers.Sc.Extensions;
using Sitecore.DataExchange.Providers.Sc.Processors.PipelineSteps;
using Sitecore.Publishing;
using Sitecore.Services.Core.Diagnostics;
using Sitecore.Services.Core.Model;

[RequiredPipelineStepPlugins(typeof(PublishItemSettings), typeof(DataLocationSettings), typeof(EndpointSettings))]
public class PublishItemPipelineStepProcessor : UpdateSitecoreItemStepProcessor
{
    protected override void ProcessPipelineStep(PipelineStep pipelineStep, PipelineContext pipelineContext, ILogger logger)
    {
        if (pipelineStep == null) throw new ArgumentNullException(nameof(pipelineStep));
        if (pipelineContext == null) throw new ArgumentNullException(nameof(pipelineContext));
        if (logger == null) throw new ArgumentNullException(nameof(logger));

        PublishItemSettings publishItemSettings = pipelineStep.GetPlugin<PublishItemSettings>();
        IEnumerable<ItemModel> itemsToPublish;
        if (publishItemSettings.ItemModel != null)
        {
            itemsToPublish = new [] { publishItemSettings.ItemModel };
        }
        else
        {
            logger.Debug($"No direct item reference was specified in the pipeline step, so we are looking in the DataLocation field instead. (pipeline step: {pipelineStep.Name}, pipeline step identifier: {pipelineStep.Identifier})");
            itemsToPublish = GetTargetObjectAsItemModels(pipelineStep, pipelineContext, logger);
        }

        if (itemsToPublish == null || !itemsToPublish.Any())
        {
            logger.Error($"No available items to publish. (pipeline step: {pipelineStep.Name}, pipeline step identifier: {pipelineStep.Identifier})");
            return;
        }

        EndpointSettings endpointSettings = pipelineStep.GetEndpointSettings();
        string sourceDatabaseName = endpointSettings.EndpointFrom?.GetItemModelRepositorySettings()?.ItemModelRepository?.DatabaseName;
        if (string.IsNullOrEmpty(sourceDatabaseName))
        {
            logger.Error($"No available item repository from which to publish. (pipeline step: {pipelineStep.Name}, pipeline step identifier: {pipelineStep.Identifier}, endpointSettings.EndpointFrom?.Name: {endpointSettings.EndpointFrom?.Name}");
            return;
        }
        Database sourceDatabase = Database.GetDatabase(sourceDatabaseName);
        IEnumerable<Database> targetDatabases = publishItemSettings.TargetDatabaseNames.Select(Database.GetDatabase);
        foreach (ItemModel itemModel in itemsToPublish)
        {
            Guid itemId = itemModel.GetItemId();
            Item item = sourceDatabase.GetItem(ID.Parse(itemId));
            if (item == null)
            {
                logger.Error($"Unable to retrieve item to publish from {sourceDatabase.Name} database ({itemId}). (pipeline step: {pipelineStep.Name}, pipeline step identifier: {pipelineStep.Identifier})");
                continue;
            }
            PublishManager.PublishItem(item, targetDatabases.ToArray(), sourceDatabase.Languages, publishItemSettings.PublishSubitems, false, false);
            logger.Debug($"Published item {itemId} to {string.Join(", ", publishItemSettings.TargetDatabaseNames)}. (pipeline step: {pipelineStep.Name}, pipeline step identifier: {pipelineStep.Identifier})");
        }
    }
}

5. Update standard values

  1. In Sitecore, navigate to the standard values of your custom pipeline step template you created back in step 1
  2. Populate the Converter Type and Processor Type fields with your custom types, for example:
    • Custom.DataExchange.Providers.Sc.PipelineSteps.PublishItemPipelineStepConverter,Custom.DataExchange
    • Custom.DataExchange.Providers.Sc.PipelineSteps.PublishItemPipelineStepProcessor,Custom.DataExchange
  • Thank you for sharing your approach with codes! Makes perfect sense for me. – Igor Evstratov Jun 8 at 15:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.