1

When I start "MongoDB Contacts to xConnect Migration" pipeline with 1 thread all works fine. But when start with more then 1. I have an error after random count of processed contacts.

24104 14:15:53 ERROR [Data Exchange] Collection was modified; enumeration operation may not execute. (pipeline: Read Contacts from MongoDB Pipeline, pipeline step: Iterate MongoDB Contacts and Run Pipelines, pipeline step identifier: 4d6400a4-4e78-4555-80d7-46aafd07afe9)
24104 14:15:53 ERROR [Data Exchange] at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
   at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
   at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable`1 source, Func`2 predicate)
   at Sitecore.DataExchange.Processors.PipelineSteps.BasePipelineStepProcessor.GetMissingRequiredPlugin[T](IHasPlugins obj)
   at Sitecore.DataExchange.Processors.PipelineSteps.BasePipelineStepProcessor.AreRequiredPluginsSet()
   at Sitecore.DataExchange.Processors.PipelineSteps.BasePipelineStepProcessor.StartProcessing(PipelineStep pipelineStep, PipelineContext pipelineContext, ILogger logger)
   at Sitecore.DataExchange.Processors.Pipelines.PipelineProcessor.ProcessPipelineStep(PipelineStep pipelineStep, PipelineContext pipelineContext, ILogger logger)
   at Sitecore.DataExchange.Processors.Pipelines.PipelineProcessor.ProcessPipeline(Pipeline pipeline, PipelineContext pipelineContext, ILogger logger)
   at Sitecore.DataExchange.Processors.PipelineSteps.IterateAndRunPipelinesStepProcessor.ProcessPipelines(PipelineStep pipelineStep, ICollection`1 subPipelines, PipelineContext pipelineContext, ILogger logger)
   at Sitecore.DataExchange.Processors.PipelineSteps.IterateAndRunPipelinesStepProcessor.<>c__DisplayClass4_0.<IterateWithThreads>b__0(Object element, Int32 threadId)
   at Sitecore.DataExchange.Helpers.EnumeratorHandlerHelper.RunThread(IEnumerator data, Int32 threadId, EnumeratorHandlerHelperContext context)

Does someone know what i configured wrong?

  • Which pipeline step was used to set threads? – Vlad Shpak Mar 4 '19 at 22:30
  • @VladShpak this "Iterate MongoDB Contacts and Run Pipelines" – pblrok Mar 5 '19 at 8:39
1

There is a bug in Data Exchange 2.1 with regard to Multithreading. Sitecore Support will advise that you downgrade to Data Exchange 2.0.2. But, after having this same issue I have determined that you only need TWO DLL's from Data Exchange 2.0.2.

  1. Go to the xDB Migration Download 2.0.2 Page and download the "Data Exchange Framework" package.

  2. Extract the zip to an empty directory and locate the following DLL's:

  • Sitecore.DataExchange.DLL
  • Sitecore.DataExchange.Local.DLL
  1. Copy these DLL's to your Sitecore CM's bin folder.

  2. Rerun Migration. This will solve the multi-threading issue.

Bonus Content: If you are connecting to a Mongo Cloud Service Provider, such as ObjectRocket or SearchStax, the packaged Mongo Driver of 2.6.1 is not sufficient to maintain cloud connections. You will need to go and download Mongo Drivers for version 2.9.3.

3

This issue is with the product itself, not related to the configuration.

BaseHasPlugins class is not a thread-safe. Basically, it holds the list of plugins associated with some pipeline-step (in this case).

When you are running it with multiple threads: while one thread is using the method public IPlugin GetPlugin(Type type) which enumerates over a list of internal plugins, another thread is trying to add/modify a new plugin.

Please contact Sitecore Support to resolve it.

Workaround: You can inherit the current pipeline step processor, and override

protected override bool AreRequiredPluginsSet() { return true; }

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.