6

We are implementing an EXM solution for sending bulk mail, but have come across an issue in scaled environments when using a Dedicated Dispatch Server. We have offloaded all dispatch to the DDS by modifying the DispatchNewsletter pipeline on the CM (as per instructions in the doco at https://doc.sitecore.com/developers/exm/90/email-experience-manager/en/the-exm-pipelines.html#UUID-82aa3c7a-d998-c6b2-bbc4-cac8ba5e54ee_id__DispatchNewsletter_pipeline) replacing the SendEmail processor with WaitForDispatchToFinish. All mail is being sent by DDS as anticipated.

The CM DispatchNewsletter pipeline looks like this.

<DispatchNewsletter patch:source="Sitecore.EmailExperience.ContentManagementPrimary.config">
    <processor type="Sitecore.EmailCampaign.Cm.Pipelines.DispatchNewsletter.CheckPreconditions, Sitecore.EmailCampaign.Cm" resolve="true"/>
    <processor type="Sitecore.EmailCampaign.Cm.Pipelines.DispatchNewsletter.MoveToQueuing, Sitecore.EmailCampaign.Cm" resolve="true"/>
    <processor type="Sitecore.EmailCampaign.Cm.Pipelines.DispatchNewsletter.DeployAnalytics, Sitecore.EmailCampaign.Cm" resolve="true"/>
    <processor type="Sitecore.EmailCampaign.Cm.Pipelines.DispatchNewsletter.PublishDispatchItems, Sitecore.EmailCampaign.Cm" resolve="true"/>
    <processor type="Sitecore.EmailCampaign.Cm.Pipelines.DispatchNewsletter.QueueMessage, Sitecore.EmailCampaign.Cm" resolve="true"/>
    <processor type="Sitecore.EmailCampaign.Cm.Pipelines.DispatchNewsletter.MoveToProcessing, Sitecore.EmailCampaign.Cm" resolve="true"/>
    <processor type="Sitecore.EmailCampaign.Cm.Pipelines.DispatchNewsletter.LaunchDedicatedServers, Sitecore.EmailCampaign.Cm"/>
    <processor type="Sitecore.EmailCampaign.Cm.Pipelines.DispatchNewsletter.SendTestMessage, Sitecore.EmailCampaign.Cm" resolve="true"/>
    <processor type="Sitecore.EmailCampaign.Cm.Pipelines.DispatchNewsletter.WaitForDispatchToFinish, Sitecore.EmailCampaign.Cm" resolve="true" patch:source="Sitecore.EmailExperience.ContentManagement.Patch.config">
        <TimeToWaitBetweenChecks>1000</TimeToWaitBetweenChecks>
    </processor>
    <processor type="Sitecore.EmailCampaign.Cm.Pipelines.DispatchNewsletter.MoveToSent, Sitecore.EmailCampaign.Cm" resolve="true"/>
    <processor type="Sitecore.EmailCampaign.Cm.Pipelines.DispatchNewsletter.NotifyDispatchFinished, Sitecore.EmailCampaign.Cm" resolve="true"/>
    <processor type="Sitecore.EmailCampaign.Cm.Pipelines.DispatchNewsletter.FinalizeDispatch, Sitecore.EmailCampaign.Cm" resolve="true"/>
</DispatchNewsletter>  

The DDS DispatchNewsletter pipeline looks like this.

<DispatchNewsletter patch:source="Sitecore.EmailExperience.EmailProcessing.config">
    <processor type="Sitecore.EmailCampaign.Cm.Pipelines.DispatchNewsletter.CheckPreconditions, Sitecore.EmailCampaign.Cm" resolve="true"/>
    <processor type="Sitecore.EmailCampaign.Cm.Pipelines.DispatchNewsletter.SendMessage, Sitecore.EmailCampaign.Cm" resolve="true">
        <Sleep>2000</Sleep>
    </processor>
    <processor type="Sitecore.EmailCampaign.Cm.Pipelines.DispatchNewsletter.FinalizeDispatch, Sitecore.EmailCampaign.Cm" resolve="true"/>
</DispatchNewsletter>

When sending, we have noticed that the campaign will not transition to a "Sent" status, even though all the mail has successfully been delivered. It will stay in the "Sending" status. This occurs for both the CustomSMTP and EmailCloud providers. A pause and resume of the campaign results in the same problem.

We performed some Pipeline debugging/dotPeeking and it appears that the code is never exiting the WaitForDispatchToFinish processor. Looking at the DDS logs we can see a continual polling for Status (initiated from the WaitForDispatchToFinish processor) and reporting "Status=Stopped". There is nothing in the logs that indicates any level of failure on the CM or DDS.

This was all tested on Sitecore 9.0.1 ( with the EXM cumulative hotfix https://kb.sitecore.net/articles/467253 ) in a fully scaled environment on Azure Paas.

Is there something we are missing with our implementation to completely disable dispatch on the CM and offload all mail dispatch to the DDS? We know there are other options to minimise dispatch on the CM, however are looking to disable it completely for a clearer separation of concerns in our Sitecore roles, which should be possible?

Update

I made the suggested changes from comments, and re-tested. We're seeing some campaigns getting stuck still. Dug into dotpeek and looks to me like WaitForDedicatedServers in the WaitForDispatchToFinish processor expects the num of DDS servers to be "sending" > 0 at least once. So theory is that if the DDS completes before there is a successful status check (EG. due to small send list or initial status calls failing etc) it seems to get stuck even though the mail is delivered. Pausing/Resuming the affected campaigns results in the same as DDS status is always "Stopped" (it's already dispatched everything).

  • Great detail. Seems like you have everything in order. Could be a bug in 9.0.1 that hasn't been realized. Let me dotPeek around. Will report if Jacob doesn't beat me to it. – Pete Navarra Dec 20 '18 at 23:19
  • Is there a reason your DDS has a 2000 ms sleep? – Pete Navarra Dec 20 '18 at 23:20
  • Thanks @PeteNavarra . Sleep on DDS is at 2000 atm for no other reason than....that was OOTB config. Thanks for the heads up, I had missed that. We'll optimise! – George Tucker Dec 21 '18 at 0:17
  • Yeah, OOTB configurations are not optimized for speeeeeed. =) – Pete Navarra Dec 21 '18 at 0:59
  • Huh, that's odd.. There are no known bugs around this. I would try to lower the sleep (just set it to e.g. 5 or even 0) and increase TimeToWaitBetweenChecks to e.g. 5000 – Jacob Nielsen Dec 21 '18 at 9:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.