EXM 3.4 EDS connection timeout exceptions

Question

At the moment we are sending emails using engagement plans and automated actions for EXM. Sometimes we get Sitecore.EDS.Core.Exceptions.ConnectionTimeoutException exceptions.

For example checked now we sent 1800 emails and 30 of them failed with this exception.

What can be wrong? Is there any suggestion how to nail this down?

We are using:

• Sitecore.NET 8.2 (rev. 160729)
• Email Experience Manager 3.4.0 rev. 161028
• Sendgrid as MTA.

Exception details are here:

at Sitecore.EDS.Core.Net.Smtp.ChilkatConnectionPool.<GetConnectionAsync>d__6.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.EDS.Providers.CustomSmtp.DispatchProvider.<ValidateDispatchAsync>d__2.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.EDS.Core.Dispatch.DispatchManager.<ValidateDispatchAsync>d__7.MoveNext() --- End of inner exception stack trace --- at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification) at Sitecore.Modules.EmailCampaign.SendingManager.CheckMailServer(String& error) at Sitecore.Modules.EmailCampaign.SendingManager..ctor(Boolean checkConnection, MessageItem message, Boolean isService, ILogger logger, ISenderManager senderManager, IDispatchManager dispatchManager) at Sitecore.Modules.EmailCampaign.SendingManager..ctor(Boolean checkConnection, MessageItem message, Boolean isService, ILogger logger) at Sitecore.Modules.EmailCampaign.Core.Dispatch.SendingHelper.SendStandardMessage(String messageId, RecipientId recipientId, Boolean async, ILogger logger) at Sitecore.Modules.EmailCampaign.Core.ClientApiLocal.SendStandardMessage(Guid messageId, RecipientId recipientId, Boolean async) at Sitecore.Modules.EmailCampaign.ClientApi.SendStandardMessage(Guid messageId, RecipientId recipientId, Boolean async) ---> (Inner Exception #0) Sitecore.EDS.Core.Exceptions.ConnectionTimeoutException: Unable to obtain connection within the given number of retries. at Sitecore.EDS.Core.Net.Smtp.ChilkatConnectionPool.<GetConnectionAsync>d__6.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.EDS.Providers.CustomSmtp.DispatchProvider.<ValidateDispatchAsync>d__2.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.EDS.Core.Dispatch.DispatchManager.<ValidateDispatchAsync>d__7.MoveNext()<---

Answer 1

This is most likely a configuration issue.

I'm guessing the NumberThreads setting is larger than the maxPoolSize?

NumberThreads (Sitecore.EmailExperience.ContentManagement.config): Specifies the number of threads to use when dispatching. Each thread creates a dispatch task responsible for sending an email campaign to one contact.

exm/eds/connectionPoolSettings/maxPoolSize (Sitecore.EDS.Providers.CustomSMTP/SparkPost/Dyn.config): Specify the number of connections that can be created in the connection pool.

If you have more threads generating emails than available connections, you can run into a timeout.

For reference, see The EXM configuration settings and The EXM connection pool settingson doc.sitecore.net.

MaxGenerationThreads (Sitecore.EmailExperience.ContentManagement.config) In addition, but unrelated to your problem, the MaxGenerationThread settings can be used to throttle the dispatch speed.

MaxGenerationThreads acts as a semaphore to limit the number of dispatch tasks that can be running concurrently.

As an example, if NumberThreads and MaxGenerationThreads are both set to 16, 16 dispatch tasks will be processed concurrently. If MaxGenerationThreads is 8, and NumberThreads is 16, only 8 of the dispatch tasks will be processed concurrently while the other 8 tasks are blocked waiting to be processed.

A high value in NumberThreads and MaxGenerationThreads will (possibly) increase the dispatch speed of a campaign, but it will also have a higher CPU load.

Answer 2

My recommendation is to put EXM into Debug mode and review the contents of the EDS.log file that is created. Any issues with the Email Delivery Service will be logged in the EDS.log.

The Chilkat library which Sitecore EDS depends on doesn't always fail gracefully. When it does fail, it fails within the Async Task, which is generating the above error message, which is denoted by the System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess() stack trace. Now, the EXM DispatchManager is smart enough to allow the failure to occur, send all of the other email messages, and then, when the dispatch is completely, places the Dispatch in a Paused state, to allow you to fix whatever issue might have been caused, so that you can Resume the send.

However, the messaging and logging from this process is generally difficult to find, and you won't find it in the EXM.log, but in the EDS.log rather.

I have seen instances where failure on the MTA side bubbles up to this type of error (which doesn't explain a lot).

Factors that can go into an error on the MTA side:

• Badly formed email addresses
• Connection/Latency Issues
• Account Issues

The Chilkat output that ends up in the EDS.log when in debug mode, while difficult to read, does lend itself to being a bit more transparent with what's going on.