11

We use the EXM with Sitecore Email Cloud on our project and going to push 4 million emails a day.

Currently we working on setting up a production servers structure.

Is it needed to use dispatch server in case of using Sitecore Email Cloud?

Are there any requirements or recommendation for servers structure for our needs? If someone has an experience of using an EXM to send so many emails, please share which infrastructure you built.

Background: Sitecore 8.2 Update 6 with EXM v.3.5.1 rev. 171103

19

Is it needed to use dispatch server in case of using Sitecore Email Cloud?

Yes, Sitecore Email Cloud is simply the mail transit provider. You still need dispatch servers to generate the emails and send to the Sitecore EDS service. The process of generating the emails can be taxing based on how complicated your emails are and if personalization rules are in play.

Are there any requirements or recommendation for servers structure for our needs?

Scaling to 4 Million emails a day is possible. The experience that I have had in the past got approximately 1 million emails in about 3 hours. Doing 4 in a day is totally do able.

Here is document about scaling EXM: https://doc.sitecore.net/email_experience_manager/setting_up_exm/configuration/configuring_exm_in_a_scaled_environment

The reality is, this document only covers how to scale to one server. But to get to 4 million a day, you're going to need multiple EXM Dispatch Servers. I would recommend starting off with 3-4 Dispatch Servers. Also, I recommend disabling dispatch on the Primary CM role so as not to bog down content authoring.

As for hardware, I would definitely make sure the dispatch servers have a lot of CPU and a good amount of RAM. Cache as much as you can in Sitecore caches to keep from having to make trips to the DB.

Solr optimization is key. EXM spends a lot of time in the indexes (analytics index, master/web indexes, etc). Making sure that your Solr implementation is sufficient and scaled appropriately will help in making sure that your dispatches are optimized. Any latency at all between Solr (and Mongo) will slow down dispatch times.

Ensure that you aren't doing anything CRAZY in the <SendEmail> pipeline. This is the pipeline that every email will call that generates the email. Any customizations to this pipeline will slow down processing. This also calls other pipelines such as <modifyHyperlink> and <modifyImageLink>so you'll want to also make sure that you aren't doing anything crazy here either.

Separate Processing and Reporting Roles

Make sure that you also separate out your Processing and Reporting roles. This will help offset additional processing that 4 million emails will cause on your Primary CM if you do not. You may also find that having only one processing server might not be enough. Consider scaling to 2 if processing times are slow. An indicator of slow processing times is when a dispatch completes, but Sitecore is slow to report the Sent events on contacts and in the Email Report.

Tools for Monitoring Dispatch

On each dispatch server, you can see stats on how the server is performing. You can reach the Dispatch Summary page at /sitecore/admin/dispatchsummary.aspx

On this report, the GetBody() statistic is what you want to try and optimize as much as you can. This method is what each email being generated calls, which reaches out to the CD cluster to generate the email with personalization, and then downloads the HTML into the email body. Any delays in communicating with the CD environment will also impact delivery.

When to Scale

The Primary CM role can generally handle most triggered emails, drip campaigns, and small batches. I have been successful with campaigns as large as 100K contacts on one Primary CM, but it wasn't fast, and authoring was a bit slower due to all of the dispatching and processing occuring.

If you are scaling up slowly, it is fine to start off with just a primary CM. If you find the authoring server begin to choke, separate out the processing and reporting roles. That should allow you to scale the Primary CM up more.

As you increase number of emails going out over time, and authoring chokes again, that is when I would deploy a dedicated dispatch server and then turn dispatch off on primary CM.

Then, to scale up from there, you are really just duplicating dispatch servers, and then down the line maybe add another processing server.

But thats slow growth.

If you have business requirements where you know you are sending out millions of emails on day one. Then plan for multiple dispatch servers out of the gate.

  • 1
    Thanks for your answer and explanation! It helped us to understand what we need and saved our time. – Artsem Prashkovich Dec 14 '17 at 14:26
  • Great information on this. What level of personalization where you doing in these email campaigns? Wondering if you got this level of throughput while also doing personalization or if these were just token replacements ExM was doing and therefor not needing to execute any personalization rules. – ToddB Aug 20 '18 at 23:08
  • @ToddB We used token personalization, which is always available. We were not using component personalization. Had we done so, the throughput would be much lower. Some of the load when doing component personalization is offloaded to the Content Delivery server, but every message has to be generated when doing so.. So, you could see a decrease in performance on large distributions. Where component personalization is more useful is on transactional emails (or emails done through Marketing Automation) where there isn't a demand for a fast dispatch to millions of users. – Pete Navarra Aug 29 '18 at 18:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.