Can ECM 2.2 be upgraded with analytics data to EXM 3.3 at all?

Question

After reading this written by Sitecorehacker on the Sitecore Slack channel I fell off my seat:

"The "upgrade" from 3.2 to 3.3.. goes silent and walks away I kid, but not really. The upgrade form 3.2 to 3.3 includes a breaking change that resets and disregards all previous analytical email data. More of an issue for people with established ECM installs that have upgrade to EXM"

Check it out here

That means it's a breaking issue for a current upgrade project. Does anyone know if it is even possible to go from ECM 2.2 to EXM 3.3 while somehow keeping your email analytics data?

Answer 1

As the product owner, let me confirm.

In version 1.x --> 3.2 the Email reporting stack was built on a foundation that was inconsistent, inaccurate and not able to scale with the future of Sitecore. The reporting UI was poor and did not meet the expectations of nearly any of our customers.

This means we had to take huge steps to make our customers happy in the long term.

It was not a case of putting a new UI on the old data foundation, because 70% of the data points our customers told us they needed, did not exist, we needed to capture them and store them in a scalable way.

There was no consistent or reliable way for us to extract the old email data and "convert" it into the new data. you would have ended up with your old campaigns showing half complete reports, with most likely different results to what the marketer saw before. Sad, I know, but a consequence of bad decisions (years ago) and a changing world.

So, when you upgrade from 3.2 --> 3.3+, the data for opens and clicks, will be in the rDB, (and you still have all your old sent messages, templates, list associations etc.) however you will have no way to view that data in the new reporting UI. You are welcome to either build something yourself or extract it for sentimental purposes. Your web analytics are also not affected (i.e showing channel attribution for clicks and value).

The new reporting foundation now captures all of the data you are likely to need, in a reliable and accurate way. It leverages the xDB correctly and then aggregates them into the existing Sitecore Analytics stack so that we were able to show them in plenty of pretty charts, in many different ways.

The benefits of the new approach:

• Pretty charts, in HTML5 (not Silverlight)
• Highly accurate, scalable data
• Integrates seamlessly with the Experience Profile
• Reduces load on analytics platform
• Standardizes data framework for channels (web/email)
• Allows Contact level lists & charts in version 3.4
• Huge savings in development and testing overheads over time

Hopefully that summarizes the decisions made and makes you feel more comfortable, if you need to discuss it directly with a customer and need support, please reach out to your local regional contact and they can schedule a time with me.

I trust you see the value in the new approach and that it was a difficult decision to make.

Kind Regards Anthony

Answer 2

Two issues really exist:

1. Backwards Compatibility
2. Analytics Redux

Backwards Compatibility

There is a series of upgrade scripts, procedures, and steps in order to upgrade ECM 2.2 to EXM 3.0. I will edit with link to the Upgrade Doc.

However, as of EXM 3.1, backwards compatibility to utilize ECM links, clicks, opens, and bounces was removed. Along with it, reliance on any of the old AutomationStates table.

The real issue is that the links created out of ECM contain ec_as and ec_camp. These correspond to Automation State and Campaign Id.

In EXM, those were removed and now ec_message_id and ec_contact_id.

Well, if you have access to the old Analytics database from 7.x, you can derive a Membership Profile username. But, EXM utilizes xDB contacts. The upgrade scripts creates user accounts in xDB, but the identifier is domain\username and OOTB EXM wants to use email address as primary identifier. So, without close consideration paid attention to, it's possible to have two Experience Profile accounts for the same person, but different identifiers which causes a host of issues.

So, with EXM 3.3, there's no way to get an xDB Contact from the ec_as query param without heavy reverse engineering.

Similarly, you can use the AutomationStates table and Automation table to derive the email message ID.

Knowing that, you can create pipeline processors for <emailOpen>, <handleBounce>, and <redirectUrl> to massage the params and find the contact I'd and message id. But again, you have to build the backwards compatibility.

Lastly, ECM OOTB utilizes the modules_shell site and a ASPX to handle RegiaterEmailOpen. In EXM, this page doesn't exist, and instead moved to a handler (ASHX) that lives in /sitecore. So we had to rebuild the old page functionality but refactor it for EXM and xDB.

Analytics Redux

EDIT: Anthony describes the story around analytics way better than I. Refer to his response.

SUMMARY

We were contracted by a client doing just that to rebuild the backwards compatibility from ECM to EXM 3.3 It was not an easy lift, and resulted in a number of Support tickets. EXM 3.4 will be coming out sometime in the near future which will support Sitecore 8.2, and I've been told that it's a pretty light weight upgrade from EXM 3.3.

Therefore, if coming from ECM 2.2, and going to EXM 3.3, I advocate working with a Sitecore partner to really hone in on what matters most and figuring out a path.

That's about as much advice as I can give.