EXM A/B test statistics not displayed

Question

Sitecore Version: 9.1 Update 1

In EXM, we created an A/B test campaign. We choose a winner manually (eg. Variant A) for this test. Now once this test is finished, we are unable to see the variant-wise report. After conducting a test, only overall campaign results are displayed. i.e stats displayed in the interface(Sent email campaigns) are for both variants combined. Which means we can’t test one particular element/module and analyse in detail?

Any help will be much appreciated.

Answer 1

I've contacted with Sitecore support for this and they have confirmed there is no out of the box report that retrieves detailed statistics for a/b test variants and registered a feature request to add it in future versions of Sitecore.

In the meantime, we can retrieve this data directly from the collection database. The information about the a/b variant is stored in the "Events" row of the "Interactions" table. It is called: "TestValueIndex". So the example of the query that for example counts the send interactions for "B" variant of the "'e6889dfb-ea7b-4393-8844-a56c29fc55fc'" email campaign will look like that:

SELECT COUNT(*) FROM [xdb_collection].[Interactions]
CROSS APPLY
OPENJSON([xdb_collection].[Interactions].[Events]) WITH
(
OdataType varchar(200) '$."@odata.type"',
MessageId varchar(200) '$.MessageId',
TestValueIndex varchar(200) '$.TestValueIndex'
)
WHERE
OdataType = '#Sitecore.EmailCampaign.Model.XConnect.Events.EmailSentEvent'
AND
MessageId = 'e6889dfb-ea7b-4393-8844-a56c29fc55fc'
AND
TestValueIndex = '1'

To get full results, such a query needs to be applied for both shards.