1

How to check which experiment ensured Goal completion when I have multiple experiments with the same Goal that a Guest could get?

I do not want to bucket as their traffic volumes are low.

For example, a homepage banner with the Primary Goal to complete a booking, Also a different popup on a page does the same.

A User is exposed to both, however, they click on the banner and start doing the booking, then exit and then are exposed to the popup, do not click it but then complete the booking... The Primary Goal is achieved and the Guest was exposed to both. However, in the Dashboards how is this counted? Double? Or against which one if single?

1 Answer 1

0

In your example, reaching the Goal is accounted for in both experiments, because of you don`t split traffic.

From Sitecore CDP documentation:

Why running multiple A/B Test can blur your test results?

There are some considerations to make before you endeavor running multiple tests at the same time.

Standard wisdom is that running overlapping tests can cause issues with interpretation of the results because of the chance of cross-contamination (the effect of a site visitor seeing a variant from Test A and that influencing the outcome of Test B and so on). The likelihood of this being a practical issue very much depends on a) the number of overlapping tests, b) the amount of traffic that are exposed to the overlapping tests, and c) the nature of the overlap.

You need to provide you target different tests to different buckets of visitors, then there is guaranteed to be no overlap between tests.

I advise you to pay more attention to the hypothesis you want to confirm with your tests, and to set up experiments based on that. Apply the following when declaring a hypothesis:

  • Avoid ambiguity in terms of cause and effect,
  • Test only one change at a time,
  • Estimate the desired effect.

P.S. If your main problem at this moment is low traffic (you can not afford to wait until statistical significance is reached and a winning variant is declared), you can setup optimized testing (multi-armed bandit algorithm). You can find more information in official documentation: https://doc.sitecore.com/personalize/en/users/sitecore-personalize/running-an-experiment-with-optimized-testing.html

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.