Earlier version of item in multivariate test showing up as not found in preview

Question

I set up a multivariate test by creating a new version of an item and changing the datasource of a certain render, I had no issues getting the pages to show up in the "review and start" section, but after starting the test and doing the following:

1- Go to Experience optimization
2- Click on Active tests and select the test I just created
3- Click on review and start

I get that the earlier variation is showing up as not found in the preview box as in the picture below

I used to get PhantomJS errors, but now they are fixed and not showing up anymore in the logs, does anyone know what the underlying issue might be?

Answer 1

Yes. What you’re doing can’t be done.

1) Tests run on ID level. Item ID. There’s no information collected that would allow you to distinguish multiple versions of an item

2) Only the latest publishable version of an item ever makes it to “web” when you publish.

You need to create multiple copies of your item to test.