Google Analytics vs Sitecore Interactions

Question

I'm running Sitecore 8.1 (Original Release), and I've generated a report that is tracking how many times visitors have seen a variation of an internal ad that is on my site. The basic premise of the campaign is that we put a wiki-banner style ad at the top of every page that Sitecore serves up. We want to track how many times each version of that ad is displayed.

I've written some logic that marks which variation is displayed to the user as part of their interaction on the website. I've also written a report to go back through the interactions and report on how many times each variation is displayed.

Here's my problem: The number of interaction records is much smaller than what Google Analytics has.

Google Analytics: 1.2 million PageViews

Sitecore: 884,000 Interaction Records

This disparity between the 2 systems has caused my leadership (and me as well) to question whether or not I'm reporting on the right information.

What is going on with this? How should the numbers compare between the 2 systems? Am I comparing the wrong numbers?

Answer 1

OK, so I've spoken with Sitecore support about this issue, and they were able to help me come up with the answer. The reason for the difference is because of the way that I was counting the interactions. I was running the following query to find out how many pageviews I had (according to Mongo):

db.Interactions.count({"StartDateTime":{$gt: ISODate("2017-03-27T00:00:00:00.000-06:00")}})

According to Sitecore support, this is not the correct query that I should have been running. This counts the interactions, but there could be multiple page views in a single interaction. So in order to get the correct numbers, you have to use a different piece of data in the mongo db collection. Here's a couple of examples.

This will find all of the interactions that just had a single page view.

db.Interactions.find({"StartDateTime":{$gt: ISODate("2017-03-27T00:00:00:00.000-06:00")},"VisitPageCount": 1 })

This will find all of the interactions that have more than 1 page view.

db.Interactions.find({"StartDateTime":{$gt: ISODate("2017-03-27T00:00:00:00.000-06:00")},"VisitPageCount": {$gt: 1}})

So, the moral of the story is that in order to find out how many page views a visitor experienced, you have to write a query to add the VisitPageCount for each interaction record.

I hope that this helps someone else understand how to interpret the analytics data that is stored in Mongo.

Answer 2

Another thing to consider- and this could potentially count for a good deal of page views is how Sitecore's algorithm automatically detects bots. A page view is considered a bot if:

• No mouse movement is observed
• No touch movement is observed (for mobile)

This is achieved by the visitorIdentification control. This ultimately calls ~/layouts/system/VisitorIdentification.js.

If you review this JavaScript file, you will see that on mousemove or touchstart, a css page is requested: ~/layouts/system/VisitorIdentificationCss.aspx.

When this page is requested, the user is classified as a human. Therefore to put it concisely, you are not a robot if you move your mouse or touch your screen.

Therefore, we can imagine a scenario where users click to visit a site, but then do not interact with it- perhaps opening a background tab, then closing their browser. There is also the possibility of keyboard-only users, especially those that rely on screen-readers. These users will be seen as bots based on Sitecore's automatic bot detection.

This setting can be disabled. However, then the only method of determining bots is via the Sitecore.Analytics.ExcludeRobots.config, which allows you to classify bots by user agent strings or IP adresses.

All and all, this may not account for 26% of traffic missing, but has to account for some of it.

Answer 3

This depends in large part on how your Google Analytics view is set up. If you are using a master view with no filters, Google will report all traffic including robots. Sitecore, by default, eliminates robot traffic from its reporting. This alone can account for the difference you're encountering.

From the Sitecore documentation:

Not all incoming requests are counted by Sitecore Analytics. It distinguishes unique users with ASP.NET sessions and browser cookies. Also, it skips robots' sessions. Thus, the number of unique visits in Sitecore Analytics is usually less than Google Analytics reports show.

Answer 4

You will always have variance between two analytics systems, depending on the situation that can range from 0-20%, beyond that, I'd really try to undersand the cause. And generally speaking what I've seen and heard Google Analytics does seem to always show more page views than Sitecore. Also the crawler identification is probably working slightly differently.

The key thing you want to validate is, if the variance is consistent. I.e. week to week, is the variance roughly same? or is there some specific period of time when it's off. At that point you might have had an issue in collecting the Sitecore Analytics.

Also, I'd check if Sitecore.Analytics.ExcludeRobots.config (or similiar) file contains for example internal IPs that have been configured to be filtered.