Experience Analytics are not showing Online Interactions

Question

I have a Sitecore solution (Sitecore 9.0.1 revision 171219 on Azure in xP1 topology) that as far as I can tell all roles are working as expected and communicating. I am able to fire goals, create contacts, and add engagement value.

On the Sitecore Launch page, I see that stats are collecting and reporting appropriately.

However, when I go into Sitecore's Experience Analytics, I would expect to see similar graphs and numbers to the one on the launch pad.

Additionally, the graph under it, which I believe is basically the same graph as on the Launchpad also shows up empty.

So, I wanted to know what the difference was in the queries.

The Launchpad Queries include:

https://sitecoresite/sitecore/api/ao/aggregates/all/799B0CD6A8114772BDC46DC607106A62/all?&dateGrouping=by-week&&keyTop=5&keyOrderBy=visits-Asc&dateFrom=22-10-2017&dateTo=22-01-2018&keyGrouping=by-key https://sitecoresite/sitecore/api/ao/aggregates/all/0751EFD8564F40A38083EF5C6E194247/all?&dateGrouping=by-week&&dateFrom=22-10-2017&dateTo=22-01-2018&keyGrouping=collapsed

However, the Experience Analytics queries look like:

https://sitecoresite/sitecore/api/ao/aggregates/all/0751EFD8564F40A38083EF5C6E194247/-3191897772735422194?&dateGrouping=collapsed&&dateFrom=14-01-2018&dateTo=22-01-2018&keyGrouping=by-key https://sitecoresite/sitecore/api/ao/aggregates/all/0751EFD8564F40A38083EF5C6E194247/-3191897772735422194?&dateGrouping=by-auto&&dateFrom=14-01-2018&dateTo=22-01-2018&keyGrouping=by-key

The difference is in highlighted here:

There are two requests being made with a negative value number instead of "all". And it's these two requests that are causing these graphs to be blank. I am pretty sure the existence of the negative numbers is by design (as these are the same requests in my xP0 environment that are working), but it is these two requests which are not providing any metrics.

Question: Why is this occurring and how can it be fixed?

Other Notes:

• This does not appear to be an issue in Sitecore 9.0.1 rev. 171219 on Azure using xP0 topology.
• No errors in any logs are being generated on any of the roles.
• No XHR request errors either.
• Using Azure Search for indexes on all roles. (Not Solr)
• I am using Self-Signed Certificates

I also have information showing in Experience Profile for Contacts showing visits and values.

Update 1:

I was given a clue to ensure that my connection strings for certificate thumbnails included the AllowInvalidClientCertificates=true. They did not. I have updated all connection strings and generated some visits but that appears to have had no effect.

While deploying my xP1 environment to Azure PaaS I used the provided 9.0.1 ARM templates. Yesterday, I noticed that Lucene had been configured for OOTB on the PRC, DDS, and REP roles. This was causing a handful of performance issues out of those roles. Changing the Search provider to Azure has cleared up all performance issues seen from those roles, but has also not fixed this particular issue.

I am wondering if because there was some lucene issues during processing, if by switching to Azure without rebuilding the reporting database is causing some issues. Therefore, I am going to attempt to do a rebuild of the reporting database (not looking forward to it since this is in Azure PaaS, but we'll see what happens).

Update 2:

I have determined the root issue of this is because there are NO SEGMENTS in my Segments table in the Reporting Database.

I have tried to deploy segments over and over and over in various different ways and configurations and Segments will NOT deploy in an Azure PaaS xP1 configuration. I have tested Azure PaaS xP0 and this is not an issue.

There are Zero Errors in the logs.

When Trying to Deploy a Single Segment

CM Log

2018-01-23T16:12:09  PID[13960] Information ManagedPoolThread #4 16:12:09 INFO  Job started: Index_Update_IndexName=sitecore_master_index
2018-01-23T16:12:09  PID[13960] Information ManagedPoolThread #4 16:12:09 INFO  Job ended: Index_Update_IndexName=sitecore_master_index (units processed: )
2018-01-23T16:12:11  PID[13960] Information 7752 16:12:11 INFO  AzureSearch Query [sitecore_master_index]: &search=datasource_1:(sitecore)&$filter=(latestversion_1 eq true and (path_1/any(t:t eq '154d56cc0de243c7bbc0a25bd7ffd901')) and (path_1/any(t:t eq '154d56cc0de243c7bbc0a25bd7ffd901')) and latestversion_1 eq true and not search.ismatchscoring('group_1:(154d56cc0de243c7bbc0a25bd7ffd901)', null, 'full', null))&queryType=full&$skip=0&$top=20&$count=true
2018-01-23T16:12:17  PID[13960] Information 8260 16:12:17 INFO  [Experience Analytics]: Deploying segment 'All visits by entry page' with id 96b651b5-c0c6-45f3-a043-fdfb03758008 for dimension 24307e3b-951f-463c-9310-475757ea2871 using reference data client
2018-01-23T16:12:17  PID[13960] Information ManagedPoolThread #3 16:12:17 INFO  Job started: Index_Update_IndexName=sitecore_master_index
2018-01-23T16:12:17  PID[13960] Information ManagedPoolThread #3 16:12:17 INFO  Job ended: Index_Update_IndexName=sitecore_master_index (units processed: )
2018-01-23T16:12:17  PID[13960] Information ManagedPoolThread #3 16:12:17 INFO  Job started: Index_Update_IndexName=sitecore_master_index
2018-01-23T16:12:17  PID[13960] Information 8260 16:12:17 INFO  [Experience Analytics]: Deploying segment 'All visits by entry page' with id 96b651b5-c0c6-45f3-a043-fdfb03758008 for dimension 24307e3b-951f-463c-9310-475757ea2871 using reference data client
2018-01-23T16:12:17  PID[13960] Information ManagedPoolThread #3 16:12:17 INFO  Job ended: Index_Update_IndexName=sitecore_master_index (units processed: )
2018-01-23T16:12:17  PID[13960] Information ManagedPoolThread #3 16:12:17 INFO  Job started: Index_Update_IndexName=sitecore_master_index
2018-01-23T16:12:17  PID[13960] Information 8260 16:12:17 INFO  AUDIT (sitecore\admin): Execute workflow command. Item: master:/sitecore/system/Marketing Control Panel/Experience Analytics/Dimensions/Pages/By entry page/All visits by entry page, language: en, version: 1, id: {96B651B5-C0C6-45F3-A043-FDFB03758008}, command: /sitecore/system/Workflows/Segment/Initializing/Deploy, previous state: Initializing, next state: /sitecore/system/Workflows/Segment/Deployed, user: sitecore\admin
2018-01-23T16:12:17  PID[13960] Information ManagedPoolThread #3 16:12:17 INFO  Job ended: Index_Update_IndexName=sitecore_master_index (units processed: )
2018-01-23T16:12:17  PID[13960] Information 8260 16:12:17 INFO  AzureSearch Query [sitecore_testing_index]: &search=This_Is_Equal_ConstNode_Return_Nothing
2018-01-23T16:12:18  PID[13960] Information ManagedPoolThread #3 16:12:17 INFO  Job started: Index_Update_IndexName=sitecore_master_index
2018-01-23T16:12:18  PID[13960] Information 8260 16:12:18 INFO  AzureSearch Query [sitecore_testing_index]: &search=This_Is_Equal_ConstNode_Return_Nothing
2018-01-23T16:12:18  PID[13960] Information 8260 16:12:18 INFO  AzureSearch Query [sitecore_testing_index]: &search=This_Is_Equal_ConstNode_Return_Nothing
2018-01-23T16:12:18  PID[13960] Information ManagedPoolThread #3 16:12:18 INFO  Job ended: Index_Update_IndexName=sitecore_master_index (units processed: )
2018-01-23T16:12:19  PID[13960] Information 5716 16:12:19 INFO  AzureSearch Query [sitecore_master_index]: &search=datasource_1:(sitecore)&$filter=(latestversion_1 eq true and (path_1/any(t:t eq '154d56cc0de243c7bbc0a25bd7ffd901')) and (path_1/any(t:t eq '154d56cc0de243c7bbc0a25bd7ffd901')) and latestversion_1 eq true and not search.ismatchscoring('group_1:(154d56cc0de243c7bbc0a25bd7ffd901)', null, 'full', null))&queryType=full&$skip=0&$top=20&$count=true
2018-01-23T16:12:19  PID[13960] Information ManagedPoolThread #2 16:12:19 INFO  Job started: Sitecore.ListManagement.Operations.UpdateListOperationsAgent
2018-01-23T16:12:19  PID[13960] Information ManagedPoolThread #2 16:12:19 INFO  Job ended: Sitecore.ListManagement.Operations.UpdateListOperationsAgent (units processed: )
2018-01-23T16:12:29  PID[13960] Information ManagedPoolThread #1 16:12:29 INFO  Job started: Sitecore.ListManagement.Operations.UpdateListOperationsAgent
2018-01-23T16:12:29  PID[13960] Information ManagedPoolThread #1 16:12:29 INFO  Job ended: Sitecore.ListManagement.Operations.UpdateListOperationsAgent (units processed: )

Processing Log (Same Time Frame)

2018-01-23T16:11:46  PID[3624] Information 6572 16:11:46 INFO  Cache created: 'ExperienceAnalytics.Sites' (max size: 1MB, running total: 1682MB)
2018-01-23T16:12:27  PID[3624] Information 5544 16:12:27 INFO  [Path Analyzer](.) Map EnsureAllDeployed started.
2018-01-23T16:12:27  PID[3624] Information 5544 16:12:27 INFO  [Path Analyzer](.) Map EnsureAllDeployed complete.
2018-01-23T16:13:27  PID[3624] Information 5544 16:13:27 INFO  [Path Analyzer](.) Map EnsureAllDeployed started.
2018-01-23T16:13:28  PID[3624] Information 5544 16:13:28 INFO  [Path Analyzer](.) Map EnsureAllDeployed complete.

This has left me with no choice but to log a Sitecore Support Ticket.

Update 3

Steps to repeat:

Using:

• Sitecore Azure Toolkit 2.0
• SXP 9.0.1 ARM Templates from the GitHub

• SXP 9.0.1 171219 XP Scaled Packages

  1. Deploy to Azure xP1
  2. Open up Azure SQL Server Client Ports
  3. Open SQL Server notes that Segments table is empty.
  4. Deploy Segments from Control Panel
  5. Segments table still empty.

So, from my explorations, here is what I am seeing:

• 279 rows in refdata-db.xdb_refdata.Definitions.
• 12 rows in xdb_refdata.DefinitionTypes
• 856 rows in xdb_refdata.DefinitionCultures
• 283 rows in xdb_refdata.DefinitionMonikers
• 16 rows in xdb_processing_pools.GenericProcessingDefinitions;
• 0 rows in all other xdb_processing_pools
• Contacts and Interactions show up in my shards
• Experience Profile is showing Contacts that I have identified along with triggered goals and value that has been added.
• 0 Rows in all of my Reporting DB Fact_SegmentMetric tables and 0 Rows in all of my Segment tables
• Deploy of Marketing Definitions of Segments completes and is an immediate completion with no movement in any log when that happens.
• Deploy of Marketing Definitions of Goals completes after some time, and I see the CM Log doing processing and publishing goals but no movement in any of the other logs.
• Deploy of Marketing Definitions of Page Events stops after about 5 minutes. The screen says there was an error and to check the Sitecore logs. I check Sitecore logs and there are no errors in the logs, and it appears Async tasks are still processing because I see log scroll from publishing page events and doing work on them. No other log messages (error or otherwise) in any other log.
• All I see for logging in the Processing Server is deploying of PAth Analyzer Deploy Maps
• Visits are recording on the Contact.
• Contact Facet data is saving and showing in Experience Profile
• Personalization off of a previously triggered goal is working.
• Experience Analytics show 0 data and no errors, no graphs, no visits.

Answer 1

After many days of research, I have determined it all boiled down to missing a step. Mistakes happen.

Post Deployment Steps

Once the deployment is complete, there are a few steps that you should take.

• Mandatory Steps
  • Building Indexes
    • Rebuild All Indexes from Control Panel
  • Build Link Databases
    • Rebuild the Link Databases from Control Panel
  • Deploy Marketing Definitions
    • From Control Panel, Deploy all Marketing Data

Building the Link Databases was the step I missed. By missing it, Definitions never really got deployed. And they would NOT get deployed until Link Databases were rebuilt. Took me 6 installs and some head scratching to figure this one out.

Answer 2

I faced similar issue in Sitecore 9.2. In addition to Pete's answer, you should also ensure that the below 2 configs in your xconnect instance are updated to enable tracking of anonymous users as in snapshot below (Sitecore 9 has anonymous user tracking disabled by default)-

• App_data\Config\sitecore\SearchIndexer\sc.Xdb.Collection.IndexerSettings.xml
• App_Data\jobs\continuous\IndexWorker\App_Data\config\sitecore\SearchIndexer\sc.Xdb.Collection.IndexerSettings.xml

Answer 3

I had the same issue.

No analytics for like 10 last days, despite the fact that absolutely nothing was changed on the server.

I rebuild xDB index, I redeployed marketing definitions - nothing helped.

I checked that there was no data in reporting database since 10 days so I started to rebuild it. It was extremely slow.

In the end I found out that it was corrupted Device Detection file that was on the servers - looks like there must have been some issue as all CD servers and CM server had corrupted file.

With that corrupted file data was being processed but extremely slow - at rate ~100 interactions per day, so really not noticeable.

The moment I replaced the old Device Detection database file with a correct one, everything started to work again and after couple of hours all the data was processed and in reporting db, without any server restarts needed.