xDB Data Migration Tool Error

Question

I'm using xDB Data Migration Tool 2.0 to migrate data from MongoDB (Sitecore 8.2) to xconnect (Sitecore 9.0.2). The tool runs without errors for around 60k records. Then it breaks with Errors like the following :

Operation #90, ReferenceNotFound, Interaction, LegacyXdb (pipeline: Process Single Interaction from MongoDB Pipeline, pipeline step: Add Interaction Model to xConnect Batch, pipeline step identifier: 7cd619a9-1ce0-44da-bf42-22736247667c)

I've confirmed that the JSON file for the facets has been deployed to the xconnect service and the xconnectSearch service.

Does anyone know what could be causing this?

Answer 1

I opened up a support ticket and Sitecore Support recommended the following steps :

1. Stop Sitecore and xConnect instances
2. Since it's pre-production, clear all the data from all the tables in Shard0, Shard1 dbs.
3. Clear all the data from InteractionLiveProcessingPool of Processing.Pools db.

4. Perform the query below against Analytics db in Mongo. This query should return contact with the duplicate identifiers.You have to get rid of the duplicate identifiers before migrating the data.

5. Apply the solution from the kb article: https://kb.sitecore.net/articles/397292

6. Clear the data inside [WebsiteRoot]\App_Data\Submit Queue folder

7. Clear all the logs on Sitecore and xConnect intsances.
8. Run xConnect and Sitecore
9. Run the migration.

Query to find duplicates in Mongo :

db.getCollection('Contacts').aggregate([{$group: {_id: "$Identifiers.Identifier",count:{$sum: 1}}},{$match: {count: {"$gt": 1}}}]);

Query to delete the duplicate record. Note that this will return a lot of records that have a null Identifier, those are anonymous contacts you may want to keep those :

 db.getCollection('Contacts').remove({"Identifiers.Identifier": "YOUR_DUPLICATE_ID"},true);

After deleting the duplicate record running the migration was successful