Have you tried Sitecore's Data Exchange Framework module? It is designed for integrating with third-party tools and the optional Sitecore Provider includes out of the box support for syncing/updating xDB contacts with/to/from a third-party system.
The documentation for the Sitecore Provider covers many of the components that are included out of the box for syncing data with Sitecore, though I believe that there are more included and available to you once you have the module installed.
Overview of DXF
Sitecore's Data Exchange Framework is essentially a two-way data transfer tool that is designed to be highly versatile, extensible, consistent in structure and fully scalable.
Put more simply, it's an integration tool that isn't just an integration tool: it's so much more.
In their Documentation, Sitecore describes DXF by stating that the:
Data Exchange Framework is designed to facilitate the transfer of data
between two systems.
The system being read from is called the source system. The system
being written to is called the target system.
What Can DXF Do?
Here is a high-level of some of the things that DXF can do:
- Sitecore to
- Integrate Sitecore with an external (even third-party) system/source
- File (CSV, Excel, text, ... if you can parse it then you can use it)
- Other third-party services
- Other Sitecore system
- And more
- Import/Export/Migrate/Update/Sync/Delete/Create Sitecore data via Integration with an external system/source
- Import/Export/Migrate/Update/Sync/Delete/Create xDB data via Integration with an external system/source
- Create an integration between two non-Sitecore systems/sources
- Sitecore doesn't even need to be the source or target of the data; DXF can be used to create integrations that have nothing to do with Sitecore
- Dynamics CRM
- DXF is used for the Dynamics CRM Connect module, which serves as Sitecore's CRM Connect suite for integrating CRM systems with Sitecore XP
- The Dynamics CRM Connect module includes several tools that provide out of the box support for integrating with a CRM
In their Documentation, Sitecore provides the following table that identifies three examples of what DXF can do:
DXF is fully scalable, and can be run outside of Sitecore to reduce load via the Remote SDK.
Scaling DXF by running it remotely is similar to splitting a Sitecore instance by role (e.g. CM, CD, Reporting, Processing), except that no additional Sitecore licensing is required!
How DXF Works
Overview of Key Components
The Data Exchange Framework essentially runs by mapping data between two endpoints as part of an execution process called a pipeline batch.
Pipeline batches essentially define zero or more DXF pipelines that should execute in the defined sequence when the batch is run.
DXF pipelines are different from Sitecore pipelines, and are defined in Sitecore. Each pipeline contains zero or more pipeline steps that are somewhat analogous to a typical Sitecore pipeline's processors.
Each pipeline step executes a specific piece of logic for its pipeline. For example, a pipeline named "Read Foos data from Bar CRM" may have the high-level goal of reading "Foo" data from a CRM named "Bar". The first pipeline step in the pipeline may be "Read Foos from REST API", referencing an endpoint item named "Bar CRM REST API Endpoint Settings". The endpoint item would define the settings for connecting to the Bar CRM's REST API. The second pipeline step may be "Iterate Foo data and Run Sync Pipeline", which has the job of looping through all of the retrieved "Foo" data and running a second pipeline for synchronization for each "Foo" that was read.
Real-World Example (Open Source)
For Sitecore Hackathon (and we have continued development since), my team members (Pete Navarra and Hetal Dave) and I created an Azure Provider module for DXF named PaaSPort that provides everything necessary to rapidly configure Azure resource visualization inside of the Sitecore Content Editor. We also have API reference documentation, including descriptions of everything that plugs into DXF and more.
Note: I do not write this section to self-promote, but rather to give you a glimpse into a real-world use-case of DXF that has already been implemented and that you can use as an example to help learn about DXF. If you have any questions or issues, feel free to email the team at firstname.lastname@example.org.
The PaaSPort module's Azure provider is designed to retrieve data from Azure services and the included demo creates Sitecore items to represent the various resources (Resource Groups, SQL Servers, Storage Accounts, CDN Profiles, etc.) as Sitecore items. Of course, Because PaaSPort is a module for DXF, Sitecore doesn't have to be the destination for the data. The PaaSPort module's Azure provider could be used to export this information to a file, send it to a third-party system, and much more.
In the following screenshot, you can see the endpoints, the pipeline batch, and the pipelines with their pipeline steps, all of which are used to retrieve Azure
Resource Group data from the new Azure client services and create items to represent that data in Sitecore:
Additional analysis of the items in the screenshot:
- The "Get Resource Groups" endpoint defines all of the settings for connecting to Azure via the Azure client services.
- The "Sitecore Database" endpoint is included with the Sitecore Provider out of the box, and is used to "send" data to Sitecore
- Pipeline Batches
- The "Azure Resource Group to Sitecore Sync Batch" pipeline batch defines all of the pipelines that should be run to bring the Resource Group data into Sitecore
- The displayed batch only points to the "Retrieve Resource Groups from Azure" pipeline, though other pipelines could be added, like in the scenario described by the below which describes another valid configuration of a pipeline batch:
- Rename the Batch to "Azure to Sitecore Sync Batch"
- Add additional Pipelines to retrieve Azure SQL Servers
- Add additional Pipelines to retrieve Azure Storage Accounts
- Set the Batch to run each of the following pipelines, in order
- Retrieve Resource Groups from Azure
- Retrieve SQL Servers from Azure
- Retrieve Storage Accounts from Azure
- The "Iterate Results from Azure and Run Pipeline" step loops through the read data objects and calls the "Azure Resource Group to Sitecore Sync" pipeline for each object that was read
- The "Azure Resource Group to Sitecore Sync" pipeline is where the mapping and saving magic happens
- Note that in each pipeline, each pipeline step has a single task (the "Iterate Results..." pipeline also has a single task, though slightly harder to see at first: run pipeline for each object that was read)
The "Beyond" Section
There is much more to DXF than what is included in this post. The documentation for DXF is pretty excellent and the Getting Started tutorial is thorough, easy to follow, and detailed/specific enough that I would even recommend it for developers new to working with Sitecore. It does take several hours, but as you run through it, you will learn all of the concepts and components necessary to understand DXF and you will have prepared yourself for expanding your implementation to support the integration that you're looking for.
Getting Started with DXF
While a bit lengthy, the Getting Started guide is pretty straightforward to follow, and takes you end-to-end through creating your own custom provider to begin syncing data.
Once you have completed your run through the guide (which also gets you started using the Sitecore Provider), you can start using the Sitecore Provider to create the integration for your contacts.
Provided and Supported by Sitecore at no Additional Cost
One of the great things about the Data Exchange Framework module is that it doesn't incur any additional licensing costs, and since it is provided by Sitecore it is also supported by Sitecore under your existing Sitecore Support plan.