5

We've written and tested two-way, Contact/Lead integration between Sitecore and a third-party CRM and it generally works as expected. The problem is how to deal with updates that come in into Sitecore via integration for a contact that is logged in and has an open session.

When the session ends, the integration update is overridden. The question is, how does Sitecore, or other third-parties for that matter, deal with Contact updates coming in via INT this conflict. We've researched this without as much as uncertified Sitecore developers can (old/badly-written blog articles) and didn't find much.

There ARE integration modules for other CRM apps, so how do they make it work? We have a theory but want to validate it before moving forward. Any ideas??

4
  • What version of Sitecore are you using? Mar 30 '17 at 14:49
  • Eight point two (stackexchange doesn't seem to like number-only comments).
    – D Diehl
    Mar 30 '17 at 14:59
  • are you using DEF module? Mar 30 '17 at 15:03
  • Do you mean the Data Exchange module? If so, no.
    – D Diehl
    Mar 30 '17 at 15:29
3

Have you tried Sitecore's Data Exchange Framework module? It is designed for integrating with third-party tools and the optional Sitecore Provider includes out of the box support for syncing/updating xDB contacts with/to/from a third-party system.

The documentation for the Sitecore Provider covers many of the components that are included out of the box for syncing data with Sitecore, though I believe that there are more included and available to you once you have the module installed.

Overview of DXF

Sitecore's Data Exchange Framework is essentially a two-way data transfer tool that is designed to be highly versatile, extensible, consistent in structure and fully scalable.

Put more simply, it's an integration tool that isn't just an integration tool: it's so much more.

In their Documentation, Sitecore describes DXF by stating that the:

Data Exchange Framework is designed to facilitate the transfer of data between two systems.

The system being read from is called the source system. The system being written to is called the target system.

What Can DXF Do?

Here is a high-level of some of the things that DXF can do:

  • Sitecore to X
    • Integrate Sitecore with an external (even third-party) system/source
      • CRM
      • File (CSV, Excel, text, ... if you can parse it then you can use it)
      • Other third-party services
      • Other Sitecore system
      • And more
    • Import/Export/Migrate/Update/Sync/Delete/Create Sitecore data via Integration with an external system/source
    • Import/Export/Migrate/Update/Sync/Delete/Create xDB data via Integration with an external system/source
  • X to Y
    • Create an integration between two non-Sitecore systems/sources
    • Sitecore doesn't even need to be the source or target of the data; DXF can be used to create integrations that have nothing to do with Sitecore
  • Dynamics CRM
    • DXF is used for the Dynamics CRM Connect module, which serves as Sitecore's CRM Connect suite for integrating CRM systems with Sitecore XP
    • The Dynamics CRM Connect module includes several tools that provide out of the box support for integrating with a CRM

In their Documentation, Sitecore provides the following table that identifies three examples of what DXF can do:

enter image description here

Scaling DXF

DXF is fully scalable, and can be run outside of Sitecore to reduce load via the Remote SDK.

Scaling DXF by running it remotely is similar to splitting a Sitecore instance by role (e.g. CM, CD, Reporting, Processing), except that no additional Sitecore licensing is required!

How DXF Works

Overview of Key Components

The Data Exchange Framework essentially runs by mapping data between two endpoints as part of an execution process called a pipeline batch.

Pipeline batches essentially define zero or more DXF pipelines that should execute in the defined sequence when the batch is run.

DXF pipelines are different from Sitecore pipelines, and are defined in Sitecore. Each pipeline contains zero or more pipeline steps that are somewhat analogous to a typical Sitecore pipeline's processors.

Each pipeline step executes a specific piece of logic for its pipeline. For example, a pipeline named "Read Foos data from Bar CRM" may have the high-level goal of reading "Foo" data from a CRM named "Bar". The first pipeline step in the pipeline may be "Read Foos from REST API", referencing an endpoint item named "Bar CRM REST API Endpoint Settings". The endpoint item would define the settings for connecting to the Bar CRM's REST API. The second pipeline step may be "Iterate Foo data and Run Sync Pipeline", which has the job of looping through all of the retrieved "Foo" data and running a second pipeline for synchronization for each "Foo" that was read.

Real-World Example (Open Source)

For Sitecore Hackathon (and we have continued development since), my team members (Pete Navarra and Hetal Dave) and I created an Azure Provider module for DXF named PaaSPort that provides everything necessary to rapidly configure Azure resource visualization inside of the Sitecore Content Editor. We also have API reference documentation, including descriptions of everything that plugs into DXF and more.

Note: I do not write this section to self-promote, but rather to give you a glimpse into a real-world use-case of DXF that has already been implemented and that you can use as an example to help learn about DXF. If you have any questions or issues, feel free to email the team at sitecore.theinjectables@gmail.com.

The PaaSPort module's Azure provider is designed to retrieve data from Azure services and the included demo creates Sitecore items to represent the various resources (Resource Groups, SQL Servers, Storage Accounts, CDN Profiles, etc.) as Sitecore items. Of course, Because PaaSPort is a module for DXF, Sitecore doesn't have to be the destination for the data. The PaaSPort module's Azure provider could be used to export this information to a file, send it to a third-party system, and much more.

In the following screenshot, you can see the endpoints, the pipeline batch, and the pipelines with their pipeline steps, all of which are used to retrieve Azure Resource Group data from the new Azure client services and create items to represent that data in Sitecore:

enter image description here

Additional analysis of the items in the screenshot:

  • Endpoints
    • The "Get Resource Groups" endpoint defines all of the settings for connecting to Azure via the Azure client services.
    • The "Sitecore Database" endpoint is included with the Sitecore Provider out of the box, and is used to "send" data to Sitecore
  • Pipeline Batches
    • The "Azure Resource Group to Sitecore Sync Batch" pipeline batch defines all of the pipelines that should be run to bring the Resource Group data into Sitecore
    • The displayed batch only points to the "Retrieve Resource Groups from Azure" pipeline, though other pipelines could be added, like in the scenario described by the below which describes another valid configuration of a pipeline batch:
      • Rename the Batch to "Azure to Sitecore Sync Batch"
      • Add additional Pipelines to retrieve Azure SQL Servers
      • Add additional Pipelines to retrieve Azure Storage Accounts
      • Set the Batch to run each of the following pipelines, in order
        • Retrieve Resource Groups from Azure
        • Retrieve SQL Servers from Azure
        • Retrieve Storage Accounts from Azure
  • Pipelines
    • The "Iterate Results from Azure and Run Pipeline" step loops through the read data objects and calls the "Azure Resource Group to Sitecore Sync" pipeline for each object that was read
    • The "Azure Resource Group to Sitecore Sync" pipeline is where the mapping and saving magic happens
    • Note that in each pipeline, each pipeline step has a single task (the "Iterate Results..." pipeline also has a single task, though slightly harder to see at first: run pipeline for each object that was read)

The "Beyond" Section

There is much more to DXF than what is included in this post. The documentation for DXF is pretty excellent and the Getting Started tutorial is thorough, easy to follow, and detailed/specific enough that I would even recommend it for developers new to working with Sitecore. It does take several hours, but as you run through it, you will learn all of the concepts and components necessary to understand DXF and you will have prepared yourself for expanding your implementation to support the integration that you're looking for.

Getting Started with DXF

While a bit lengthy, the Getting Started guide is pretty straightforward to follow, and takes you end-to-end through creating your own custom provider to begin syncing data.

Once you have completed your run through the guide (which also gets you started using the Sitecore Provider), you can start using the Sitecore Provider to create the integration for your contacts.

Provided and Supported by Sitecore at no Additional Cost

One of the great things about the Data Exchange Framework module is that it doesn't incur any additional licensing costs, and since it is provided by Sitecore it is also supported by Sitecore under your existing Sitecore Support plan.

7
  • I remember looking at DEF a while ago but we chose not to use it because it pulls changes from the source in batch mode (triggered when the pipeline executes). We're triggering the changes based on events in the CRM and Sitecore. I guess we could have written the events to a message queue and had DEF read them in batch from there...but that seemed way overkill at the time. Plus we already have a middleware message processor between Sitecore and the CRM, so we have persistence. Another two queues seemed overkill.
    – D Diehl
    Mar 30 '17 at 18:02
  • If I understand correctly, you could do it without the queue too. Imagine that an event is fired on the CRM, causing the CRM to make a call to an exposed service in your Sitecore solution. The service that is called then executes a pipeline batch with the job of syncing the passed contact(s) or the contact(s) retrieved by requesting from the middleware message processor or the CRM. From the other side, an event is fired in Sitecore that triggers a handler or processor that triggers the same (or another) pipeline batch to request the contact(s) from the middleware message processor or CRM... Mar 30 '17 at 18:09
  • That may be the solution. The Sitecore-to-CRM integration is working fine. It's the CRM-to-Sitecore that has a problem.
    – D Diehl
    Mar 30 '17 at 18:17
  • Here's a related question: If I call ContactManager.TryLoadContact() for an account that's logged in, I get a successful lock. Should it not return .AlreadyLocked? If it's successful, I would think that it could have found the contact in Shared Session State and things would have worked beautifully, but it isn't. So, can i assume that even though the contact is in SharedSessionState, it's not locked?
    – D Diehl
    Mar 30 '17 at 18:23
  • Conceptually yes, that is what it should be doing. When a session is started and a contact is created/identified, Sitecore creates a Lease entry for the contact with the contact ID in MongoDB (collections). The Lease locks the contact. However, remember that site users and contacts are not the same and are not integrated or linked with one another, by default. For Sitecore to be able to resolve an authenticated user's contact then you will need to make sure that you are calling Sitecore.Analytics.Tracker.Current.Session.Identify(identifier) on each request where your contact is tracked. Mar 30 '17 at 18:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.