It's a difficult one; a question I've considered many times over in the years I've been working with Sitecore.
I can start with some numbers. I've done perhaps 30 or so projects that involved data integration of some kind. 29 of which, I ended up importing the data. I've only ever done 1 project where I've implemented a full Data Provider - and I'll ...
I've used DataProviders fairly extensively and I'd say that they're good if:
1. Your API is fast (not too many calls to get the info, easily cached etc)
2. You need data to be up do date (to reasonable caching margins)
3. You're not going to have masses of items within each folder
The biggest issue I've found is how hungry Sitecore is getting children - ...
A good example is exposing information from a PIM (product information manager) as native Sitecore items in the Sitecore tree. With this, production information can be accessed via the same API's as you'd use for accessing content managed via Stiecore.
I think there are some good individual examples here of useful Data Provider usage, and some good arguments about when/why you should not use them, but I'd like to summarize the situations in which a Data Provider can work well, and provide some concrete examples.
A data provider can potentially be a good option if:
The data is local to the Sitecore ...
I have used them before to inject custom Security on an item level. In my particular case the security requirements couldn't be fulfilled by Sitecore security, so my data provider would perform the calculation instead and return items appropriately.
I wrote about the implementation details here: http://jockstothecore.com/item-provider-for-security/
Sitecore Commerce Connect employs a similar approach. From the Commerce Connect Integration Guide:
3.1.5 ID Mapping By design, the remote product repository is always regarded as the main repository, which by default owns the products.
That makes the ID of the products and artifacts in the external system
the primary key.
In Sitecore, the IDs of the ...
In the past, I have recommended a data provider while working with a healthcare client. They had a member database that contained HIPAA (Health Insurance Portability and Accountability Act) protected data that was not suitable for storing in Sitecore, however, was the source of truth for pieces of information that needed to be accessed within the confines of ...
I have used Data Providers for PIMs, employee/personell/staff/etc. directories, and also for large quantities of data points coming from a source that is managed in another system but referenced by Sitecore items.
There are a few really great features of Data Providers that I haven't seen anyone else mention yet:
You can easily tie the "items" that they "...
The Sitecore 6.6 scaling guide has this to say about the IDTable:
You can store persistent mappings of arbitrary keys to Sitecore IDs in the Sitecore IDTable.
By default, this is only used by the Sitecore CMS WebDAV functionality to keep locks on media items in the CM environment and does not need to be synchronized with the CD environment.
Yes. But to do so, you need to invoke a feature that I am willing to bet only 1% of the 1% knows about.
For a dataProvider, you can disable groups of functionality. In your case you want to disable publishing.
For completeness, I'm showing a few examples. All in your <dataProvider> element:
You should try to write a function to generate deterministic guids. Found this: http://kechengpuzi.com/q/s2642141 as a possibility (example is based on paths, but you can use your id as input - if needed with a pre- or suffix).
It actually comes down to creating a hash and calling the Guid constructor with that.
I did not copy the example code here as ...
Your root item (probably a real Sitecore item) has to be set as a bucket. For your child items (the virtual ones from your provider), you need to set the Is Bucketable value to true on the 'Standard Values' of the template (so not on the template itself).
Definitively read this https://stackoverflow.com/questions/36692431/display-items-in-bucket-with-...
We have a situation were we will probably be moving data outside of Sitecore into a custom database and write a data provider on it. Might become a read-only data provider even.
We are facing a tree structure with products that has grown over a million items. The back-end systems with the original data cannot be used in a live environment for several ...
If you have data in a back-end system that has network access limitations, using a Data Provider allows Sitecore to publish the data as Items, sparing you the need to worry about VPNs or latency when accessing that data from a Content Delivery server.
It's not the only path you can take, and there are many caveats, but for front-end performance, it's ...
They are appropriate when you need to integrate with external data sources and want to take advantage of Sitecore's item processing, tracking and manipulation.
One example would be a university:
We have many different data sources that we need to work with, one of which is a Course Database. Each course gets it's own page and content associated with it.
You can do this by overriding Sitecore.Resources.Media.MediaProvider.
In the GetMediaUrl you'll have access to the MediaItem to check the template or a field. You can then return either the base.GetMediaUrl (i.e. the Sitecore one) or build and return an Azure CDN link to the image/resource.
I think this is a bug with Sitecore.
If you look more closely at their documentation you can see that in their sample code they have a TryCatch that expects a NullReferenceException:
I peeked inside the uCommerce Sitecore DLLs and found the class DataProviderReinitializeTask<T> which contained code for clearing the dataprovider cache.
To use in any location you'll need code like this:
var context = ObjectFactory.Instance.Resolve<UCommerce.Sitecore.ISitecoreContext>();
var dataProviderMaster = context.DataProviderMaster;
Sitecore Client Services is the one you need to be looking at. There might be a way with DXF as well but I am not too familiar with it.
As for Sitecore Client Services:
Sitecore.Services.Client provides a service layer on both the server and the client side of Sitecore applications that you use to develop data-driven applications.
I have had to do similar in the past have used deterministic GUIDs. I use the method described and shared here:
This algorithm follows RFC 4122, and is designed not to clash with GUIDs that are generated in other namespaces - so it won't clash with Guid.NewGuid() for example.
The IDTable is used to store persistant mappings of external keys to Sitecore IDs. Be aware though, that IDtable lookups requires an additional query to the database (caching aside), and may impact performance.
There are alternatives to using the IDTable, for example Sitecore Commerce Connect takes a different approach, by generating deterministic GUIDs from ...