I am looking for the best way to synchronize profile data from an external system.
We authenticate users using an external authentication service. We use the Sitecore virtual user mechanism to log into Sitecore. Once a user is logged in we need to call a backoffice service to get user profile information. We intend to sync this information with the Sitecore Contact (in XDb).
I want the synchronization of profile data to run as an asynchronous process because the backend service is slow and takes some time to respond. Also potentially we can have a lot of people login at the same time.
We do not need the profile data immediately (but as soon as possible).
The obvious way to fire an asynchronous task would be to use a Sitecore Job. This would mean that I would fire of a job to synchronize profile data everytime a user logs in. I am afraid that this will have a negative impact on performance when lots of user log in and It is not clear to me how these Jobs are prioritized.
I think I am looking for some sort of queuing system (more or less like the Sitecore publishing queue) to handle these tasks. The event queue does also not seem to meet my requirements since this is actually more a messaging than a queuing mechanism.
What would be the best solution to handle this requirement? Preferably keeping as close to Sitecore as possible.