I am trying to understand the flow of data from the time a user opens/ click a link in the e-mail to the time it is displayed.
I am listing my understanding below.
Connecting the dots backwards:
Sitecore has provided an image at https://doc.sitecore.com/developers/90/platform-administration-and-architecture/en/email-marketing.html.
So, the data comes from xConnect. In the background, xConnect collects data from xDB index and xDB Collection database (both the shards).
Sitecore xConnect doesn't index PII data. So, I understand that data under section highlighted below would be coming from xDB Collection database.
The data on the upper section of the page or under Overview section would be coming from xDB index. So, the data would be getting Search Indexer under xConnect.
How data flows from e-mail: In the root manager, we provide the URL of the website on the CD instance of the Sitecore. Once a user opens the mail the 1 px image gets downloaded from the CD instance and thus the opening of mails are tracked. Similarly, the links in the e-mail are resolved through the CD instance of the Sitecore and thus the clicks and corresponding details are captured.
On the CD instance, the request handler handles the request and executes the corresponding pipelines.
What I am not able to understand:
What happens to the data in between the handler on the content delivery and the data reaching xDB databases or the indexer in xConnect?
When the processing and reporting databases come in the picture?
Campaigns table in EXM master also has fields to hold the click, bounce, recipients and related data. When they are used and when they are populated?
Thanks in advance.