First of all, when enabling indexing of anonymous contacts in the xDB index, it's important to keep in mind that the size of the index and the amount of data that needs to be processed will increase significantly. And it will increas not only index but also the size of the
xDB SQL database. This is because for public websites, the majority of traffic comes from anonymous visitors, and enabling indexing for these visitors means that data will need to be collected and indexed for a much larger number of interactions.
In general, you can expect that the size of the
xDB index and the amount of data that needs to be processed will approximately double when enabling indexing of anonymous contacts. This can have an impact on the performance and resource usage of both
Azure Cognitive Search and the
xConnect Search Indexer, and it's important to ensure that both are sized appropriately to handle the increased load.
Regarding the specific settings in the
sc.Xdb.Collection.IndexerSettings.xml file, both
BatchSize can affect the indexing performance and should be considered when sizing appropriately for the indexing load. The
ParallelizationDegree setting determines how many parallel threads the
xConnect Search Indexer uses to index data, while the BatchSize setting determines how many documents are processed in a single batch.
To ensure that Solr / Azure Cognitive Search and the xConnect Search Indexer are sized appropriately, you may want to consider the following general approach:
Review the documentation for Solr / Azure Cognitive Search and xConnect Search Indexer to understand the recommended hardware and software requirements.
Monitor the resource usage (such as CPU, memory, disk usage, and network bandwidth) on the servers hosting Solr / Azure Cognitive Search and xConnect Search Indexer to identify any bottlenecks or performance issues.
Test the indexing load with the new configuration settings and monitor the system resources to ensure that the system can handle the additional load.
Adjust the configuration settings as needed to optimize the performance and ensure that the system can handle the expected indexing load.
It's important to note that sizing appropriately for indexing can be a complex process that depends on many factors, including the size and complexity of your xDB data, the number of anonymous contacts you expect to index, and the overall system configuration.