There can be only one
If I misunderstood the above statement and you actually have 3 completely separate xDB collection databases, then you might be having a harder time as that is not a supported scaling model for the xDB database in 9.0 and 9.1
Here is the horizontal scaling documentation: https://doc.sitecore.com/developers/90/platform-administration-and-architecture/en/scale-horizontally.html
It's all the same to xDB
However, if you are running a supported model with a single xDB collection database, you should be fine. Ultimately, regardless of the number of zones, there is only one xDB behind xConnect so whichever xConnect you connect to will get the same data because you are connecting to the same underlying data store.
Searching against the index
From an indexing perspective, any calls that trigger searches against the xDB Search Index could be impacted depending on how you have decided to scale your indexing. However, in a typical setup, all your indexing endpoints have the same indexes in them so effectively your lookups should behave the same regardless of the xConnect endpoint.
Remember that there is only one xConnect Search Indexer. From the docs:
Although it is possible to scale out the xConnect Search service,
there can only be one instance of the xConnect Search Indexer. In
Microsoft Azure, the indexer web job is automatically configured to
run on one instance with the ability to fail over to another instance.
Keep impact low
I would recommend having CM connect to whichever zone is going to have the least impact to your content delivery endpoints. While the likelihood of your xConnect requests from CM causing a performance hit is low, if you have a choice, that would be how to go.