To answer your questions directly, no, you should not install CES or the Search API on your CD servers.
As for your second question, yes, it should query your CM Search API.
It is even recommended that you install CES and the Search API on a different server so that working with Sitecore won't affect Coveo Enterprise Search performance (or the other way around).
The reasoning behind this is that you already have different indices for the master and web databases. When querying against the index, if Sitecore is using the Web database for its content, Coveo for Sitecore will automatically pick the Web index (usually called
Coveo_web_index). So there is no "CM server" or "CM Search API", just a "CES Server" and "Search API" that can be used to query either CM or CD content.
For more information on the topic, I would suggest that you have a look at our deployment guide, which contains many topics on various processes of the installation. More importantly, the on-premises only topics.
As a side note, I would also suggest that you try out Coveo Cloud, it is way easier to configure and has much more features! Development of Coveo Enterprise Search has been stopped for a couple of years now, and it comes at its end-of-life in 5 months. You are also missing out on many Coveo for Sitecore 5 features!