"The remote server returned an error: (401) Unauthorized" Error while accessing item binary data in Coveo with Windows Auth

Question

I have Windows Authentication on the IIS of sitecore site (for limited external access). Coveo is running remotely on another server. When trying to access item binary data p_Args.CoveoItem.BinaryData from <coveoPostItemProcessingPipeline> I get 401 error. Is there a way to pass credential when accessing this item?

Error thrown my processor:

ManagedPoolThread #10 17:35:49 ERROR An exception occurred while trying to fetch the HTML content of the document http://ca.site.com/en/page-to-index.
Exception: System.Net.WebException
Message: The remote server returned an error: (401) Unauthorized.
Source: System
at System.Net.HttpWebRequest.GetResponse()
at
Coveo.SearchProvider.Processors.HtmlItemContentFetcher.FetchItemContent(String p_Url)
at
Coveo.SearchProvider.Processors.HtmlContentInBodyWithRequestsProcessor.Process(CoveoPostItemProcessingPipelineArgs p_Args)

Answer 1

I ended up doing the following:

1. Create a separate IIS site pointing to original site's directory.
2. Allow anonymous access on it
3. Update ServerUrl path in Coveo.SerchProvider.config to new site
4. Set hostname on the server appropriately
5. On CES server, added hostname for new site

With this when indexing, Coveo is able to bypass authentication on sitecore instance by hitting the 2nd site. And since they share same DBs and files, it seems to be indexing correctly.