It is a difficult one to answer completely. Putting in answer instead of comment because of the length.
Based on this link
https://doc.sitecore.com/en/developers/91/platform-administration-and-architecture/configure-a-search-and-indexing-provider.html
Sitecore supports 3 search providers at the moment. Good to first quick check with Sitecore support if a Custom Search provider is possible or not. Means are there any hard-coded references of these 3 options in core Sitecore dlls.
We have below,
For Solr = webroot\App_Config\Sitecore\ContentSearch
For Azure = webroot\App_Config\Sitecore\ContentSearch.Azure
Maybe for the custom search providers we may duplicate one of these folders and explore the configuration and provide custom classes wherever necessary.
>>What mainly drives a particular index to show on the Indexing Manager?
Upon checking the backend of the indexing manager dialog i.e. Sitecore.ContentSearch.Client.Forms.IndexingManagerWizard
webroot\sitecore\shell\Applications\ContentSearch\IndexingManager.xml
Indexes information is fetched from ContentSearchManager.Indexes
This leads to.
public static IEnumerable<ISearchIndex> Indexes => (IEnumerable<ISearchIndex>) ContentSearchManager.SearchConfiguration.Indexes.Values;
=>
Checked AddIndex
method in Sitecore.ContentSearch.ContentSearchConfiguration
which actually adds the indexes information based on Sitecore.ContentSearch.{SearchProvider}.Index.{Database}.config (e.g Sitecore.ContentSearch.Azure.Index.Master.config)
or your copy of this file to implement the custom indexing used by your provider.
Basically we need to satisfy the required custom types used in above config specific to search provider needs.
Like we need to have custom CloudSearchProviderIndex (implementing ISearchService) and CompositeSearchService (implementing ISearchIndex)
These interfaces are used in ContentSearchConfiguration
.
Good to explore Sitecore.ContentSearch.Azure.dll
in dotpeek for all these existing types and have our custom types for custom search provider accordingly.
