Sitecore 9.0 update 1 Azure search is not stable

Question

Using Sitecore 9 update 1.

followed Sitecore documentation to implement the site search in Azure PaaS https://doc.sitecore.net/sitecore_experience_platform/setting_up_and_maintaining/search_and_indexing/configure_azure_search

As Azure Search can contain up to 1000 fields I have created custom search index for the large data in Sitecore which is imported from third party and I added the items into custom index.

using simple queries to return the content from the new index, the search is not stable and break multiple times during the day, sometimes it return data and other times not.

I noticed that document count in Azure changed after each publish

querable filter:

 var queryable = context.GetQueryable<SearchResultItem>();
            queryable = queryable.Filter(i => i.Paths.Contains([MyItem].ID));
            queryable = queryable.Filter(i => i.TemplateId == Templates.[MyItem].TemplateID);
            queryable = queryable.Filter(i => i.Language == Sitecore.Context.Language.Name);
            var indexresults = queryable.OrderBy(x => x.Name);
            List<Item> resultList = new List<Item>();
            resultList = queryable.Select(n => n.GetItem()).ToList();

Answer 1

I am running Sitecore 9.0 Update1 with Azure Search, and added custom index configurations in as well. The only time I found indexes are not returning results is when they are being re-indexed. So I recommend to check your indexing strategy, what do you currently have configured in your custom index "sitecore-product-index"?

Basic indexing strategies for Azure Search

One option is to only set the index strategy to manual, and this would be a good one for you to test first. The index will only rebuild when you manually reindex from your CM server (control panel > indexing manager), or you add code elsewhere to rebuild your index.

<strategies hint="list:AddStrategy">
    <strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/manual" />
</strategies>

If you only want it to rebuild on publish (or a manual build), which is recommended if it might be outdated after item changes are published, then set this:

<strategies hint="list:AddStrategy">
    <strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/onPublishEndAsync" />
</strategies>

You could add other strategies, but its good to start with a basic one, and add only if you find you need the index to be updated more regularly. Which brings me to the next point:

Azure Search Index SwitchOnRebuild

If you do update an index regularly or it needs to be available for searching at all times (critical functionality), then another option with Azure Search is to enable SwitchOnRebuild, which basically adds two copies of each index, and Sitecore automatically uses the one that isn't being built to do searches (zero index downtime). https://doc.sitecore.net/sitecore_experience_platform/setting_up_and_maintaining/search_and_indexing/indexing/zero_downtime_index_rebuild_in_azure_search

However, this was only made available with Sitecore 9.0 Update 2. So you will have to upgrade your version to use it.