Azure Search duplicate results for multilingual countries in same site with different language versions

Question

We have a multi-lingual site where the group of countries (say for example Asia Region) is hosted in one Site folder.

Thus Home, Pages, Settings have different language versions (zh-HK, en-HK, en-AU)

We configured Azure search for this region with index: "site_asia_web_index" with a crawler root configured to say /sitecore/brandsite/asia

However when I search for some term in the Australia site it returns the result for pages in en-HK, en-AU and even zh-HK. The non-Australia results ultimately go to 404 page.

Some of the important part of my code snippet just to make sure it is correct:

IEnumerable<Tuple<string, string, string>> mustConditions = filters.Where(x => x.Item3.Equals("must"));
System.Linq.Expressions.Expression<Func<AzureSearchResult, bool>> baseQuery = PredicateBuilder.True<AzureSearchResult>(); 
IEnumerable<Tuple<string, string, string>> shouldConditions = filters.Where(x => x.Item3.Equals("should"));

mustQuery = mustQuery.And(l => l.Language == Sitecore.Context.Language.Name);
baseQuery = baseQuery.And(contentQuery).And(mustQuery).And(templateQuery);
IQueryable<AzureSearchResult> queryRunner = context.GetQueryable<AzureSearchResult>(new CultureExecutionContext(Sitecore.Context.Language.CultureInfo)).Where(baseQuery);

This is the only filter I have for the language. Am I missing something here?

Note: using Sitecore Version 8.2 Update 5 and we already have the Sitecore patch with apiVersion=2015-02-28-Preview

Answer 1

I do recall a bug in 8.2 update4 where the language search in Azure did not work properly with dashes in the language. The results would be for any part of the "language-culture". E.g. searching for fr-BE would give results for fr-BE, nl-BE, fr-FR and so on. This seems like your case (en-HK, en-AU and zh-HK all share at least the language or the culture).

This bug was fixed in 8.2 update 7. It can be found in the release notes of that version: https://dev.sitecore.net/Downloads/Sitecore%20Experience%20Platform/82/Sitecore%20Experience%20Platform%2082%20Update7/Release%20Notes (reference number 171546)

When you use Azure search queries, to search for items in a specific language that contains a hyphen in its name (for example, en-AU), the results incorrectly contain items for all languages.​

If you cannot upgrade, I would advice to contact Sitecore support to get a patch for your version.