Search indexing not working as expected

Question

I'm new to search and indexing. We are implementing this for a search feature on the website for the end user (not the internal search) and using Solr 4.10.3 with sc8.0 u5

1. Solr is setup with all the cores/indexes
2. I have created a patch config - MySite.ContentSearch.Solr.MySiteIndexConfiguration.config

which has custom configuration node

<indexConfigurations>
   <mysiteSolrIndexConfiguration type ..
   <indexAllFields>true</indexAllFields>
...
</indexConfigurations>

3. Also included required template & field IDs in respective nodes

The search works only partially. These are the issues I see:

(1) There are no results from Home page. Though I have included the home page template, its inherited template and all the other templates and fields where the term being searched, exists.

(2) To troubleshoot, I set <indexAllFields>false</indexAllFields> and removed all included template & field IDs from this config. But surprisingly, the search still gives results. How come !!!. Should it not return null or empty?

This is the config for web index

<index id="mysite_web_index" 
...
<configuration ref="contentSearch/indexConfigurations/mysiteSolrIndexConfiguration" />
<crawler type="Sitecore.ContentSearch.SitecoreItemCrawler, Sitecore.ContentSearch">
                <Database>web</Database>
                <Root>/sitecore/content/Home/MySite</Root>
              </crawler>

code:

using (var context = ContentSearchManager.GetIndex("mysite_web_index").CreateSearchContext())
{
  var query = context.GetQueryable<SearchResultItem>()
                    .Where(x => x.Content.Contains(searchText));
}

Answer 1

The search you are performing is on the Content property of the SearchResultItem:

[IndexField("_content")]
public virtual string Content { get; set; }

As you can see this value maps on the IndexField _content, which is a build-in computed field that aggregates the content. This is why your indexAllFields setting doesn't make any difference.

To see why your IncludeTemplates are not working, we would need to see that config ;) Make sure it looks something like this:

<indexConfigurations>
  <ProductIndexConfiguration ref="contentSearch/indexConfigurations/defaultSolrIndexConfiguration">
    ...
    <documentOptions ref="contentSearch/indexConfigurations/defaultSolrIndexConfiguration/documentOptions">
       <include hint="list:AddIncludedTemplate">
          <Product>{843B9598-318D-4AFA-B8C8-07E3DF5C6738}</Product>
          ...
       </include>
    </documentOptions>
  </ProductIndexConfiguration>
</indexConfigurations>