Do I need a SearchCrawler to add and update items in a custom Solr Index?

Question

I have an custom index where I will store data that doesn't exist in Sitecore, it has the following configuration:

<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <contentSearch>
      <configuration type="Sitecore.ContentSearch.ContentSearchConfiguration, Sitecore.ContentSearch">
        <indexes hint="list:AddIndex">
          <index id="custom_index" type="Sitecore.ContentSearch.SolrProvider.SolrSearchIndex, Sitecore.ContentSearch.SolrProvider">
            <param desc="name">$(id)</param>
            <param desc="core">$(id)</param>
            <param desc="propertyStore" ref="contentSearch/indexConfigurations/databasePropertyStore" param1="$(id)" />
            <configuration ref="contentSearch/indexConfigurations/customSolrIndexConfiguration">
              <indexAllFields>true</indexAllFields>
              <fieldMap ref="contentSearch/indexConfigurations/customSolrIndexConfiguration/fieldMap"/>
              <documentOptions type="Sitecore.ContentSearch.SolrProvider.SolrDocumentBuilderOptions, Sitecore.ContentSearch.SolrProvider">
              </documentOptions>
            </configuration>
            <strategies hint="list:AddStrategy">
              <strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/manual" />
            </strategies>
            <!--
    <locations hint="list:AddCrawler">
              <crawler type="Custom.Sitecore.Extensions.ContentSearch.CustomSearchCrawler, Custom.Sitecore.Extensions"></crawler>
            </locations>
            -->
          </index>
        </indexes>
      </configuration>
    </contentSearch>
  </sitecore>
</configuration>

I'm trying to add items to the index via code (see this question: Programmatically adding items to a custom Solr Index)

However whatever I try I can only get the index to update when using my CustomSearchCrawler (commented out above in the configuration) which returns items to index using the GetItemsToIndex() method.

I want to manage the indexed items outside of a Search Crawler as I don't want to add lots of items at once. I want to be able to add and delete items at will from a SiteCron Job as needed.

I've read a lot of posts on SSE and blogs and as I understand it I want to execute the Refresh(), Update() or Add() methods to add new items or the Delete() methods to delete items. Rebuild() deletes everything in the entire index and creates it again - I don't want to do this.

I have tried the following to add items with no luck:

1) Using Refresh()

var tempItem = new SitecoreIndexableItem(item);
ContentSearchManager.GetIndex("custom_index").Refresh(tempItem);

2) Using Add()

var tempItem = new SitecoreIndexableItem(item);
ContentSearchManager.GetIndex("custom_index").Operations.Add(tempItem);

3) Using Update()

var tempItem = new SitecoreIndexableItem(item);
ContentSearchManager.GetIndex("custom_index").Operations.Update(tempItem, ContentSearchManager.GetIndex("custom_index").CreateUpdateContext(), ContentSearchManager.GetIndex("custom_index").Configuration);

4) Using IndexCustodian UpdateItem()

IndexCustodian.UpdateItem(ContentSearchManager.GetIndex("custom_index"), new SitecoreItemUniqueId(item.Uri));

5) Using IndexCustodian Refresh()

var tempItem = new SitecoreIndexableItem(item);
IndexCustodian.Refresh(ContentSearchManager.GetIndex("custom_index"),tempItem);

I don't see anything in the main log or crawler or search logs but nothing is created in my index. When I use the IndexCustodian it creates a new job in /sitecore/admin/jobs.aspx that appears to be updating the index but I don't see anything.

So the question is do I need a Crawler or should the above code work without one? If it should work how do I debug this and is anything in my config or code wrong?

Answer 1

In order to add the data to the index, you should use the Sitecore SecurityDisabler.

I have tested it locally on the IndexCustodian.Refresh() method and I can see the data being added to the index.

using (new SecurityDisabler())
{
    var item = Factory.GetDatabase("master").GetItem(new ID("{110D559F-DEA5-42EA-9C1C-8A5DF7E70EF9}"));

    var tempItem = new SitecoreIndexableItem(item);

    var index = ContentSearchManager.GetIndex("test_sitecore_core");

    IndexCustodian.Refresh(index, tempItem);
}

From my investigation about the use of Crawlers, please see below the analysis:

Assuming that the default Sitecore Crawler (SitecoreItemCrawler) is being used;

Case 1

If the crawler has a root like /sitecore/content/home and the item I want to add to the index via code is the Home item itself or a sub item.

Result: The item will be added to the index.

Case 2

If the crawler has a root like /sitecore/content/home, the item I want to add to the index via code is the Item AA and Item AA is not a sub item of Home item.

Result: The item will not be added to the index.

For your case, since you are using a custom crawler, the crawler should grabs the objects from your source, pushes them into the indexable object and sends them down the index provider. So, you will need the crawler but without specifying the root.

I think you are already following this article.