Import large amount of items from excel

Question

I am working on building a sitecore backend page that the editors are using for importing a large amount of news items from an excel document that looks like this:

title  description  date  link  imagemediapath

Currently I am iterating each row and inserting items individually as I need to create/update them in folders based on Date field like Year/Month/newsitem. I am using query search on the sitecore-master-index to verify if a news already exists.

Is there a way to optimize the import process on higher environments, I've found some posts to try to disable indexing and sitecore events programmatically and then rebuild it, but I am not sure how to do that.

Also if there are any other approaches or automatic ways to accomplish this it would help me a lot.

Answer 1

Disabling statistics when updating the item will improve performance, make sure you don't need this data:

• item.Editing.EndEdit(updateStatistics:true, silent:true);

You can also wrap your code with BulkUpdateContext and EventDisabler usings, again, keep in mind that item:saving and item:saved events won't be fired.

For pausing indexing (that will also improve performance) you can do the following:

IndexCustodian.PauseIndexing(); // your update/create items logic IndexCustodian.ResumeIndexing();

You can also achieve that with PowerShell Extensions, here a great example from Michael West: https://gist.github.com/michaellwest/453165b6f92db2b6add8553fa291679d

Answer 2

Your code for bulk creating/updating of items could look something like this:

    using (new SecurityDisabler())
    using (new DatabaseCacheDisabler())
    using (new EventDisabler())
    using (new BulkUpdateContext())
    {

      try
      {
        //Pause indexing
        IndexCustodian::PauseIndexing()
    
        //create/delete/update your items
         
        Item newItem = parentItem.Add("name", template);
        newItem.Editing.BeginEdit();
        newItem["SomeField"] = "SomeText";
        newItem.Editing.EndEdit();
      
      }
      catch(Exception ex){
        //log the exception etc
      }
      finally
      {
        //Resume indexing
        IndexCustodian::ResumeIndexing()
      }
    }

Depending on how your import process is implemented, you could make scheduled job to execute it or make it happen on button press. Put the code in loop to update more at once. Publish items if necessary. You can observe jobs.aspx to check for indexing to happen only once.