We had a similar problem in a production installation that had a significant number of calculated Lucene fields, plus a number of very active content authors with a lot of publishing and index updating activity.
Essentially Sitecore was pushing events at Lucene faster than it could process them.
Here are some suggestions:
Try switching your index update ...
You cannot. Lucene is a "local filesystem" technology.
If you want something that is shared between multiple Content Delivery server instances; you need to switch from Lucene to SOLR.
If you're using the recommended Sitecore ContentSearch APIs this is (mostly) a relatively simple matter of switching your configuration files and setting up the SOLR index ...
The fields are being indexed, but they are not all marked as storageType="YES" in the default Lucene configuration. This means that the fields will be tokenized in the index, but not stored, or visible in Luke. Note that they are still searchable even tho they are not stored.
If you want to be able to see the values in Luke, you will need to change the ...
This looks like a bug with the OnPublishEndAsync strategy.
It becomes apparent when authors start creating new versions of content each time they lock and edit an item.
It has been fixed in 8.1 but still resides in the version you are using.
See this knowledge base article
The issue was that in my efforts to trim the index of waste, i.e. remove fields that I'm not accessing, I removed a certain special field (though it is not marked in any way as being important, though by name it makes sense in hindsight). I removed the "_uniqueid" field from the index configuration. Specifically here:
To answer your question, there are no differences in indexes between SOLR and Lucene (with the exception of the swap cores for SOLR, see Bonus Answer). What you are seeing is probably a product of too many configs activated.
Let me explain.
In Sitecore 8.2, the following indexes are active OOTB using Lucene:
From what I remember if you perform a full index rebuild from the Control Panel, index files will be removed. You don't need to do this manually.
And yes, sometimes indexes may become corrupted for multiple reason. Like network connectivity issues or application restart. Rebuilding the whole index may be the easiest option in some cases.
The sitecore_analytics_index stores information about interactions, contacts, and some other xDB entities. The more data you collect, the bigger the index will grow. The more visits you have in a given time period, the more frequent index reads and writes will be.
Whether or not you should switch from Lucene to Solr depends on whether the high hard disk ...
There's a bit of plumbing you need in place first. Assuming you're already somewhat familiar with the Content Search API, the core of your question is answered below.
Custom Search Result Item
This is needed in order to access the Title field you are referencing. Example implies a field name of "My Title".
public class CustomSearchResultItem : ...
One approach that we have used is to index the content of all data sources in a single computed field. You can search that field when searching the index. This can probably be improved, but it's a start.
I believe this is also the approach the are using in the Habitat project.
Copied/inspired from: http://kamsar.net/index.php/2014/05/indexing-subcontent/
The main thing you need to set up is the Instance Names in your ScalabilitySettings.config file.
The names can be anything they just need to be consistent between the CM and CD.
You can leave these blank and Sitecore will create it's own unique names which can cause issues if you switch IIS sites, so it's safer if you create the names yourself.
On the ...
If you open the showconfig page (hostname/sitecore/admin/showconfig.aspx) it will tell you all the indexes that Sitecore is loading specially the one that is causing you problem. It will show you something like this:
<index id="sitecore_web_index" type="Sitecore.ContentSearch.SolrProvider.SolrSearchIndex, Sitecore.ContentSearch.SolrProvider" patch:source=...
If you want to boost field during the indexing time, you can use the "boost" property of field in the index configuration. For example:
Below is another solution how you can boost an item based on keywords, which user entered:
I guess you are going to use multilist type for the Tags field. The multilist field is a reference field and contains IDs of ...
Lucene will automatically apply that kind of weighting. It will give each record a "score" based on how well the document matches the term. A lot will be dependent on how you have crafted your search query of course.
You can influence the score by boosting. This is the how that affects the query:
Lucene allows influencing search results by "boosting" in ...
Depending on your Sitecore version, you can have both Solr and Lucene enabled at the same time. Each index configuration will define which provider is used. I have done this in v8.0+
You will need to make sure that both the Sitecore.ContentSearch.Lucene.DefaultIndexConfiguration.config and Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config files ...
First, you need to define your own index configuration. That's where you can define the fields that will only be used by your custom indexes.
Create a file named e.g. Sitecore.ContentSearch.Lucene.Index.CustomIndexConfiguration.config under /App_Config/Include with the following content:
In the section of your index (<configuration ref="contentSearch/indexConfigurations/defaultLuceneIndexConfiguration" />) you should point towards a custom config instead of the default one. You need to create this config though, and there you can add the computed field.
Here's a blog post describing this: http://ggullentops.blogspot.be/2015/11/...
If it is a custom config then it should also contain the _uniqueid field.
<field fieldName="_uniqueid" storageType="YES" indexType="TOKENIZED" vectorType="NO" boost="1f" type="System.String" settingType="Sitecore.ContentSearch.LuceneProvider.LuceneSearchFieldConfiguration, Sitecore.ContentSearch.LuceneProvider">
Setting ContentSearch.VerboseLogging to true can help give you diagnostic info on search index activity. Or a set of memory dumps when the thread appears to hang.
You may benefit from using the verbose logging along with this patch to turn up the details . . .
You need to create on /sitecore/system/Settings/Buckets/Facets a new facets add field name (must be lowercase).
You have the field Minimum Number ( Minimum Number of Items - The minimum number of items that the facet must appear in before it is shown in the search results [shared, standard value].
Set the value 0 and it will appear in your search results....
I believe the issue is that the indexing runs without a HttpContext and not in the correct site context either.
The missing site context is why you have to use SiteResolving = true as otherwhise it doesn't know the site and have to use the full path (i.e. /en/site1/Home/...).
The missing HttpContext means it cannot determine the protocol/scheme (as there ...
It's worth bearing in mind that you'll always have the ID of the item in the Index, so you would still be able to generate a URL at runtime once you have your search result. This may also be preferable on cutting down the complexity of your solution.
However, to address your question:
With Lucene, the URL should be populated by default in the index. ...
I would use a different field for your computed field, so it's easier to boost. if this is not possible, you'll have to use some tricks, as you can't distinguish the values coming from a different field.
The trick: Add a token to your value so you can identify and boost it
Assuming that for some reason you need the original value of your taxonomy, add a ...
Is using a single, reused context a good approach?
Yes. It is the recommended approach.
Are there any pitfalls?
Yes. New documents added to your indexes will not show, until you re-open a new Search Context. So you may need to tie in your SearchContext to index update events.
Are there any good code examples of doing this, possibly with a dependency ...
It should be possible, but probably not that simple - at least not if you want to use a different analyzer at search time.
Please note that I haven't tested any of the stuff below.
Changing the analyzer (both indexin and searching)
You should be able to change the analyzer of the _content field like this:
The EventQueue - the medium of communication between your Sitecore instances - is timestamp-based, which can lead to issues if you are using replication and something happened that caused your SQL instance to re-initialize/failover or if the EventQueue's last processed timestamp becomes corrupted.
In order to ensure that this is not the issue, connect to ...
I second Mark's answer. In a "huge" content delivery farm you really want to consider a centralized search provider like Solr.
Coveo is another good option as an alternative to Solr. The starter edition is free, and if your indexes are small enough it may do the job for you with minimal fuss.
I think your Lucene index needs a trim. Especially if it hasn't been this slow always, but just gradually started degrading little by little.
Since it's not a production server, my first recommendation would be to issue a full Index Rebuild and let it run for however long it needs. If you see a really slow rebuild speed, your problem lies elsewhere ...