Error opening New Searcher. Core is missing: `Sitecore_Analytics_Index` [closed]

Question

Our Sitecore Solr has issues with the Sitecore analytics index and the error message is :

Error opening New Searcher. Core is missing.

We tried to look in to the error but could not gather much information on why this is happening.

This is a production environment and we need to make sure that adding the Solr core again with same index files helps before going through deploying process.

The error is as follows:

null:org.apache.solr.common.SolrException: SolrCore 'sitecore_analytics_index' is not available due to init failure: Error opening new searcher
    at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:884)
    at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:247)
    at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:412)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
    at org.eclipse.jetty.server.Server.handle(Server.java:497)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
    at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
    at org.apache.solr.core.SolrCore.(SolrCore.java:815)
    at org.apache.solr.core.SolrCore.(SolrCore.java:658)
    at org.apache.solr.core.CoreContainer.create(CoreContainer.java:637)
    at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:381)
    at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:375)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:148)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
    ... 1 more
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
    at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1657)
    at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1769)
    at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:911)
    at org.apache.solr.core.SolrCore.(SolrCore.java:788)
    ... 9 more
Caused by: org.apache.lucene.index.CorruptIndexException: codec header mismatch: actual header=423661802 vs expected header=1071082519 (resource=BufferedChecksumIndexInput(MMapIndexInput(path="xxx/solr/data/sitecore_analytics_index_sec/data/index.20180908031501818/_1f8x.cfs") [slice=_1f8x.fnm]))
    at org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:197)
    at org.apache.lucene.codecs.CodecUtil.checkIndexHeader(CodecUtil.java:256)
    at org.apache.lucene.codecs.lucene50.Lucene50FieldInfosFormat.read(Lucene50FieldInfosFormat.java:115)
    at org.apache.lucene.index.IndexWriter.readFieldInfos(IndexWriter.java:908)
    at org.apache.lucene.index.IndexWriter.getFieldNumberMap(IndexWriter.java:924)
    at org.apache.lucene.index.IndexWriter.(IndexWriter.java:854)
    at org.apache.solr.update.SolrIndexWriter.(SolrIndexWriter.java:78)
    at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:65)
    at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:273)
    at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:116)
    at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1626)
    ... 12 more
    Suppressed: org.apache.lucene.index.CorruptIndexException: codec footer mismatch (file truncated?): actual footer=0 vs expected footer=-1071082520 (resource=BufferedChecksumIndexInput(MMapIndexInput(path="xxx/solr/data/sitecore_analytics_index_sec/data/index.20180908031501818/_1f8x.cfs") [slice=_1f8x.fnm]))
        at org.apache.lucene.codecs.CodecUtil.validateFooter(CodecUtil.java:416)
        at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:331)
        at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:379)
        at org.apache.lucene.codecs.lucene50.Lucene50FieldInfosFormat.read(Lucene50FieldInfosFormat.java:164)
        ... 20 more

Answer 1

would you verify that the core exists In the apache-solr\server\solr folder, do you see a sitecore_analytics_index folder? if not you can use the following instructions to add the new index:

1. Duplicate master folder index in apache-solr\server\solr dircetory
2. Rename the copied folder into "sitecore_analytics_index" and delete "data" folder and core.properties files from the new folder
3. Navigate to solr "core Admin" and add new core for "Sitecore_analytics_index"

Answer 2

From the feedback received from @Haneen Daoud, I have looked at my corrupted cores in slave Solr. I see all the folders for the corrupted cores are present and understood from the logs that one of the segments of index is corrupted because of 100% memory utilization happened which is because of few error logs.

I see conf, data folders and core.properties text file in the /var/Solr/data/Sitecore_analytics_index core. Data folder under path /var/Solr/data/Sitecore_analytics_index has the index folder and index.properties, replication.properties text files. I have deleted this data folder and restarted solr and this fixed the issue.