12

Every now and then my drive fills up with temp files from SOLR, and I can't figure out why that is happening. Here's what I know:

I have a 50gb partition and my analytics index has numDocs:43,287,203 and maxDoc:43,316,981. We currently have 15 cores that live on that partition. The analytics index is usually around 30gb but every so often it fills the drive. We restart the SOLR services to dump the temp disk space usage, and that seems to fix the issue. This is a basic install of SOLR (we haven't done any tweaking to the configs). We are on SOLR 5.4.0.0.

Questions:

  1. Is there a way to limit the merges it makes so it won't eat up my disk space?
  2. If not, what are the recommended disk space sizes for these merges? I can get more disk space if I need it, but I need to know how much to ask for.

Thanks for your help!

Improve this question
5
  • What merges are you doing? The question is not clear on that
    – Richard Seal ♦
    Commented Oct 25, 2016 at 16:34
  • 1
    I'm definitely not an expert on this. But according to this answer, there is no way to limit them. stackoverflow.com/questions/13167767/…
    – Mark Cassidy ♦
    Commented Oct 25, 2016 at 16:36
  • We're not doing any merges. The merges that I'm talking about are whatever SOLR does behind the scenes. I'm not sure what the merges are doing, but the best that we can tell, it's during some of those merges that the large growth is happening.
    – Keith VanderVeen
    Commented Oct 25, 2016 at 16:40
  • What Solr version are you on?
    – Klaus Petersen
    Commented Oct 25, 2016 at 17:17
  • I updated the question with the current version.
    – Keith VanderVeen
    Commented Oct 25, 2016 at 17:27

1 Answer 1

Reset to default
8

This can be a number of things, but it sounds like the merge processes might be the thing. Look into this https://cwiki.apache.org/confluence/display/solr/IndexConfig+in+SolrConfig for more information.

To limit disk space, you should use very LOW merge factors (segmentsPerTier in tieredmergepolicy - mergefactor in LogByteSizeMergePolicy). It will make indexing slower though (might be tremendously slower).

If you have a growing index, you need more disk space. You need to use some sort of capacity management to foresee growth and align this with document growth and expectations of this to know what to ask for, but with indexes and growing contentsize taken into account, you just need to ask for a lot more than you expected.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.