5

I am tasked with importing a large bulk of transaction data and make it available to my Sitecore solution. We're talking roughly 10 million transaction records that we need to be able to utilise real-time on the site. I've already ruled out importing all of this into a Sitecore Bucket or so; it would add massive overhead to the solution which we don't need right now.

We have SOLR, MongoDB and MS SQL Server available as our remaining storage options.

I'd like to keep MS SQL server out of the mix for now - and have it focusing on other tasks. But when it comes to MongoDB vs SOLR, it becomes more difficult to see exactly why I would choose one over the other. Looking at System Properties Comparison MongoDB vs. Solr - they have most things in common .

They are both:

  • Document Stores (SOLR is a Document Store specialised for searching)
  • Schema-free (we have about 20 different transaction types, more will come)

There are a few things MongoDB offers, but none seem relevant to our solution:

  • User/Role access rights
  • Immediate Consistency

So I'm left wondering; why would I choose one over the other? Are there performance benefits to consider? (MongoDB being written in C++, SOLR in Java) - scalability concerns? Anything I've overlooking?

To clarify; I'm looking for the key thing - should one exist - that sets these two technologies apart and would allow me to weigh in one over the other. Or reassurance that in this case, it's a case of tomatoes vs tomatoes.

Update: Edited to clarify requirements

  • We are receiving between 2500 and 3000 new transactions per day
  • 99% of our queries will be of the nature "give me all transactions for entity x between dates start and end
  • On administrator discretion, a transaction migration can be triggered during the day. It's important that data doesn't go offline while this takes place
  • Data doesn't live here. It is sourced from elsewhere and worst-case can be re-integrated from ground up in a disaster scenario
  • Migration happens nightly. We don't need a real-time view of transaction data
Improve this question
2
  • Some good, if not a little dated, information here stackoverflow.com/questions/3215029/…
    – Christian Hagelid
    Commented Oct 17, 2016 at 12:25
  • 1
    Is this dataset static, or will it be added to as time goes on? Is your operations team prepped to manage a cluster of servers? And preferred OS for the operations team?
    – Laver
    Commented Oct 17, 2016 at 13:18

3 Answers 3

Reset to default
6

Personally I've always seen and treated Solr as a transient index, not a persistent storage solution. Whereas MongoDB has always been treated as a storage solution, and not much of an index. But this is my personal treatment, not saying that Solr can't be used as a persistent store.

If you need to access the data directly as Key/Value pairs and there isn't much querying going on, then I'd say go for MongoDB.

If you need to query the data to retrieve a subset then perhaps I'd lean towards Solr for it's more sophisticated indexing.

Multi-lingual data being searched, searching bodies of text data, faceting results? Totally choose Solr.

From personal experience setting up a reliable 3+ server Solr Cloud cluster configuration was pretty challenging at first, and has resulted in many hours of troubleshooting at times when the cluster wasn't performing as well as it should have been (or just randomly failing unexpectedly).

I'd say that Mongo's clustering / redundancy capabilities are more polished and better documented than Solr, so if operational ease is a factor then perhaps that'd sway you? Mongo you also have better access to official commercial support.

As an alternative you would probably be best off by combine the two. MongoDB cluster for persistent storage and key-value retrieval, more simplified Solr setup to index and allow fast access via more complex queries.

Edit based on clarifications:

If you can rebuild the data from an external source and can only pick one technology, then it sounds like Solr is your choice (for the above reasons). You should make sure to weigh up the pain of re-indexing from scratch as part of a disaster recovery plan to be sure you're comfortable with that operation should some sort of disaster occur, but Solr should be as stable and secure.

Improve this answer
2

I honestly think it doesn't matter which one you use, I'd go with whatever system you're most comfortable developing with. The persistence of the data doesn't matter since you need to be able to re-run a full import anyway, so you can always regenerate it from the source if needed.

All things being equal (implementation time, performance, hosting cost) I'd lean toward Solr simply because it presents you with more options in the future. You said 99% of your searches will be getting the most recent documents in a timeframe. If to grow your functionality beyond that, you have Solr's text searching, faceting, and boosting capabilities you can take advantage of.

Solr gives you some out of the box caching too. If you're running the same query over and over again, or querying the same fields over and over again, you'll be able to take advantage of these right away. Here's a good blog post explaining how they work: https://teaspoon-consulting.com/articles/solr-cache-tuning.html

Improve this answer
2

from a feature perspective - I would say tomato/tomato - but you have more features exposing the inner of the documents as searchable/statistics etc - but it might not be relevant.

The main advantage of Mongo would be the consistency when you run updates nightly - you can be sure of the distribution. This might be relevant if the data accuracy is very important

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.