10

We are soon going be pulling thousands of items from a PROD instance back into a QA Environment. These packages have thousands of items and take a VERY long time to install, sometimes even beyond 8 hours to install into Sitecore.

My question is, is there a way to speed up the installation of these packages OR is there a better way to bring these items down from our PROD instance back into our QA instance faster than a package?

Looking to see what the community thinks is the fastest & most efficient way and we will heavily explore that option.

Improve this question

3 Answers 3

Reset to default
10

To pull content from a PROD instance back to a QA instance I would highly recommend using Sitecore Sidekick - Content Migrator

This tool uses Rainbow Serialization under the hood to quickly migrate content from one Sitecore database to another.

From Jeffs blog:

The Content Migrator will move content in a multi-threaded way. The system is designed to be as fast as the Sitecore Item API can go. When the operation starts it spins up two separate thread pools, one for transmission of serialized item data from the remote server and another to ingest that data into Sitecore. Since the inhibiting factor here is the rate in which Sitecore can write item data to the database you can virtually eliminate the penalty of pulling items over the network in most cases (the obvious exceptions being transmission over a slow connection).

A test was done for speed vs a standard Sitecore Package. The migration and package contained 24k items:

The results were:

  • Content migrator: 41 seconds
  • Sitecore Package Install: 70 minutes

So it is a much faster option than standard Sitecore packages.

Cons: You would have to install Sidekick on both PROD and QA servers, I don't really view this as a con because its a great tool, but for some this could cause a problem.

Content Migrator

Improve this answer
4
6

Make a copy of databases you need to port items from and restore them on your QA instance.

You will lose time only for transferring the database from one server to another.

I did it many times and I can recommend this approach. No modules, no additional setup needed.

Not sure if this fits your needs. In my case there were no differences between production and qa databases so I could do it like that.

Improve this answer
2
  • I should have mentioned that this is not an option for us on this particular project. Otherwise, I completely agree, this would be the best way to go.
    – Toby Gutierrez
    Commented Nov 9, 2016 at 21:48
  • @TobyGutierrez - which DB's would you pull down to QA, if say you wanted QA to completely be restored/match Prod? Seems like there are multiple relevant DB's to Sitecore.
    – Don Cheadle
    Commented Jul 23, 2019 at 4:54
3

I've used Unicorn to manage very large content transfers.

We had a situation where we needed to move gigabytes of (mostly) media data, but also thousands of articles. Serializing the whole thing with Unicorn took an almost insignificant amount of time. Then we zipped the whole thing up.

Reverse; move the whole thing to local environment; sync the Unicorn configurations involved - pretty much as fast as can be.

Unicorn syncs with events disabled by default. I don't think you can get much faster than this.

Improve this answer
1
  • Serialization seems also like a very viable option as well. We don't use Unicorn at the moment, but the serialization seems promising.
    – Toby Gutierrez
    Commented Nov 9, 2016 at 21:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.