Slow build performance using TDS (Helix-inspired)

Question

I am using TDS to maintain my Sitecore items in a Helix-inspired solution with around 50 smaller projects (currently with 17 TDS projects). When I try to deploy my TDS projects locally, it takes +9 minutes for the 17 projects, which is a pain in a multi-developer setup, in which I often have to push items from other developers (without knowing which of the TDS projects they have commited to - if we only had a single TDS project I could simply sync that project).

We are using TDS version 5.5.0.20 and my Gulp build configuration is:

return gulp.src([location + "/**/tds/**/*.scproj"])
  .pipe(foreach(function (stream) {
      return stream
        .pipe(debug({ title: "Building project:" }))
        .pipe(msbuild({
            targets: ["Deploy"],
            configuration: config.buildConfiguration,
            logCommand: false,
            verbosity: "minimal",
            stdout: true,
            errorOnFail: true,
            maxcpucount: 0,
            toolsVersion: 14.0,
            properties: config.tdsMsbuildProperties
        }));
  }));

Any tips on how to optimize the deployment of TDS items across multiple projects?

Answer 1

TL;DR

Some possible solutions to reduce build/deployment time:-

1. Reduce the number of projects in your solution by 'compacting' projects.
2. Reduce the number of projects in your solution by having a Tests-project-free sln file.
3. Create an sln file with the explicit ProjectDependencies removed. This would only be for local builds. It doesn't help you for your deployments after a pull from source control.
4. Utilize the Delta Deployment feature in TDS, deploying only items changed after a set date (like one week ago).
5. Remove any other unnecessary things being run for local builds.

Long Version

1. When developing with Helix, a common pain that people have reported is the lengthy build times due to the large amounts of projects in the solution. Naturally, the more projects, the longer the build/deployment time. As TDS serializes items into projects, this again means that there will be more projects in your solution.

If the number of projects is becoming a pain, I would suggest considering how you could reduce the number of projects, but still maintain the Helix principles in your solution. This can be done by utilizing Hedgehog's FxCop rules which perform code analysis based on your namespaces.

2. Maybe you want to consider duplicating the sln file, having one for general development (removing the Tests projects), and one complete one where the Tests are still included. For the Tests-free sln, Visual Studio may run builds faster.

3. As for the deployment of the solution, the Habitat.TDS repository adds custom 'Project dependencies' to force a correct deployment order within the solution (this is done in the original repo with custom gulp scripts, deploying Foundation -> Feature -> Project layers). This custom deployment order should always be maintained when:-

   • performing an initial deployment
   • performing a deployment when you've retrieved code and items from other developers (like when you get the latest from source control through a git pull)

However, during your own development (in between source code retrieval....just developing locally), you theoretically don't need this order for just builds (because you're not doing deploys). It's not ideal, but you could duplicate your sln file, remove all the custom project dependencies (the sections in the raw sln file that have ProjectSection(ProjectDependencies)), and develop with that. This way Visual Studio builds only projects with the actual references, and doesn't unnecessarily go through the entire solution. I've tested this on the example site, and found a little bit of improvement in build times.... but again, it's not an ideal scenario. Here's a PowerShell script that can create this sln file for you:- https://github.com/HedgehogDevelopment/Habitat/blob/TDS-latest/scripts/CreateQuickBuildSlnFile.ps1

4. Utilize the 'Deploy Items Changed After' option on the Properties -> General Tab. (You could set this in your local TdsGlobal.config.user file using the <IncludeItemsChangedAfter> node). This ignores the deployment of items from AFTER the set date. Maybe set the date to 1 week prior. This can greatly speed up the local deployment process. You want to be careful about this though, as it uses the __Updated field on the items to do the comparison out of the box....so you could customize it to check source control for updated .item files instead.

5. Minimize the other things that cause the build to take longer. In one solution, I saw that the developers were running certain gulp tasks that weren't needed for every build. They also had Razor View compilation. Now for these, we moved them out to the 'Release' configuration, so they were still run, but only for CI builds, not local developer builds (which are run on the 'Debug' configuration).

All up, using some combinations of the above, we found that we were able to reduce some build times from 14 minutes down to about 2 minutes. Again, it depends on your solution, but these are some things you may want to consider.

Answer 2

I would recommend not using gulp to perform the deployments. Or maybe you can modify the script, as it's currently calling MSBuild on every TDS project in your solution.

If you setup the solution to have the appropriate build order, then a single call to deploy the solution would save time for a full solution deploy. You can set the ProjectDependencies on each TDS project to get the Solution Build order you desire. See the Build Order section of the README here for the correct Build Order on the Habitat.TDS repo. Then deploy either using Visual Studio's "Deploy Solution", or by calling MSBuild -> Deploy on the sln file itself (not each project) . This would save you time on:-

• the overhead of multiple calls to MSBuild (which, in your example, happens multiple times in your foreach loop)
• the evaluation of dependent projects to only happen once.
• TDS (from v5.5) saves time by not attempting to reinstall the Sitecore Connector in between each project. (It does it once for the solution instead).