FakeDB runs into error when unit tests executed in parallel

Question

I am facing a weird problem with Sitecore 9.2.0 and FakeDb 2.0.1. Some of the unit tests fails randomly with the following error.

System.NullReferenceException
Object reference not set to an instance of an object.
   at Sitecore.Data.Fields.DefaultFieldTypeManager.GetFieldType(String name)
   at Sitecore.Data.Templates.TemplateField.Builder.SetType(String type)
   at Sitecore.FakeDb.Data.DataProviders.FakeDataProvider.BuildTemplate(DbTemplate ft, TemplateCollection templates)
   at Sitecore.FakeDb.Data.DataProviders.FakeDataProvider.GetTemplates(CallContext context)
   at Sitecore.Data.DataProviders.DataProvider.GetTemplates(CallContext context, DataProviderCollection providers)
   at Sitecore.Data.DataSource.GetTemplates()
   at Sitecore.Data.Engines.TemplateEngine.AddStaticTemplates(TemplateDictionary result)
   at Sitecore.Data.Engines.TemplateEngine.InternalGetTemplates()
   at Sitecore.Data.Engines.TemplateEngine.GetTemplate(ID templateId)
   at Sitecore.Data.Managers.TemplateProvider.GetFieldId(String fieldName, ID templateId, Database database)
   at Sitecore.Collections.FieldCollection.get_Item(String fieldName)
   at Sitecore.Data.Items.BaseItem.get_Item(String fieldName)

When I run the test individually they succeed but some of them fail when executed in parallel with other test. All the tests are using Db items and they are created using AutoData fixture

[Theory]
[AutoData]
public void GetField_Should_Return_Something(Db db, string name, ID itemId)
{
    //arrange
    db.Add(new DbItem(name, itemId));
    var item = db.GetItem(itemId);
    var myClass = new MyClass();

    //act
    myClass.Act(item)
}

Has anyone faced this issue or any idea how to fix it? I have tried it with FakeDB 1.7.0, using different way of creating Db (NSubstitute, using new Db, etc) no luck.

Answer 1

I'm afraid the parallel test execution is not supported. The scenario looks familiar to me, but I've never found a proper solution to this problem and ended up disabling this feature.

If you use xUnit, you can add the following setting to the App.config file of your test project. That is definitely not the most elegant fix, but it will stabilize your tests for sure. There is a bunch of tests in FakeDb which pass only if it's false:

<appSettings>
    <add key="xunit.parallelizeTestCollections" value="false" />
</appSettings>

The test execution will be fast enough anyway. I run tests with and without parallel execution. Parallel is faster but both are acceptable (run 1300 tests in less than half a minute). I assume that's a fair trade-off. Just compare:

Finished:    Sitecore.FakeDb.Tests
=== TEST EXECUTION SUMMARY ===
Sitecore.FakeDb.AutoFixture.Tests    Total:   91, Errors: 0, Failed: 0, Skipped: 0, Time:  4,519s
Sitecore.FakeDb.NSubstitute.Tests    Total:    4, Errors: 0, Failed: 0, Skipped: 0, Time:  1,269s
Sitecore.FakeDb.Serialization.Tests  Total:   42, Errors: 0, Failed: 0, Skipped: 0, Time:  4,284s
Sitecore.FakeDb.Tests                Total: 1175, Errors: 0, Failed: 0, Skipped: 2, Time: 18,573s
                                            ----          -          -           -        -------
                               GRAND TOTAL: 1312          0          0           2        28,644s (131,594s)

Parallel enabled:

Finished:    Sitecore.FakeDb.Tests
=== TEST EXECUTION SUMMARY ===
Sitecore.FakeDb.AutoFixture.Tests    Total:   91, Errors: 0, Failed: 1, Skipped: 0, Time:  2,002s
Sitecore.FakeDb.NSubstitute.Tests    Total:    4, Errors: 0, Failed: 0, Skipped: 0, Time:  6,120s
Sitecore.FakeDb.Serialization.Tests  Total:   42, Errors: 0, Failed: 0, Skipped: 0, Time:  1,862s
Sitecore.FakeDb.Tests                Total: 1175, Errors: 0, Failed: 8, Skipped: 2, Time:  7,323s
                                            ----          -          -           -        -------
                               GRAND TOTAL: 1312          0          9           2        17,307s (114,920s)