I believe linked items by default are loaded lazily, which is probably what is causing this.
If you first access the property outside the VersionCountDisabler then the items are loaded without it.
An alternative solution to you own answer could be to try and use SitecoreFieldSettings.DontLoadLazily on the properties in question. The items should then all be loaded and mapped inside the VersionCountDisabler.
// Attribute configuration
[SitecoreField(Setting = SitecoreFieldSettings.DontLoadLazily)]
public IList<MyLinkedItem> LinkedItems { get; set; }
// Fluent configuration
Map(x => x.Field(y => y.LinkedItems ).Setting(SitecoreFieldSettings.DontLoadLazily));