MSTest Runner not behaving after upgrade from 6.5 to 7.0

Yesterday, I upgraded our TeamCity server from 6.5 to 7.0.  All of our MSTest biuld steps have now stopped working.

I have specified 10 test assemblies to run, for a total of around 400 tests.  What I have found is that if I reduce the number of assemblies, it will run, but the more assemblies that add, the slower the step runs, until it finally stops working.

Our build times prior to the upgrade were around 3 minutes.  With a couple assemblies in the MSTest step, it will run, but take twice the time.  It doesn't appear to matter which assemblies I add and remove, only the number which are present.

Has anybody else encoutered a similar issue?

With all 10 assemblies specified, after about 20 minutes the MSTest step will fail with the following output:

[10:50:54]Step 2/3: MSTest (17m:16s)
[10:50:54][Step 2/3] Starting: D:\TeamCity\buildAgent\plugins\dotnetPlugin\bin\JetBrains.BuildServer.NUnitLauncher.exe #TeamCityImplicit
[10:50:54][Step 2/3] in directory: D:\TeamCity\buildAgent\work\47d0ecb36bbf7ade
[11:08:08][Step 2/3] 2012-03-01 11:08:08,804 [1484] ERROR JetBrains.BuildServer.NAntLoggers.NUnitLauncher2 - Exception of type 'System.OutOfMemoryException' was thrown.
[11:08:08][Step 2/3] System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
[11:08:08][Step 2/3]    at JetBrains.TeamCity.Utils.FsScanner.Wildcard.IsMatch(String str) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\Wildcard.cs:line 80
[11:08:08][Step 2/3]    at JetBrains.TeamCity.Utils.FsScanner.AntPatternState.Enter(String component, MatchResult& match) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\AntPatternState.cs:line 96
[11:08:08][Step 2/3]    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.Any(AntPatternState[] state, String component, AnyPredicate predicate, AntPatternState[]& newState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 89
[11:08:08][Step 2/3]    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 138
[11:08:08][Step 2/3]    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148
[11:08:08][Step 2/3]    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148
[11:08:08][Step 2/3]    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148
[11:08:08][Step 2/3]    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148
[11:08:08][Step 2/3]    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148
[11:08:08][Step 2/3]    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148
[11:08:08][Step 2/3]    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148
[11:08:08][Step 2/3]    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148
[11:08:08][Step 2/3]    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148
[11:08:08][Step 2/3]    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148
[11:08:08][Step 2/3]    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148
[11:08:08][Step 2/3]    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148
[11:08:08][Step 2/3]    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFiles(IFileSystem fs, IDirectoryEntry root, String[] includes, String[] excludes) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 27
[11:08:08][Step 2/3]    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFiles(String root, String[] includes, String[] excludes) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 19
[11:08:08][Step 2/3]    at JetBrains.BuildServer.CommonLoggers.Impl.DirecrotyWildcardResolverImpl.FindFiles(String[] includeRules, String[] excludeRules) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\CommonLoggers\src\Impl\DirecrotyWildcardResolverImpl.cs:line 23
[11:08:08][Step 2/3]    at JetBrains.TeamCity.MSTestSupport.Executable.MSTestTargetGenerator.GenerateMSTestCommand(IMSTestExeArguments msTestArgs, String resultsFile) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\MSTestSupport\src\Executable\MSTestTargetGenerator.cs:line 68
[11:08:08][Step 2/3]    at JetBrains.TeamCity.MSTestSupport.Executable.MSTestTargetGenerator.GenerateTargetContent(IMSTestExeArguments mstestArgs) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\MSTestSupport\src\Executable\MSTestTargetGenerator.cs:line 47
[11:08:08][Step 2/3]    at JetBrains.TeamCity.MSTestSupport.Executable.MSTestExeTestRunnerFactory.CreateRunner(ITestRunArguments args) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\MSTestSupport\src\Executable\MSTestExeTestRunnerFactory.cs:line 29
[11:08:08][Step 2/3]    at JetBrains.BuildServer.NAntLoggers.RunnerFactory.FindTestRunner(ITestRunArguments arguments) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\NUnitBootstrap\src\RunnerFactory.cs:line 45
[11:08:08][Step 2/3]    at JetBrains.BuildServer.NAntLoggers.RunnerFactory.CreateTestRunner(ITestRunArguments arguments) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\NUnitBootstrap\src\RunnerFactory.cs:line 30
[11:08:08][Step 2/3]    at JetBrains.BuildServer.NAntLoggers.NUnitLauncher2.Run2(String[] args) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\NUnitBootstrap\src\NUnitLauncher2.cs:line 98
[11:08:08][Step 2/3]    at JetBrains.BuildServer.NAntLoggers.NUnitLauncher2.Run(String[] args) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\NUnitBootstrap\src\NUnitLauncher2.cs:line 54
[11:08:08][Step 2/3] Exception of type 'System.OutOfMemoryException' was thrown.    at JetBrains.TeamCity.Utils.FsScanner.Wildcard.IsMatch(String str) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\Wildcard.cs:line 80    at JetBrains.TeamCity.Utils.FsScanner.AntPatternState.Enter(String component, MatchResult& match) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\AntPatternState.cs:line 96    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.Any(AntPatternState[] state, String component, AnyPredicate predicate, AntPatternState[]& newState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 89    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 138    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFilesRec(IDirectoryEntry directory, ArrayList result, AntPatternState[] includeState, AntPatternState[] excludeState) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 148    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFiles(IFileSystem fs, IDirectoryEntry root, String[] includes, String[] excludes) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 27    at JetBrains.TeamCity.Utils.FsScanner.DirectoryScanner.FindFiles(String root, String[] includes, String[] excludes) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\Utils\src\FsScanner\DirectoryScanner.cs:line 19    at JetBrains.BuildServer.CommonLoggers.Impl.DirecrotyWildcardResolverImpl.FindFiles(String[] includeRules, String[] excludeRules) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\CommonLoggers\src\Impl\DirecrotyWildcardResolverImpl.cs:line 23    at JetBrains.TeamCity.MSTestSupport.Executable.MSTestTargetGenerator.GenerateMSTestCommand(IMSTestExeArguments msTestArgs, String resultsFile) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\MSTestSupport\src\Executable\MSTestTargetGenerator.cs:line 68    at JetBrains.TeamCity.MSTestSupport.Executable.MSTestTargetGenerator.GenerateTargetContent(IMSTestExeArguments mstestArgs) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\MSTestSupport\src\Executable\MSTestTargetGenerator.cs:line 47    at JetBrains.TeamCity.MSTestSupport.Executable.MSTestExeTestRunnerFactory.CreateRunner(ITestRunArguments args) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\MSTestSupport\src\Executable\MSTestExeTestRunnerFactory.cs:line 29    at JetBrains.BuildServer.NAntLoggers.RunnerFactory.FindTestRunner(ITestRunArguments arguments) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\NUnitBootstrap\src\RunnerFactory.cs:line 45    at JetBrains.BuildServer.NAntLoggers.RunnerFactory.CreateTestRunner(ITestRunArguments arguments) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\NUnitBootstrap\src\RunnerFactory.cs:line 30    at JetBrains.BuildServer.NAntLoggers.NUnitLauncher2.Run2(String[] args) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\NUnitBootstrap\src\NUnitLauncher2.cs:line 98    at JetBrains.BuildServer.NAntLoggers.NUnitLauncher2.Run(String[] args) in c:\BuildAgent\work\dc6fdb6ba3889d18\src\NUnitBootstrap\src\NUnitLauncher2.cs:line 54
[11:08:10][Step 2/3] Process exited with code -1

Please sign in to leave a comment.