Parallel test execution (ordering)
We've recently started using the parallel tests feature in TeamCity, but we're having trouble with proper resource utilization.
In addition to enabling the "parallel tests" feature in TeamCity, we are also using the parallel test execution feature in MSTest.
Runner type: .NET
Command: test
Our main pain point here is, that test execution times range from few seconds to 15-20 minutes.
Specifically we are looking for a way to avoid risking that the most expensive (read: longest running) tests are executed last (or at the end of the batch), to avoid having to wait a long time for a single test to complete at the end.
If we are really unlucky, a 16 core machine will start the most expensive test 15 minutes as the last test, potentially doubling the overall execution time of the batch, since the remaining 15 cores are just idling during this time...
Is there someway to ensure that the most expensive tests are executed first in the batch - given the history of execution times recorded by TeamCity? This would ensure that the batch will always complete as-a-hole at the earliest time possible.
Right now overall execution time of the batch, and the machine utilization % is all over the place... very unpredictable.
Please advice, any help would be greatly appreciated.
Please sign in to leave a comment.
As a workaround, it should be possible to reorder the tests from your code with the means of the testing framework (apply to MS documentation for info on how to do that). That would not be an ideal solution, but if you really need to run the slow tests first and you know which ones are the slow ones, it should work.