.NET command "dotnet test" fails with exit code 1 and does not recognize any tests after update from .NET 6 to .NET 7-rc1
After updating our solution from .NET 6 to .NET 7-rc1, all build steps that use the "dotnet test" command to run tests with NUnit do not seem to recognize any tests. After a few steps, the build fails with "Number of tests 0 is 100% less than 1318 in build #xxx; exit code 1 (Step: Test - XYZ)". Also, the "Tests" tab is missing in the build result (it is present in the previous builds when it was still .NET 6)
Our project has several build steps that uses the .NET runner with the "test" command, in Debug configuration. These steps were not modified during the update.
Looking at the build log output, I do see a difference in how the test results are formatted on the console (see below). Maybe the test runner does not recognize the new format, and thinks there were no tests at all?
Before, the last line was "Process finished with positive exit code 1 (some tests have failed). Reporting step success as all the tests have run." but now it is simply "Process exited with code 1 (Step: ......)"
From previous run (.NET 6) I see the following pattern at the end of step:
[14:24:45] : [dotnet test] Total tests: 5
[14:24:45] : [dotnet test] Passed: 3
[14:24:45] : [dotnet test] Failed: 1
[14:24:45] : [dotnet test] Skipped: 1
[14:24:45] : [dotnet test] Total time: 5.3118 Seconds
[14:24:45]W: [dotnet test] Test Run Failed.
[14:24:45]W: [dotnet test]
[14:24:45]W: [dotnet test] Build FAILED.
[14:24:45] : [dotnet test] 0 Warning(s)
[14:24:45] : [dotnet test] 0 Error(s)
[14:24:45] : [dotnet test]
[14:24:45] : [dotnet test] Time Elapsed 00:00:05.81
[14:24:46]W: [dotnet test] Process exited with code 1
[14:24:46]W: [dotnet test] Process finished with positive exit code 1 (some tests have failed). Reporting step success as all the tests have run.
After updating to .NET 7-rc1 I see the following pattern.
[12:40:49] : [dotnet test]
[12:40:49] : [dotnet test]
[12:40:49] : [dotnet test]
[12:40:49] : [dotnet test] Failed! - Failed: 1, Passed: 3, Skipped: 1, Total: 5, Duration: 2 s - Acme.Tests.dll (net7.0)
[12:40:49]W: [dotnet test] Process exited with code 1
[12:40:50]E: [dotnet test] Process exited with code 1 (Step: Test - Dashboard - Abstractions (.NET))
In both cases, at least one test failed, so the "dotnet test" command returned exit code 1. Previously the step runner did recognize as "normal" and still pass the step. After the update, the runner seems to think it was a more fatal error, and aborts everything.
Here are the NuGet versions used in one such test project:
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.2" />
<PackageReference Include="Microsoft.TestPlatform.TestHost" Version="17.3.2" />
<PackageReference Include="NUnit" Version="3.13.3" />
<PackageReference Include="NUnit3TestAdapter" Version="4.2.1">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
...
</ItemGroup>
I can run the test without issues using the Resharper extension in Visual Studio.
The file global.json:
{
"sdk": {
"version": "7.0.100-rc.1.22431.12",
"rollForward": "latestMinor"
}
}
The NUnit Console runner tool version is 3.15.2
The version of TeamCity is 2022.04.4 (build 108763)
Please sign in to leave a comment.
Hello Christophe, and thanks for reporting the issue.
At the time of writing, TeamCity doesn't officially support .NET 7, but we plan to support it when it releases in November. I can confirm that NUnit tests are not being reported in .NET 7 builds. Please follow TW-77992 for further updates on that issue.
However, in my tests, the build isn't failing. It would help to investigate why it is failing for you if you could provide the full build log (go to the Build Log page > Download log > select Full build log or Zip). Feel free to upload it to https://uploads.jetbrains.com/ and let me know the upload ID. The uploaded data will not be available to the general public and will only be accessible by JetBrains.
Cheers,
Anatoly
The build "fails" because there is at least one unit test that failed, so the process "dotnet test" returned with exit code 1 which is considered a failure but the step runner. Before, the same exit code 1 was "discarded" because the runner saw that all tests where executed, and considered it a success instead.