Service messages processing interval

Hello,

I've started using NUnit 3.0 console in order to run tests in parallel. This NUnit version sends a lot of service messages to TC about tests results. And just before finishing build, my build script sends service message with "{build.status.text}" in order to see tests results summary in builds overview page.

The problem is, that "{build.status.text}" shows incorrect numbers.

For example finished build shows:
Tests passed: 120, ignored: 6

But in build log are correct values (printed by nunit-console):
Tests run: 124, Passed: 124, Errors: 0, Failures: 0, Inconclusive: 0
Not run: 8, Invalid: 0, Ignored: 8, Skipped: 0

When launched unit tests (which are very fast) the difference of numbers is even bigger:
Tests passed: 374, ignored: 2

Tests run: 549, Passed: 549, Errors: 0, Failures: 0, Inconclusive: 0
Not run: 2, Invalid: 0, Ignored: 2, Skipped: 0

I've tried redirecting nunit-console output to file and counted service messages. Looks like, that everything is correct ("##teamcity[testStarted" count equals 549).

So, I guess, that probably TeamCity fails to process all messages before build finishes..

Is there any parameter, which specifies time interval for scanning build log for new messages (right now, looks like, it's set to 5 or 10 seconds)? Or maybe you have any other ideas what might be wrong?

Thank you,
Andrew

5 comments
Comment actions Permalink

It's very crucial for me to start using parallel tests, but first I need to get correct tests result. Any ideas how to fix it?

0
Comment actions Permalink

Hi Andrew,

Sorry for delay. Since TeamCity 9.0 multiple tests with the same name within the same build are considered a single test with an invocation count. This change results in drop of test number counters in builds which have multiple runs for the same test. Please check is this the case?

0
Comment actions Permalink

Hi Alina,

I'm not running the same test multiple times in the same build. I'm just running them once, but in parallel. All tests have different names.

0
Comment actions Permalink

Any updates? ?:|

0
Comment actions Permalink

Hi Andrew,

Sorry for delay. If tests run in parallel then flowId should be used to distinguish separate processes. It seems that NUnit 3.0 console does not use flowId at the moment, see the related forum thread.
I've added comment to the related issue: https://youtrack.jetbrains.com/issue/TW-38047, please watch and vote for it.

0

Please sign in to leave a comment.