Colons (:) in test service messages are confusing Team City.
Hello,
I've recently worked on having automated tests output service messages so that Team City is able to understand and display the tests' results.
To do this, I've added a command line build step that runs perl tests using a formatter that outputs Team City service messages (http://search.cpan.org/~thaljef/TAP-Formatter-TeamCity-0.04/).
What I've discovered is that whenever a test name holds a colon (:), Team City apparently gets confused about the test's name and suite it ran in.
It's as if any text before the colon is removed from the test's name and ends up in the testcase's name instead.
For example, here are messages printed (all on stdout) by a build step of a simple test suite:
##teamcity[testSuiteStarted name='test3.t']
##teamcity[testStarted captureStandardOutput='true' name='One is one.']
##teamcity[testFinished name='One is one.']
##teamcity[testStarted captureStandardOutput='true' name='Two: is 2.']
##teamcity[testFinished name='Two: is 2.']
##teamcity[testStarted captureStandardOutput='true' name='Three is: 3.']
##teamcity[testFinished name='Three is: 3.']
##teamcity[testSuiteFinished name='test3.t']
All tests successful.
Files=1, Tests=3, 0 wallclock secs ( 0.11 usr 0.02 sys + 0.08 cusr 0.01 csys = 0.22 CPU)
Result: PASS
With these messages, here's how Team City interprets the information:
| Actual Test Name |
Displayed test name |
Displayed suite name |
|---|---|---|
| One is one. | One is one. | test3.thttp://sjx86m91.sanjose.ibm.com:8111/viewLog.html?currentGroup=test&scope=test3.t%23teamcity%23*%23teamcity%23*&pager.currentPage=1&order=DURATION_DESC&recordsPerPage=20&filterText=&status=&buildTypeId=bt29&buildId=952&tab=testsInfo |
| Two: is 2. | is 2. | test3.t: Two |
| Three is: 3. | 3. | test3.t: Three is |
You can see visually in the attached screenshot how the information is displayed.
This has been seen in TC 6.0.1 build 15861.
Are there any possible workarounds or fixes in later version?
Thanks!
Martin
Attachment(s):
Screenshot - TeamCity test names bug.png
Please sign in to leave a comment.
Hello Martin,
TeamCity uses ': ' as a separator for the test suite. Unfortunately, there is no way to change this behaviour so far.
As a workaround, you can remove space after ':'
I don't think we have a request in our tracker regarding this separator, feel free to add it.
Kind regards,
KIR
Another workaround is to put the name part that contains the colon ':' into brackets (any kind of).
For instance, if your test name is 'Two: is 2.', then putting it into brackets will make TC parse it properly:
test3.t: Two: is 2. => suite: test3.t: Two, test name: is 2.
test3.t: [Two: is 2.] => suite: test3.t, test name: [Two: is 2.]
Since this report, the NUnit 3.x Teamcity Eventlistener has been used and solved this for the majority of cases with replacing colons by default with
<colon>. This can now be configured by setting theTEAMCITY_COLON_REPLACEMENTenvironment variable, see https://github.com/nunit/teamcity-event-listener/issues/68.My personal preference is to configure this variable to be set to
ː, which isU+02D0(MODIFIED LETTER TRIANGULAR COLON).There are still some issues though, and I have referred them to the solution mentioned by @Sergey Pak above from my report here, perhaps that could be an approach for the long haul:https://github.com/nunit/teamcity-event-listener/issues/72.