Colons (:) in test service messages are confusing Team City.

Hello,

I've recently worked on having automated tests output service messages so that Team City is able to understand and display the tests' results.
To do this, I've added a command line build step that runs perl tests using a formatter that outputs Team City service messages (http://search.cpan.org/~thaljef/TAP-Formatter-TeamCity-0.04/).

What I've discovered is that whenever a test name holds a colon (:), Team City apparently gets confused about the test's name and suite it ran in.

It's as if any text before the colon is removed from the test's name and ends up in the testcase's name instead.

For example, here are messages printed (all on stdout) by a build step of a simple test suite:

##teamcity[testSuiteStarted name='test3.t']
##teamcity[testStarted captureStandardOutput='true' name='One is one.']
##teamcity[testFinished name='One is one.']
##teamcity[testStarted captureStandardOutput='true' name='Two: is 2.']
##teamcity[testFinished name='Two: is 2.']
##teamcity[testStarted captureStandardOutput='true' name='Three is: 3.']
##teamcity[testFinished name='Three is: 3.']
##teamcity[testSuiteFinished name='test3.t']
All tests successful.
Files=1, Tests=3,  0 wallclock secs ( 0.11 usr  0.02 sys +  0.08 cusr  0.01 csys =  0.22 CPU)
Result: PASS

With these messages, here's how Team City interprets the information:

Actual Test Name
Displayed test name
Displayed suite name
One is one. One is one. test3.thttp://sjx86m91.sanjose.ibm.com:8111/viewLog.html?currentGroup=test&scope=test3.t%23teamcity%23*%23teamcity%23*&pager.currentPage=1&order=DURATION_DESC&recordsPerPage=20&filterText=&status=&buildTypeId=bt29&buildId=952&tab=testsInfo
Two: is 2. is 2. test3.t: Two
Three is: 3. 3. test3.t: Three is



You can see visually in the attached screenshot how the information is displayed.

This has been seen in TC 6.0.1 build 15861.

Are there any possible workarounds or fixes in later version?

Thanks!

Martin



Attachment(s):
Screenshot - TeamCity test names bug.png
3 comments
Comment actions Permalink

Hello Martin,

  TeamCity uses ': ' as a separator for the test suite. Unfortunately, there is no way to change this behaviour so far.
  As a workaround, you can remove space after ':'

  I don't think we have a request in our tracker regarding this separator, feel free to add it.

  Kind regards,
  KIR

0
Comment actions Permalink

Another workaround is to put the name part that contains the colon ':' into brackets (any kind of).

For instance, if your test name is 'Two: is 2.', then putting it into brackets will make TC parse it properly:

test3.t: Two: is 2. => suite: test3.t: Two, test name: is 2.

test3.t: [Two: is 2.] => suite: test3.t, test name: [Two: is 2.]

0
Comment actions Permalink

Since this report, the NUnit 3.x Teamcity Eventlistener has been used and solved this for the majority of cases with replacing colons by default with <colon>. This can now be configured by setting the TEAMCITY_COLON_REPLACEMENT environment variable, see https://github.com/nunit/teamcity-event-listener/issues/68.

My personal preference is to configure this variable to be set to ː, which is U+02D0 (MODIFIED LETTER TRIANGULAR COLON).

There are still some issues though, and I have referred them to the solution mentioned by @Sergey Pak above from my report here, perhaps that could be an approach for the long haul:https://github.com/nunit/teamcity-event-listener/issues/72.

0

Please sign in to leave a comment.