Running NUunit under NCover Loss of Test History

Before implementing NCover in our NAnt build script TC tracked our NUnit test results in the TEST_INFO table and the TC web interface displayed at Tests tab to view the results.

We have now implemented NCover in our NAnt script, meaning the NUnit tests are run via NCover within TC. Now TC is no longer tracking our NUnit test results in TEST_INFO table and the Tests tab no longer appears in the TC web interface.

I've read several blogs concerning NCover, NUnit and TC integration with no mention of this issue. In fact, Tech Brothers blog has a screen shot showing the Test Tab and Code Coverage tab, so it appears to work.

Have you heard or seen this issue with any other customers? I did not see anything mentioned in forum. Any suggestions?

Thanks,

Sean

Edited by: Sean Gilbert on Oct 3, 2008 3:26 AM

10 comments
Comment actions Permalink

I was informed to give the TC NUnit runner a try instead of NUnit itself under NCover.

http://www.jetbrains.net/confluence/display/TCD3/.NETTestingFrameworks+Support#.NETTestingFrameworksSupport-TeamCity3.1

I am testing this now. However, after reading the documentation there is no means to define the NUnit /config option in TC NUnit Runner. I'm not yet sure this is going to be an issue. In addition, we use NUnit 2.4.7 and TC 3.1.1 TC NUnit runner appears to only support up to 2.4.6. It may come down to us downgrading to NUnit 2.4.6 instead of upgrading to TC 4.0 for now.

0
Comment actions Permalink

After getting the command line details for TC NUnit Launcher NUnit 2.4.7 is supported (seen below).

JetBrains.BuildServer.NUnitLauncher.exe {v1.1|v2.0|ANY} {MSIL|x86|x64} Plugin{|Plugin...} [(;| )]]>[...]]
Plugin:
NUnit-2.2.10
NUnit-2.2.9-VSTS
NUnit-2.4.1
NUnit-2.4.6
NUnit-2.4.7
NUnit-Auto

I found a good example of the command line arguements on the TC Blog.
teamcitydev.blogspot.com/.../...port-features.html

I now have the NUnit Tests History back but no NCover data now, still playing with it. I'm gettting close.

I get the following error when TC NUnit Launcher runs my tests (this does not happend when NUnit runs by itself).
ERROR JetBrains.TeamCity.NUnitLauncher.ErrorLoggerImpl - Unhandled exception in remote appdomain: System.Threading.ThreadAbortException: Thread was being aborted.

Call is the following:
${teamcity.dotnet.nunitlauncher} v2.0 x86 NUnit-2.4.7 /category-exclude:${unit.test.tc.exclude} $\$\${assembly.test}

NCover Task:
]]>

Edited by: Sean Gilbert on Oct 3, 2008 8:42 PM

0
Comment actions Permalink

I'm now getting successful builds but still no code coverage for my project. It does appear from my NCover log file that NCover is profiling Teamcity and not my project. Please see a sample form the NCover log.

Anyone have any suggestions? Again, I'm trying to run NCover using TC NUnit Launcher instead of nunit-console.exe.



(0x00000480) MESSAGE: Class Load Complete -


Class[JetBrains.BuildServer.CommonLoggers.IRPCLogger]
(0x00000480) MESSAGE: Class Load Start -


Class[JetBrains.BuildServer.CommonLoggers.Impl.XMLRPCLogger]
(0x00000480) MESSAGE: Method Load Start -


Method[.ctor]
(0x00000480) MESSAGE: Method Signature => .ctor(JetBrains.BuildServer.CommonLoggers.IAgentUtil agentUtil) : void
(0x00000480) MESSAGE: IL Body at 0x04C53484 len(0x00000096)
(0x00000480) MESSAGE: Sequence Points
(0x00000480) MESSAGE: Offset NextOffset Line Col EndLine EndCol Flags
(0x00000480) MESSAGE: ============ ========== ========== ========== ========== ========== ==========
(0x00000480) MESSAGE: 0(0x0000) 0x0006 14 5 14 46 0x00010000
(0x00000480) MESSAGE: 6(0x0006) 0x0012 16 7 16 42 0x00010000
(0x00000480) MESSAGE: 18(0x0012) 0x001E 17 7 17 44 0x00010000
(0x00000480) MESSAGE: 30(0x001E) 0x0024 19 7 19 51 0x00010000
(0x00000480) MESSAGE: 36(0x0024) 0x002C 0 0 0 0 0x00020000
(0x00000480) MESSAGE: 44(0x002C) 0x002E 0 0 0 0 0x00020000
(0x00000480) MESSAGE: 46(0x002E) 0x0043 21 9 21 105 0x00010000
(0x00000480) MESSAGE: 67(0x0043) 0x004E 22 9 22 43 0x00010000
(0x00000480) MESSAGE: 78(0x004E) 0x0055 23 9 23 32 0x00010000
(0x00000480) MESSAGE: 85(0x0055) 0x0061 24 9 24 32 0x00010000
(0x00000480) MESSAGE: 97(0x0061) 0x0068 25 9 25 25 0x00010000
(0x00000480) MESSAGE: 104(0x0068) 0x006A 16707566 0 16707566 0 0x00010201
(0x00000480) MESSAGE: 106(0x006A) 0x0079 28 9 28 105 0x00010000
(0x00000480) MESSAGE: 121(0x0079) 0x0089 30 9 30 52 0x00010000
(0x00000480) MESSAGE: 137(0x0089) 0x0089 0 0 0 0 0x00020000
(0x00000480) MESSAGE: 137(0x0089) 0x0000 32 5 32 6 0x00010000
(0x00000480) MESSAGE: Method Load Complete -


Method[.ctor]

0
Comment actions Permalink

I just found a post by Adam Tybor dealing with this same issue.
http://www.intellij.net/forums/message.jspa?messageID=5207204#5207204

From this issue Adam created an item in the issue tracker system.
http://jetbrains.net/tracker/issue/TW-4501

It does not appear that this has been resolved or the suggested workaround actually works.

0
Comment actions Permalink

That was my issue Sean. I can't say I ever found a solution other than double running which sucks.

To get the pretty TC xUnit support I run everything using TC's nunit launcher instead of regular nunit. Then periodically (nightly / weekly) I run just the coverage with regular nunit and upload the output as a TC artifact.

The path I was going down was trying to invoke the TC nunit runner directly. From what I can remember TC first calls a nunit launcher to bootstrap the correct TC modified nunit runner. So when using ncover it actually attaches to nunit runner instead of the second app domain that nunit invokes for the testing.

If you can find a way to invoke the nunit runner directly instead of going through the nunit launcher it should work.

However since this issue was filed TC came out with their new api which can capture testing info from Console output instead of using the HTTP RPC calls. Take a look at Gallio, it just got TC support and has support for ncover and partcover I believe. I haven't tried the Gallio / TC stuff yet but it might be worth a shot.

0
Comment actions Permalink

Thanks for your quick reply Adam.

I agree running the unit tests twice sucks. I’d like to stay away from that if at all possible. Your idea to run NCover separately and upload the coverage artifacts has potential, but again, requires a second run of the unit tests. However, it may be an interim solution for now. Thanks!

I will take a look at Gallio.
http://www.gallio.org/

0
Comment actions Permalink

That is expected behaviour, because TeamCity NUnit launcher spawns child process to run the tests under fixed .NET framework and runtime (x64 or x86)

Please try running tests with ANY MSIL keys for that process. For that case no child process should be created.

0
Comment actions Permalink

I figured it out. Set this option in the ncover task:

profiledProcessModule="JetBrains.TeamCity.PlatformProcessRunner.msil.exe"

Such that yours looks like:

]]>

<mkdir dir="$/coverage" failonerror="false"/> <!-- run the unit tests and generate code coverage --> <property name="tools.dir.tmp" value="${tools.dir}"/> <if test="${not path::is-path-rooted(tools.dir)}"> <property name="tools.dir.tmp" value="../../${tools.dir}"/> </if> <property name="nunitpath" value="${lib.dir}/${lib.nunit.basedir}/bin/nunit-console.exe"/> <property name="nunitargs" value=""/> <if test="${not property::exists('teamcity.dotnet.nunitlauncher')}"> <property name="nunitpath" value="C:\Program Files\TeamCity\buildAgent\plugins\dotnetPlugin\bin\JetBrains.BuildServer.NUnitLauncher.exe"/> <property name="nunitargs" value="v2.0 MSIL NUnit-2.4.8"/> </if> <ncover program="${tools.dir.tmp}/${tools.ncover.basedir}/ncover.console.exe" commandLineExe="$" commandLineArgs="$ ${proj.name.unix}.dll" profiledProcessModule="JetBrains.TeamCity.PlatformProcessRunner.msil.exe" workingDirectory="$"
assemblyList="${proj.srcproj.name.unix}"
logFile="$/coverage/coverage.log" excludeAttributes="System.CodeDom.Compiler.GeneratedCodeAttribute" typeExclusionPatterns=".?\{.?\}.*?" methodExclusionPatterns="get_.?; set_.?" coverageFile="$/coverage/coverage.xml"
coverageHtmlDirectory="c:\coverage"
/>

</target>

Obviously I am using variables, but you get the idea. This tells NCover to profile TeamCity's runner process, not the NUnit launcher.

0
Comment actions Permalink

Sweet! I will give this a shot!

0
Comment actions Permalink

This worked for me! Thanks for posting the solution Schley.

0

Please sign in to leave a comment.