Code coverage yielding "jumping" results

Hello Yann,

If you use Ant runner, I suppose that your build doesn't recompile existing classes (and as result, coverage metadata information is not built).
Please check that you have a kind of "clean" target in the beginning of your build with coverage.

Hope this helps,
KIR

Hello Kirill,

thanks for your reply.

Actually clean-target exists and completely removes build.home as well as build-test.home directory.

Could it be a problem that our build-dir is one level above checkout-directory?

see build.log from failed EMMA run:

: Checking for changes
: Clearing temporary directory: ..\temp\buildTmp
: Checkout directory: C:\BuildAgent\work\ee614a8be8775242
: Updating project sources
: Updating C:\BuildAgent\work\ee614a8be8775242 from svn
: Starting build process in C:\BuildAgent\work\ee614a8be8775242
: delete (<1s)
: Deleting directory C:\BuildAgent\work\build
: delete (<1s)
: Deleting directory C:\BuildAgent\work\build-test
: mkdir (<1s)
: Created dir: C:\BuildAgent\work\build
: Adding debug=true parameter to javac task
....
: classpath=C:\BuildAgent\work\build;C:\BuildAgent\work\ee614a8be8775242\project\lib\annotations.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\backport-util-concurrent-3.1.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\commons-codec-1.3.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\commons-el-1.0.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\commons-lang-2.3.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\commons-logging-1.1.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\freemarker-2.3.10.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\kxml2-2.1.8.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\log4j-1.2.15.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\ognl-2.6.11.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\oraclejdbc10g10.2.0.4.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\oro-2.0.8.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\packtag-3.2.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\poi-3.0.2.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\report-205.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\retrotranslator-runtime-1.2.4.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\sitemesh-2.2.1.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\spring.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\struts2-core-backport-2.1.0.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\struts2-sitemesh-plugin-2.1.0.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\struts2-spring-plugin-2.1.0.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib\xwork-backport-2.1.0.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib-test\junit-4.4.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib-test\selenium-java-client-driver.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib-test\selenium-server.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib-test\servlet-api.jar;C:\BuildAgent\work\ee614a8be8775242\project\lib-test\spring-test.jar;C:\BuildAgent\plugins\ant\lib\ant-launcher.jar;C:\BuildAgent\lib\buildServerRuntimeUtil.jar;C:\BuildAgent\plugins\antPlugin\lib\buildServerAntRuntime.jar;C:\BuildAgent\plugins\ant\lib\ant-antlr.jar;C:\BuildAgent\plugins\ant\lib\ant-apache-bcel.jar;C:\BuildAgent\plugins\ant\lib\ant-apache-bsf.jar;C:\BuildAgent\plugins\ant\lib\ant-apache-log4j.jar;C:\BuildAgent\plugins\ant\lib\ant-apache-oro.jar;C:\BuildAgent\plugins\ant\lib\ant-apache-regexp.jar;C:\BuildAgent\plugins\ant\lib\ant-apache-resolver.jar;C:\BuildAgent\plugins\ant\lib\ant-commons-logging.jar;C:\BuildAgent\plugins\ant\lib\ant-commons-net.jar;C:\BuildAgent\plugins\ant\lib\ant-icontract.jar;C:\BuildAgent\plugins\ant\lib\ant-jai.jar;C:\BuildAgent\plugins\ant\lib\ant-javamail.jar;C:\BuildAgent\plugins\ant\lib\ant-jdepend.jar;C:\BuildAgent\plugins\ant\lib\ant-jmf.jar;C:\BuildAgent\plugins\ant\lib\ant-jsch.jar;C:\BuildAgent\plugins\ant\lib\ant-junit.jar;C:\BuildAgent\plugins\ant\lib\ant-launcher.jar;C:\BuildAgent\plugins\ant\lib\ant-netrexx.jar;C:\BuildAgent\plugins\ant\lib\ant-nodeps.jar;C:\BuildAgent\plugins\ant\lib\ant-starteam.jar;C:\BuildAgent\plugins\ant\lib\ant-stylebook.jar;C:\BuildAgent\plugins\ant\lib\ant-swing.jar;C:\BuildAgent\plugins\ant\lib\ant-testutil.jar;C:\BuildAgent\plugins\ant\lib\ant-trax.jar;C:\BuildAgent\plugins\ant\lib\ant-vaj.jar;C:\BuildAgent\plugins\ant\lib\ant-weblogic.jar;C:\BuildAgent\plugins\ant\lib\ant-xalan1.jar;C:\BuildAgent\plugins\ant\lib\ant-xslp.jar;C:\BuildAgent\plugins\ant\lib\ant.jar;C:\BuildAgent\plugins\ant\lib\junit-3.8.1.jar;C:\BuildAgent\plugins\ant\lib\xercesImpl.jar;C:\BuildAgent\plugins\ant\lib\xml-apis.jar;C:\Programme\Java\jdk1.5.0_15\lib\tools.jar;C:\Programme\Java\jdk1.5.0_15\jre\lib\rt.jar;C:\Programme\Java\jdk1.5.0_15\jre\lib\jsse.jar;C:\Programme\Java\jdk1.5.0_15\jre\lib\jce.jar;C:\Programme\Java\jdk1.5.0_15\jre\lib\charsets.jar;C:\Programme\Java\jdk1.5.0_15\jre\lib\ext\dnsns.jar;C:\Programme\Java\jdk1.5.0_15\jre\lib\ext\localedata.jar;C:\Programme\Java\jdk1.5.0_15\jre\lib\ext\sunjce_provider.jar;C:\Programme\Java\jdk1.5.0_15\jre\lib\ext\sunpkcs11.jar
: Added @NotNull assertions to 18 files
: EMMA: Instrumenting classes (1s)
: EMMA commandline:
: EMMA: processing instrumentation path ...
: EMMA: instrumentation path processed in 281 ms
: EMMA:
: EMMA: metadata merged into {in 656 ms}
: copy (<1s)
: Copying 1 file to C:\BuildAgent\work\build
: copy (<1s)
: Copying 1 file to C:\BuildAgent\work\build
: mkdir (<1s)
: Created dir: C:\BuildAgent\work\build-test
: Adding debug=true parameter to javac task

...

: EMMA commandline:
: EMMA: processing input files ...
: EMMA: locking coverage output file ...
: EMMA: 2 file(s) read and merged in 0 ms
: EMMA: writing report to ...
: EMMA: runtime coverage data merged into {in 156 ms}
: EMMA: locking coverage output file ...
: EMMA: runtime coverage data merged into {in 250 ms}
: EMMA: locking coverage output file ...
: EMMA: runtime coverage data merged into {in 359 ms}
: EMMA: locking coverage output file ...
: EMMA: runtime coverage data merged into {in 422 ms}
: EMMA: locking coverage output file ...
: EMMA: runtime coverage data merged into {in 781 ms}
: EMMA: writing report to ...
: Process exit code: 0
: Publishing artifacts (<1s)
: Paths to publish:
: Publishing artifacts 'coverage.txt' to root artifacts directory
: Publishing artifacts 'coverage' to root artifacts directory
: Publishing artifacts 'coverage.ec' to root artifacts directory
: Publishing artifacts 'coverage.em' to root artifacts directory
: Build finished

Thanks!
Yann

Hello Yann,

Which version of TeamCity do you use? In 3.1.1 we've made a change which fixed coverage issue when working directory didn't match checkout directory.

Regards,
KIR

Hello Kirill,

we're using Version 3.1.1 (build 6828)

Thanks,

Yann

Hello Yann,

Could you please take a look at coverage.* files in artifacts of the builds which have different coverage results. If builds identical, both coverage.em and coverage.ec files should have the same size.
Please also try to run builds with enabled clean checkout option. What about subsequent coverage results in this case?


Regards,
KIR

Hello Kirill,

Coverage artifact file sizes actually differ from build to build (btw, artifact sizes (kB) could be right-aligned in popup).

We have another build configuration w/ Ant+EMMA which works just fine, but it compiles in a subdir into the project root.

For now it seems "Clean all files before build:" and changing checkout mode to "server" runs the build correctly more often, but not continuously..

Greetings,

Yann

Hello Yann,

I'm sorry for the delay with the answer, I've lost your answer in my mailbox.
I'd really appreciate if you send me 2 full build logs - with successful and with failed coverage.
I hope I can gain some information by comparing them.

Kind regards,
KIR at JetBrains com