Teamcity Enterprise (5.1.5) code coverage and emma

Did you enable EMMA coverage in TeamCity web UI, or emma task was configured in your build script?

Hi Pavel,

I configured emma task in build.xml script:
<target name="compile_test" depends="jar">
            <delete dir="${dir.test.bin}"/>
          <mkdir dir="${dir.test.bin}"/>
          <javac srcdir="${dir.root}/test" destdir="${dir.test.bin}" debug="on" deprecation="on" source="1.5" target="1.5" encoding="UTF-8">
            <classpath>
                <pathelement location="${gl.path.build.tmp}/${gl.jar.file}_${DSTAMP}_jdk${compile.source}.jar"/>
                <pathelement location="${dir.test}/dependencies/crimson.jar"/>
                <pathelement location="${dir.test}/dependencies/junit.jar"/>
                <pathelement location="${dir.test}/dependencies/serializer.jar"/>
                <pathelement location="${dir.test}/dependencies/xalan.jar"/>
                <pathelement location="${dir.test}/dependencies/xalan.jar"/>
                <pathelement location="${dir.test}/dependencies/xercesImpl.jar"/>
                <pathelement location="${dir.test}/dependencies/xml-apis.jar"/>
                <pathelement location="${dir.test}/dependencies/xsltc.jar"/>
                <pathelement location="./${dir.test}/dependencies/tools"/>
                <pathelement location="${dir.root}/dependencies/lib.common/log4j-1.2.15.jar.jar"/>
                <pathelement location="${dir.root}/dependencies/lib.common/commons-logging-1.0.4.jar"/>
                <pathelement path="${dir.root}/dependencies/lib.jr3.7/"/>
            </classpath>
        </javac>
      </target>

    <target name="test" depends="compile_test">
        <copy
            file="${gl.path.build.tmp}/${gl.jar.file}_${DSTAMP}_jdk${compile.source}.jar"
            tofile="${gl.path.build.tmp}/emma.jr.pdf.jar">
        </copy>
       <emma>
             <instr instrpath="${gl.path.build.tmp}/emma.jr.pdf.jar"
                mode="overwrite"
                metadatafile="${dir.coverage}${file.separator}coverage.em"
                >
                       <filter includes=""/>
            </instr>
        </emma>
        
           <mkdir dir="${dir.testout}${file.separator}visualtestout"/>
        <junit
            fork="yes"
            printsummary="yes"
            showoutput="true"
            maxmemory = "1024M">

         <jvmarg value="-Dfile.encoding=UTF-8"/>
         
         <test name="regression.Regression" fork="true"/>    
        
        <formatter type="plain" usefile="false" />

        <classpath>
            <fileset dir="${dir.test.lib}">
                <include name="*.jar"/>
            </fileset>
            <fileset dir="${dir.lib}">
                <include name="*.jar"/>
            </fileset>
            <fileset dir="${dir.root}/dependencies/lib.common">
                <include name="*.jar"/>
            </fileset>
            <fileset dir="${dir.root}/dependencies/lib.jr3.7">
                <include name="*.jar"/>
            </fileset>
            <fileset dir="${dir.root}/build-tmp">
                <include name="emma.jr.pdf.jar"/>
            </fileset>
            <fileset dir="${dir.root}/misc/tools">
                <include name="emma.jar"/>
            </fileset>    
            <fileset dir="${dir.root}/misc/tools">
                <include name="emma_ant.jar"/>
            </fileset>             

            <pathelement location="${dir.test.bin}"/>
        </classpath>

        </junit>
        
        <!--<copy
            file="${basedir}/coverage.ec"
            tofile="${dir.root}/coverage/coverage.ec">
        </copy>-->
        <echo message="${dir.coverage}"/>
        <emma>
            <report sourcepath="${dir.coverage}" depth="method" > <!--sourcepath="${dir.root}/src"-->
                <fileset dir="${dir.coverage}" >
                    <include name="coverage.em" />
                </fileset>
                <fileset dir="${dir.coverage}" >
                    <include name="coverage.ec" />
                </fileset>
                <txt outfile="${dir.coverage}/coverage.txt" />
                <xml outfile="${dir.coverage}/coverage.xml" />
                <html outfile="${dir.coverage}/coverage.html"  />
            </report>
        </emma>
        <echo message="${dir.coverage}"/>
       </target>

Setting from teamcity project, which is calling "test" target:

If you have your own Emma task in build script, you should not use TeamCity coverage, otherwise you'll have you class files instrumented two times. Try to disable TeamCity own coverage and run your build again.

Thx Pavel, coverage data was generated. But how can I show it to user?
Now, coverage.txt ... were generated to build folder. Do I need to set some specific path?
--I solved this problem. Just copied files to root folder.

But the question I still have: how can I show some stats in overview tab?

If you need both coverage statistics and report, you'll have to use TeamCity coverage only (you need to disable emma task in your build script).
Alternatively you can publish coverage report produced by emma as an artifact and define custom tab on the build to show it: http://confluence.jetbrains.net/display/TCD6/Including+Third-Party+Reports+in+the+Build+Results but in this case you won't have coverage statistics numbers.
I would suggest to go with the first option, to my opinion it is simpler.

Thx, Pavel. Probably I will do as you proposed.

Hi Pavel.

>If you need both coverage statistics and report, you'll have to use  TeamCity coverage only (you need to disable emma task in your build  script).

Please explain me, if it is not hard for you, what i have to remove/add in my build script and Teamcity properties, to make Teamcity generate coverage (I want show stats on Overview tab to users).

You should call the target from your build script which runs tests only. In the TeamCity web UI you can choose IDEA coverage or Emma coverage. IDEA coverage is preferrable because it does not modify class files on disk. Then you need to provide packages where you want to gather coverage and regexps where you do not want to have coverage (usually test classes should be excluded). After that TeamCity should be able to gather coverage automatically. Let me know if it works for you.

Hi Pavel. Thank you for monitoring this thread.

You wrote:
>Then you need to provide packages where you want to gather coverage and  regexps where you do not want to have coverage (usually test classes  should be excluded).

Tell me please, how I can provide this packages? What I have to do particularly?

Do you mean coverage.lst, which contains smth like this:
+com.pdf.jr3_1_0.EditionType
+com.pdf.jr3_1_0.JrColor
+com.pdf.jr3_1_0.JrGraph
....

I set Emma coverage runner in Teamcity project properties.
Also I changed target of my build script (classes for which I want have coverage are located in emma.jr.pdf.jar):
<target name="compile_test">
        <delete dir="${dir.test.bin}"/>
        <mkdir dir="${dir.test.bin}"/>
          
        <copy todir="${dir.root}/test/tools">
            <fileset dir="${dir.root}/src/src.jr3.7"/>
        </copy>
        <copy todir="${dir.root}/test/tools">
            <fileset dir="${dir.root}/src/src.jr3.1"/>
        </copy>
          
        <javac srcdir="${dir.root}/test" destdir="${dir.root}/test/classes" debug="on" deprecation="on" source="1.5" target="1.5" encoding="UTF-8">
            <classpath>
                <pathelement location="${gl.path.build.tmp}/${gl.jar.file}_${DSTAMP}_jdk${compile.source}.jar"/>
                <pathelement location="${dir.test}/dependencies/crimson.jar"/>
                <pathelement location="${dir.test}/dependencies/junit.jar"/>
                <pathelement location="${dir.test}/dependencies/serializer.jar"/>
                <pathelement location="${dir.test}/dependencies/xalan.jar"/>
                <pathelement location="${dir.test}/dependencies/xalan.jar"/>
                <pathelement location="${dir.test}/dependencies/xercesImpl.jar"/>
                <pathelement location="${dir.test}/dependencies/xml-apis.jar"/>
                <pathelement location="${dir.test}/dependencies/xsltc.jar"/>
                <pathelement location="./${dir.test}/dependencies/tools"/>
                <pathelement location="${dir.root}/dependencies/lib.common/log4j-1.2.15.jar.jar"/>
                <pathelement location="${dir.root}/dependencies/lib.common/commons-logging-1.0.4.jar"/>
                <pathelement path="${dir.root}/dependencies/lib.jr3.7/"/>
                <fileset dir="${dir.root}/dependencies/lib.jr3.7">  
                    <include name="**/*.jar" />  
                </fileset>
                <fileset dir="${dir.root}/dependencies/lib.common">  
                    <include name="**/*.jar" />  
                </fileset>
            </classpath>
        </javac>
      </target>

    <target name="test" depends="compile_test">
        <junit
            fork="yes"
            printsummary="yes"
            showoutput="true"
            maxmemory = "1024M">

         <jvmarg value="-Dfile.encoding=UTF-8"/>
         
        <test name="regression.Regression" fork="true"/>
        <test name="regression.Regression_jr370" fork="true"/>         
        
        <formatter type="plain" usefile="false" />

        <classpath>
            <fileset dir="${dir.test.lib}">
                <include name="*.jar"/>
            </fileset>
            <fileset dir="${dir.lib}">
                <include name="*.jar"/>
            </fileset>
            <fileset dir="${dir.root}/dependencies/lib.common">
                <include name="*.jar"/>
            </fileset>
            <fileset dir="${dir.root}/dependencies/lib.jr3.7">
                <include name="*.jar"/>
            </fileset>
            <fileset dir="${dir.root}/build-tmp">
                <include name="emma.jr.pdf.jar"/>
            </fileset>
            <fileset dir="${dir.root}/misc/tools">
                <include name="emma.jar"/>
            </fileset>    
            <fileset dir="${dir.root}/misc/tools">
                <include name="emma_ant.jar"/>
            </fileset>               

            <pathelement location="${dir.test.bin}"/>
        </classpath>

        </junit>
        
    </target>

Log i have got:
[04:44:42]: compile_test

[04:44:42]:     [compile_test] mkdir

[04:44:42]:         [mkdir] Created dir: C:\BuildAgent\work\d27cb84568f8a8bd\trunk\test\classes

[04:44:42]:     [compile_test] copy

[04:44:42]:         [copy] Copying 10 files to C:\BuildAgent\work\d27cb84568f8a8bd\trunk\test\tools

[04:44:42]:     [compile_test] copy

[04:44:42]:         [copy] Copying 9 files to C:\BuildAgent\work\d27cb84568f8a8bd\trunk\test\tools

[04:44:42]:     [compile_test] Adding debug=true parameter to javac task

[04:44:45]:         [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.234 sec

Btw, I tried Idea coverage runner - it works for me. But if you can, explain me pls, what I am doing wrong with emma.
Thx