Processing coverage report from Android - Possible?

Hi folks,

We are using TeamCity for continuous test/build cycles for all our Android apps, and it's working well... but....

Android can generate EMMA code coverage reports natively, and in fact HAS to generate it natvely (on the device) as that's where the code is executed.  We can produce both HTML and XML output from the EMMA coverage execution, but there doesn't seem to be a way for us to have TeamCity interpret these results.

The same happens for JUnit tests (which are also run on-device) in which we produce a series of XML files that are parsed by TeamCity such that the test results are "integrated" into the reports provided by TC.

So my question is:

If we are able to supply a complete "coverage.xml" file to TeamCity, is it possible to have TC parse this to integrate the coverage reporting?

In understand that we can select to have EMMA coverage reports generated, but this obviously won't work for us because we are already generating the reports from execution on the device.

I don't see any UI options to do this, but thought (hoped) there may be some config options or plugins that might help.

Any ideas?

Thanks,

Jason.

7 comments
Comment actions Permalink

Jason,

What build runner do you use?

0
Comment actions Permalink

Hi Maxim,

It's just a standard ANT runner, but the ANT script it runs is and Android ANT script which does 4 things:

1. Compiles the code with instrumentation for EMMA and builds the APK (application file)
2. Installs the APK on a device (or emulator) attached to the build server
3. Executes the tests ON the device and produced JUnit reports and EMMA reports ON the device
4. Pulls the reports from the device and places them on the file system of the build server.

Because the code execution happens on the device, I need a way to be able to have TC just parse the output of the EMMA coverage run after the build.  In exactly the same way TC currently processes JUnit test output (from a series of XML files pulled from the device).

Hope this makes sense.

Cheers,

Jason.

0
Comment actions Permalink

Jason,

Since you can generate HTML report you can setup a custom report tab in TeamCity to see it. In this case you should publish your HTML report as artifact in your build (for this you can setup artifact paths on the build configuration general settings page) and configure a report tab.

And if you want to highlight the code in IDE with coverage data you need to generate also "coverage.ec" and "coverage.em" files.

0
Comment actions Permalink

Thanks,

We already have the HTML artifact viewable, but it's not so much about viewing the data, rather it's about TC being aware of it.  As in the screenshots here:

http://www.jetbrains.com/teamcity/features/code_quality.html

TC is aware of the coverage data associated with a given class and presents this in the reporting for a build.  This is ideally what we would like to see.

It's not a deal-breaker at all.. just would be nice to have everything "inside" TC rather than having to pop out to an external HTML file that doesn't hook into TC at all.

Thanks for your attention nontheless.

Cheers,

Jason.

0
Comment actions Permalink

Oh, and we do also have the .ec/.em files.  Is there somewhere I can put these so TC can interpret them?

0
Comment actions Permalink

Sorry for the delay in replying.

Seems TC cannot parse your report automatically, but you can provide your own values to be drawn in chart. Please read custom chart documentation for more information.

As to .ec/.em files - if you publish them as artifacts in your build you will be able to highlight coverage data in IDE. E.g. in IntelliJ IDEA you can do it via "Analyze -> Show Code Coverage Data" menu option (and in the near TC 6.5 release you will be able to do it right from the "TeamCity -> Code Coverage" menu item). Please note that TeamCity Integration plugin is required for it.

0
Comment actions Permalink

TeamCity CAN parse the coverage report and store the statistics; it just expects  coverage.txt to appear in a different location. Unfortunately, neither Android Ant nor TeamCity allow you to configure the location of coverage.txt. Android Ant puts it in the bin/ directory, and TeamCity expects it in the checkout directory.
This is what I did:
1. Create a file custom_rules.xml (which is automatically included in build.xml) with the following contents:
<?xml version="1.0" encoding="UTF-8"?>
<project name="SpeedcamAppTest-CustomRules" default="help">
<target name="tc-emma">
<if condition="${emma.enabled}">
<then>
<echo level="info">Saving a copy of coverage.txt in ${out.absolute.dir}/..</echo>
<copy file="${out.absolute.dir}/coverage.txt" tofile="${out.absolute.dir}/../coverage.txt"/>
</then>
</if>
</target>
</project>

This defines a build step 'tc-emma' which copies the file coverage.txt to the checkout directory.
2. Tell TeamCity to execute that step, by setting the build target to "clean emma debug install test tc-emma"
3. Tell TeamCity to use the Emma Code Coverage runner
4. Last but not least, tell TeamCity to exclude all file from instrumenting, else TeamCity will instrument the files as well as Android Ant. Do this by specifying "-ix -*" as Coverage instrumentation parameters.

Now the coverage reports are used to gather statistics in TeamCity. I couldn't get Android Ant to retain the coverage.em and coverage.ec files for me, but I use the standard Android Ant coverage.html in a custom Tab in TeamCity.

0

Please sign in to leave a comment.