Test reporting from Ant (service messages)

Hi!

I've tried to get our custom integration tests to report correctly in TeamCity (v5.1.1). This is the original code used for them:

     <target name="run-integration-test">

          <echo message="##teamcity[testStarted name='${target-server}']" />

          <trycatch property="tried">

               <try>

                    <antcall target="integration-test-${target-server}" />

               </try>

               <catch>

                    <echo message="##teamcity[testFailed name='${target-server}' message='Integration test for ${target-server} failed.' details='${tried}']" />

               </catch>

          </trycatch>

          <echo message="##teamcity[testFinished name='${target-server}' flowId='${target-server}']" />

     </target>



This target is run many times with different values for ${target-server}.

The problem is that failures logged to the build log with "##teamcity[testFailed..." are not registered anywhere. This *might* have something to do with the fact that testStarted and testFinished are not visible in the build log, at least when viewed from TeamCity's UI. Does anyone have any idea why that is?

I've tried to do workarounds hinted at from http://youtrack.jetbrains.net/issue/TW-5059 and http://devnet.jetbrains.net/message/5217575, but to no avail. Does anyone have an actually working workaround for this problem? Or a guide on how to correctly report test failures from Ant?

Thanks!
/Jonatan
8 comments
Comment actions Permalink

Hi Jonatan

As Yegor suggested in the issue, try to add the same flowId='${target-server}' to all three service messages.

Michael

0
Comment actions Permalink

Hi Michael,

Thanks for your answer. That was actually the first thing I tried, but unfortunately it did not work in this case. The result was exactly the same as without flowIds.

I also had no luck changing the echos to concats (with or without flowIds).

It just hit me that inside one of the antcalls, we run JUnit tests which probably will mess up the "flow", but the flowIds should take care of this if I understood them correctly.

/Jonatan

0
Comment actions Permalink

I've rechecked that, and in my lab it works correctly.
Could you post full build.xml please.

Thanks

0
Comment actions Permalink

Hi,

The ant script running the tests can be found here and the build.xml from which it is called is here. The project is open source so, if need be, you can grab it from svn at http://dev.vaadin.com/svn/versions/6.5.

The test script calls a build script on a remote server, this script starts a server, runs a test or two, and stops the server.

Any hints as to what might be wrong is highly appreciated.

/Jonatan

0
Comment actions Permalink

Thanks for the configs.

I played with them in my lab, but unfortunately cannot reproduce the issue.
Please let me some time to perform few additional tests.

Thanks
Michael

0
Comment actions Permalink

Hi again,

Any news on the additional tests?

Thanks,
/Jonatan

0
Comment actions Permalink

Hello Jonatan,

  The following script fails the test as expected for me:

<project>
<target name="run-integration-test">

          <echo message="##teamcity[testStarted name='sss' flowId='12321' ]" />
          <echo message="##teamcity[testFailed name='sss' flowId='12321'  message='Integration test for sss failed.' details='details']" />
          <echo message="##teamcity[testFinished name='sss' flowId='12321' ]" />

     </target>
</project>

  Please note, that flowId attribute should be set for all the <echos> and should have the same value.

  Hope this helps,
  KIR

0
Comment actions Permalink

Thanks for your efforts. It actually works now after updating to 6.0.2 (build number 15857) in case anyone stumbles upon this thread.

/Jonatan

0

Please sign in to leave a comment.