Best approach to using custom test framework


  For many reasons we are in the process of creating a custom automated test framework.  I would like to have TeamCity execute tests using this custom framework.  What is the best approach?  Is there an API I can use?  We are only just starting to use TeamCity as well, so I do not know much about it.

Pat O

1 comment
Comment actions Permalink


There are at least two sides of integration: launching the tests and reporting their results.

You will probably have a way to run the tests from the command line. For a start in TeamCity you can use the same approach to run the testing process from the build using either command line runner or invoking the process from a build script like Ant or MSBuild. Later on, you can consider writing a TeamCity plugin that will add own type of runner with specific settings and launching logic.

As to reporting the results, the best way seems to use "service messages". This way the process can detect that it is run under TeamCity build and print specially formatted text lines into standard output. TeamCity will then parse the lines and display tests running (e.g. like JUnit or NUnit tests). The progress can be reported on-the-fly not wating for the build to finish to display the results.

Other integration options include providing HTML reports that can be displayed on the build results and (if necessary) custom statistics that can be reported into TeamCity so that users can get idea of the metrics trends.


Please sign in to leave a comment.