TeamCity and TestComplete Integration a Walkthough

Many people ask how we can integrate TeamCity and TestComplete to run automated regression tests, however the advice seems to be contradictory or incomplete as many people fail to appreciate that others might be going thought the same pain and forget to post the solutions that worked for them. I aim to correct this with a working solution.

This walkthrough will outline the steps we took to achieve a working solution. Most of these instructions are valid both for invoking TestComplete/TestExecute from the Command Line and from the MSBuild extension, however I am going to concentrate on the Command Line method. The choice of the command line method was partially influenced by the fact it is possible to produce a detailed Log of the TestComplete run in a rich graphical manner allowing for the screenshots and other additional information to be presented in the TeamCity web interface.

Prerequisites

In order to achieve this there are some prerequisites:-

·         You are able to have an unlocked interactive session on a server/workstation/virtual machine in your network. We used a Console session on a virtual machine.

·         You have a spare TeamCity Build Agent and at least one build configuration available.

·         You have a spare TestComplete/TestExecute license for the automated regression test. This is so that you don’t have a license failure during the automated test.

Setting up the TestComplete environment

We started with a Virtual Machine which could only be accessed by a limited number of users via RDP.

1.       Create an account appropriate to your runtime environment to run the tests in. This will be the account the TeamCity Agent will run under.

2.       Setup TestComplete/TestExecute as normal and make sure that your tests will run in the environment.

3.       Set the account you setup in step 1 to be the Auto-Logon account for the machine, this will ensure that the machine always has a console session signed in with the relevant permissions to run your application. We used the AutoLogon utility from SysInternals to set this up.

4.       Download an MHT to HTML convertor/extractor we used this one ( http://www.legroom.net/software/extractmht )

5.       Install this on the TestComplete server.

6.       Install a TeamCity Build Agent on the machine. The agent will be responsible for running TestComplete in a UI safe environment.

a.       Specify the TeamCity Build Server parameters as if you were connecting a normal agent.

b.      Do not install it as a service – this is key to the success of your tests running.

c.       Check that the agent has not been created as a service using the Services console in windows. If it is present then you can safely disable the service.

d.      Browse to the Agents bin folder there you will see an Agent.BAT script file.

 i.      Create a new batch file, call it AgentRestart.BAT

 ii.      In this batch file type
                 call Agent.BAT stop
                 call Agent.BAT start

 iii.      Save the batch file.

e.      Go to Scheduled Tasks / Task Scheduler

i.      Create a new Task

ii.      Give it an Action which runs the AgentRestart batch file

iii.      Give it a Trigger so that the task runs on Login of the auto-logon account previously setup

iv.      Optionally add a second trigger which runs periodically. Say at midnight each night

v.      Select the options to allow the task to run only when the auto-logon account is logged on.

7.       Reboot the machine, this will log the Auto-Logon user in and start the TeamCity agent.

8.       Using Remote Desktop (or your remote control software of choice) connect to the Virtual machine with the Auto-Logon credentials

a.       If using RDP this will steal the console session – you will need to put the session back on to the console when you have finished. You can use the command
                 TSCON 1 /dest:console
where the 1 in this example is your RDP session. This should be OK because you have previously rebooted the machine and the autologon user will be assigned session 1 (the default for the console session)

9.       Manually run a test using TestComplete to ensure that everything is OK.

10.   You need to have a mechanism in place to clean up any remaining open applications once TestComplete has finished, as if there is a failure then TestComplete can leave some apps in the state at the point where testing ended. Causing reference errors whilst running the tests.

a.       You can reboot the workstation

b.      You can kill any open instances of the apps under test before the next run

c.       You can kill any open instances of the apps under test as a final step in the test configuration ( a bit more involved in setting this up)

11.   You are now ready to configure TeamCity.

Setting Up TeamCity to Schedule TestComplete

1.       Log onto TeamCity using an administrator account.

2.       Open the Agents Console.

3.       Select the new agent created in the Test Complete setup and Authorise it

4.       Place the Agent in its own pool so that it will not be available for general build purposes.

5.       Create a Project / Build configuration for running the test

a.       Add the necessary steps to ensure that your test environment has the desired version of the applications under test installed and any dependencies such as databases have been updated.

b.      Add a command line build step to run TestComplete/TestExecute.

i.      Specify the TestCompleteProjectSuite to run

ii.      Specify that it should save a MHT style test log, save this log to the TeamCity checkout directory (or subfolder)

iii.      Specify that TestComplete should exit once the run is complete.

iv.      Specify that Testcomplete should run in silent mode and that it should save execution errors to an error log. Save the error log in the TeamCity checkout directory or subfolder.

c.       Add a command line build step to run the MHT to HMTL extract tool and extract the HTML and associated files to a subfolder.

i.      Set the step to always run even if some steps have failed.

d.      Add a PowerShell build step. (This step will correct the internal paths of the extracted HTML so that the resulting web pages will run inside TeamCity.

i.      Set the step to always run even if some steps have failed.

ii.      Use the following script. Replacing the log directory with the location you extracted the test log to in the previous step.
(Get-Content %teamcity.build.checkoutDir%\log\root.xml) | Foreach-Object {$_ -replace "http:\/\/localhost\/", ""} | Set-Content %teamcity.build.checkoutDir%\log\root.xml

e.      Add an artifact to the build configuration to expose the extracted log folder.

f.        Optional: Add an artifact to the build configuration to expose MHT version of the log

g.       Optional: Add an artifact to the build configuration to expose the error log.

6.       In the Project add some Report Tabs

a.       Add a new Report Tab pointing to the index.htm inside the log folder artifact. This should provide a rich interactive graphical log in the TeamCity build status web page. Which depending on your log settings can contain the snapshot images just like TestComplete inbuilt log viewer.

b.      Optional: Add a new Report Tab pointing to the error log text file if you have decided to expose this.

7.       Configure the Project/Build configuration to only run from the TestComplete Agent Group.

8.       Configure triggers as appropriate either on a schedule or on the successful completion of a dependant build configuration.