TeamCity and TestComplete Integration a Walkthough

Completed

Many people ask how we can integrate TeamCity and TestComplete to run automated regression tests, however the advice seems to be contradictory or incomplete as many people fail to appreciate that others might be going thought the same pain and forget to post the solutions that worked for them. I aim to correct this with a working solution.


This walkthrough will outline the steps we took to achieve a working solution. Most of these instructions are valid both for invoking TestComplete/TestExecute from the Command Line and from the MSBuild extension, however I am going to concentrate on the Command Line method. The choice of the command line method was partially influenced by the fact it is possible to produce a detailed Log of the TestComplete run in a rich graphical manner allowing for the screenshots and other additional information to be presented in the TeamCity web interface.

Prerequisites

In order to achieve this there are some prerequisites:-

·         You are able to have an unlocked interactive session on a server/workstation/virtual machine in your network. We used a Console session on a virtual machine.

·         You have a spare TeamCity Build Agent and at least one build configuration available.

·         You have a spare TestComplete/TestExecute license for the automated regression test. This is so that you don’t have a license failure during the automated test.

Setting up the TestComplete environment

We started with a Virtual Machine which could only be accessed by a limited number of users via RDP.

1.       Create an account appropriate to your runtime environment to run the tests in. This will be the account the TeamCity Agent will run under.

2.       Setup TestComplete/TestExecute as normal and make sure that your tests will run in the environment.

3.       Set the account you setup in step 1 to be the Auto-Logon account for the machine, this will ensure that the machine always has a console session signed in with the relevant permissions to run your application. We used the AutoLogon utility from SysInternals to set this up.

4.       Download an MHT to HTML convertor/extractor we used this one ( http://www.legroom.net/software/extractmht )

5.       Install this on the TestComplete server.

6.       Install a TeamCity Build Agent on the machine. The agent will be responsible for running TestComplete in a UI safe environment.

a.       Specify the TeamCity Build Server parameters as if you were connecting a normal agent.

b.      Do not install it as a service – this is key to the success of your tests running.

c.       Check that the agent has not been created as a service using the Services console in windows. If it is present then you can safely disable the service.

d.      Browse to the Agents bin folder there you will see an Agent.BAT script file.

 i.      Create a new batch file, call it AgentRestart.BAT

 ii.      In this batch file type
                 call Agent.BAT stop
                 call Agent.BAT start

 iii.      Save the batch file.

e.      Go to Scheduled Tasks / Task Scheduler

i.      Create a new Task

ii.      Give it an Action which runs the AgentRestart batch file

iii.      Give it a Trigger so that the task runs on Login of the auto-logon account previously setup

iv.      Optionally add a second trigger which runs periodically. Say at midnight each night

v.      Select the options to allow the task to run only when the auto-logon account is logged on.

7.       Reboot the machine, this will log the Auto-Logon user in and start the TeamCity agent.

8.       Using Remote Desktop (or your remote control software of choice) connect to the Virtual machine with the Auto-Logon credentials

a.       If using RDP this will steal the console session – you will need to put the session back on to the console when you have finished. You can use the command
                 TSCON 1 /dest:console
where the 1 in this example is your RDP session. This should be OK because you have previously rebooted the machine and the autologon user will be assigned session 1 (the default for the console session)

9.       Manually run a test using TestComplete to ensure that everything is OK.

10.   You need to have a mechanism in place to clean up any remaining open applications once TestComplete has finished, as if there is a failure then TestComplete can leave some apps in the state at the point where testing ended. Causing reference errors whilst running the tests.

a.       You can reboot the workstation

b.      You can kill any open instances of the apps under test before the next run

c.       You can kill any open instances of the apps under test as a final step in the test configuration ( a bit more involved in setting this up)

11.   You are now ready to configure TeamCity.

Setting Up TeamCity to Schedule TestComplete

1.       Log onto TeamCity using an administrator account.

2.       Open the Agents Console.

3.       Select the new agent created in the Test Complete setup and Authorise it

4.       Place the Agent in its own pool so that it will not be available for general build purposes.

5.       Create a Project / Build configuration for running the test

a.       Add the necessary steps to ensure that your test environment has the desired version of the applications under test installed and any dependencies such as databases have been updated.

b.      Add a command line build step to run TestComplete/TestExecute.

i.      Specify the TestCompleteProjectSuite to run

ii.      Specify that it should save a MHT style test log, save this log to the TeamCity checkout directory (or subfolder)

iii.      Specify that TestComplete should exit once the run is complete.

iv.      Specify that Testcomplete should run in silent mode and that it should save execution errors to an error log. Save the error log in the TeamCity checkout directory or subfolder.

c.       Add a command line build step to run the MHT to HMTL extract tool and extract the HTML and associated files to a subfolder.

i.      Set the step to always run even if some steps have failed.

d.      Add a PowerShell build step. (This step will correct the internal paths of the extracted HTML so that the resulting web pages will run inside TeamCity.

i.      Set the step to always run even if some steps have failed.

ii.      Use the following script. Replacing the log directory with the location you extracted the test log to in the previous step.
(Get-Content %teamcity.build.checkoutDir%\log\root.xml) | Foreach-Object {$_ -replace "http:\/\/localhost\/", ""} | Set-Content %teamcity.build.checkoutDir%\log\root.xml

e.      Add an artifact to the build configuration to expose the extracted log folder.

f.        Optional: Add an artifact to the build configuration to expose MHT version of the log

g.       Optional: Add an artifact to the build configuration to expose the error log.

6.       In the Project add some Report Tabs

a.       Add a new Report Tab pointing to the index.htm inside the log folder artifact. This should provide a rich interactive graphical log in the TeamCity build status web page. Which depending on your log settings can contain the snapshot images just like TestComplete inbuilt log viewer.

b.      Optional: Add a new Report Tab pointing to the error log text file if you have decided to expose this.

7.       Configure the Project/Build configuration to only run from the TestComplete Agent Group.

8.       Configure triggers as appropriate either on a schedule or on the successful completion of a dependant build configuration.

6 comments
Comment actions Permalink

Hi Darren,

Thank you for sharing your solution. Interested users also can vote for the related feature request: https://youtrack.jetbrains.com/issue/TW-18642

0
Comment actions Permalink
The feature request only goes as far as TeamCity being able to consume the native TestComplete xml reports format.Much of the walkthrough is getting TestComplete to run UI tests from TeamCity, so is still relevant even if this was implemented.The extracted html report will provide far more information that parsing the raw xml file. See attachment.

Attachment(s):
TestComplete_Log_In_TeamCity.PNG
0
Comment actions Permalink

Hello, Darren

Thank you for your contribution. I have tried your suggestions and cannot figure out how tests can interact with GUI. All works fine, connection to DB, other console logic, but when test tryes to click smth it cannot and i don't have any screenshots of the situation. What's wrong?

My Agent is setting on a virtual machine with Windows 2008 Server OS. If you have any ideas how can I to solve my problem, please respond.

0
Comment actions Permalink

Rodion Mochalov, you will probably need an active GUI session that is provided via RDP.  Depending on your companies security policies, you may also have to figure out a way to keep the RDP session from going in to Windows Sleep Mode.  In any event, try RDP'n into the vm you're using, and launching the TeamCity project so you can actively watch your application under test.  It should work then.

 

Our setup requires an active GUI session to interact with our webpages and toolbar forms.  Our setup looks like this:

 

I RDP in to Parent_VM_1

From Parent_VM_1, I RDP in to

 - TestComplete_VM_1

 - TestComplete_VM_2

 - TestComplete_VM_3

etc, etc

 

Our test vm's are setup to never go in to sleep mode.  We do not use auto-login's because of security.  We reboot our vm's several times a day and provide manual logins.

 

By logging in to the Parent_VM_1, then logging in to the TestComplete_VM's, this will allow you to close out of Parent_VM_1 RDP session and still maintain an active GUI session with the remote TestComplete VM's.

 

0
Comment actions Permalink

 

Another way is to use RDPMan from Microsoft. This allows one pc to provide access to all your VMs by RDP and will give you thumbnails for each vm. This can be on any server or desktop which can see the VMs. Keep this app open will keep all the VM desktops alive with a connected virtual keyboard and mouse.

 

1
Comment actions Permalink

Hi Darren

Thank you very much for this article.  I follow and have successfully setting my Test Complete to run from a Team City Build agent on a virtual machine with now window.

However, the virtual desktop size is much smaller than the normal size when I log in.  How could I change the desktop resolution?  Or I have to change my test cases to cater for this smaller window size?

Thanks in advance

 

0

Please sign in to leave a comment.