Distributing testing over multiple agents

We currently have a product that its build and testing runs around 2 hours.  We would like to get the time down.  One way we want to try is to spread the testing out over multiple agents.  The issue that I have is how in team City would we set that up.  Currently we have all our build and tests chained together with snapshots:  the build process is:

test 1
test 2
test 3
test 4

So when a build is run I want the compile to kick off.  then the database step.  Then I want to run the test steps on different machines.  then if all is good run the installers and label that changeset.  I cant seem to find of a way to have this process flow setup in TeamCity.  Can anyone help?

Comment actions Permalink

wow no one has any suggestions?

Comment actions Permalink

I guess I'll take a swing at it and see if I can help.

We currently use a multi-threaded framework for testing (MbUnit) this may be an option for you (I highly recommend using a framework that will do parallel processing, it's much easier than the other solution).

However if that is not:

Build Config 1 { compile, database }

Build Config 2 { test 1 }

Build Config 3 { test 2 }

Build Config 4 { test 3 }

Build Config 5 { test 4 }

Build Config 6 { installers,  label }

So the idea is that Build config 2, 3, 4 and 5 trigger on build config 1 being complete, the disadvantage is that with 3 build agents you'll get 3 tests running then the 4th, but it should still be much faster. Also it will eat a lot of build configurations if you're running on the free version.

Build config 6 had a dependency on 2, 3, 4, 5 being completed. You can probably have it queue on Build config 2 being completed and it will simply queue (If I'm not mistaken, I haven't set up queues like this before).

The other option is to look into scripting job triggering using ANT or something similar.

Comment actions Permalink

Thanks, William

in your example config 6 (installer) should depend from all the tests - to ensure that setup build become successful only if all test suites are passed.
Also, this installer build configuration should have VCS trigger enabled with Trigger on changes in snapshot dependencies option - this allows to start the whole build chain on any change in code or any of tests.

David, if you have any additional questions - feel free to ask.


Comment actions Permalink

Hi Micheal,

It would be nice if teamcity allowed you to spin up several ec2 instances and run all the tests split across those ec2 instances all from one build configuration.  This way all the tests are reported all in one build type and not spread across 10 build configurations.  Any thoughts of adding something like this?  Is there a youtrack ticket already for something like this I can vote on?  I can't stress how huge this would be for us and I can only image how many people would want this, faster test results without a lot of extra work.  Get this done and you will crush Jenkins :).


Comment actions Permalink

I suppose TW-16423 is the closest one.
I'll duscuss this with the team once again.



Please sign in to leave a comment.