parallelizing tests on teamcity

 I just tried to google and I cannot believe what I see. Tickets hanging for almost 10 years with high priority, multiple are assigned and still nothing is done about it, still no freaking support for parallel execution... because somebody in TC believes that it's not a good idea?! WTF?

I was absolutely sure that such obvious thing is possible out of the box, I guess my opinion was way off about TC.

What's possible other than switching from TC or writing something ourselves? In my case build has to be done on one PC (because of size and time), then if because of some dumb decision in TC I have to run these test on different machines to get parallel execution then I'm ok with that. If that's possible, how can I do that? If that's possible then while each of our multicore build agents run 5+ minute long tests will that build agent become unavailable for other builds because 1 core out of 8+ cores is waiting on IO?! This kind of dumb limitation make people turn away looking for alternatives.

Here's some visual approximation what I mean:

Right now it looks like this:

But if that "Release Mac Build" config had a build step that was "run multiple commands in parallel, complete all or fail" and then it would look something like that:

Hi Pablo,

I'm not entirely sure that I understand what you mean. You say "parallelizing tests" in the title. Parallel tests is a feature of the test runners themselves, and are supported where the test runner supports parallel tests. For example, JUnit has no native support, while NUnit does, and so does TeamCity respectively.

What you seem to describe is instead parallelizing different build steps (what you call tasks) inside a single build configuration. This is not directly possible as build steps are supposed to be executed in order. If you have different build steps that can be run in parallel, you should convert them into different build configurations which can be run in parallel, then set up snapshot dependencies so that they run in the appropriate order if required. This is our approach to different tasks which depend on each other.

Different build configurations can run in parallel in different build agents, as you have already mentioned. You can also set up multiple agents in a same device so that the results from one can be used by the other agents in the same device without having to transfer large loads of information.

@Pablo XX: Sorry, Pablo, you are losing the point! Look: https://blog.jetbrains.com/teamcity/2012/04/teamcity-build-dependencies-2/
Especially: "TeamCity Snapshot Dependencies" - so it is possible to do parallel steps in this way, but it requires:

a) extra build configurations 

or (likely and)

b) extra agents

So TC sales would not be interested in parallel steps within the same build configuration - as TC pricing is based on number of agents and build configurations.

If real parallel steps functionality developed - it means Jetbrain is losing money! Hopefully I am wrong.

@Alexopenmind:

I would like to point out that I had explained how it works in the response above yours. Multiple build configurations running in several agents is the way TeamCity is set up to run tasks in parallel. Also, with the Professional Edition, free of charge, we allow for up to 20 build configurations and 3 agents, which should cover most small projects.

So the "parallel steps" is already developed, implemented and working, and (to a certain extent) available for free. The admin just needs to adapt and move from multiple build steps, to multiple build configurations.

@Denis Lapuente

We have many cases where we would like to run build steps in parallel on the same agent. Often it's a good way to make full processing power of an agent. Sometimes you have build steps like uploading large debug symbols or packaging and uploading files to an FTP somewhere, that can be run while something else like unit tests are being run.

Sure you can install multiple agents on one host, but in a lot of cases creating multiple build configurations with snapshot dependencies becomes impracticable. This is especially true when you use branch builds functionality. It's convenient when you have one build configuration off one repo able to build any dev branch automatically from a naming pattern, but if you split this config up into half a dozen, it becomes a nightmare. It's also a problem when you want to branch the project itself for release versions.

I just end up having to manually create parallelisation using my own scripting.

Hi Anuradha,

The situation you describe in the first paragraph is almost by definition what TeamCity considers a build chain: A series of build configurations tied with snapshot dependencies that do independent things and can be run in parallel.

Creating snapshot dependencies should take care of the branches by itself. Once you set up a snapshot dependency, triggering a build on a specific branch will carry that branch throughout the chain (in all builds that have a VCS set up, even through multiple different VCSs, if the branch exists in them).

Maintaining the different builds and their dependencies does have some maintenance overhead, but "manually create parallelisation using own scripting" also has quite a bit, if not more, while doing the maintenance for the dependencies allows to:
-parallelize tasks not just on a same agent's machine but also on multiple agents, be it on the same machine or on multiple.
-have a more granuralized overview of the different components that form part of a complex build.
-reuse components in multiple builds
-reuse components from many other builds, not just those on the chain

If the steps are independent to each other (or just have common dependencies), and you want to run them in parallel, a build configuration has a very small overhead in comparison to a build step and offers parallelization along all the advantages mentioned above.

This said, we provide scripting runners precisely for the situations where teamcity's functionality doesn't match your scenario. If you consider it easier or better for your workflow to have a single build with scripts performing all tasks rather than multiple builds, please feel free to do so.

Also, please consider voting and watching this issue: https://youtrack.jetbrains.com/issue/TW-13849. Feel free to join the discussion adding any comment you might consider relevant.

@Denis Lapuente The use case I have for wanting parallelised builds (and this is what I suspect other that ask for it have) in one configuration can't really be addressed by multiple build  configurations though:

1. We want to make full use of the CPU power of modern physical agents. A decent physical machine will be using less than a quarter of its capacity for most of the time during a one hour build of our product. Splitting large legacy builds across multiple build configurations and agents will require a lot of resources. If we had a well architectured product that could build on Linux and use, say Docker containers, then it would be better to spin up (or down) lots of tiny VMs to meet demand. But with a product that's been around a long time and builds on Windows, this is not so easy.
2. We need to support several previous releases, with dev branch builds on all of them. If a single release required half a dozen smaller build configurations, multiplied out to half a dozen supported releases, all with dev branches, the number of agents required to get decent throughput is not practical. We already have close to 100 build configurations to manage as it is.

We script our own parallel building solutions, but it would be very useful if TeamCity had the functionality.

Hi Anuradha,

Thanks for the comments, but I'd like to point out a couple things:
-Docker also works on Windows and we now have support for it: https://blog.jetbrains.com/teamcity/2017/10/teamcity-docker-images-for-windows/ and https://confluence.jetbrains.com/display/TCD10/Integrating+TeamCity+with+Docker
-Other Windows-capable services, such as VMWare, are also supported and provide very similar functionality.
-Parallelization works in the extent of which the tools that run the builds support it. We don't restrict tools that support it from using it. This obviously extends to custom scripts such as what you are using.

-If you need to maintain a very large amount of build configurations for different branches, products and versions, you can use versioned settings: https://confluence.jetbrains.com/display/TCD10/Storing+Project+Settings+in+Version+Control, which might be more helpful. Older versions can be stored in the versioned settings if you need, so that you can simply modify them at need.

Also we don't deny its utility, that's why it works with the different build configurations running in multiple agents system.

As I already mentioned, please refer to the issue in the tracker. Any discussion towards its request should go there.

Hi Dennis,

Running steps in parallel is always in high demand and must be as easy as in Jenkins, e.g.:

  steps {
    parallel(
      a: {
        echo "This is branch a"
      },
      b: {
        echo "This is branch b"
      }
    )
  }

No non-obvious and not immediately visible snapshot dependencies - in one quick look I see the whole parallel graph of executions.

It's quite strange that you are trying so hard not to see this obvious lack of very useful functionality in TC.

Dear Serguei,

If you re-read the conversation, I've already pointed out that we already have a feature request to implement "it", have linked to the request, and have spent the rest of the time explaining how the requirement can be approached with the current tools already implemented, so your personal attack is uncalled for.

Also, the functionality is available in TeamCity, as I've already explained through the rest of the topic. Maybe not as "easy" as in jenkins, but available. The Kotlin DSL, with proper enough handling, will even let you have a similar syntax. Change "parallel steps" for "build configurations", and you can achieve basically the same rather easily, setting up a chain. Do not translate jenkins "steps" to TeamCity's "steps", because they aren't the same base concept.

If you have further comments about the feature itself, and not on how to achieve the specific task with TeamCity, please forward them to the feature request in the tracker (https://youtrack.jetbrains.com/issue/TW-13849). Discussion about its merits and drawbacks will happen there with the developers.

Hi Denis,

Let me just to quote a guy Pablo XX who started this topic:
"I just tried to google and I cannot believe what I see. Tickets hanging for almost 10 years with high priority, multiple are assigned and still nothing is done about it, still no freaking support for parallel execution... because somebody in TC believes that it's not a good idea?! WTF?"

That's basically it. :-)

Denis Lapuente I think you are missing the point here. Sure TC "can already" execute steps in parallell, but that requires a setup of build chains.
TC is already lacking in overview in terms of step configuration. Trying to see the flow here becomes uneccessarily complex (read: impossible).
This simply is not an acceptable solution if you just want to have "execute these 2 steps in parallel, and wait for both to finnish before running the next one"
And then reading "TW-13849 Created 12 years ago"

Hi Christian,

Since the last answer is over 4 years old, some things have changed in the "step configuration overview" realm that you mentioned. We introduced the build chains kotlin dsl extension, so what Serguei mentioned a few messages ago is now possible: https://www.jetbrains.com/help/teamcity/kotlin-dsl.html#Build+Chain+DSL+Extension .