Parallel Build Configurations with dependencies
Answered
I have build config A which does test setup that (among other things) starts a Docker container.
The following build configs B, C, D run some tests within the Docker container from A.
I would like B, C, D to run in parallel after A has run.
How can I set up my builds in a way to make this possible? I've set up a snapshot dependencies in B,C,D but it looks like it requires things to be run on the same agent, which doesn't help with parallelization.
Please sign in to leave a comment.
https://www.jetbrains.com/help/teamcity/install-multiple-agents-on-one-machine.html
Would that be useful for you?
Additionally, it could help if you could provide some more information about your use case (such as what you are trying to achieve and why).
Anatoly, thank you for your response.
In Jenkins, I have something like this:
"Test Setup" does a docker compose which brings up a docker container `test-container`.
Then I have 3 different build configs that run different tests using command line.
Stuff like
These all require test-container to be running.
(1) Run 'Tests Setup' as the first build step and run all of the 'Run Tests' commands in the second build step as background tasks, something like this:
docker exec test-container pylint blah blah 30 &
docker exec test-container pytest blah blah &
docker exec test-container isort blah blah &
wait
The downside is that it will likely mix the output of those three in a single build log.
Or (2) run each for the 'Run Tests' commands on a different build agent in its own container.
If that doesn't work for you, could you clarify why they have to run in the same container?
Yep, I ended up going with option (2). This runs the setup more times than is needed, but the runtimes end up being the same anyway if run in parallel.
Thank you for your response!
I've seen many requests over the years to add support for parallel steps in TeamCity and I'm not sure it's ever going to happen… but I'll post this anyway.
In my case I have a build project. As you'd expect, it has multiple steps.
Once my build is complete - it then has deploy steps (using Octopus Deploy). I have multiple deploys (to different envs/targets), but all deploy the same build packages.
I want to run my deploys in parallel.
Can I? Is there some way to acheive this?
Native support for parallelism within a single build configuration is limited. Here are a workaround to accomplish running deploys in parallel.
You can split the deployment steps into separate build configurations that run in parallel, using dependencies between configurations. Here's how:
- Create individual build configurations for each deployment step
- Set the build dependency between your main build and the deployment builds, so that they trigger after the main build completes.
- Configure the deployment build configurations to run in parallel by adjusting the build triggers and dependencies.
TeamCity can run multiple build configurations in parallel if there are enough agents available.
Best Regards,
Tom