Parallel Build Configurations with dependencies


I have build config A which does test setup that (among other things) starts a Docker container.

The following build configs B, C, D run some tests within the Docker container from A.
I would like B, C, D to run in parallel after A has run.

How can I set up my builds in a way to make this possible? I've set up a snapshot dependencies in B,C,D but it looks like it requires things to be run on the same agent, which doesn't help with parallelization.

Hi! It is not possible to run multiple builds simultaneously on the same build agent, but it is possible to run multiple build agents on one machine (albeit it is not the recommended setup):
Would that be useful for you?

Additionally, it could help if you could provide some more information about your use case (such as what you are trying to achieve and why).

Anatoly, thank you for your response.

In Jenkins, I have something like this:

"Test Setup" does a docker compose which brings up a docker container `test-container`.
Then I have 3 different build configs that run different tests using command line.

Stuff like

  • docker exec test-container pylint blah blah
  • docker exec test-container pytest blah blah
  • docker exec test-container isort blah blah

These all require test-container to be running.

I see the two options:

(1) Run 'Tests Setup' as the first build step and run all of the 'Run Tests' commands in the second build step as background tasks, something like this:

docker exec test-container pylint blah blah 30 &
docker exec test-container pytest blah blah &
docker exec test-container isort blah blah &

The downside is that it will likely mix the output of those three in a single build log.

Or (2) run each for the 'Run Tests' commands on a different build agent in its own container.
If that doesn't work for you, could you clarify why they have to run in the same container?

Yep, I ended up going with option (2). This runs the setup more times than is needed, but the runtimes end up being the same anyway if run in parallel.


Thank you for your response!


Please sign in to leave a comment.