Our project is compiled for multiple platforms. During the build the data needs to be processed and the source code needs to be compiled. This takes approximately 7 hours per platform.
Currently we sync from perforce in a first build configuration (checkout to custom folder) and we execute processing to generate project solution etc. The processing takes approximately 5 mins on average but a clean perforce checkout can take 4-5 hours due to its size.
During the weekend we clean the perforce folder, but on working days perforce is just syncing incrementally. So the first build configuration takes between 5-10 mins on average.
Due to the long build times we want to add additional build agents. Although the system above has worked well in the past on a single agent, it doesn't allow for parallel execution over multiple build agents. We could dedicate a single build agent per platform but this feels like a waste of resources.
If the first build configuration is A, we have configurations B, C, D and E which all depend on A. It is possible to launch builds manually on the other agents, but the automatic triggering will result in all the builds being queued up on the first available agent (the one that has been assigned A). I've tried to change settings but nothing seems to be doing what we want. We would basically want to tell A-B, A-C, A-D and A-E are build sets and overall it is a lot faster to assign A to multiple build agents.
Basically what is 'wrong' in our setup is that the perforce checkout A should be part of B, C, D and E but that would mean that each build agent wants to create 4 perforce workspaces (one for each platform) per agent. If we want to clean all that would take 4x 4-5 hours per clean perforce checkout.