How to branch a snapshot dependency graph with reverse parameter injection?

I am having problems creating an aggregate build that schedules separate build chains run with different build parameter reverse injections.  

Lets say I have a two step chain, “Build”, and “Upload”.

  •  “Build” takes a parameter “configuration” and “Upload” takes a parameter, “destination”.
  • “Upload” depends on “Build”.
     

I then create two aggregate builds, depending on "Upload", with specific parameters

  • “BuildReleaseUploadTest”.  It has parameters “reverse.dep.Build.configuration=release” and “reverse.dep.Upload.destination=test”.
  • “BuildReleaseUploadPublic”.  It has parameters “reverse.dep.Build.configuration=release” and “reverse.dep.Upload.destination=public”.

I can run these aggregates and the correct thing happens.

But:  Lets say I want to run both aggregates.  Build “release” and upload to “test” and "public".  I create a new aggregate:

  • “BuildReleaseUploadTestAndPublic”
    It depends on both “BuildReleaseUploadTest” and “BuildReleaseUploadRelease” aggregate builds.

This is where the system breaks down.  When I run this, it will run both of the dependend aggregate builds, but both of them will depend on a single instance of the ‘Upload’ build.

What I expect:

A “Build runs once in ”Release".  Two “Upload” builds are then run, both depending on the release build, but with different “destination” parameter.

What happens:

A single “Upload” build is built.
Teamcity runs it with conflict parameters:

  • conflict.BuildReleaseUploadTest.destination=test
  • conflict.BuildReleaseUploadTest.destination=public
     

Discussion

Teamcity notices that there are different parameters being applied to the build through the chain, but instead of forking the build into two actual build based on the differeing parameters, it just notices this and carries on.

The above is a very simple problem.  We could simply duplicate the “Upload” step.  Base it on a template and then instantiate two versions based on the “destination” parameter.  But what if we want not just “Release” Build?  Then we have to also duplicate the “Build” for “Test” and “Release” configuratrion.   And then we need two other cases of the upload.   Very soon the permutations explode and it becomes unmanageable.
 

And while it is possible to ask the operators to trigger two build chains separately, what if the chain is more complex?  What if the final aggregate step I suggestsed isn't the last one, but an intermediate step somewhere along the line?

This is what the chain looks like in the dependency graph (build steps have different names from the above)

I am not able to tell TC that the “Chain Test Process” step needs to be run twice with different arguments.


Is there any clean way to do this, or similar things, using TeamCity's declerative apprach to build chains?
 

1
1 comment

Hi! The matrix builds feature can help here.

  • “Build” takes the parameter configuration.
  • “Upload” is a matrix build configuration
    • Matrix build feature settings have one parameter named destination with as many destinations as required.
    • To make the destinations more discoverable and to simply it for the user who runs the “Upload” configuration, you can optionally add the following parameters on the build configuration's Parameters tab:
      • Name: destination
        Kind: Configuration parameter
        Value type: Select
        Allow multiple selection: enabled
        Items: new line delimited list of destinations
        Display: Prompt
      • Name: reverse.dep.Build.configuration
        Kind: Configuration parameter
        Value type: Select
        Allow multiple selection: disabled
        Items: new line delimited list of configurations
        Display: Prompt

Things to watch out for:

  • Suppose you need to run “Upload” for multiple configurations of “Build”, say “Build debug” and “Build release”. In that case, you need to run “Upload” multiple times with different values of the reverse.dep.Build.configuration parameter. Ideally, you would probably want to make it just another parameter of the matrix build feature, but that would only work after TW-84442 is resolved.
  • If another build configuration depends on “Upload”, and the user triggers the dependent configuration via web UI, they need to remember to specify the parameter values required by the Upload. The dependent configuration will not automatically prompt those just because running “Upload” would prompt them. You can also add the prompt parameter to the dependent configuration to work around that. Ideally, it should be prompted by the dependent config automatically. Related issue: TW-18486.
  • When a multi-select parameter is prompted in the Run build dialog, it is possible to select no value. Ideally, in the case of the “Upload” configuration, you would probably want to force the user to select at least one destination. Related issue: TW-85994.

I hope this helps!

0

Please sign in to leave a comment.