Have a build starting several other builds with parameters

Hallo

I'd like to create some builds that will be triggered manually that will start several other builds. Since the other builds only differ in some settings I'd like to pass those to the other builds:

  • Build: Trigger Promote Deployments from Dev to Test -> starts Builds: (read version from source.env and deploy that to target.env)
    • DB Deploy
    • Backend Deploy
    • Pilot Backend
    • SearchServer
    • UI
    • etc
  • Build: Trigger Promote Deployments from Test to Pre-Prod -> starts Builds: (read version from source.env and deploy that to target.env)
    • DB Deploy
    • Backend Deploy
    • SearchServer
    • UI


The Trigger Builds pass mainly some parameters (source.env and target.env) to the other builds. The other builds already have those parameters but currently are copied per trigger build.

I'm only aware of Snapshot dependencies and referencing variables via %dep.BuildId.paramName% - but those are sort of "the wrong away around". And since the reference is by BuildId that would not work since for ex. the "DB Deploy" build would reference two different snapshot builds having the same parameter as different %dep. parameter.

Also the Build+Script+Interaction+with+TeamCity seems only to work within build steps, not accross builds.

Is there a way to kick parameterized builds in TeamCity?
Thanks!

6 comments
Comment actions Permalink

Hello,

Do I correctly understand that you have one Deployment build configuration that performs different build steps depending on target environment? If yes, then I would recommend you to split this build configuration into two. You can use build configuration templates to simplify the setup.

The usual setup looks like this. For example, you have build configuration "Compile" and two build configurations "Deploy to Test" and "Deploy to Prod", both snapshot-depend on "Compile". If you see that the project on revision X was compiled successfully and want to deploy it, you can promote deployment of this revision to the selected environment.

0
Comment actions Permalink

Hallo

Thanks for the reply!

I didn't know about the promote feature. I'll look into that.

Meanwhile I came up with two ways of doing what I have in mind (not sure if it's the smartest way but well, try and try again):

Number 1:
I use the REST API from TeamCity to start a parameterized build (this is a bit geeky, but works)

Here is the build script that does this (in case someone has less concerns for geekyness):

logfile="curl.log" > $logfile teamcity_host=%teamcity.serverUrl% teamcity_build_queue_rest_url=$teamcity_host/app/rest/buildQueue build_ids="%build.ids%" echo "Build Ids: " | tee -a $logfile echo "$build_ids" | tee -a $logfile echo "" | tee -a $logfile src_env=%source.env% echo "source env: $src_env" | tee -a $logfile target_env=%target.env% echo "target env: $target_env" | tee -a $logfile echo "" | tee -a $logfile echo "Start sending requests..." | tee -a $logfile for build_id in $build_ids ; do     build_xml="<build>     <buildType id=\"$build_id\"/>     <comment>        <text>triggered from promote build: %system.teamcity.buildConfName%</text>     </comment>     <properties>         <property name=\"src.env\" value=\"$src_env\"/>         <property name=\"target.env\" value=\"$target_env\"/>     </properties> </build>"     echo "POST request XML:" | tee -a $logfile     echo "$build_xml" | tee -a $logfile     echo "" | tee -a $logfile     echo curl to $teamcity_build_queue_rest_url | tee -a $logfile     curl -k -S -s -u teamcityrest:teamcityrest $teamcity_build_queue_rest_url --request POST --header "Content-Type:application/xml" -d "$build_xml" >> "$logfile"     echo "" | tee -a $logfile     echo "" | tee -a $logfile done

  • teamcityrest:teamcityrest is the username password for REST calls.
  • build_ids is a parameter containing the build Id's to start

This works. But the drawback is there is no visible connection between those builds in TeamCity.

Number 2:
So after more digging I found the section on Overriding Dependencies Parameters

This allows to change parameters on a dependant build.

In short: i definded a parameter "reverse.dep.*.source.env" and use snapshot dependencies which themselves use the parameter "source.env" in their builds. So I can use the same build to repeat the deployment (or promotion) from environment to environment:

I'll attach some screenshots that may be simpler to understand.

You can even define "source.env" as parameter with a specification so you get prompted if you start that build manually (screenshot 4-).



This has the advantage that I can see what builds are part of an environment (the list of snapshot dependencies). And I can easly add another promote build.

I'm still not sure if using build templates might be smarter. I'll do some more experiments on that too.



Attachment(s):
4-prompted-params.png
3-trigger-build-params.png
2-build-experiments-deps.png
1-build-experiment-overview.png
0
Comment actions Permalink

Hi Werner,

It's not a good practice to change the logic of the build inside build script. In this case it will be difficult to interpret the results of the build and also the statistics of the builds (for example average build time) will be uninformative.
The described solutions look rather difficult for understanding and maintaining. Could you please describe why suggested approach with separate build configurations does not fit in your case?

0
Comment actions Permalink

Hallo

I understand. Therefore I tried to come up with solution number two - which of course does a similar thing: it changes parameters up the chain.

The main reason for this is simply put the masses of builds I would need to create.

Currently there are

  • Release Build (creates files: app.ear, sql.zip, app.tests)
  • deploy sql.zip
  • deploy app.ear to CI Server
  • Run app.tests


Then there are additional environments: Development, Test1, Test2, Loadtesting, Education, Emergency, Production (to name just a few)

So I wanted to create a "promote" build:

  • look at source server -> version
  • deploy sql.zip to target server
  • deploy app.ear to target server
  • run app.tests


If I create seperate build configs I would need to create 6 builds: dev to test1, dev to test2, test2 to loadtesting, ... etc (but all builds do exactly the same thing).
Since we currently maintain about 40 applications that would mean 240 builds (and there are no microservices yet).

If i'm able to just parameterize the promotion build (for example using the reverse.dep.* feature) I just create 40 promote builds and 6 builds that pass the parameters. (Well probably the build that will run app.tests will survive - for the build monitors - so the numbers are a bit wrong).

The goal I try to achieve is promote a set of applications from one stage to another, but the set is not the same. That set is managed in the trigger builds as snapshot dependencies (which read the reverse.dep.* parameter).
These trigger builds then run automatically. If these where seperate builds I run into trouble when to run app.tests (as those often require several running applicaitons depending on their test scope)

Since the promote builds have no logic other than "from source to target" I feel like its ok to pass parameters up the chain. Since no artifacts change, only deployed versions on servers.


Well. Thats my 2 cents. Any feedback welcome :)

0
Comment actions Permalink

Thank you for detailed explanation.
I think that your second solution (using reverse.dep.*  and prompted parameters) is the appropriate one in this case.
We have the related requests: https://youtrack.jetbrains.com/issue/TW-3661, https://youtrack.jetbrains.com/issue/TW-2608, please vote for them.

0
Comment actions Permalink

mkay! I'll try following that path :-)

Thanks a lot!

0

Please sign in to leave a comment.