We are at the position that we would like to move our UI based build pipeline configuration to Kotlin DSLs. But for some reasons I cannot really conclude which way of doing it would be best. I was hoping there are some more advanced real world examples that people can share with me or give some guideance based on your expertise.
We have three independent products which live in three different GitHub Enterprise repositories. For each product we have a hierachy like this:
- Build (build configuration; compiles the project for all pushes and pull requests and publishes artifacts)
- Nightly (build configuration; compiles the main branch and produces a setup.exe)
- Release (build configuration; same as nightly, just with some special flags on the actual build script)
- Tests (sub-project)
-- Linting (build configuration; does linting of all PRs before they can get merged)
-- Static Checks (build configuration; does a full recompile of the product and does some nightly checks of the main branch with SonarQube)
-- Unit Tests (build configuration; takes the artifacts from 'Build' and runs our continuous unit test suite)
-- Unit Tests Daily (build configuration; takes the artifacts from 'Nightly' and runs some special daily test suites)
For each of these build configurations we use a build template which is then parameterized on project level. In rare cases we have some additional build steps added to the individual configurations.
Solution in Kotlin?
I had two main approaches in mind where I cannot really find out what's the better solution:
a) We make the Kotlin DSLs part of the product git repositories.
This has the benefit that we can utilize the product branching strategy to introduce changes. If a new feature needs to adjust the pipeline, we can introduce it as part of this feature branch. Or if over time for releases the configurations change (new MS Build toolsets, new package managers etc.) the old correct scripts are still in the repo and used by TC if we need to make a hotfix on an old release.
The cons are that we cannot share easily common configurations across the products. Either we maintain copies of the common parts and need to be careful on changes. Or we introduce a common repository and consume some additional scripts as git submodules (not sure if this is even possible).
Another thing where I'm not sure how TC behaves is with the new Build Chain DSL. If different branches define different pipelines with different configurations, how would the UI look like in this case. Does it show a unification of all build configs from all branches and only a few are triggered based on the settings? Or does it only consider the main branch for the display, and performs other additional steps under-the-hood?
b) One common build repository for all products
We could create a new single repository for our product suite where we maintain our scripts. The whole hierarchy above can almost be generated from a parameterized template. This way we can easily share common scripts and steps among the products.
On the "con" side we loose the benefit of the product branching. If Version 1 (release/1.0) of Product 1 requires MSBuild 14 and Version 2 (develop) of Product 1 MSBuild 15 how can this be properly arranged? One solution could be that we do not use branches for this main repo but rather make conditions in the DSL where we set parameters/variables depending on some checks against the branches used.
Another con is that if you have a tough time synching new features that change the build pipeline. You have PR in the main repo, and a PR in the buildscript repo that need to be linked together somehow. Also TC will need to understand via some configs or script logic that certain branches require different behaviors. After the PRs are merged, you need to clean these things again.
I hope somebody will be able to share some insights on how you would use the Kotlin DSL and Build Chain DSL in such a setup.