Good Real World examples of Kotlin DSL

Answered

Hi everyone. 

We are at the position that we would like to move our UI based build pipeline configuration to Kotlin DSLs. But for some reasons I cannot really conclude which way of doing it would be best. I was hoping there are some more advanced real world examples that people can share with me or give some guideance based on your expertise. 

Current Situation

We have three independent products which live in three different GitHub Enterprise repositories. For each product we have a hierachy like this: 

Product (Project)
- Build (build configuration; compiles the project for all pushes and pull requests and publishes artifacts)
- Nightly (build configuration; compiles the main branch and produces a setup.exe)
- Release (build configuration; same as nightly, just with some special flags on the actual build script) 
- Tests (sub-project)
-- Linting (build configuration; does linting of all PRs before they can get merged)
-- Static Checks (build configuration; does a full recompile of the product and does some nightly checks of the main branch with SonarQube)
-- Unit Tests (build configuration; takes the artifacts from 'Build' and runs our continuous unit test suite)
-- Unit Tests Daily (build configuration; takes the artifacts from 'Nightly' and runs some special daily test suites) 

For each of these build configurations we use a build template which is then parameterized on project level. In rare cases we have some additional build steps added to the individual configurations. 

Solution in Kotlin?

I had two main approaches in mind where I cannot really find out what's the better solution:

a) We make the Kotlin DSLs part of the product git repositories.

This has the benefit that we can utilize the product branching strategy to introduce changes. If a new feature needs to adjust the pipeline, we can introduce it as part of this feature branch. Or if over time for releases the configurations change (new MS Build toolsets, new package managers etc.) the old correct scripts are still in the repo and used by TC if we need to make a hotfix on an old release. 

The cons are that we cannot share easily common configurations across the products. Either we maintain copies of the common parts and need to be careful on changes. Or we introduce a common repository and consume some additional scripts as git submodules (not sure if this is even possible). 

Another thing where I'm not sure how TC behaves is with the new Build Chain DSL. If different branches define different pipelines with different configurations, how would the UI look like in this case. Does it show a unification of all build configs from all branches and only a few are triggered based on the settings? Or does it only consider the main branch for the display, and performs other additional steps under-the-hood?

b) One common build repository for all products

We could create a new single repository for our product suite where we maintain our scripts. The whole hierarchy above can almost be generated from a parameterized template. This way we can easily share common scripts and steps among the products.

On the "con" side we loose the benefit of the product branching. If Version 1 (release/1.0) of Product 1 requires MSBuild 14 and Version 2 (develop) of Product 1 MSBuild 15 how can this be properly arranged? One solution could be that we do not use branches for this main repo but rather make conditions in the DSL where we set parameters/variables depending on some checks against the branches used.

Another con is that if you have a tough time synching new features that change the build pipeline. You have PR in the main repo, and a PR in the buildscript repo that need to be linked together somehow. Also TC will need to understand via some configs or script logic that certain branches require different behaviors. After the PRs are merged, you need to clean these things again.

 

I hope somebody will be able to share some insights on how you would use the Kotlin DSL and Build Chain DSL in such a setup. 

1 comment
Comment actions Permalink

Hi, we are currently using (a) + introducing common features using a shared maven dependency (in pom.xml), TeamCity downloads this dependency from provided location during configuration recompilation. We successfully tested it, but it's not fully in production yet.

We had some issue with versioning since pom.xml seems to not support "latest" keyword or such, and to increase version in all configurations on every change would be an insane hassle. When we build our library in TC, as a second step we run a script that looks up through APIs where it is used and forces a recompilation of settings of those TC configurations. Hmm, I don't think we thought of submodules...

a) TeamCity in UI will display settings based on master, and if UI editing is enabled it will only modify master.

iirc I created a proposal to have it a selectbox to switch between branches, or at least display info that it relates to master, not sure if or when it would be implemented

b) I'm not sure but wouldn't branches still work? I'd assume TC should be able to use branch matching with the one being built. But considering there will be some shared part there might be some practical issues? Not sure right now

"The whole hierarchy above can almost be generated from a parameterized template. This way we can easily share common scripts and steps among the products." this should work in (a) too, it's just a question of how the templates/common script suite would be delivered (from the same repo x submodule? x dependency...), unless I'm missing something

 

0

Please sign in to leave a comment.