Composite build without revision synchronization for dependencies
Hi. I'm trying to setup a pipeline for running all kinds of tests against pull requests into my repository. Repository is git repo hosted with Bitbucket server and TeamCity is Enterprise 2019.1.5 (build 66605). There are a few key aspects to the task:
- There's a lot of tests. One way or the other tests should run in parallel to achieve reasonable execution time. The tests are already partitioned as separate TeamCity build configurations, each having good enough execution time.
- There's much less build agents available for the task, so it's not impossible for a particular build to spend quite some time (up to 1-2 hours) in a build queue.
- The result of the testing should be reported to Bitbucket as a single aggregate value. I.e. if there are, say, 3 individual builds from p.1 with two passing and one failing then Bitbucket should receive single "failed" build status.
- Ideally, pipeline should be triggered by a feature branch change (refs/pull-requests/123/from in Bitbucket lingo), but checkout merges of the feature branch into target branch (refs/pull-requests/123/merge in Bitbucket lingo).
Given above requirements, I experimented with Composite Build Configuration (https://www.jetbrains.com/help/teamcity/composite-build-configuration.html) as it seemed perfect fit for the job. So I set up single composite build with proper builds from p.1 as snapshot dependencies, "Pull Requests" and "Commit status publisher" build features. It works perfectly, except for one thing...
The only thing I cannot seem to be able to work around is the fact that VCS roots in the dependencies collect changes when the build chain is added to the build queue. This means that because of p.2 (non-negligible max time spent in build queue) some builds end up running against a little bit dated sources. Ideally, I would like to be able to run the builds against the latest sources.
So my question is if there's any way to disable revision synchronization for dependencies? Or maybe I could approach the whole problem in some completely different way without using snapshot dependencies?
Cross-posted on SO: https://stackoverflow.com/q/59166356/262403
Please sign in to leave a comment.
Hi Ilya,
Nice work getting your build chains set up! Yes, you can disable revision synchronization since version 2019.1.
This topic is covered in our documentation, https://www.jetbrains.com/help/teamcity/build-chain.html#BuildChain-DisablingRevisionsSynchronizationBetweenChainParts with a little bit more information here, https://www.jetbrains.com/help/teamcity/build-dependencies-setup.html#BuildDependenciesSetup-AdvancedSnapshotDependenciesSetup.
Hi Eric, thanks for the encouragement!
It looks like Disabling Revisions Synchronization Between Chain Parts feature covers slightly different use case than mine: "<the feature> is useful when you need to run a dependent build without synchronizing its code revision with its dependencies (preceding builds in a chain)." I probably should have explained my goal a bit better for it seems to be not too common one and, perhaps, violates some core assumptions of build chains' use cases.
To demonstrate my situation with simplified example, let's say there's B -> A build chain, i.e. when the whole chain is triggered by A, B runs first and then A runs. Part of build chain behavior that I'm trying to avoid is this one: "When the builds are added to the queue, TeamCity starts checking for changes in the entire build chain and synchronizes them - all builds have to start with the same sources snapshot." It's exactly the synchronization I'd like to avoid. Instead, I want B to collect changes at the time it gets out of the queue and starts running, not when it gets queued an hour or two before actual execution. This would imply that given (B1,B2) -> A chain the builds B1 and B2 could end up executing against different source versions in the general case -- I'd be fine with that.
After carefully studying the documentation and running some experiments, I'm feeling "all builds have to start with the same sources snapshot" is probably very core feature of build chains. But is there any way around it?
One way I can think of working around this is checking out sources "manually" in the build scripts instead of relying on a VCS root. This wouldn't be a deal breaker, of course, though it would be less optimal: VCS roots provide important goodness like reporting changes captured by the build, showing revisions of all roots used by the chain in single place, agent-side caching of git repos, etc. So I was hoping there might be some other existing feature that I'm missing or some misunderstanding on my part.
It's probably also useful to clarify why I'm looking to use build chains if I don't need the builds to necessarily use the same sources. Specifically, the following features of composite builds make them an otherwise perfect tool for my problem:
A composite build does not occupy an agent.
Please let me know if I'm understanding your objective correctly. You are hoping to trigger a group of independent builds from a single feature branch change, and once all of the builds are completed, you wish to report pass/fail results on all of these independent builds in a single place?
That's the gist of it, yes. To avoid potential ambiguity in wording, I want to report single aggregated pass/fail result. This single result should be reported to a single place, yes. By "reporting" I mean rather specific thing: https://developer.atlassian.com/server/bitbucket/how-tos/updating-build-status-for-commits/. I achieve this with Commit Status Publisher plugin on the composite build now.
I could take care of the aggregation "by hand" using TeamCity REST API as it exposes all the necessary building blocks. But I'd rather not to, at least not until I know for sure there's no easier way. Because then I'd have to also take care of UI aspect that's now completely covered by Composite Build: especially, providing convenient ability to drill down to individual builds, re-run only failed ones, see a list of failed tests in a single place, etc. I.e. there are a lot of important things Composite Build takes care of for me, they are not exactly trivial to re-implement.
I think I understand your concept, but I don't think it is currently possible using snapshot dependencies; just as you have found in your experiments. The Build Chains were not really designed with this use case in mind. At this time, I think the only way to achieve this would be using the Rest API to trigger the group of builds into the queue and then aggregate by hand like you mentioned.
Thanks, Eric. My primary concern was to avoid wasting effort building something that's already offered by TeamCity out of the box. I knew from the beginning my use case is likely not a mainstream one, so I'm not too disappointed. Will keep an eye on TeamCity development.