Build reusage does not always reuse builds even if a build has "No changes"
We have a monorepo with some microservices in it, we have problems getting Teamcity to reuse builds even though it reports that the build has "no changes".
Each service has it's own build configuration with appropriate checkout rules, the UI correctly indicate changes in accordance with the checkout rules.
In some cases it reuses build correctly, but more often it will rebuild with the "No Changes" message.
A few notes
- We use the Kotlin DSL checked into the same repository
- builds are trigged by the VCS trigger
- These are not personal builds
- We use git
- we have "teamcity.vcsTrigger.analyzeFullHistoryForMergeCommits = false"
Please advice with what could be the issue :)
Below are examples of it working and not working, we cannot tell why it sometimes work and why it sometimes does not work...
Here it mostly works although there is one build on the PR branch that still has the "No changes" message.
Our trunk is called development, the other branch is a short lived PR branch.
Below is an example of when it fails for no apparent reason
Please sign in to leave a comment.
It seems the Teamcity 22.04.4 has resoled this issue, we no longer see all these "No Changes" builds
Which TC version did you use before the upgrade? I can't recall any related bug fixes in 2022.04.4. The full list of changes is available here: https://www.jetbrains.com/help/teamcity/teamcity-2022-04-4-release-notes.html
TC can only re-use builds that it finds suitable: https://www.jetbrains.com/help/teamcity/snapshot-dependencies.html#Suitable+Builds. Whenever you see a build with no changes in your build chain, please check against this list.
Hi Mikhil
We upgraded from vesrion 2021.1.4 so it was a major version and some more...
I haven't been able to pinpoint it in the change logs either.
We still see some "No Changes" builds but it is difficult to find the reason why the previous build was not suitable.
We have seen it happen when artifacts are missing from the latest build on default branch (although the log states that it should have been created)
Sometimes a build overtake an other build, so the dependency was built at a time where the default branch had not built the dependency, but when the main build started it had and showed no changes, even though we have the following configuration
I really whished that there was a clear explanation why a build was not reused when it has "No changes"
> We still see some "No Changes" builds but it is difficult to find the reason why the previous build was not suitable.
The best approach is to use the 'Compare Builds' functionality. Compare the build that was expected to be reused, and the one TC rebuilt. And make sure that the one that was supposed to be reused meets all the criteria mentioned here: https://www.jetbrains.com/help/teamcity/snapshot-dependencies.html#Suitable+Builds
> We have seen it happen when artifacts are missing from the latest build on default branch (although the log states that it should have been created)
Artifacts could be missing because of cleanup rules, make sure this is not the case: https://www.jetbrains.com/help/teamcity/teamcity-data-clean-up.html
We do use that function which I think is great but it does not always work.
There are still cases where you see "No Changes" and no difference in "Compare Builds", where you have to dig deeper to find out the reason, as the case I mentioned above.
Again I which there was a way that always worked for identifying what made this build not reuse while other builds with the same configuration did, fx as you mention that a cleanup job has removed a artifact of some build down the pipeline.
So far I have been able to track things down but it is time consuming when app developers ask why their build triggered more than they expected and there by took way more time.
You right, there's room for improvement in this area. Alas, this is not easy to implement. There are dozens of candidates for reuse in each build configuration. Storing all this information in-memory could be troublesome, and the bigger the server, the more severe this problem may become. It's also not easy to understand how and what we should show to help users troubleshoot such cases.
Still, I created this feature request: https://youtrack.jetbrains.com/issue/TW-78768/Provide-a-way-to-inform-users-about-why-a-particular-build-could-not-be-reused. Feel free to vote and leave comments. Your feedback can help us understand how to implement this feature.