Mono repo build with dependencies between folders
We have a git repository containing folders for different npm packages (monorepo).
The folder structure is:
/packages/packageA
/packages/packageB
packageB depends on packageA
We have one build configuration per package. Each build config uses the same VCS root, AND (to avoid unnecessary builds) each has a VCS trigger rule which excludes all folders except their own (so the trigger rules for packageA are: -:. and +:/packages/packageA, to avoid triggering when pacakgeB is changed)
So far so good.
As I mentioned before, there are dependencies between the packages BUT, these dependencies are not through source, but through an NPM registry, to which each build configuration publishes. So each build configuration does a build, then publishes the built package to an NPM registry.
So packageB depends on packageA through its usage of packageA from the NPM registry.
The problem: if there is a commit with changes to BOTH the source of packageA AND packageB (so there are modified files in both packages/packagesA and packages/packagesB), then both configurations are triggered simultaneously.
This leads to the build of packageB to fail, since it needs the updated version of packageA from the NPM registry, which is only ready after the build config of packageA has run to an end, and the new packageA has been published to the NPM registry
(Note: in reality our system has more than 2 packages, but our problem is easiest explained in this stripped down scenario)
What is the correct way to handle this scenario?
Please sign in to leave a comment.
Hello Support Team,
Can this request be managed under my account? or let us know how to do that?
Zoltan is a Cognex employee, and this request and questions are in relation with our Enterprise subscriptions.
Thanks
-Ken
Hi guys,
thanks for the heads up. While filtering at the trigger level works for only triggering under certain conditions, if all your projects are independent from each other, can be built without reusing parts of the code from the others, we would start by recommending checkout rules instead of filters. By completely removing projects from the build configurations, TeamCity can safely track which code is used by each build configuration, even if they share the same VCS Root. More info on here: https://www.jetbrains.com/help/teamcity/vcs-checkout-rules.html
Once you have this set up, it seems clear that B has a dependency on A in the example you posted. Even if the artifacts do not come through TeamCity, and they are pulled from npm, there is a dependency in that you need A to run before B. For this situation, we would recommend setting B to have a snapshot dependency on A.
With the combination of the snapshot dependency and checkout rules, you would have the following scenario:
-When a commit just for A arrives, the trigger on A will trigger it, build, do everything, publish the artifact
-When a commit just for B arrives, the trigger on B will trigger it. The snapshot dependency will trigger also a build from A, but since checkout rules exclude the commit from being relevant for A, as long as your dependency is configured to reuse builds when a suitable one is available, no new build will be produced and the last one will be reused, being able to simply run B.
-When a commit arrives that includes changes from A and B, or several commits arrive in very short order, both A and B will be published to the build queue. TeamCity should understand that the A triggered by A and the A triggered by the dependency of B are equivalent and only one instance of A will run, along an instance of B, that will wait until A is ran. If any change in A should include a build of B to use the updated library, then you could simply remove the trigger from A and the trigger on B would do the work.
Would that work for you?
Thanks for the tip, it works,
We have only one issue. Build A commits back into git some files with version numbers at the end of the build.
Because of this if build B is started it always starts build A. There are always pending changes after a build A.
Is there a solution for that?
(We can not add those files to checkout rules.)
Hi, and sorry for the delay, this are being a few hectic days.
For any given set of builds tied by dependencies, TeamCity determines a given commit to use throughout the process at the very start of the first build (you can find this in the build logs, it's done during the "Collecting changes" step right at the start, before the builds are assigned to agents. Creating commits mid build does not impact builds further down the line in the chain, but means that prior builds cannot be reused since there will always be pending changes, which is what I understand you mention as a problem in your scenario
What does the commit? It is often a good idea to separate the committing part of the build from the one doing the actual work, as it would make this easier. Understanding what the commit does and when it's best to do it would help us in suggesting a solution around it.
After a successfull build the build commits the package.json and the packagelock.json with the new package version. So somitimes the only change is a version number.
Sorry for the delay, I had to put this in the backlog due to load issues. To be clear, there is no straight solution to this. There are clearly two concepts at conflict here: You want to trigger when there are changes in the repo, but then commit changes and prevent from triggering on those commits.
With the recent introduction of conditional steps, the easiest possibility to achieve this would be to have a first build step that checks the last commit, and if it's the one you recently performed, sets some flag, then the rest of the steps aren't run. This would also clear the build for all future dependencies since it would consider that has already run with those changes. It would have the problem that it would fill the build results page and report successful builds from builds that were never run. It would make sense to at least flag them via a status message as such.
Using the same logic (first build step that checks for the last commit), you could simply stop/cancel the build, and set the dependency to run even if the dependency fails. This would in turn create some amount of failed/cancelled builds.
There is the possibility to have a master build, which would be one that picks up the vcs changes, then it triggers builds via REST as required. This would stop the need for snapshot dependencies, since it could use logic to trigger builds as required, to make some wait for others, etc, but it would require a complex script.
There is also the ability to set builds to use "shared resources": https://www.jetbrains.com/help/teamcity/shared-resources.html . These can prevent two builds from running at the same time, so you could put a shared resource for both A and B (following the example above), which would make one wait for the other. This doesn't guarantee that A would run before B, though, since the ordering in the build queue could be the other way, and the way of ordering, as discussed, is snapshot dependencies, which in turn would still lead to the same problem faced currently. You could use build queue priorities to try to manage this, but it still doesn't guarantee that A would run before B, just make it more likely: https://www.jetbrains.com/help/teamcity/ordering-build-queue.html#OrderingBuildQueue-ManagingBuildPriorities
Again, as I stated at the beginning, there is no easy "solution" for your scenario. I'll try to get some further feedback from the dev team to see if I missed something, we'll hopefully be able to provide some feedback earlier this time.
Another option is to use the checkout rules to ignore the commit from the buildserver. Assuming that the buildserver commits the new version number and is using a unique Git account, the checkout rule can be adjusted to ignore the commit of the teamcity git user. This will prevent the triggering of a new build. However it does not work perfect, because teamcity handles the checkout rules inconsistent: The checkout rule will prevent the immediate triggering of the build but it will flag the build as having Pending Changes, as a consequent the start of any buildchain that has a snapshot depency to this build will trigger an unnecessary rebuild.