Do not build subdirectory if it has not been changed

Answered

Permanently deleted user

Created July 11, 2016 10:22

I have a repo with two projects, say A and B. I also have a build configuration in TC that builds both A and B and packs them into artifacts.

My goal is to optimize the process and build project A only when files in A/ directory has changed, otherwise get built A from previous artifacts.

I think it could be done by splitting this build configuration on BuildA and BuildB with snapshot dependency between them. However, I need to redefine what "suitable build" means. In particular, I need to say TC: "if difference between this revision and previously built one (by BuildA) is in directory B/, go on, use previous BuildA as suitable".

I think it is impossible to do right now, but it seems to be very common scenario when you do not want building unchanged parts. Is there another way? May be there are some plugins?

16 comments

Alina Mishina

Created July 11, 2016 11:59

Hi Anton,

Yes, you can split your build configuration on BuildA and BuildB.

Can these build run in parallel or are they dependent (e.g. BuildB requires BuildA artifacts)?

If they can run in parallel, you can create the following chain: create additional build configuration e.g. "Package", that publishes both artifacts and artifact and snapshot-depends on BuildA and BuildB with option "Do not run new build if there is a suitable one" enabled. In this case, if you start Package build configuration and there is already suitable build A or B available, it won't rerun. Suitable build is the build that uses the same sources as the entire queued build chain being processed (for more details see the section and this one).

If BuildB depends on BuildA, you can create dependency between them. In this case, if you start BuildB and there is already BuildA on the same sources available, it won't rerun.

If you have any further questions, please feel free to ask!

Permanently deleted user

Created July 12, 2016 09:17

Thanks for your answer, Alina!

Yes, let's assume that A and B are independent. I can do as you said:

BuildA <---- Package
BuildB <----/

where arrow is snapshot dependency. Also let's assume that we have following commits in VCS:

rev2 Changes only in /B directory!
rev1 Changes in /A

where rev1 is parent of rev2. Also we know that BuildA already have successful build of rev1 Changes in /A.

When I trigger Package build configuration with revision rev2 it queues BuildA and BuildB both with revision rev2 (as snapshot dependency works). And Project A will honestly build revision rev2, because there is no suitable builds.

However, I don't want him to do it because it takes 1 hour to perform BuildA :) I would like Package build configuration to think a little bit and say "Wait, I already have built rev1 by BuildA. git diff between rev2 and rev1 shows that changes were only in directory B. It means I can use artifacts from BuildA with revision rev1."

Can I do something like that?

Alina Mishina

Created July 12, 2016 16:18

Hello Anton,

You can configure VCS checkout rules to checkout and monitor only changes from A directory in A build configuration and only changes from B directory in B build configuration. If there are no changes in the monitored part of the repository in build configuration A, build A won't be triggered. Will it work for you?

Permanently deleted user

Created July 13, 2016 14:04

Hi, Alina,

Unfortunately no, triggers are not suitable for our case. We let snapshot dependency decide what is needed to be built.

Our build chain looks like that right now (where arrow is snapshot dependency, we do not use triggers):

Build Application <---- Deploy <---- Integration tests <---- Acceptance tests
Build Tests <---------------------------/

In order to answer on question "Can we deploy particular revision to production?" we run "Acceptance tests" build configuration with this revision and it queues all build configurations it depends on if needed.

Our acceptance test developer often asks "Why do I need to wait for "Build Application" (that takes 1 hour) when I changed tests only?". Is there a way to help him and take previously built application artifacts?

Alina Mishina

Created July 19, 2016 14:26

Hi Anton,

>Unfortunately no, triggers are not suitable for our case. We let snapshot dependency decide what is needed to be built.

I suggested you to use VCS checkout rules. So that Build Application build configuration will checkout only Application sources and Acceptance tests only tests code. In this case if you start Acceptance tests build configuration, then Build Application won't start if there is no changes in Application sources.

Permanently deleted user

Created July 20, 2016 09:40

That's great, I didn't know about that, thank you!

Permanently deleted user

Created July 20, 2016 12:01

However, there is a small problem when I create a new branch from already built one. For example there is revision rev1 and branch master pointing on it. All build chain already have built this revision on this branch. But if I create new branch (pointing on the same revision rev1) called some_branch and trigger build chain for this branch, it rebuilds everything despite the fact that there is no changes.

Why does it work in such way? Can I change this behavior?

Alina Mishina

Created July 21, 2016 13:42

Please try to set teamcity.vcsTrigger.runBuildOnSameRevisionInEveryBranch=false either as a parameter in build configuration - to affect a particular build configuration, or in internal.properties to affect all build configurations.

For more details see the related ticket: https://youtrack.jetbrains.com/issue/TW-23744.

Permanently deleted user

Created July 22, 2016 11:50

Thanks for your answer!

Howerver it seems to be ignoring this flag. Here is the screenshot:

Builds Build App #15 and Build Tests #16 was triggered despite the fact it wasn't changes in them.

My settings look like that:

I also restarted TeamCity so settings be applied.

Alina Mishina

Created August 17, 2016 12:59

Is the issue still actual? Could you please try the following internal property: teamcity.vcsTrigger.analyzeFullHistoryForMergeCommits=true. It takes all merged commits into account when calculating if the merge commit is matched by checkout rules. A detailed explanation can be found here: https://youtrack.jetbrains.com/issue/TW-24747#comment=27-814689.

Anton Iakimov

Created October 19, 2018 14:36

Hi Alina and everyone!

Is this issue sorted?

I have the same situation:

There is:

A (VCS checkout rule - A/)
B (VCS checkout rule - B/)
And C, which has A and B as snapshot dependency and checkout the whole repo
When I run C - it triggers an A build on new branch even if there are no changes to its parent (default branch)

I added both

teamcity.vcsTrigger.runBuildOnSameRevisionInEveryBranch=false
teamcity.vcsTrigger.analyzeFullHistoryForMergeCommits=true

On both A and C, but it doesn't help.

Anton Iakimov

Created October 19, 2018 15:00

Looks like it works with VCS trigger, but not with a manual run.

Is it possible to have the same with a manual run?

Meaning this scenario:

There is a new branch with changes in B/ folder
Running C build manually should run a B build on this branch
But it should not run A build, because there are no changes in A/ folder. It should take the suitable one from the default branch. "Suitable" meaning the last commit, where A/ was touched.