Build chains - clarification how they work

Update: Builds B and C are being added to the queue (they are removed so quickly that it's hard to spot them when using a VPN), so the only question is - where is the dll that A is dependent on being pulled from?

A has a file reference to C:\Dev\Projects\ProjectB\bin\release\B.dll and the folder "C:\Dev\Projects\ProjectB" has been deleted - so the .dll is no longer on disk

Thanks

Richard

Hello Richard!

I assume that you have snapshot and artifact dependencies, both for A on B and A on C. Is that correct? 

Let me explain how the snapshot dependencies work first. Essentially, snapshot dependency is meant to guarantee that the dependency build has been started at least once with the current settings of dependent build. In your example, whenever build B (or C) is started, TeamCity will calculate a hash function taking VCS revision, build parameters and several other important variables as input. For this hash, TeamCity will then check if there already was a build with stored hash equal to the currently calculated - if there is one, then it means it was started under the same context and may be reused. In this case, newly-started build will be replaced with a link to the older build (so yes, it will always be added to the queue but if there is a replacement it will be removed from the queue shortly after).
This behavior may be changed in the snapshot dependency settings.

Now to the artifact dependencies. If snapshot dependency ensures that the build itself was executed (that is, it depends on the operation that build implements), artifact dependency is linked to the build results (artifacts). Artifacts are stored on the Data Directory of TeamCity (https://www.jetbrains.com/help/teamcity/teamcity-data-directory.html), specifically, on the following path:

<Data Dir>\system\artifacts\<project ID>\<build configuration ID>\<build ID>

When artifact dependency is resolved, TeamCity will first locate the build according to the rule set up on the dependency (last successful, last executed, etc.), then will access the artifacts listed on the path above. These files are then provided into the dependent build context.

I am not sure if I follow you on the file references part; artifact dependency will copy the files into the checkout folder of dependent build, so if you had an absolute path dependency on the C:\Dev\... paths, artifact dependency would have nothing to do with that. Could you please elaborate on the dependencies setup? (e.g. the exact settings of the dependencies, and where/how do you reference the .dll(s)?)

Fantastic answer Fedor.

I don't have any artefact dependencies set up - just snapshots from A on B and A on C.

In ProjectA's (c:\Dev\ProjectA) references, we have a file reference to c:\Dev\ProjectB\bin\Release\ProjectB.dll.

I deleted ProjectB's folder (c:\Dev\ProjectB) thinking that when ProjectA is built, the snapshot dependency would cause TeamCity to try to use C:\Dev\ProjectB, find it was missing and cause the checkout and build of ProjectB.

I can see that using artefact dependencies would work in this case - deleting the ProjectB folder - (the artefacts are held outside the build tree), so wouldn't matter if the original folders were deleted. But as I don't have any artefact dependency setup I don't see how it can be resolving the file reference to ProjectB.dll

This setup will change, but it's what we have at the moment

Thanks again for the in-depth answer above

Richard

Richard,

Is it correct that build B would put the output (.dll) into the C:/Dev/ProjectB? This way, I can see it working (essentially, you have own artifact logic which does not rely on the artifact dependencies this way). The downside, indeed, is that the build will not track the removal of its output folder (and/or corruption of data inside, if for instance someone would remove the .dll). 

I am also not sure why A did not fail while the .dll was physically missing from disk. Could you please upload the build log to https://uploads.jetbrains.com and share the resulting upload ID with me? (Maybe it will show how the reference was resolved)

Thanks again Fedor, ProjectB does not put the dll anywhere other than it's own "bin\Release", so ProjectA *should* have failed to build. Looking at the log, I can see that it's pulling from the ProjectA\bin\release folder - it had been built before - which explains why it worked.

Should I be using an artefact dependency as well as a snapshot dependency?

Thanks

Richard

Hello Richard,

Understood, thank you for the details! Yes, I believe artifact dependency usage would be helpful, as you could:
1) ensure the artifacts from the same build chain are used (there is a revision rule for that specifically);
2) trace back the artifacts used in build A back to the exact build of B;
3) no longer worry that someone (be it user actions or automatic cleanup) removes the .dll from B checkout folder. 

For any questions or concerns, please feel free to reach out and I will gladly assist.