Teamcity artifacts
I am currently looking at teamcity build pipelines as a replacement for build steps, primarily to support concurrent execution of a few build steps. Build steps and templates very copy/paste friendly and let me add additional environments easily. The build pipeline system seems like it will add a lot of complexity and room for error.
Basically my question is about how to handle maven artifacts. When everything builds in one job, we have no need to publish artifacts to a maven repository. This becomes more tricky with pipelines. I can't find it right now, but I saw somewhere in the teamcity documentation that said build artifacts were not a good fit for systems with existing dependency management (i.e. this implies maven). It uses bash script output as a good useage of artifacts. I did manage to get teamcity artifacts working by copying the maven /target directories, but I am worried that the system wasn't designed to handle the amount of artifacts maven generates. For example, our maven builds generate 1GB of artifacts, which then get copied between build agents. Will teamcity be able to handle this load or should we be using Nexus instead? I'd like to avoid publishing artifacts to an external repository because it doesn't bring us a lot of value for our team.
TLDR: What are your experiences with teamcity's ability to scale its artifact feature with number of files and size of artifacts?
Please sign in to leave a comment.
Hi Jamie,
A few comments:
-Build artifacts work fine with large artifacts (it might take more time depending on your infrastructure, but that's it). In some situations it might not be a good fit if you already have dependency management because using our artifact dependencies might hide other issues in the build scripts, such as artifacts not being available in repositories, or downloading artifacts that are already present due to the dependency mechanism, conflicts in versions, etc. Also if you are using places like maven, nexus, nuget, artifactory... you already have an artifact dependency system, and using teamcity for it as well would make it redundant.
-Using the artifact system works well to produce full packages within TeamCity, but if you intend to run the builds somewhere else (such as in local machines), you need to ensure that they can access the artifacts as well. This can be done, but in some situations might lead to have different build scripts for the TeamCity server and local machines. Using TeamCity as an artifact server works here, but dedicated servers usually offer advanced options as well.
-If you want to run concurrent steps/builds but want to have access to the same files/folders, first you need to keep in mind that OSs sometimes restrict multiple simultaneous accesses to the same file. If this isn't a concern for you, you can always set up multiple agents in the same machine, create (or copy/move) the artifacts to a fixed folder and use them in other builds from it.
-On the other hand, we have tested the system extensively and use it internally for big and small projects. It should work. Should you find any issues, of course feel free to share them with us.
It's a given that separating your builds from running in a single process to running on multiple processes across multiple machines is going to make things more complicated and error prone, although we usually have enough checks to ensure that things are properly set up, at least at the level we can control.
Hope this helps. If you have any further question or face any issue while setting it up, please don't hesitate to ask.