Optimizing "Resolving Artifacts" when there are 2000 small files.
Hey,
I have a few buildconfigs, that as their artifacts have several folders with an enormous amount of small files.
Other buildconfigs have these as their dependencies. It turns out that there's a quite substantial (seems like a second, maybe two), overhead per
resolving of any file, nomatter how small.
I'm wondering if there are smart ways to optimize this. I could imagine:
- I make my buildconfig zip up all those artifacts, and only produce a zip
- Teamcity could be smarter, making a buildagent be able to ask for "all artifacts matching * please!", and it could then receive a
nicely zipped/tarred up package.
Since #1 is the only one I can actually make happen, I'll probably need to go down that route, but I was wondering
if people have better / less invasive solutions to speed this up.
Thanks, Lucas
Please sign in to leave a comment.
Right now I think first option is the only way to go. You can however simplify things a bit. You can publish zip file but configure artifact dependency to automatically extract files from the zip (see artifact path syntax: http://www.jetbrains.net/confluence/display/TCD5/5.Dependencies#5.Dependencies-Add%2FEditArtifactDependency). In this case the whole zip will be downloaded to the agent and then unpacked, this should work faster.
Also if you'll start publishing zip - the build publishing artifacts will run faster too.
I'm trying to optimize some artifacts I have as well and am looking to do this. Can you clarify something though? Is there a build in mechanism to have teamcity publish my artifacts in zips or do I need to zip them before passing them off to Teamcity?
No, the zip should be prepared by your build script. Please watch/vote for this issue: http://youtrack.jetbrains.net/issue/TW-3277