Optimizing "Resolving Artifacts" when there are 2000 small files.


I have a few buildconfigs, that as their artifacts have several folders with an enormous amount of small files.
Other buildconfigs have these as their dependencies.  It turns out that there's a quite substantial (seems like a second, maybe two), overhead per
resolving of any file, nomatter how small.

I'm wondering if there are smart ways to optimize this. I could imagine:

- I make my buildconfig zip up all those artifacts, and only produce a zip
- Teamcity could be smarter,  making a buildagent be able to ask for "all artifacts matching * please!",  and it could then receive a
  nicely zipped/tarred up package.

Since #1 is the only one I can actually make happen, I'll probably need to go down that route, but I was wondering
if people have better / less invasive solutions to speed this up.

Thanks, Lucas

Comment actions Permalink

Right now I think first option is the only way to go. You can however simplify things a bit. You can publish zip file but configure artifact dependency to automatically extract files from the zip (see artifact path syntax: http://www.jetbrains.net/confluence/display/TCD5/5.Dependencies#5.Dependencies-Add%2FEditArtifactDependency). In this case the whole zip will be downloaded to the agent and then unpacked, this should work faster.

Also if you'll start publishing zip - the build publishing artifacts will run faster too.

Comment actions Permalink

I'm trying to optimize some artifacts I have as well and am looking to do this.  Can you clarify something though?  Is there a build in mechanism to have teamcity publish my artifacts in zips or do I need to zip them before passing them off to Teamcity?

Comment actions Permalink

No, the zip should be prepared by your build script. Please watch/vote for this issue: http://youtrack.jetbrains.net/issue/TW-3277


Please sign in to leave a comment.