How can I speed up TeamCity resolving artifacts?

Hello there,

I have been using TeamCity 4.5, and have been having some issues with TeamCity transferring information to build agents, be it artifacts or source code.  First, with resolving artifacts, I have a build configuration that has close to 2 GB in 4,000 files which it publishes as artifacts and when I try to access these artifacts from another build configuration, it takes a very long time (over 3 hours) to download the artifacts into the checkout directory (and we are using a gigabit network with all participating build machines using gigabit NICs).  Is there anyway I can speed up this artifact download?

The source code transfer isn't too bad...I am guessing there are improvements I can make on my build LAN, but by comparison to the artifacts mentioned above, 2GB of source code from Perforce generally takes 10-15minutes transferring from the master to the build agent.

Any suggestions would welcome....thanks for any help in advance

5 comments
Comment actions Permalink

Is it possible to pack your artifacts in single archive file? TeamCity's artifact dependencies can extract files from the archive automatically (http://www.jetbrains.net/confluence/display/TCD4/5.Dependencies#5.Dependencies-addEditDependency). In this case archive will be downloaded to an agent and then unpacked.

As for source code transferring, usually full checkout is done if there are no sources on agents, after that TeamCity sends incremental patches. However if you've choosen option "clean all files before build" then full checkout will be performed every time.

0
Comment actions Permalink

Actually, yes...it should be no problem....let me give that a try...So reducing the number of files should speed it up?

0
Comment actions Permalink

I think so, because every single file is an HTTP request to server.

0
Comment actions Permalink

thanks for the tip...I am still in the process of modifying my scripts to convert the build-outputs to singular zip files, so I will let you know how that works out....

As for file transfers, do all transfers (sources, artifacts) under TeamCity occur over http?  Is there a feature to get TeamCity to transfer using other protocols between master and agent.  The reason I ask is because from my testing on my Gigabit LAN I am finding that ftp is faster than other protocols like smb and nfs and I was wondering if its possible to get TeamCity to transfer content using a different protocol to get better performance.  Or would this be better suited as a feature request?

Thanks again!

0
Comment actions Permalink

Thanks!  It Worked!  Archiving single files as artifacts makes a huge difference!

0

Please sign in to leave a comment.