Hosting Build Agent Data Directory on Network Filesystem

Currently in the process of deploying a build farm infrastructure to host TeamCity Server and build agents remotely but would like to get some recommendations/guidance on the use of network shares.  Our initial thinking is to have TeamCity Server software run on local disk while TeamCity data directory be hosted on a NetApp vfiler.  This particular vfiler will have at least 1TB of available space with room for growth and ultimately being hosted on a Filer almost exclusive for our use. Our user base will be approximately 300+ developers using the web UI and running builds at any given time of the day being that we have developers in multi-city locations.

With that said, what kind of issues would we potentially encounter with that many users hitting the web UI and artifacts being saved on network drive?

Any other factors we need to take into consideration?

Thank you for the help,
Oscar

2 comments
Comment actions Permalink

We do exactly this, and it works well. The only problem we have is the long maintenance period, which can be over 24 hours with a sizeable set up. We have moved the maintenance to the weekends, but it's still too long and we're not sure how to reduce the time.

0
Comment actions Permalink

To better understand what we're trying to do, here is more context on how we are planning on using a build server:

-----------------------------------------------
build server specifications:
VM: 8 vCPU's, 48GB or RAM, 400GB HDD
VMware ESX server: PowerEdge R610 E5649, 8 core Intel(R) Xeon(R) CPU E5649 @ 2.53GHz

Database:
PostgreSQL (remote server)

number of build projects:
~100

number of build configurations:
~600

number of the builds running daily:
On any given day, we can have at least half of those builds run either through remote runs or triggered for a specific time

amount of data generated by the builds (size of the build log, number and output size of unit tests, number of inspections and duplicates hits etc.):
based on our usage and growth of to that amount of builds we are looking at hosting about 500GB with potential to grow even more
we are planning to use total storage space of 1TB on NetApp vfiler (qtree)

cleanup rules configured:
90% of our build projects have the following settings:
clean everything older than the 5th successful build
clean history more than 30 days older than the last build and older than the 10th successful build
clean artifacts older than the 2nd + successful build
prevent dependency artifacts cleanup

7%  of our build projects
clean history more than 30 days older than the last build
clean artifacts older than the 14th + successful build
prevent dependency artifacts cleanup

3% of remaining build projects
clean history older than the 100th successful build
clean artifacts older than 20th + successful build
prevent dependency artifacts cleanup


number of agents and their utilization percentage:
60 (none running on build server)

number of users having TeamCity web pages open:
~200

number of users logged in from IDE plugin:
~400

number and type of VCS roots as well as checking for changes interval for the VCS roots:
~300
Checking interval is using global server settings of 800 seconds

total size of the sources checked out by TeamCity daily:
by default, all build configurations are set to do check out at build agent side


---------------------------------------------------------

I'd like to get any recommendations or advice on what we think our environment is going to be based on our current usage and the consolidation of other TeamCity instances to one central build server.

Thank you again for the help,
Oscar

0

Please sign in to leave a comment.