Save and restore data on agent before & after builds
We just started using TeamCity after a while of not using it.
Our project is built using the Unity (game engine).
During the build process, a certain "data" folder is created which is a very slow operation.
I'd like to cache this folder so that it will exist on the target build agent before every build. In a single agent environment, this is not an issue, although a clean checkout sometimes occurs (not sure why) and so i'd like to have a solution for this.
The system will work in the following way:
- When a build is triggered, copy the data folder to a particular location on the agent
- Build the project
- Store the new data folder somewhere for next build.
What is the best, reliable (and fast) way of achieving this ?
Please sign in to leave a comment.
The easiest way would be for you to modify the build configuration and wrap your actual "build steps" with an starting and finishing command line scripts that perform exactly those tasks. So if you currrently have, say, 3 build steps in your build configuration, you simply add one at the start and one at the beginning, so you have 5, and the internal 3 have the information they need.
I'm not too familiar with Unity's build process, so maybe understanding how this "data" folder is created might shed some light into more accurate help. Is it always created? Is it only created if it doesn't exist? Does it update when the source code updates? (For the last one, I guess yes, and only the changes from the affected source code need to be created, but please correct me if my assumption is wrong)
If the data folder contains static binaries, or static resources, a common scenario is to have the approach i mentioned but simply mounting network shares in those scripts. Another option is to keep them in a separate VCS Root that just keeps the static resources.
Unity generates a Library folder when opening the project. The way it works is by "importing" all data files into a custom format. so, the process depends on the size of the project (and our project is fairly large).
I would like to cache this Library folder somewhere. I was thinking of using TeamCity's built-in "artifacts" mechanism and store that folder (zipped) as a build artifact. However, since this file is very large, zipping it and copying it from Build agent --> Server may take a long time.
Ideally, i would like to cache it somewhere on the build agent. is there any local artifacts storage location on the agent, or can i simply use any folder that i like ?
We do have an artifacts cache, but it's local, for published artifacts and it's not as reliable as you would seem to require, as the cache might be cleared at some point. The best option would be to simply use any folder you consider appropriate outside the work folder that you can later reference
We have the same case (building Unity projects in TeamCity). We have solved this issue by setting up a Unity cache server on a machine that is accessible from all the TeamCity build agents, and configuring all Unity build to use this.
Hi Eirik, glad to see another Unity developer on these forums :)
I have developed a plugin (on the asset store) geared towards developers / studios working with CI and building from the command line.
It would be awesome if u could have a few minutes of your time to help me out (by giving me some feedback on the actual usefulness of this tool).
If you are interested, drop me an email to liortal53@gmail.com so i can explain it all. Thanks !!