One of my platforms often fails to build with TeamCity, it comes up with an error message the mentioned a particular environment variable may not be setup properly (it points to a license manager for the given platform), however I have double checked this variable and it is setup correctly. I loaded up filemon/procexplorer to monitor what the failing TeamCity process does, and it does appear to launch the license manager program (that it claims it can't find) and accesses a bunch of files, but the build still fails. I can restart the team city build agent process and the problem persists, but a combination of restarting the license manager/restarting PC will fix it.
If I launch the build process manually it works. The build agent runs under a user account (not system account), and I'm using the same user account locally (when it works fine).
I've also tried running the failing build under the system account and even that works.
I've just written an application that shows me all the running processes on a system, and allows me to 'clone' the environment, this basically spawns a command shell with all the environment variables set from the selected process, I've tried using this on the failing TeamCity build process, I can then issue the build command and it works fine.
I have now run out of ideas as to how I can track this down - can anyone offer me some hints? Is there a way to get TeamCity to operate in some sort of interactive mode, where it'll spawn the build process in a command window and leave the window open after? Even with that I'm not sure where I'd go next to track this down. It doesn't appear to be connected to which user launches the process, it doesn't appear to be linked to environment variables that are currently set, so what else could cause it to fail in this way?
By the way, I'm sure the problem will be something to do with the specific compiler/license manager I'm using, and no fault of TeamCity, I just need some help getting to the bottom of it.