In our CI environment, some build agents are connected to test equipment which from time to time may fail or crash. When this happens, all builds running on that agent which attempt to make use of the test equipment will then fail until the equipment problem is corrected. What I'm looking for is a neat way to automate running checks that agents will run in between all normal builds, and if the checks fail, the agent will be automatically disabled - or at least it would not accept any new builds through some other mechanism until the connected equipment is alive again. Some kind of pre-build check that runs agent-side and doesn't produce a build-failure if the equipment/environment is not ready to be used yet.
Does anyone else have a similar need? Does such a thing exist for TeamCity? Or can anyone think of a way to set it up with minimal obscurity?