I have implemented automatic tear-down and recreation of our build agents in virtual machines to ensure they remain clean and fresh. The agents are regenerated each weekend. The agents are split over three pools as follows:
- Pool A: 1 agent
- Pool B: 1 agent (never regenerated)
- Default Pool: Remaining 10 agents
In a small subset of the rare occasions that the regeneration has fails over the weekend, the agent-pool associations will get messed up. For example, this week the Pool A agent ended up in the Default Pool, and one of the agents from the Default Pool ended up in Pool A. My current best guess is that TeamCity associates agents with pools via IP address, and that because the agents were missing for a couple of days they lost their DHCP leases; as a result, when the agents were regenerated, an agent in the Default Pool was leased the IP address that was previously allocated to the Pool A agent and so became associated with the CES pool of agents.
Is my guess correct, or is there some alternative mechanism that is causing this behaviour?
Many thanks in advance