overriding Agent selection algorithm

I would like to be able to determine the order some agents are used in the scheduling process.

I have 3 agents that are connected to physical test equipment.

Agent 1 & 3 are connected to equipment A
Agent 2 are connected to equipment A & B

Scenario:
I schedule 2 configurations , one to use equipment A and the second one to use equipment B
When they are scheduled it can happen that Agent 2 is grabbed first to do testing using A thereby starving B since it has to wait for B.
The 2 other Agents are sitting idle that could have tested using A.

Is there any way to do this ?

11 comments

Morne,

Sorry for the delay in replying.

So far there is no way to establish such "smart" assignment. We have a feature filed to address settings like you describe - please consider voting for it (so far it's not very popular).

Currently used algorythm is to calcualte build setimate (based on the builds history) and assign the build to hte fasts idle agent.

Current workaround is to use existing configuraiotn means (like limiting agent to run only specific build configuraitons).

0

Yegor,

Do you or someone on your team have an answer to my question? What I'd like to see is an implementatino of https://youtrack.jetbrains.com/issue/TW-37860, but in its absense, perhaps I could write a plugin to suit my needs.

As an aside, would it be possible to publish the logic of the current algorithm to TeamCity docs? I had a few users ask me how TeamCity selects the agent, and I could only answer in generalities.

Thanks,
Oleg.

0

Hi Oleg,

The algorithm of agents selection is the following:

1. If there were no builds on agents then CPU rank is used to select the agent.

2. If there were previous builds on agents then the estimate time for this build configuration is calculated. Estimated build duration is based on the average run time of the latest builds (small weights are assigned for older build configurations, large - for latest) in the history of a build configuration.

You can use StartingBuildAgentsFilter class to choose a build agent for a queued build.

0

Hi Alina,

Thank you for the details! I will try that extension point.

When you say "latest builds", how many builds is that? Does that exclude personal/remote runs and cancelled builds?

0

The latest 25 builds are used. Personal builds and canceled builds are not taken into account, as well as individual builds which duration differs significantly from other builds.

0

Hi Alina,

I implemented a plugin by overriding filterAgents method in StartingBuildAgentsFilter. It works as far as I can tell by applying my filter logic and selecting the expected agent. It appears that filterAgents is called continuously or frequently while the build is sitting in the queue. The result is that the user only sees the agents what my filter allows in "Can run on" column. I think this is fine and by design.

Is there an extension point that I can add a filter right as the build pops off the queue? Let's say TC built-in algorithm determines that 5 agents are compatible, the build pops off the queue or is about to and then my filter kicks in to further filter the agents down to one. Essentially, I'm looking at build history for a given build and selecting the last agent the build ran on, but I only want to do that if it's not running a build now.

0

Hi Oleg,

Yes, the filter is invoked continuously and it should work fast to avoid delays. As to calling your filter only when build is about to start, let's see how it works actually.

Say we have some idle agents which can run builds. TeamCity takes these agents, and starts processing queue from the beginning. For each queued build TeamCity first determines which agents are compatible with it. After that TeamCity calls StartBuildPreconditions to determine if there are any additional conditions which can prevent build from starting. Then if there are no such conditions filters are called one by one. Result of the previous filter is passed to the next one. Then TeamCity takes the list of agents left after filtration and simply takes the first agent from the list, assigns queued build to the agent and then removes agent from the list of available agents. After that TeamCity performs operation with subsequent queued build until all idle agents become assigned.

So, if you really want TeamCity to prefer one agent over another, make sure your filter returns agents in correct order - more preferrable agent first.

0

Hi Pavel,

Thank you for shedding light on agent selection. This really helps, and I will push my preferred agent to be the first in the list.

Could you confirm that the algo that Alina described takes place before filterAgents is called in my plugin? I need my filter to be the last in the chain.

I could not find a way to query a given build config for the last build that has an actual agent assigned (some builds may have "N/A" for agent). I ended up using SBuildType.getHistoryFull and walking back in history until I find an agent. I realize that querying the entire history is inefficient, especially if we have many builds and poor cleanup policy. Anything you could recommend here?

Could I also bug you to answer my question? Thank you!

0

To make sure your filter is a last one in the list it should implement jetbrains.buildServer.util.positioning.PositionAware interface and return PositionConstraint.last() in getConstraint() method.

As to retrieving latest build on agent, you can try using BuildHistory.processEntries method. It can be a bit more effective, especially if you return false in ItemProcessor once you found necessary build. Although, with big amount of builds in queue this way still can be quite slow.

0

Pavel,

Thank you for your suggestions. I used them all and created the plugin that works as we need.

0

Please sign in to leave a comment.