How to fool the "Build queue optimizer"

Answered

I'm working on a culprit-finding plugin. If a build has failed, and it covered multiple changes, the intermediate changes are automatically queued for rebuild.

However, some of the queued builds are often removed from the queue by the "Build queue optimizer":

[2015-09-25 10:59:43,461]   INFO -   jetbrains.buildServer.SERVER - Build queue optimizer substituted 2 builds in the queue with builds containing more changes
[2015-09-25 10:59:43,461]   INFO - tbrains.buildServer.ACTIVITIES - Build removed from queue: 475217, comment: substituted by build with more changes
[2015-09-25 10:59:43,465]   INFO - tbrains.buildServer.ACTIVITIES - Build removed from queue: 475216, comment: substituted by build with more changes

Can this be prevented somehow?

I imagine the Build queue optimizer can be fooled by making the queued builds look different to him. How? Currently, they only differ in their change number (BuildCustomizer.setChangesUpTo()). Would it help if they differed in their build parameters for example?

Are the rules of the queue optimization described anywhere?

13 comments
Comment actions Permalink

Hello,

Build queue optimizer rules are described in this section in the online documentation.
You can disable queue merging by specifing configuration parameter teamcity.buildQueue.allowMerging=false (see the related comment).

0
Comment actions Permalink

Hi Alina,

That's very helpful, thank you. I'll play with the teamcity.buildQueue.allowMerging parameter. Setting this per build configuration may or may not be granular enough (we may need to set it per individual builds, so as to protect them from merging, but still use the default optimisation for the other builds). I'll see where I get with that.

If it turns out that I do need to set this per build, can you think of how to achieve that with the rules you've sent? Here are my comments/questions about the rules:

     TeamCity optimizes the build queue as follows:

    • if a similar build exists in the queue, a new build (on the same change set and with the same custom properties) will not be added

my builds are not of the same change set, so I believe this rule does not apply

    • if an automatically triggered build chain has more changes than a build chain that is already queued, the latter will be replaced with the automatically triggered build chain, if such replacement will not delay obtaining the build chain results (based on the estimated duration)
this is my case: multiple chains, triggered automatically (by a plugin, to be more precise), getting merged; what if each of the builds had slightly different custom properties (as in rule 1)? would they still be merged?
    • while a build chain is in the queue, TeamCity tries to replace the queued builds with equivalent started builds.

Thanks,
Sam
0
Comment actions Permalink

Hello Sam,

>What if each of the builds had slightly different custom properties (as in rule 1)? would they still be merged?

Builds with different custom properties are considered as different builds and should not be merged.

0
Comment actions Permalink

Thanks, that should be sufficient.

0
Comment actions Permalink

Hi, I'm having the same problem here, build is skipped if later build is finishing than existing build with scheduler trigger (TC vers 9.1.3)

Question to QA Platform,

1) What TC version are you using?

2) Are you using schedule trigger as well? If yes, how you overcome it? Could you share?

Question to Alina,

1) What is Custom properties?

2) Besides Custom properties, any way to make the build differently?

 

0
Comment actions Permalink

Hi Li,

 

1) We're using 9.1.3. But I don't think this is specific to this version.

2) No; the situation I described was when we were triggering builds using the OpenAPI (we were writing a TC plugin).

 

I wonder if you've hit this problem: https://youtrack.jetbrains.com/issue/TW-45168. The description is lengthy, and a little hard to follow, but it describes a real problem.

 

Regards,

Sam

0
Comment actions Permalink

Hi Sam,

Sorry for delay. 

  1. If you trigger a build manually (using Run... button) you can specify build properties on Parameters tab. Build with parameters values differ from default values is called build with custom properties/parameters.
  2. Builds are considered different if any parameters or build configuration settings changed.


Also please see the comment, it's recommended to upgrade to one of the latest TeamCity versions.

0
Comment actions Permalink

Hi Alina,

I think there's been a confusion. You've answered my question long ago (30 Sep 2015). It's just that Li Lee Lui then chimed in and I tried to help him out.

But Li Lee Lui had two questions for you; see his 26 July 2016 post.

 

0
Comment actions Permalink

Hi Alina,

Option 1 that you suggested are working, but this failed to run automatically 24hrs x 7 days.

For option 2, my parameter changes is per weekly basic. So, this is not suitable for my problem statement too.

Let me know if you have another way to overcome this.

Thanks.

0
Comment actions Permalink

Hi Alina,

If I have one build running and another sitting in the queue, when does TeamCity check if two builds have the same parameters before optimizing the queued build away?

0
Comment actions Permalink

Hello Oleg,

TeamCity optimizes builds only while they are in the queue. If the build is started, TeamCity does not take this build into account any more.

0
Comment actions Permalink

So does it optimize every now and then as long as there is one build in the queue? What events trigger optimization?

0
Comment actions Permalink

Alina may correct me but I believe this happens when a build is added to the queue. At that point, TC looks around and merges it with a similar queued build (if available).

BTW, this is logged into the server log so you may inspect it and see.

0

Please sign in to leave a comment.