How to fool the "Build queue optimizer"
I'm working on a culprit-finding plugin. If a build has failed, and it covered multiple changes, the intermediate changes are automatically queued for rebuild.
However, some of the queued builds are often removed from the queue by the "Build queue optimizer":
[2015-09-25 10:59:43,461] INFO - jetbrains.buildServer.SERVER - Build queue optimizer substituted 2 builds in the queue with builds containing more changes
[2015-09-25 10:59:43,461] INFO - tbrains.buildServer.ACTIVITIES - Build removed from queue: 475217, comment: substituted by build with more changes
[2015-09-25 10:59:43,465] INFO - tbrains.buildServer.ACTIVITIES - Build removed from queue: 475216, comment: substituted by build with more changes
Can this be prevented somehow?
I imagine the Build queue optimizer can be fooled by making the queued builds look different to him. How? Currently, they only differ in their change number (BuildCustomizer.setChangesUpTo()). Would it help if they differed in their build parameters for example?
Are the rules of the queue optimization described anywhere?
Please sign in to leave a comment.
Hello,
Build queue optimizer rules are described in this section in the online documentation.
You can disable queue merging by specifing configuration parameter teamcity.buildQueue.allowMerging=false (see the related comment).
Hi Alina,
That's very helpful, thank you. I'll play with the teamcity.buildQueue.allowMerging parameter. Setting this per build configuration may or may not be granular enough (we may need to set it per individual builds, so as to protect them from merging, but still use the default optimisation for the other builds). I'll see where I get with that.
If it turns out that I do need to set this per build, can you think of how to achieve that with the rules you've sent? Here are my comments/questions about the rules:
TeamCity optimizes the build queue as follows:
my builds are not of the same change set, so I believe this rule does not apply
Thanks,
Sam
Hello Sam,
>What if each of the builds had slightly different custom properties (as in rule 1)? would they still be merged?
Builds with different custom properties are considered as different builds and should not be merged.
Thanks, that should be sufficient.
Hi, I'm having the same problem here, build is skipped if later build is finishing than existing build with scheduler trigger (TC vers 9.1.3)
Question to QA Platform,
1) What TC version are you using?
2) Are you using schedule trigger as well? If yes, how you overcome it? Could you share?
Question to Alina,
1) What is Custom properties?
2) Besides Custom properties, any way to make the build differently?
Hi Li,
1) We're using 9.1.3. But I don't think this is specific to this version.
2) No; the situation I described was when we were triggering builds using the OpenAPI (we were writing a TC plugin).
I wonder if you've hit this problem: https://youtrack.jetbrains.com/issue/TW-45168. The description is lengthy, and a little hard to follow, but it describes a real problem.
Regards,
Sam
Hi Sam,
Sorry for delay.
Also please see the comment, it's recommended to upgrade to one of the latest TeamCity versions.
Hi Alina,
I think there's been a confusion. You've answered my question long ago (30 Sep 2015). It's just that Li Lee Lui then chimed in and I tried to help him out.
But Li Lee Lui had two questions for you; see his 26 July 2016 post.
Hi Alina,
Option 1 that you suggested are working, but this failed to run automatically 24hrs x 7 days.
For option 2, my parameter changes is per weekly basic. So, this is not suitable for my problem statement too.
Let me know if you have another way to overcome this.
Thanks.
Hi Alina,
If I have one build running and another sitting in the queue, when does TeamCity check if two builds have the same parameters before optimizing the queued build away?
Hello Oleg,
TeamCity optimizes builds only while they are in the queue. If the build is started, TeamCity does not take this build into account any more.
So does it optimize every now and then as long as there is one build in the queue? What events trigger optimization?
Alina may correct me but I believe this happens when a build is added to the queue. At that point, TC looks around and merges it with a similar queued build (if available).
BTW, this is logged into the server log so you may inspect it and see.