RFE: Limit number of simultaneously running builds vs. hanging agents

Yann Cébron wrote:

set limit to 1, agent disconnects in the evening while build is still running, build will have warning sign "probably hanging" but next scheduled build will NOT run before manually cancelling hanging build due to of max running builds = 1
proposal: do not count in hanging builds for max running builds
comments?

The problem is that "hang detection" is not 100% reliable (because it is
done via something like "last 5 mins w/o any messages" heuristic), thus
it is completely possible that things can go wrong with come
configurations. Although in general this is doable.

--
Alexey Gopachenko
JetBrains Inc.
http://www.intellij.com
"Develop with pleasure!"

so would you consider it safe to skip counting builds hanging for longer than e.g. 1 hour continuously?

Yann Cébron wrote:

so would you consider it safe to skip counting builds hanging for longer than e.g. 1 hour continuously?

I suppose. Anyway, feel free to add feature request to our tracker.
We'll see how it fits.

--
Alexey Gopachenko
JetBrains Inc.
http://www.intellij.com
"Develop with pleasure!"

You can also disable hanging builds detection for the build configuration.

--
Pavel Sher

Yann,

Simple "not counting" for hanging builds does not seem a universal approach.

e.g. consider we have 4 agents and 2 projects, each has limit of simultaneous build set to 2.
Now a build gets a check-in that just hangs the build. With the suggested approach in a couple of build-times we will get all 4 agents running hanging builds and no other build can start.

Setting execution timeout for a build seems to be a recommended approach here.

Hello Yegor,

I agree and have execution timeout set on the build configuration, but it seems it is not applied to hanging builds.

Greetings,

Yann

Yann,

I agree and have execution timeout set on the build configuration, but it seems it is not applied to hanging builds.

I managed to reproduce the problem only when the agent disconnects abruptly. Please vote/watch: http://jetbrains.net/tracker/issue/TW-5771

Actually, there is one more issue that requests dropping the builds running on abruptly disconnected agents even if there is no execution timeout set. If you think this is the desired behavior, please vote.

Thanks, that is exactly my observation as well.