Mutually exclusive builds and dependencies on sources

Hi JetBrains,

I have 2 build configs that should not be executed in parallel (a testing
one, relying on the DB, and a DB upgrade one). Would setting reciprocal source
dependencies help with this constraint?

Is the new source dependency meant to take care of all the use cases previously
discussed under the "reverse" dependencies and resource sharing topics?

Thx,
Andrei


6 comments
Comment actions Permalink

OK, upgraded to the latest 4.0 EAP for this, just to find it falls short
of solving my problem: I cannot set reciprocal source dependencies.

We discussed this scenario a few months ago and 4.0 was supposed to provide
a resource sharing mechanism (AFAIR this was the latest approach to be implemented)
that would help with this kind of setup. Is there something else I should
be looking at, apart from source dependencies?

To recap, my setup has two builds, A & B that should not be running at the
same time. I don't care much about versions of the sources but I do need
to have them executed sequentially. Basically, when B starts while A runs
(B is a DB update build), A will most likely fail, so I need B to wait until
A is done before starting. I want B to start with the latest sources, not
some old ones, so it applies the changes to the DB. After that, a dependent
build is triggered to run the tests again, against the newly updated DB.

How do I achieve that?

Thx,
Andrei


0
Comment actions Permalink

Resource locking and source dependencies are two different features. There will not be resource locking in TeamCity 4.0. Anyway you can try the following setup:
- make A and B compatible with one agent only
- in build configuration A add dependency by sources to B

Compatibility with one agent guarantees that there will not be A & B builds running simultaneously.
Dependency by sources A->B will allow you to avoid situations when your tests from build A runs with incompatible database schema.

--
Pavel Sher

0
Comment actions Permalink

Hello Pavel,

PS> Resource locking and source dependencies are two different features.
PS> There will not be resource locking in TeamCity 4.0.

That's quite sad... From the looks of it, the discussions went from a simple
use case to a very generic one that ended up being dropped altogether...

PS> Anyway you can
PS> try the following setup:
PS>
PS> - make A and B compatible with one agent only

That means losing the whole benefit of multiple agents. Our build A takes
20+ minutes, having it run on multiple agents is one of the key points in
the whole setup...

PS>
PS> - in build configuration A add dependency by sources to B
PS>
PS> Compatibility with one agent guarantees that there will not be A & B
PS> builds running simultaneously.
PS>
PS> Dependency by sources A->B will allow you to avoid situations when
PS> your tests from build A runs with incompatible database schema.

This is not solving my problem, it's solving another one that I didn't have.
The DB update takes less than a minute, we've already solved that through
build wait periods.

So, basically, one release later, my only option is still writing a plugin
or hacking something in the build scripts...

Oh well...

Best,
Andrei


0
Comment actions Permalink

There is one thing I do not understand. If build B is so fast, why not run it as a part of build A?

--
Pavel Sher

0
Comment actions Permalink

Hello Pavel,

PS> There is one thing I do not understand. If build B is so fast, why
PS> not run it as a part of build A?

There are other types of builds similar to A that can run in parallel (and
multiple instances of A can run at the same time), we wouldn't want the DB
upgrade part to be executed by all of them, we'd end up with lots of failures
because of that (basically we'd have to have a single such build running
at a time or they'd nuke each other). So we defined B as a separate build,
monitoring the DB sources, and we trigger the other builds from it after
it's done (to ensure the code still runs OK with the updated DB). If the
other ones are triggered by code changes, no DB upgrade is needed/performed,
so they can run concurently.

The problem occurs if somebody checks in DB changes while the other builds
(type A) are running. They'll end up failing, because B does not wait for
them to finish.

So, basically, what we need is something that says to build B to wait until
no type A builds (and this can include more than one type) are still running.
And, vice-versa, tell type A builds to not start until B is complete (this
is not that crucial for us right now, since B would be finished by the time
A finished getting the changes/compiling, so we don't see this problem often,
although it could be a valid use case). I thought the source dependencies
would do just that, but then I'd have build B running with the sources build
A was running, so basically it would miss its changes (if I understand it
correctly).

It shouldn't be hard to implement a wait dependency between builds and it's
been requested by multiple users on the forums a few months back (and discussed
a fair bit), that's why I'm dissapointed that it's been upgraded to a very
generic resource sharing use case (supposed to take care of this case as
well) that ended up being dropped altogether.

Best,
Andrei


0
Comment actions Permalink

Hi Andrei,

We have this request in our tracker, but in sank to deep and wasn't included in our plans. Sorry about it :(

http://jetbrains.net/tracker/issue/TW-3798

Regards,
KIR

0

Please sign in to leave a comment.