Lost Change Sets

I'm seeing agent repositories getting out of sync with CVS/the server.

I reported this earlier as http://jetbrains.net/jira/browse/TW-592, but it still seems to be happening for me.

The clearest example of this for me came when one of the two agents for a particular build would always fail and the other one would succeed. I did a comparison of the source in buildAgent/work and saw that the two had different versions of several files which had been checked in a couple days back. I was able to fix the broken agent by doing a "Reset Source", but this obviously is not a very good solution. Nor is always doing a clean source checkout.

This is obviously a tough one for me to capture. I can see the two builds for the agents which occurred right around where one of the suspect files gets mentioned. One starts at 12:54, while the other one starts at 12:58. The first one includes the changes to the file in question, while the other one doesn't. This is as I would expect, but it looks like the second agent never got those changes sent to it.

So my question is, what exactly is supposed to happen with the following scenario:

CVS Rep Agent 1 Agent 2

T0 All three have identical filesets for the particular build
T1 Checkin File A and B
T2 Build Triggered (Set S1)
T3 CI C and D
T4 Still Building BT (Set S2)
T5 Build Succeeds
T6 Build Fails
T7 CI E and F
T8 BT (S3)
T9 CI G and H
T10 Still Building BT (S4)

So I would guess that S1 would be A and B.

Should S2 be A, B, C, D or just C&D? How to indicate that there is a dependency in the checkin if just the latter? What if A and C were the same file?

Should S3 be C, D, E & F or just E & F? If E & F, when does it ever get changes C & D?

What about S4? Does it get G & H? Does it get A, B, G & H because the build at T1 has now succeeded? Does it get A, B, E, F, G & H?

Sorry if this isn't clear, but it does get complicated. Is there a good page in the documentation to explain the change sets get handled when you have overlapping builds/checkins? Is this what "One Build At A Time" is for? Is this happening because of forced builds? Should a build server ever be out of sync with another build server well after a checkin?

I can open another issue for this if that is useful, though I think I'd need much more information before I can fill it with anything useful.

--Tim

4 comments
Comment actions Permalink

I reported this earlier as http://jetbrains.net/jira/browse/TW-592, but it
still seems to be happening for me.


I reopened it. Some problem related to cvs patches was fixed, but it seems
it was another one.

Should S2 be A, B, C, D or just C&D? How to indicate that there is a
dependency in the checkin if just the latter? What if A and C were the
same file?


S2 should contain all changes, A, B, C, D. Otherwise there is a bag.

If you have logs from these agents, please attach them to the request, and
log from the server too.

Thanks!

--
Olesya Smirnova
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"


"Tim McNerney" <intellij@oneofus.org> wrote in message
news:3207512.1156791742516.JavaMail.itn@is.intellij.net...

I'm seeing agent repositories getting out of sync with CVS/the server.

>

I reported this earlier as http://jetbrains.net/jira/browse/TW-592, but it
still seems to be happening for me.

>

The clearest example of this for me came when one of the two agents for a
particular build would always fail and the other one would succeed. I did
a comparison of the source in buildAgent/work and saw that the two had
different versions of several files which had been checked in a couple
days back. I was able to fix the broken agent by doing a "Reset Source",
but this obviously is not a very good solution. Nor is always doing a
clean source checkout.

>

This is obviously a tough one for me to capture. I can see the two builds
for the agents which occurred right around where one of the suspect files
gets mentioned. One starts at 12:54, while the other one starts at 12:58.
The first one includes the changes to the file in question, while the
other one doesn't. This is as I would expect, but it looks like the second
agent never got those changes sent to it.

>

So my question is, what exactly is supposed to happen with the following
scenario:

>

CVS Rep Agent 1 Agent 2

>

T0 All three have identical filesets for the particular
build
T1 Checkin File A and B
T2 Build Triggered (Set S1)
T3 CI C and D
T4 Still Building BT (Set
S2)
T5 Build Succeeds
T6
Build Fails
T7 CI E and F
T8 BT (S3)
T9 CI G and H
T10 Still Building BT (S4)

>

So I would guess that S1 would be A and B.

>

Should S2 be A, B, C, D or just C&D? How to indicate that there is a
dependency in the checkin if just the latter? What if A and C were the
same file?

>

Should S3 be C, D, E & F or just E & F? If E & F, when does it ever get
changes C & D?

>

What about S4? Does it get G & H? Does it get A, B, G & H because the
build at T1 has now succeeded? Does it get A, B, E, F, G & H?

>

Sorry if this isn't clear, but it does get complicated. Is there a good
page in the documentation to explain the change sets get handled when you
have overlapping builds/checkins? Is this what "One Build At A Time" is
for? Is this happening because of forced builds? Should a build server
ever be out of sync with another build server well after a checkin?

>

I can open another issue for this if that is useful, though I think I'd
need much more information before I can fill it with anything useful.

>

--Tim



0
Comment actions Permalink

I reported this earlier as

http://jetbrains.net/jira/browse/TW-592, but it

still seems to be happening for me.


I reopened it. Some problem related to cvs patches
was fixed, but it seems
it was another one.


Logs added. This is from use of the current EAP. Which begs the question, is there going to be a newer EAP any time soon?

Should S2 be A, B, C, D or just C&D? How to

indicate that there is a

dependency in the checkin if just the latter? What

if A and C were the

same file?


S2 should contain all changes, A, B, C, D. Otherwise
there is a bag.


Great. That's what I'd hope for.

Is the current state of one of the build agents kept on the filesystem, the database or in memory? Is there any way I can look at it and monitor it? Can I change my logging levels to print out what changes are being pushed to a buildAgent?

--Tim

0
Comment actions Permalink

>Which begs the question, is there going to be a newer EAP any time soon?
I hope... We're planning to do it tomorrow.

Is the current state of one of the build agents kept on the filesystem,
the database or in memory? Is there any way I can look at it and monitor
it? Can I change my logging levels to print out what changes are being
pushed to a buildAgent?


Server log contains before and after versions of building patch for each
one.
I added possibility do not delete patch file from the agent so you'll be
able to view applied changes. (You'll have to add system property to build
named agent.save.patch with value 'true'). It will be available in the next
eap
--
Olesya Smirnova
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"


"Tim McNerney" <intellij@oneofus.org> wrote in message
news:20998218.1156870413763.JavaMail.itn@is.intellij.net...
>> > I reported this earlier as
>> http://jetbrains.net/jira/browse/TW-592, but it
>> > still seems to be happening for me.
>>
>> I reopened it. Some problem related to cvs patches
>> was fixed, but it seems
>> it was another one.
>

Logs added. This is from use of the current EAP. Which begs the question,
is there going to be a newer EAP any time soon?

>
>>
>> > Should S2 be A, B, C, D or just C&D? How to
>> indicate that there is a
>> > dependency in the checkin if just the latter? What
>> if A and C were the
>> > same file?
>>
>> S2 should contain all changes, A, B, C, D. Otherwise
>> there is a bag.
>

Great. That's what I'd hope for.

>

Is the current state of one of the build agents kept on the filesystem,
the database or in memory? Is there any way I can look at it and monitor
it? Can I change my logging levels to print out what changes are being
pushed to a buildAgent?

>

--Tim



0
Comment actions Permalink

>Which begs the question, is there going to be a
newer EAP any time soon?
I hope... We're planning to do it tomorrow.

Is the current state of one of the build agents

kept on the filesystem,

the database or in memory? Is there any way I can

look at it and monitor

it? Can I change my logging levels to print out

what changes are being

pushed to a buildAgent?


Server log contains before and after versions of
building patch for each
one.
I added possibility do not delete patch file from the
agent so you'll be
able to view applied changes. (You'll have to add
system property to build
named agent.save.patch with value 'true'). It will be
available in the next
eap


Awesome. I look forward to the new build. If there are no major regressions and a few minor improvements, we'll probably roll out to a larger audience and run in parallel with our CruiseControl build server.

I'll use the logging to try and capture the problem in the wild for you guys.

--Tim

0

Please sign in to leave a comment.