Server infinite loop

I have the server stuck in a loop, trying to stop a build on an agent, after poking at it a little I restarted both the build machine and the server, when I startup the server again on the mainpage it just keeps looping on stopping the build.

#1.5.0.3183 Cancelled (Tests passed: 510, ignored: 94) No artifacts No changes started: 16 Sep 08:26 Stopping

Below are my logs,

Thanks,
Orn

8 comments
Comment actions Permalink

Why do you think the server was in infinite loop? Have you noticed high CPU load or the server was not responsive?

Could you please attach agent logs too?

--
Pavel Sher

0
Comment actions Permalink

Attached are the build agent logs.

What we see on the mainpage after canceling the build is the following, it just seems to be stuck trying to unload the build.

#1.5.0.3183 Cancelled (Tests passed: 510, ignored: 94) No artifacts No changes
started: 16 Sep 08:26
started: 16 Sep 08:26
Stopping

And the green circular icon just keeps spinning, with a tag "Build is running"

And the warning icon gives this tag
"Build agent .114 was disconnected while running a build.
Please check the agent and network connectivity. Either ensure the agent is connected or stop the build.
Last message was received on: 22 Sep 12:31 (16m:35s ago)"


On the server looking at the cpu load tomcat6.exe is taking 90-99% cpu with about 90MB memory.

Thanks,
Orn



Attachment(s):
buildAgent.zip
0
Comment actions Permalink

Please do the following: http://www.jetbrains.net/confluence/display/TCD3/Reporting+Issues#ReportingIssues-hang

and attach here thread dump from the server.

--
Pavel Sher

0
Comment actions Permalink

Here is a dump from AdaptJ



Attachment(s):
tc_dump.txt
0
Comment actions Permalink

It seems the server hangs on loading build log, probably the log file is broken. Try to locate this log on disk and rename the file. Build logs are stored in the .BuildServer/system/messages folder. There is a number of directories with files with names ]]>.msg3. Build id you can discover from the link to your build.

--
Pavel Sher

0
Comment actions Permalink

We solved the problem already as we needed a build for the testers, probably not recommended but we went into the database after stopping the server and wiped the

TEAMCITY.RUNNING

table clean, then we added the build in this table

TEAMCITY.PAUSED_BUILD_TYPES

after that we were able to start the build again from the main page.

Thanks,
Orn

0
Comment actions Permalink

Hello,

The problem will be fixed in the upcoming Calcutta EAP build.

Regards,
KIR

0

Please sign in to leave a comment.