TeamCity killing spawned processes?

We have recently moved our CI system from hudson to teamcity and have been in general very pleased but I have one issue that is making my integration builds tough to run.

The build is a multi step process.

Step 1 : Build a deployment package using maven
Step 2 : Start packaged server (spawns an OSGI daemon)
Step 3 : Run tests against deployed server using maven
Step 4 : Cleanup started server

This works most of the time (~ 80%), the other 20% of the time the server started gets summarily killed within a second or so of Step 3 starting. I can tell they happen at the same time beacuse I can compare build step log entries with the last entry in the apparently killed (probably kill -9) server process. When we were running similar tests under hudson we had to set a "buildId=dontKillMe" property when we spawned the process to stop hudson from reaping that process, is there a similar mechanism in teamcity? And if there is that still doesn't really explain why it works most of the time. Any thoughts? I am having trouble diagnosing this issue from the logs and we run the builds on an EC2 node so its tough to be logged in to watch the error happen in real time.

Thanks for any help, this is a public teamcity installation so I can point you to the actual logs if that would help.

Sam Hendley

1 comment
Comment actions Permalink

Sam,

So far TeamCity does not have a feature to kill build processes until user clicks Cancel for the build on the web UI or the build is cnacelled in some other way.

That is why I guess that the server killing is not executed by TeamCity, but probably by some OS or your build script logic. Can this be related to the invocating script exit?

Seems, investigating why the server process terminates should provide more data to investigate this.

BTW, since you migrated from Hudson, I would appreciate if you can compare the tools, may be mentioning the pivotal points for choosing one over another. Is there something you like/dislike about either of the servers?

0

Please sign in to leave a comment.