My TC 5 EAP builds fail intermittently

My configuration is TC 5 EAP build 10380, Glassfish 2.1, JDK 1.6U16, Windows Server 2003 and subversion over ssh. I have builds configured to run when there are java source changes checked into the subversion repo. The first few auto builds with this configuration ran fine, then I started seeing the following error:

Problem while loading patch data stream: Read timed out

Subsequent to this all builds fail (after a long period of time - too long) with the same error message. Here's a sample of the full log for one of these problem builds:

[21:12:49]: Checking for changes
[21:39:08]: Clean build enabled: removing old files from C:\BuildAgent\work\98f25827ac48c2f9
[21:39:08]: Clearing temporary directory: C:\BuildAgent\temp\buildTmp
[21:39:08]: Checkout directory: C:\BuildAgent\work\98f25827ac48c2f9
[21:39:08]: Updating sources: server side checkout... (1h:0m:44s)
[21:39:08]: [Updating sources: server side checkout...] Will perform clean checkout
[21:39:08]: [Updating sources: server side checkout...] Clean checkout reasons
[21:39:08]: [Clean checkout reasons] Checkout directory is empty or doesn't exist
[21:39:08]: [Clean checkout reasons] "Clean all files before build" turned on
[21:39:08]: [Updating sources: server side checkout...] Transferring cached clean patch for VCS root: trunk-it
[21:39:50]: [Updating sources: server side checkout...] Building incremental patch over the cached patch
[21:59:51]: [Updating sources: server side checkout...] Transferring repository sources: 457.99Mb so far...
[22:19:51]: [Updating sources: server side checkout...] Problem while loading patch data stream: Read timed out
[22:39:52]: Will repeat attempt when server will be available, number of attempts left: 2
[22:40:02]: Updating sources: server side checkout...
[22:40:02]: [Updating sources: server side checkout...] Will perform clean checkout
[22:40:02]: [Updating sources: server side checkout...] Clean checkout reasons
[22:40:02]: [Clean checkout reasons] Checkout directory is empty or doesn't exist
[22:40:02]: [Clean checkout reasons] "Clean all files before build" turned on
[22:40:02]: [Updating sources: server side checkout...] Transferring cached clean patch for VCS root: trunk-it
[22:40:45]: [Updating sources: server side checkout...] Building incremental patch over the cached patch
[23:00:45]: [Updating sources: server side checkout...] Transferring repository sources: 457.99Mb so far...
[23:20:46]: [Updating sources: server side checkout...] Problem while loading patch data stream: Read timed out


Thoughts?


-Noah
3 comments
Comment actions Permalink

Here's a follow up stack trace from a recent build.

[21:34:38]: Problem while loading patch data stream: Read timed out

[21:34:38]: jetbrains.buildServer.agent.impl.patch.PatchStreamDownloader$1: Read timed out
jetbrains.buildServer.agent.impl.patch.PatchStreamDownloader$1: Read timed out
at jetbrains.buildServer.agent.impl.patch.PatchStreamDownloader.downloadPatch(PatchStreamDownloader.java:41)
at jetbrains.buildServer.agent.impl.patch.PatchDownloaderImpl.downloadPatch(PatchDownloaderImpl.java:141)
at jetbrains.buildServer.agent.impl.patch.PatchDownloaderImpl.copyPatchAndCheck(PatchDownloaderImpl.java:54)
at jetbrains.buildServer.agent.impl.patch.UpdateSourcesPatcherBase.copyPatchToTempFile(UpdateSourcesPatcherBase.java:65)
at jetbrains.buildServer.agent.impl.patch.UpdateSourcesFromServer.updateSources(UpdateSourcesFromServer.java:58)
at jetbrains.buildServer.agent.impl.patch.UpdateSourcesBuildStageBase.doSourceUpdate(UpdateSourcesBuildStageBase.java:74)
at jetbrains.buildServer.agent.impl.patch.UpdateSourcesBuildStageBase.doRecoverableStage(UpdateSourcesBuildStageBase.java:52)
at jetbrains.buildServer.agent.impl.runStages.RecoverableBuildStage.doLastAttempt(RecoverableBuildStage.java:101)
at jetbrains.buildServer.agent.impl.runStages.RecoverableBuildStage.doBuildStage(RecoverableBuildStage.java:58)
at jetbrains.buildServer.agent.impl.BuildRunAction.callRunStage(BuildRunAction.java:106)
at jetbrains.buildServer.agent.impl.BuildRunAction.doStages(BuildRunAction.java:80)
at jetbrains.buildServer.agent.impl.BuildRunAction.access$000(BuildRunAction.java:17)
at jetbrains.buildServer.agent.impl.BuildRunAction$1.run(BuildRunAction.java:55)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
at org.apache.commons.httpclient.ChunkedInputStream.read(ChunkedInputStream.java:152)
at java.io.FilterInputStream.read(FilterInputStream.java:66)
at org.apache.commons.httpclient.AutoCloseInputStream.read(AutoCloseInputStream.java:88)
at jetbrains.buildServer.http.ServerStreamProvider$1.read(ServerStreamProvider.java:107)
at java.io.InputStream.read(InputStream.java:154)
at jetbrains.buildServer.util.TCStreamUtil.writeBinary(TCStreamUtil.java:116)
at jetbrains.buildServer.agent.impl.patch.PatchStreamDownloader.copyStreamToFileAndCloseStream(PatchStreamDownloader.java:85)
at jetbrains.buildServer.agent.impl.patch.PatchStreamDownloader.downloadPatch(PatchStreamDownloader.java:38)
... 13 more
0
Comment actions Permalink

Hello,

  Could you please take a look at the glassfish logs, I believe they should contain some more information regarding this problem.
  Looks like server is unable to build a patch for the sources correctly.

  Another thing to try is to change checkout mode to "automatically on agent" for VCS settings of the build configuration.

  Hope this helps,
  KIR

0
Comment actions Permalink

I took your advice and switched checkout mode to "automatically on agent" for VCS settings of the build configuration and I have not had this issue since then. I'll keep monitoring. Thanks,

-Noah

0

Please sign in to leave a comment.