Difficulty w/perforce when I specify a clientspec

Hello, We're setting up builds in a new TeamCity installation and builds have been working fine when I specify my VCS info into per-build views.  But if I setup a VCS root with the name of my clientspec instead I get 2 different behaviors, neither of which I want:

1. When I don't specify a checkout directory (after all, that's what I thought my p4 client root was for), I effectively get a clean sync every time.  This is too costly for me, I need my builds to remember what they've already sync'd and not bother syncing them again.  So I tried,

2. Specifying a clientspec, *and* a checkout directory -- where the checkout directory is identical to my p4 clientspec root.  In this case, the sync takes all the time normally required to download every single file, and then it simply fails with the exception and stack trace below.

This may seem silly, but I'd really like to get this to work with a *clientspec*, not a view.  Because I want (1) a build that remembers what it already has and (2) anytime I login to the buildagent and do anything in perforce, I want that to be taken into account when the build runs next.  If I copy the view from my client-spec into the 'team-city-build' client-mapping, then I won't be able to do anything outside of TeamCity (I.e., via perforce) and have it remembered the next time TeamCity builds.

Is this at all possible?  Thanks, stack-trace below ....

--kov

[21:33:59]: Getting project sources
[21:46:43]: Could not get project sources: for build #1771
[21:46:43]: jetbrains.buildServer.vcs.VcsException: java.io.IOException: Write error
jetbrains.buildServer.vcs.VcsException: java.io.IOException: Write error
at jetbrains.buildServer.buildTriggers.vcs.perforce.DepotFile.changeOrCreateBinaryFile(DepotFile.java:114)
at jetbrains.buildServer.buildTriggers.vcs.perforce.PerforcePatchProvider$MyPythonReader.sendContent(PerforcePatchProvider.java:320)
at jetbrains.buildServer.buildTriggers.vcs.perforce.PerforcePatchProvider.loadContents(PerforcePatchProvider.java:94)
at jetbrains.buildServer.buildTriggers.vcs.perforce.PerforcePatchProvider.buildPatch(PerforcePatchProvider.java:66)
at jetbrains.buildServer.buildTriggers.vcs.perforce.PerforceSupport.buildPatch(PerforceSupport.java:195)
at jetbrains.buildServer.serverSide.impl.projectSources.SmallPatchCache.getCachedCleanPatch(SmallPatchCache.java:12)
at jetbrains.buildServer.serverSide.impl.projectSources.PatchCacheImpl.buildCleanPatch(PatchCacheImpl.java:12)
at jetbrains.buildServer.serverSide.impl.projectSources.PatchComposer.buildPatch(PatchComposer.java:95)
at jetbrains.buildServer.serverSide.impl.projectSources.PatchComposer.buildPatch(PatchComposer.java:29)
at jetbrains.buildServer.serverSide.impl.projectSources.PatchComposer.buildPatch(PatchComposer.java:93)
at jetbrains.buildServer.serverSide.impl.BuildTypeImpl.buildPatch(BuildTypeImpl.java:326)
at jetbrains.buildServer.serverSide.impl.BuildTypeImpl$$FastClassByCGLIB$$a84db719.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)
at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:700)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
at jetbrains.buildServer.serverSide.impl.auth.TeamCityMethodSecurityInterceptor.invoke(TeamCityMethodSecurityInterceptor.java:32)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.Cglib2AopProxy$FixedChainStaticTargetInterceptor.intercept(Cglib2AopProxy.java:582)
at jetbrains.buildServer.serverSide.impl.BuildTypeImpl$$EnhancerByCGLIB$$d9fa76bf.buildPatch(<generated>)
at jetbrains.buildServer.serverSide.impl.BuildStarter$2.call(BuildStarter.java:9)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
at java.util.concurrent.FutureTask.run(FutureTask.java:123)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
Caused by: java.io.IOException: Write error
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:260)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at jetbrains.buildServer.buildTriggers.vcs.perforce.DepotFile.getContentFile(DepotFile.java:164)
at jetbrains.buildServer.buildTriggers.vcs.perforce.DepotFile.changeOrCreateBinaryFile(DepotFile.java:97)
... 24 more

[21:46:43]: Could not get project sources
[21:46:43]: Loading changed files from server...
[21:46:43]: Patch is broken, can be found in file C:\BuildAgent\temp\cache\temp32196patch1771
[21:46:43]: Error while applying patch: java.io.IOException: Write error
[21:46:43]: Build finished

6 comments
Comment actions Permalink

Any response to this?  Rather than the problem itself, I'd just like to understand what TeamCity is supposed to do where my VCS = perforce and I specify a clientspec for particular VCS Root.  Particular questions are:

  1. If I specify a clientspec for my build's VCS Root, does that mean it will use the client-spec's root directory as the build-root?  It appears when I run builds with these configurations that TeamCity checks files out to a temp directory.
  2. What happens (for the above scenario for VCS Root) if I specify a Checkout Directory? Do the files get checked out into my clientspec-root then moved to the Checkout Directory?
  3. What happens if I do not specifya Checkout Directory?  Does it use my clientspec-root, or a temp-directory?



If this is documented somewhere then my apologies (and please point me to the docs)!  I cannot find such details in any documentation.

0
Comment actions Permalink

Hello, Ken

   For Perforce SCM, TeamCity supports only server-side checkout. That means, TeamCity always performs actual checkout from the Perforce on the server (to a temp cache directory) and then sends checked out sources to build agents. Even if you have a build agent (which runs the actual build) on the same machine as TC server, the sources will be sent to agent directly from the server anyway. That allows to have multiple build agents on different machines and setup Perforce client only on the server.

   Regardless whether client name or client mapping is specified in TeamCity VCS Perforce settings, TeamCity doesn't use data in Perforce client root - it always checks out sources to a temp directory in special format, and this data is not visible to user. 

   When server sends sources to the build agent, it uses checkout directory (+checkout rules, if any) as a target location for the sources. If checkout directory is empty, default checkout directory on build agent is used (something like buildAgent/work//). Please read more about checkout directory in TeamCity docs at http://www.jetbrains.net/confluence/display/TCD4 (there is a speed search above the left navigation). 

   Usually, TeamCity tries to send not the whole source tree to build agent, but only the difference since the previous checkout. To enable this, TeamCity remembers the version of sources sent to each build agent to each checkout directory. But sometimes, TeamCity sends the whole source tree:

    
  • in the case, when you enable "clean checkout" checkbox in VCS settings  
  • if checkout directory is empty or doesn't exist. This may happen after changing the VCS settings in TeamCity, because in this case  changes  
  • if previous build was interrupted/failed during checkout process. TeamCity cannot be sure that target directory contains actual sources  
  • if you have different build configurations with different VCS settings pointing to the same manually specified checkout directory. In this case this directory will contain build A sources after build A and we cannot apply patch for build B, we have to checkout sources for B entirely  
  • if user requested clean build manually, using a command to 'enforce clean checkout', either from "Actions" on build configuration toolbar, or a similar command from individual build agent page  


  There is another thing to consider. TeamCity server maintains a local cache for checked out sources (for all version controls and VCS roots). And this cache (if filled) is used as a base for full checkouts to reduce load on VCS servers. 

   By the moment, TeamCity doesn't update this cache if you change Perforce client mapping for perforce client specified in TeamCity VCS settings. I.e. if you specify the mapping in TeamCity, there are no problems. But if you specify perforce client name, you'll have to 'enforce clean checkout' each time you make changes in the client mapping of the client. Please note, the only way to clean internal TeamCity VCS cache is to invoke 'enforce clean checkout' command. I admit that this is a bug, but this how it can be workarounded by the moment.

   Hope this answers your questions,

   Kind regards,
   KIR     

0
Comment actions Permalink

Thanks, I really appreciate the detailed answer.  It isn't really the news I wanted to hear, but it does make sense due to perforce's architecture.

My last question, is there any version-control product where you don't checkout code on the build-server and ship it out to the build agents?

Thanks again,

--kov

0
Comment actions Permalink

Hello Ken,

  TeamCity supports checkout on agent for Subversion, CVS, and TFS - in this case TeamCity server will only send information about revisions to be checked out on the build agent. The actual process of update from VCS server will be performed on the build agent. In this case, your build can also commit files to VCS.

  You can also totally disable VCS handling by TeamCity and use your own scripts - there is an option "Do not checkout sources".
  Please read more at http://www.jetbrains.net/confluence/display/TCD4/VCS+Checkout+Mode

  Kind regards,
  KIR

0
Comment actions Permalink

One other question, please ....

> There is another thing to consider. TeamCity server maintains a local cache for
> checked out sources (for all version controls and VCS roots). And this cache
> (if filled) is used as a base for full checkouts to reduce load on VCS servers.

Where is this configured?  I can't find anything in the docs, and I want to specify where this cache ends up so that I can get it onto protected storage.

0

Please sign in to leave a comment.