Artifact checkin back into source control

This should be something easy to do...and yet I can not figure out how!

I'm using a buildConfiguration to generate a Sandcastle Help (.chm) file. I would like to store this help file back in with its source solution (so that its accessable right in the solution developers work with). We employ Perforce for our source control, and simply use "team-city-agent" for client mapping in the VCS root instead of clogging up Perforce Admin further with a million build-specific clients.

So the Question is: when using the built-in (to TeamCity) functionality of the team-city-agent for VCS Root mapping, how does one then submit changes to say help files produced by the build that need to be checked back into source control?


Any help regarding this question is most appreciated....

Brad

10 comments
Comment actions Permalink

For Perforce version control TeamCity exports sources on the server side and then transfers them to an agent (this is called server side checkout, and for Perforce this is the only available option). There is a feature request for agent side checkout for Perforce: http://www.jetbrains.net/tracker/issue2/TW-2103

The only workaround is to modify your build script to checkout files on the agent and checkin produced artifact.

BTW publishing of the produced .chm file as TeamCity artifact can be a better alternative. All other builds where this artifacts is required can then fetch it with help of Artifact dependencies feature of TeamCity: http://www.jetbrains.net/confluence/display/TCD4/Dependent+Build#DependentBuild-ArtifactDependency

0
Comment actions Permalink

Pavel,

[Quote]

BTW publishing of the produced .chm file as TeamCity artifact can be a better alternative. All other builds where this artifacts is required can then fetch it with help of Artifact dependencies feature of TeamCity: http://www.jetbrains.net/confluence/display/TCD4/Dependent+Build#DependentBuild-ArtifactDependency

[/Quote]

While that is a requirement (we have multiple solutions per product being built along a dependency chain, combining also to build one complex .chm); I still would like to also be able to maintain independent help files along with each specific solution.

What would be nice to have, is some way to leverage the team-city-agent to SUBMIT artifacts into any given source control AT the build Server (ie still use server side checkout and then server side checkIN). Yes I am aware that I could achieve this directly in my MSBuild scripts -- but that gives me a much less compelling reason to use TeamCity over CCNet; we're looking here for a BETTER solution than the CruiseControl way!

You mention that this is the only available option for Perforce...is there server side checkin facility in TeamCity for other SCMs? We will eventually be moving all of our teams over to TFS (just our Web group uses it currently); can I do this sort of thing with TC moving to TFS for source control?

I guess I could do something with some other outside scripting agent like PowerShell to move given .chm artifacts as they are created on the TC server to somewhere they can then be checked in to Perforce -- but that sounds complex...and "complex" is the way of CCNet, not TC! (so would prefer to find a package managed way to do what we want to do...)

Anyway, thanks for your continued help on this issue....

Brad



0
Comment actions Permalink

TFS has the same issue, though it seems that they have a solution for the next version that will address the issue.
http://www.jetbrains.net/devnet/thread/281758?tstart=0

0
Comment actions Permalink

Hello,

   TeamCity starting from version 4.5 should support checkout on agent mode (unfortunately this is not well documented yet).
   This definitely requires TFS installation on the build agent.

   Regards,
   KIR

0
Comment actions Permalink

Yup 4.5 with tfs agent side checkout is great as it uses the workspaces. It's helped tremendously with directories that we have cloacked.
Currently I use nant and tf (tfs command line) to do my checkouts when using Agent side checkout.
But checking artifacts back in (whether from the build that created the artifact or another build) does not work in tfs becuase of file locks (http://jetbrains.net/tracker/issue2/TW-8227).

0
Comment actions Permalink

I really think I need SERVER side checkin to do what I want most efficiently....not agent side checkout/in.

no?

0
Comment actions Permalink

Remember that everything is dependant on the vcs being used.
With tfs everything revolves around the workspace. And as far as I know, only the agent will have access to the workspace as it will differ among the agents.
You can still do builds, run unit tests, check for duplicate code etc. while using the server side checkout as these actions do not do any check outs or check ins. Once you start wanting to do checkouts and such then you need to start using the workspace and agent side checkouts.

0
Comment actions Permalink

For what its worth, I created an additional commandline runner chained to my help file builder build config to manage checking my build Artifacts back into VCS. While it would be nice if this could be more automated (ie driven by preconfigured buildConfigs) and more isolated (ie not require source control be running/installed on agent machines)...I'll take what I can get.

below is the powershell script I used...if it may be of use to anyone else:



update_Perfoce.ps1




Function copyFile([string]$target)

{

    $url = "http://[teamcity server]/guestAuth/repository/download/"+ $bldt +"/.lastSuccessful/"+ $arti

    $clnt = new-object System.Net.WebClient

    $clnt.DownloadFile($url, $lpth)

}

# -------------------------------------------------------------------------------------------------------

# the blow vars were populated with Param(x,y,z) definitions. I snipped the code because it was

# overly complex and not really needed. You could manually set the values within this .ps1 file


$acct          # Perforce user account
$pswd          # Perforce account password
$clnt          # Perforce workspace/client

$lpth          # Perforce or local path to be updated

$bldt          # TeamCity buildTypeID
$arti          # TeamCity Artifacts to copy back to Perforce (relative paths to TC VCS)

$desc          # description for the changelist


$tobj = 1      # is 1 if you want to make the files tempobjects
               # (writable and only keep head revision +S1w)

$addn = 0      # is 1 if you want to add new files instead of just updating

# -------------------------------------------------------------------------------------------------------


p4.exe -c $clnt -u $acct -P $pswd -C utf8 -p perforce:1666 sync -k "$lpth"
p4.exe -c $clnt -u $acct -P $pswd -C utf8 -p perforce:1666 edit -c default "$lpth"

copyFile $lpth

if ($addn = 1) { p4.exe -c $clnt -u $acct -P $pswd -C utf8 -p perforce:1666 add default "$lpth" }

if ($tobj = 1) { p4.exe -c $clnt -u $acct -P $pswd -C utf8 -p perforce:1666 reopen -c default -t +S1w "$lpth" }

p4.exe -c $clnt -u $acct -P $pswd -C utf8 -p perforce:1666 revert -a -c default

p4.exe -c $clnt -u $acct -P $pswd -C utf8 -p perforce:1666 change -o

p4.exe -c $clnt -u $acct -P $pswd -C utf8 -p perforce:1666 submit -d "$desc"


0
Comment actions Permalink

I do agree with you Brad, that the feature you've described would be insteresting.

As a ClearCase user, it would be used on my project.

Regards,

Olivier.

0
Comment actions Permalink

This thread was useful for me to come up with a similar solution to this problem so, even though it's a bit of a necro, I figured I'd add my solution to it in case others may find it useful in the future.  Brad thanks for posting that powershell script as that was the key to coming up with this solution.  In my case we actually have two separate depots, one for the development of the tech and one for its deployment.

My setup looks like this:
Perforce

  1. Main depot
  2. Deployment depot
  3. Workspace for the build agent account which has no host and a specific mapping for where the files will be that never changes (this let's me have 1 user and workspace for all Perforce agents)


TeamCity

  1. Continuous integration build
    • Runs continuous integration
    • Stores the built artifacts
    • Checks out on server
  2. Update build
    • Gets the snapshot of the deployment depot at the same time as continouous
    • Runs openForEdit.ps1 (this will open the directory I pass for edit so the next step can overwrite the artifacts)
    • Checks out on server with a custom path that matches the mapping of the workspace (I had to go the checkout on server as it seems the agent side checkout for Perforce still has some issues like it doesn't actually use the client specified and will create it's own workspace using the VCS settings, server side will avoid all that and is why I ended up having to split this process between 2 builds)
  3. Submit builds
    • Has the same VCS settings as Update but is set to not check out automatically, it only needs the settings so it knows where the files are
    • Has artifact dependency for Continouous, gets the artifacts from it and overwrites the ones in the checkout dir
    • Runs checkinArtifacts.ps1


And here's what my PowerShell scripts ended up looking like:

openForEdit.ps1

#Perforce user account, Perforce user account password, Perforce workspace/client, Perforce server address, local path to be updated
Param($account='buildagent', $password='', $workspace='TeamCity_Pipeline', $perforceServer='perforce:1666', $localPath='...')


#open the whole directory for edit and use the default changelist
p4.exe -c $workspace -u $account -P $password -p $perforceServer edit -c default $localPath



checkinArtifacts.ps1

#Perforce user account, Perforce user account password, Perforce workspace/client, Perforce server address, local path to be updated, Description for the changelist, Set to true if adding new files
Param($account='buildagent', $password='', $workspace='TeamCity_Pipeline', $perforceServer='perforce:1666', $localPath='...', $description='', $addNewFiles=$false)

#mark files for add
if ($addNewFiles = 1) { p4.exe -c $workspace -u $account -P $password -p $perforceServer add default $localPath }

#revert all unchanged file from the default changelist
p4.exe -c $workspace -u $account -P $password -p $perforceServer revert -a -c default

#resolve files as they'll be seen as out of date, accept yours only
p4.exe -c  $workspace -u $account -P $password -p $perforceServer resolve -ay -c default

#submit the default changelist with the description that was passed in
p4.exe -c $workspace -u $account -P $password -p $perforceServer submit -d $description


0

Please sign in to leave a comment.