Advise requested for running post success script

I see lots of confusion around the basic approach of what I am trying to do (create a snapshot dependency that runs a post success script that does a chmod +x on executable artifacts), so I’ll just lay the problem in more capable hands and request advise.

I am doing a continuous build with the Ant runner with a single LINUX build agent.  When the artifacts are published I would like to give some of them execute permission.

What is the best way to do this that assures that the artifacts are processed only if the continuous build succeedes and that it happens immediately after artifact publish (no other builds can sneak in before the chmod is done)?

Thanks,

--ben

9 comments
Comment actions Permalink

Ben,

Sorry for the delay in replying, we somehow overlooked the post :(


It seems that the best approach is to analyze the build state within your build script and set the attibutes as the last step within the same build.

BTW, what is the problem in always setting the attibute with no respect to the build status?

Do you set the attribute for thr files being bublished as artifacts? Can you please describe why do you need this?

0
Comment actions Permalink

Thank you for responding.

Seems there is more than a bit of misunderstanding here .. much of it undoubtly mine.

The ant script performs a chmod +x task on a *.sh file set.  When the artifacts are published [dist => dist] the permission are not preserved (just like any other cp operation would on *nix that uses umask).

Since the copy of the artifacts does not happen in the ant script, I assumed that the only way to affect the artifacts file was to have a dependent build.  If there is a better way, please educate me.

I do not really require knowing success/failure to perform the chmod +x; the important thing is that for every artifact publish (copy), the ant task or script can be run on the actual artifacts file without being interrupted due to other build triggers.

0
Comment actions Permalink

Ben,

How do you access the artifacts? To my understanding downloading a file with a web browser does not set the executable attribute to the saved file.

Is packing the artifacts into tar.gz an option for your case?

Maybe you can write a script that will set the attribute direclty in the artifacts storage under .BuildServer directory on the server, but that depends onthe way you plan to use them.

0
Comment actions Permalink

Yergor,

let me try to be very specific including code and configuration fragments so there is no ambiguity here (I hope the html editor doesn’t make too much of a mess of mark up characters.)

1. The artifacts are copied by TeamCity into the following directory: /usr/local/TeamCity/data/system/artifacts/ (-Dteamcity.data.path=/usr/local/TeamCity/data)

2. The build runner is Ant.  In CC I ran a post success task that ran a bash script to update the permissions on the artifacts and create some link:

                <artifactspublisher
                    dir="${CC_WORKDIR}/checkout/waggle/dist"
                    dest="artifacts/waggle"
                    subdirectory="dist"/>
                <execute command="${CC_WORKDIR}/latest.sh"/>


In TeamCity the artifacts appear to be copied outside of the control of Ant, and I haven’t figured out how I can easily both do an Ant build and run a bash script on the artifacts after they are copied as part of the same build.  So what I attempted to do was create a snapshot dependency which led to the frustration that led to this topic.

3. While occasionally someone may wish to get at some of the artifacts via HTTP, the primary intended use case is that the artifacts are shared over NFS.  The admin or an automated script will cd /usr/local/TeamCity/data/system/artifacts/*/<BUILD>/install/ and run deploy.sh


Can you help?



 

0
Comment actions Permalink

Ben,

I see, so the main requirement is that you can execute scripts directly in "<TeamCity data dir>/system/artifacts" directory...

I am not sure this can be achieved without some tricky solution because direct access to the directory is not considered a common use case in TeamCity.

I can suggest the folowing workarounds:
- do not use TeamCity artifacts and use a-la CC logic: upload the files into a shared directory from your build scirpt
- configure a separate build which will patch the files in the TeamCIty artifacts directory. There can be difficulties determining what directory to patch files in, but workarounds exits like choosing the latest directory or priting a small TeamCity plugin to provide the property with dependency build internal buildID.
- setup a job external to TeamCity that will set the executable attributes to the required file. Can be a cron job.
- write a plugin to TeamCity that will modify the files after they are saved in artifacts directory.

If you need help with any of the TeamCity0related approaches, please let me know.

I'd also suggest to consider changing the workflow if possible. e.g. creating a separate build configuration that will retrieve the necessary artifacts via TeamCity artifact dependencies and run the script that is usually run inside "<TeamCity data dir>/system/artifacts".

0
Comment actions Permalink

Thanks Yegor, (sorry for the typo last time)

We finally understand each other.

"I see, so the main requirement is that you can execute scripts directly in "<TeamCity data dir>/system/artifacts" directory..."

I understand that this is outside of your expected use case.  We are building and deploying a scalable, clustered enterprise server here which is very different from the typical end user facing apps that I'll bet you see most often.  However, as general enhancements TC might consider the ability to run a list of post success tasks (similar to build runner config) as well as allowing a umask type config for artifact maps.. but I digress..

The options you suggested that fit best with our needs are:

1- configure a separate build which will patch the files in the TeamCIty artifacts directory.
2- write a plugin to TeamCity that will modify the files after they are saved in artifacts directory.

These are really the same functionally, but just with different tool sets.

#1 is where I started, but I had real issues trying to figure out how to set up the snapshot dependency so that I was always triggered immediately after successful build so that the "latest" logic wasn't undermined by another VCS trigger.  If this is the easiest or best of the 2 can you give me some guidance.

#2 sounds the cleanest, I have no issues with writting Java code as long as the effort level is reasonable.  Where do I start and how do I configure it?

--ben

0
Comment actions Permalink

Ben,

> #1 is where I started, but I had real issues trying to figure out how to set up the snapshot dependency so that I was always triggered immediately after successful build so that the "latest" logic wasn't undermined by another VCS trigger.  If this is the easiest or best of the 2 can you give me some guidance.

Actually, there is no way to guarantee a build runs right after another one. You can use dependency trigger - but that will add a new build to the end of the queue. Snapshot dependency of patching build on the main build can only help if you setup to always trigger the patching build instead on the main build: this will cause both main and patching build to be put in queue and unless the queue is reordered, they will run one aftr another. Anyway, too much a "hack" here.


> #2 sounds the cleanest, I have no issues with writting Java code as long as the effort level is reasonable.  Where do I start and how do I configure it?

You can start here for the initial pointers on writing a plugin. The sample plugin that we have in distribution is implementing BuildServerListener interface and you can do the necessary updates in buildFinished event (you can get the artifacts directory via build.getArtifactsDirectory).

As a quick start you can try to use an experimental GroovyPlug plugin (you will need latest TeamCity EAP for the current version of the plgin to work). It allows to change the code and see the effect without server restart. If you download the latest plugin version (I just committed the change), you can edit <TeamCity data dir>\config\__groovyPlug__\GroovyBuildServerListener.groovy file.

0
Comment actions Permalink

Thanks,

other then having to write Java this is exactly what I was looking for.

--ben

0
Comment actions Permalink

Ah, but with the GroovyPlug you get to write Groovy instead!

(I'm installing that on my EAP server as I type...)

0

Please sign in to leave a comment.