Getting a build to run on all agents by default

Hi, I'm using TC 6.5.3

I need to get my parent pom build to run on all agents by default, since this can't be pulled by Maven.

1) I can get that with Run custom build, all agents. How can I make this the default run ?

2) I need the Agent local Maven to be used. To achieve this I have always been specifying the 'Maven home path'. but this is platform dependant.
Agents being on different platforms, I need to tell them to use the local MAVEN_HOME env var as the default build agent version 2.2.1 of Maven is not suitable.
How can I do that?

cheers

15 comments
Comment actions Permalink

Running build on several agents by a build trigger is not yet available.
We have a feature request on this that you can vote for / watch: TW-4940

Also there is a workaround suggested in the comment.

BTW, can you please describe what is the original goal for running the build on all agents?


As to the second question, you can define a parameter (e.g. agent.maven.path=c:/Tools/Maven) with the path to Maven on each agent (in conf/buildAgent.properties file of the agent) and then reference it in the build configuration's path to Maven like %agent.maven.path%.

0
Comment actions Permalink

Hi Yegor,

our use case for running a build on all agent is the following:

- the company parent pom, used by all Maven project, still have to be manually installed on each every machine that uses Maven.

This is a Maven limitation afaik, the parent pom SNAPSHOTS aren't updated the same way the dependencies are.

So for all buildagents, to make sure they have the lastest parent POM, a local install on each every of them is needed.

About the agent.maven.path value, %agent.maven.path% is the syntax in TC for env variables? This work independently from the platform, it just refers to a var declared in the buidAgent.properties?

Somehow it seems that the M2_HOME var is taken into account by the buildAgent, the same way the MAVEN_OPTS and JAVA_HOME are taken into accounts.
It even seem to take precedence over the bundled Maven instance.

On a Build Step screen, the description for the JDK Home path "If blank then path is read from JAVA_HOME environment variable or agent's own Java." makes sense.
Shouldn't it be the same for the Maven Home path? "if blank, path is read from M2_HOME", that would make more sense to me.
Even though it seems to be what's happening. I'm a bit lost here...

0
Comment actions Permalink

about M2_HOME & MAVEN_OPTS

- it's actually recognized on Windows and take precedence over the bundled maven in the buildAgent

- on Linux it's not, even though variables are set for the user under which the buildAgent runs.

0
Comment actions Permalink

nodje,

Thanks for describing the case. Seems like a valid case. For alike tasks on our internal server we use the build triggering workaround mentioned (a build triggers and triggers another build on all the agents from the script via HTTP request).

Another approach to pom.xml distribution can be adding one more step to every build configuration that runs Maven. The step would get the pom and publish it before the other steps.

> About the agent.maven.path value, %agent.maven.path%  is the syntax in TC for env variables? This work independently from the  platform, it just refers to a var declared in the buidAgent.properties?

Not sure what you mean here. For environment variables TeamCity uses %env.<name>% syntax. See more at the doc.

As to M2_HOME and MAVEN_OPTS - can you please detail the question? Whould you expect another logic or do you just want to make it more clear from UI notes?

0
Comment actions Permalink

Hi Yegor,

what I mean for MAVEN_OPTS & M2_HOME is that I would expect the same logic as the one in place for JAVA_HOME and JAVA_OPTS.
AFAIK, it's not the case right now, the buildAgent bundled instance of Maven take precedence over M2_HOME.

But it doesn't work like that in pratice:
- if I run a build without specifying the 'Maven home path' on a Linux buildAgent, it uses the bundled Maven instance without finding the MAVEN_OPTS.
- if i run it on a Windows agent, again without specifying the 'Maven home path', it uses the defined M2_HOME and fines the MAVEN_OPTS

If I could get all agents to take into accounts M2_HOME and MAVEN_OPTS, I'd never have to fill the 'Maven home path' and wouldn't have to care of which agent is enabled for that build.
That'd be even simpler than specifying an environment variable in each agent config.

Hope I made myself clear

cheers

0
Comment actions Permalink

I'm really at a loss understanding why my Linux agent logs consistently says:

Initial MAVEN_OPTS not set
Current MAVEN_OPTS not set



and my Windows agent consistently finds the env var MAVEN_OPTS:

Initial MAVEN_OPTS = -server -Xmx1024m -XX:MaxPermSize=512m
Current MAVEN_OPTS = -server -Xmx1024m -XX:MaxPermSize=512m

where env var are set on both machine for users used to run agent.

How can I control this?
0
Comment actions Permalink

On Windows env variables are set in Advanced System Settings dialog (tab Advanced). On Linux you can set environment variables in ~/.bash_profile.

You can easily find a lot of resources in the Internet regarding this subject. For example http://www.cyberciti.biz/faq/linux-unix-set-java_home-path-variable/ or http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/environment_variables.mspx?mfr=true

If these JVM arguments are specific to the build configuration, there is an easier way of specifying them - using the "JVM command line parameters" field in the runner settings.

0
Comment actions Permalink

Thank you for your answer.
Alas, I guess the env variable basics are pretty clear to me already, on all major OSs.

What I'm trying to make sense with, is the behavior of the buildAgent regarding those vars.
On my systems, it's not consistent. It may well be that I have some misconfiguration.

As said previously, Windows buildAgent would fautomatically find and take into accounts M2_HOME and MAVEN_OPTS, where Ubuntu agent would not. For some unknown reason...

I have to add that I've for long had problem getting a consistent behavior between a manual maven build and the same build triggered on a agent.
And that on all platforms. I'm still having errors with agent that I can't reproduce on CLI.

It might be env variables, different mvn repository usage, different JRE version used...

Probably my multiple configs being messy somehwere.

But I can't help believing that beiing able to reproduce Agent build with CLI is essential in debugging.
I would expect the default behavior of a newly installed buildAgent - I'm speaking for Maven builds here - should be exactly like the CLI - i.e. based on env var.

regards

0
Comment actions Permalink

> As said previously, Windows buildAgent would fautomatically find and take into accounts M2_HOME and MAVEN_OPTS, where Ubuntu agent would not. For some unknown reason...

Could you please login to your Ubuntu machine as the user under which the agent is run and execute 'env'. If you don't see M2_HOME and MAVEN_OPTS in the command output then this is the answer - these variables simply haven't been set.

Otherwise this is an unknown TeamCity issue and I'll need more details on it.

0
Comment actions Permalink

This is precisely what I was saying before, the 2 variables are set for all users on Ubuntu, including the one that runs the agent.
Which is why there seem to be a problem...

0
Comment actions Permalink

OK. That's pretty strange. Sorry for my misunderstanding.

Could you plese send me the 'env' output (executed under the agent user) and the full build log.

Thank you

0
Comment actions Permalink

env outup [excerpt]

SHELL=/bin/bash
TERM=xterm-256color
INSTALL4J_JAVA_HOME=/usr/lib/jvm/java-6-sun
USER=root
MAVEN_OPTS=-server -Xmx1024m -XX:MaxPermSize=512m
MAIL=/var/mail/root
PATH=/usr/lib/jvm/java-6-sun/bin:/opt/local/maven/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/local/mysql/bin
PWD=/root
JAVA_HOME=/usr/lib/jvm/java-6-sun
ARTIFACTORY_HOME=/var/opt/local/artifactory
LANG=en_US.UTF-8
SPEECHD_PORT=6560
M2_HOME=/opt/local/maven
SHLVL=1
HOME=/root
LOGNAME=root



build log [excerpt]: (with 'Maven home path' manually specified in the Maven build step)

[22:00:53]: Skip checking for changes - changes are already collected
[22:00:53]: Clean build enabled: removing old files from /opt/local/buildAgent/work/609cdce379dbedeb
[22:00:53]: Clearing temporary directory: /opt/local/buildAgent/temp/buildTmp
[22:00:53]: Checkout directory: /opt/local/buildAgent/work/609cdce379dbedeb
[22:00:53]: Updating sources: agent side checkout... (1m:28s)
[22:00:53]: [Updating sources: agent side checkout...] Will perform clean checkout. Reason: Checkout directory is empty or doesn't exist
[22:00:53]: [Updating sources: agent side checkout...] Cleaning /opt/local/buildAgent/work/609cdce379dbedeb
[22:00:53]: [Updating sources: agent side checkout...] VCS Root: trunk/alpha-struts (1m:28s)
[22:00:53]: [VCS Root: trunk/alpha-struts] revision: 11644_2011/10/11 16:00:40 +0200
[22:00:53]: [VCS Root: trunk/alpha-struts] Will use fast SVN update
[22:02:22]: Publishing internal artifacts (2s)
[22:02:25]: [Publishing internal artifacts] Sending build.start.properties.gz file
[22:02:22]: Using watcher: /opt/local/buildAgent/plugins/mavenPlugin/maven-watcher/maven-watcher-agent.jar
[22:02:22]: M2_HOME = /opt/local/maven
[22:02:22]: Initial MAVEN_OPTS not set
[22:02:22]: Current MAVEN_OPTS not set
[22:02:22]: PATH = /opt/local/maven/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
[22:02:22]: Starting: /usr/lib/jvm/java-6-sun-1.6.0.26/jre/bin/java -Dagent.home.dir=/opt/local/buildAgent -Dagent.name=Default agent -Dagent.ownPort=9090 -Dagent.work.dir=/opt/local/buildAgent/work -Dbuild.number=294 -Dbuild.vcs.number=11644 -Dbuild.vcs.number.1=11644 -Dbuild.vcs.number.trunk_alpha_struts=11644 -Dclassworlds.conf=/opt/local/buildAgent/temp/buildTmp/teamcity.m2.conf -Dcom.jetbrains.maven.watcher.report.file=/opt/local/buildAgent/temp/buildTmp/maven-build-info.xml -Dfile.encoding=UTF-8 -Dfile.separator=/ -Djava.io.tmpdir=/opt/local/buildAgent/temp/buildTmp -Dmaven.failsafe.debug='-javaagent:/opt/local/buildAgent/plugins/coveragePlugin/coverage-agent.jar=/opt/local/buildAgent/temp/buildTmp/coverage5472073416209115044args' -Dmaven.home=/opt/local/maven -Dmaven.surefire.debug='-javaagent:/opt/local/buildAgent/plugins/coveragePlugin/coverage-agent.jar=/opt/local/buildAgent/temp/buildTmp/coverage5472073416209115044args' -Dos.arch=i386 -Dos.name=Linux -Dos.version=2.6.32-34-generic -Dpath.separator=: -Dteamcity.agent.cpuBenchmark=458 -Dteamcity.agent.dotnet.agent_url=http://localhost:9090/RPC2 -Dteamcity.agent.dotnet.build_id=3669 -Dteamcity.auth.password=FPMNPP0rV3gBr8EDZC68FgXhK4pPGvSV -Dteamcity.auth.userId=TeamCityBuildId=3669 -Dteamcity.build.changedFiles.file=/opt/local/buildAgent/temp/buildTmp/changedFiles964557361341372646.txt -Dteamcity.build.checkoutDir=/opt/local/buildAgent/work/609cdce379dbedeb -Dteamcity.build.id=3669 -Dteamcity.build.properties.file=/opt/local/buildAgent/temp/buildTmp/teamcity.build9063999309413071619.properties -Dteamcity.build.tempDir=/opt/local/buildAgent/temp/buildTmp -Dteamcity.build.workingDir=/opt/local/buildAgent/work/609cdce379dbedeb -Dteamcity.buildConfName=alpha-struts : verify -Dteamcity.buildType.id=bt30 -Dteamcity.configuration.properties.file=/opt/local/buildAgent/temp/buildTmp/teamcity.config8192729040481534396.properties -Dteamcity.coverage.idea.baseDir=/opt/local/buildAgent/temp/buildTmp/idea2028998870347324660coverage -Dteamcity.coverage.idea.includePatterns=com.allence.* -Dteamcity.coverage.runner=IDEA -Dteamcity.idea.home=/opt/local/buildAgent/plugins/idea -Dteamcity.maven.watcher.home=/opt/local/buildAgent/plugins/mavenPlugin/maven-watcher -Dteamcity.projectName=Alpha -Dteamcity.runner.properties.file=/opt/local/buildAgent/temp/buildTmp/teamcity.runner6726502169236792168.properties -Dteamcity.tests.recentlyFailedTests.file=/opt/local/buildAgent/temp/buildTmp/testsToRunFirst4583621269848220362.txt -Dteamcity.version=6.5.4 (build 18046) -Duser.country=US -Duser.home=/root -Duser.language=en -Duser.name=root -Duser.timezone=Europe/Paris -classpath /opt/local/maven/boot/plexus-classworlds-2.4.jar: org.codehaus.plexus.classworlds.launcher.Launcher -f /opt/local/buildAgent/work/609cdce379dbedeb/pom.xml -B -U verify
[22:02:22]: in directory: /opt/local/buildAgent/work/609cdce379dbedeb
[22:02:23]: [INFO] Scanning for projects...


build log [excerpt]: (WITHOUT 'Maven home path' manually specified in the Maven build step)

[18:10:28]: Skip checking for changes - changes are already collected
[18:10:30]: Clean build enabled: removing old files from /opt/local/buildAgent/work/609cdce379dbedeb
[18:10:30]: Clearing temporary directory: /opt/local/buildAgent/temp/buildTmp
[18:10:30]: Checkout directory: /opt/local/buildAgent/work/609cdce379dbedeb
[18:10:30]: Updating sources: agent side checkout...
[18:10:30]: [Updating sources: agent side checkout...] Will perform clean checkout. Reason: Checkout directory is empty or doesn't exist
[18:10:30]: [Updating sources: agent side checkout...] Cleaning /opt/local/buildAgent/work/609cdce379dbedeb
[18:10:30]: [Updating sources: agent side checkout...] VCS Root: trunk/alpha-struts
[18:10:30]: [VCS Root: trunk/alpha-struts] revision: 11644_2011/10/11 16:00:40 +0200
[18:10:30]: [VCS Root: trunk/alpha-struts] Will use fast SVN update
[18:12:01]: Publishing internal artifacts (3s)
[18:12:05]: [Publishing internal artifacts] Sending build.start.properties.gz file
[18:12:01]: Using watcher: /opt/local/buildAgent/plugins/mavenPlugin/maven-watcher/maven-watcher-agent.jar
[18:12:01]: M2_HOME = /opt/local/buildAgent/tools/maven
[18:12:01]: Initial MAVEN_OPTS not set
...

0
Comment actions Permalink

I see a difference in PATH variable values in the env and the build log. The one in the env is longer. This most likely means that the values described by 'env' were set AFTER the agent started.
Could you please describe how/where you run the agent and set maven variables.

0
Comment actions Permalink

It's a good catch Sergey

I start the agent with a script located in /etc/inid.d
Maven variables are set in /etc/profile

After restarting the agent the variables are taken into accounts.

I don't know yet how to make sure /etc/profile is executed before /etc/init.d/buildAgent, but that's the trick.

Thank you SO MUCH for helping out on a system problem.

best rgds

EDIT:

/etc/environment seems to be a way

0
Comment actions Permalink

OK. Thank you for sharing. This is useful knowledge for future support cases.

0

Please sign in to leave a comment.