Working with 1382

I've installed and started working with build 1382. I have some questions and some comments.

First is about using a different JDK than is used for the build agent. I brought this up before (http://www.intellij.net/forums/thread.jspa?threadID=222959&tstart=15) and have followed the jira issue (http://www.jetbrains.net/jira/browse/TW-307) which says it is fixed in 1382, but I have been unable to get everything set correctly to work. I edited the config by hand, set up the as described (and in a variety of ways as not described) and none of them seem to work quite right. The only change I see from the log is that it is trying to use "java" instead of /usr/java/jdk1.5.0/bin/java", which is what I have JAVA_HOME set to on the server for running the agent. I tried adding: in the project-config.xml and in buildAgent.properties: system.JAVA_HOME_14=/usr/java/j2sdk1.4.2 But this didn't seem to work. I tried lots of different variations, though none of them seemed to do the trick. I even tried to do this explicitly (type 2 from the jira issue) and had: but this did not work, either. The first version is what I want, since we have 1.4 JDKs in different places on different machines (and different versions), but I was just trying to get either to work. As I mentioned, we do have JAVA_HOME set, to get the agent working and the java in the path is not the 1.4.2 version. Enough about that. We have several top level CVS modules for most applications. We bundle these into a CVS alias so that we get both the common modules and the application specific modules through one CVS "project". Using this works well in TeamServer if I just pass the project alias. So for example, we'd have the top level modules: app1a app1b common1 common2 And these are all contained in the CVS alias app1. We might also have: app2a app2b common1 common2 And these are contained in CVS alias app2. If we use "app1" and "app2" in the configuration, we end up with all these modules at the same level. The problem is that in general, the apps are being built against a tag or branch of common1 and common2, so I want to do a checkout of: app1a - HEAD app1b - HEAD common1 - REL1 common2 - REL1 And have them all at the same directory level. I know there was an example shown at one point that showed working with branches, but I can no longer find it, nor do I recall if it addressed this. The FAQ doesn't seem to cover branching in any detail. Can you give me some info on how to accomplish the above? Also along these lines, would it necessarily be the case that we would have to create separate toplevel projects for each variation of branches. Say I wanted to have a build with the previous example and one with: app1b - HEAD app1b - HEAD common1 - REL1_1 common2 - REL1_1 Do they need different projects? Is there a way to group these projects somehow, as they are closely related as opposed to a build for app2. A couple of comments. It would be nice to see the run time of each build on the viewType.html?buildTypeId=Apex_Build page. The reason being that it allows one to classify the failures to some extent since there is not necessarily any information other than "Failure" under the status (the only exception being "Tests failed:" as far as I can see). If I see a whole lot of builds listed and I have the build time available, I can make a reasonable estimate of which one made it furthest and which one failed immediately. The immediate failure is not likely of as much interest as the one which made it further, unless it is the most recent build. Another question. In cruisecontrol, it is possible to set up a project so that it gets requeued after a failure even if thee haven't been additional checkins. This is our preferred setting, as we occasionally have transient failures which clear up with just a rebuild. Is there a way to set this? We started using MySQL as the backend. It seems to work much better. Is this configuration preferred over hsql? I notice that the "log_messages" table is growing very quickly. Is it cleaned up through the "Data Cleanup Policy" process? I do like the policy set up. It is fairly simple, but covers most of what I want. I currently do something slightly different in CC, which is successful builds for a month and failed builds for a week, but I think this may work for me just as well. How does it work if you have a policy for "All" and then one for a specific configuration? It strikes me that having the CVS repository path as part of the hierarchy under the work directory is unnecessary and since it needs to be added to the Artifacts paths and Build file path, you end up with a lot more typing than necessary. From the example above, I'd think: %build.working.dir%/app1a/

would be sufficient, rather than: %build.working.dir%//app1a/ For that matter, I would think: app1a/]]>

would be sufficient and most likely the more common usage and that if for some reason, an absolute path is needed, it should be easy enough to pick up the leading '/' to know not to work relative to the work directory.

I'm still a bit hazy on "Properties" vs. "environment" variables and how both end up used in the build.

Pausing projects is a bit difficult as it is a few levels in. It would be nice to have the projects page offer both "Run" and "Pause" buttons.

It would be nice to be able to create administrators for particular projects, just as it is possible to set a user to view particular projects.

It would be nice to be able to do some management of the build agents from a centralized spot. Say disabling them, setting them to run at certain times of the day.

More ability to configure the Projects view would be nice. Things such as filtering out successful builds and/or broken builds with someone taking responsibility. A collapse all for builds within a project. A top level collapse for a project which shows some metrics for that project.

I think that is probably well more than enough for now. Hope I wasn't too rambling.

It's coming along very well. I hope within the next build or two to start running TS in parallel with CC.

--Tim

8 comments
Comment actions Permalink

Hello Tim,
Concerning target JDK:
I was not able to reproduce the problem - everything works ok for me. The fact that the server allows the build to be started on
this agent means that the "system.JAVA_HOME_14" property is found by the agent, reported to the server and successfully matched with
the target-jdk requirement.

The only change I see from the log is that it is trying to use "java" instead


My guess is that for some reason the file "/usr/java/j2sdk1.4.2/bin/java" is considered as non-existent, so the default value "java"
is used.
Could you please check whether this is the case?

--
Best regards,
Eugene Zhuravlev
Software Developer
JetBrains Inc.
http://www.jetbrains.com
"Develop with pleasure!"


0
Comment actions Permalink


A couple of comments. It would be nice to see the run time of each build on the viewType.html?buildTypeId=Apex_Build page. The reason being that it allows one to classify the failures to some extent since there is not necessarily any information other than "Failure" under the status (the only exception being "Tests failed:" as far as I can see). If I see a whole lot of builds listed and I have the build time available, I can make a reasonable estimate of which one made it furthest and which one failed immediately. The immediate failure is not likely of as much interest as the one which made it further, unless it is the most recent build.


Currently, this information is available on "all history" page. Probably we should put it back to
build configuration page.


Another question. In cruisecontrol, it is possible to set up a project so that it gets requeued after a failure even if thee haven't been additional checkins. This is our preferred setting, as we occasionally have transient failures which clear up with just a rebuild. Is there a way to set this?


See/Vote for http://www.jetbrains.net/jira/browse/TW-448


We started using MySQL as the backend. It seems to work much better. Is this configuration preferred over hsql?


For environments with dozen of agents and projects - definetely yes.


I notice that the "log_messages" table is growing very quickly. Is it cleaned up through the "Data Cleanup Policy" process? I do like the policy set up. It is fairly simple, but covers most of what I want. I currently do something slightly different in CC, which is successful builds for a month and failed builds for a week, but I think this may work for me just as well. How does it work if you have a policy for "All" and then one for a specific configuration?


We plan to remove this table and use external files instead. Yes, this table is cleaned when data
cleanup is run.


It strikes me that having the CVS repository path as part of the hierarchy under the work directory is unnecessary and since it needs to be added to the Artifacts paths and Build file path, you end up with a lot more typing than necessary. From the example above, I'd think:

%build.working.dir%/app1a/<dir>

would be sufficient, rather than:

%build.working.dir%/<repository_path>/app1a/<dir>

For that matter, I would think:

app1a/<dir>

would be sufficient and most likely the more common usage and that if for some reason, an absolute path is needed, it should be easy enough to pick up the leading '/' to know not to work relative to the work directory.


In fact, you can omit %build.working.dir% in the path.


I'm still a bit hazy on "Properties" vs. "environment" variables and how both end up used in the build.


We'll add some help regarding this issue. Properties are passed via -Dname=value and environment,
well.. as environment variables to the build process.


Pausing projects is a bit difficult as it is a few levels in. It would be nice to have the projects page offer both "Run" and "Pause" buttons.


Currently you may pause build configuration either from build configuration page, or from
administration interface. Do you mean you nead a possibility to pause a project with all it's build
configurations?


It would be nice to be able to create administrators for particular projects, just as it is possible to set a user to view particular projects.


Please See/Comment/Vote for http://www.jetbrains.net/jira/browse/TW-372


It would be nice to be able to do some management of the build agents from a centralized spot. Say disabling them, setting them to run at certain times of the day.


We plan to provide JMX-style configuration for build agents, but not for 1.0 release. Please
comment for http://www.jetbrains.net/jira/browse/TW-302 .


More ability to configure the Projects view would be nice. Things such as filtering out successful builds and/or broken builds with someone taking responsibility. A collapse all for builds within a project. A top level collapse for a project which shows some metrics for that project.


Please file requests for ideas you find the most attractive.


I think that is probably well more than enough for now. Hope I wasn't too rambling.


Thanks for your feedback, we appreciate it a lot.


It's coming along very well. I hope within the next build or two to start running TS in parallel with CC.


Glad to hear it!

Kind regards,
KIR


--
Kirill Maximov
Software Developer
JetBrains, Inc.
http://www.jetbrains.com
"Develop with pleasure!"

0
Comment actions Permalink

1) For each module (VCS root) you have to configure "checkout path" -
directory with the specified relative path will be created in the working
directory and sources from the module will be checked out into the
directory. For each cvs root you can configure corresponding branch name.
For example, you can create vcs roots:
checkout path: 'app' cvs: module 'App'
checkout path: 'common' module 'Common'
checkout path: empty module 'Main'

Content of module Main will be checked out into working directory. In this
directory will be created directories 'common' and 'app'. Content of the
directories will be checked out from the corresponding modules (using
specified branches).

2) Team server does not support "branches" (unlike cvs support), so, if
there are several projects and the only difference is branche name you have
to create one project per each branch (or set of branches).

And these are contained in CVS alias app2. If we use "app1" and "app2" in
the configuration, we end up with all these modules at the same level. The
problem is that in general, the apps are being built against a tag or
branch of common1 and common2, so I want to do a checkout of:

>

app1a - HEAD
app1b - HEAD
common1 - REL1
common2 - REL1

>

And have them all at the same directory level. I know there was an example
shown at one point that showed working with branches, but I can no longer
find it, nor do I recall if it addressed this. The FAQ doesn't seem to
cover branching in any detail. Can you give me some info on how to
accomplish the above?




"Tim McNerney" <intellij@oneofus.org> wrote in message
news:4050648.1151516092164.JavaMail.itn@is.intellij.net...

I've installed and started working with build 1382. I have some questions
and some comments.

>

First is about using a different JDK than is used for the build agent. I
brought this up before
(http://www.intellij.net/forums/thread.jspa?threadID=222959&tstart=15) and
have followed the jira issue (http://www.jetbrains.net/jira/browse/TW-307)
which says it is fixed in 1382, but I have been unable to get everything
set correctly to work. I edited the config by hand, set up the
<target-jdk> as described (and in a variety of ways as not described) and
none of them seem to work quite right. The only change I see from the log
is that it is trying to use "java" instead of
/usr/java/jdk1.5.0/bin/java", which is what I have JAVA_HOME set to on the
server for running the agent.

>

I tried adding:

>

<requirements>
<target-jdk name="%system.JAVA_HOME_14%" />
</requirements>

>

in the project-config.xml and in buildAgent.properties:

>

system.JAVA_HOME_14=/usr/java/j2sdk1.4.2

>

But this didn't seem to work. I tried lots of different variations, though
none of them seemed to do the trick. I even tried to do this explicitly
(type 2 from the jira issue) and had:

>

<requirements>
<target-jdk name="/usr/java/j2sdk1.4.2" />
</requirements>

>

but this did not work, either.

>

The first version is what I want, since we have 1.4 JDKs in different
places on different machines (and different versions), but I was just
trying to get either to work.

>

As I mentioned, we do have JAVA_HOME set, to get the agent working and the
java in the path is not the 1.4.2 version.

>

Enough about that. We have several top level CVS modules for most
applications. We bundle these into a CVS alias so that we get both the
common modules and the application specific modules through one CVS
"project". Using this works well in TeamServer if I just pass the project
alias. So for example, we'd have the top level modules:

>

app1a
app1b
common1
common2

>

And these are all contained in the CVS alias app1. We might also have:

>

app2a
app2b
common1
common2

>

And these are contained in CVS alias app2. If we use "app1" and "app2" in
the configuration, we end up with all these modules at the same level. The
problem is that in general, the apps are being built against a tag or
branch of common1 and common2, so I want to do a checkout of:

>

app1a - HEAD
app1b - HEAD
common1 - REL1
common2 - REL1

>

And have them all at the same directory level. I know there was an example
shown at one point that showed working with branches, but I can no longer
find it, nor do I recall if it addressed this. The FAQ doesn't seem to
cover branching in any detail. Can you give me some info on how to
accomplish the above?

>

Also along these lines, would it necessarily be the case that we would
have to create separate toplevel projects for each variation of branches.
Say I wanted to have a build with the previous example and one with:

>

app1b - HEAD
app1b - HEAD
common1 - REL1_1
common2 - REL1_1

>

Do they need different projects? Is there a way to group these projects
somehow, as they are closely related as opposed to a build for app2.

>

A couple of comments. It would be nice to see the run time of each build
on the viewType.html?buildTypeId=Apex_Build page. The reason being that it
allows one to classify the failures to some extent since there is not
necessarily any information other than "Failure" under the status (the
only exception being "Tests failed:" as far as I can see). If I see a
whole lot of builds listed and I have the build time available, I can make
a reasonable estimate of which one made it furthest and which one failed
immediately. The immediate failure is not likely of as much interest as
the one which made it further, unless it is the most recent build.

>

Another question. In cruisecontrol, it is possible to set up a project so
that it gets requeued after a failure even if thee haven't been additional
checkins. This is our preferred setting, as we occasionally have transient
failures which clear up with just a rebuild. Is there a way to set this?

>

We started using MySQL as the backend. It seems to work much better. Is
this configuration preferred over hsql?

>

I notice that the "log_messages" table is growing very quickly. Is it
cleaned up through the "Data Cleanup Policy" process? I do like the policy
set up. It is fairly simple, but covers most of what I want. I currently
do something slightly different in CC, which is successful builds for a
month and failed builds for a week, but I think this may work for me just
as well. How does it work if you have a policy for "All" and then one for
a specific configuration?

>

It strikes me that having the CVS repository path as part of the hierarchy
under the work directory is unnecessary and since it needs to be added to
the Artifacts paths and Build file path, you end up with a lot more typing
than necessary. From the example above, I'd think:

>

%build.working.dir%/app1a/<dir>

>

would be sufficient, rather than:

>

%build.working.dir%/<repository_path>/app1a/<dir>

>

For that matter, I would think:

>

app1a/<dir>

>

would be sufficient and most likely the more common usage and that if for
some reason, an absolute path is needed, it should be easy enough to pick
up the leading '/' to know not to work relative to the work directory.

>

I'm still a bit hazy on "Properties" vs. "environment" variables and how
both end up used in the build.

>

Pausing projects is a bit difficult as it is a few levels in. It would be
nice to have the projects page offer both "Run" and "Pause" buttons.

>

It would be nice to be able to create administrators for particular
projects, just as it is possible to set a user to view particular
projects.

>

It would be nice to be able to do some management of the build agents from
a centralized spot. Say disabling them, setting them to run at certain
times of the day.

>

More ability to configure the Projects view would be nice. Things such as
filtering out successful builds and/or broken builds with someone taking
responsibility. A collapse all for builds within a project. A top level
collapse for a project which shows some metrics for that project.

>

I think that is probably well more than enough for now. Hope I wasn't too
rambling.

>

It's coming along very well. I hope within the next build or two to start
running TS in parallel with CC.

>

--Tim



0
Comment actions Permalink

My guess is that for some reason the file
"/usr/java/j2sdk1.4.2/bin/java" is considered as
non-existent, so the default value "java"
is used.
Could you please check whether this is the case?


Wow. That was it. I wasn't expecting you guys to be so careful. I did something wrong the first time, so I then tried using nonsense strings with several variations so I could be sure that it was the value being passed and not picked up via some other source and which particular value I was passing. Habit of mine as it is very easy to grep for it in logs and I was expecting the errors would make it easy to determine that the value had made it through.

--Tim

0
Comment actions Permalink

1) For each module (VCS root) you have to configure
"checkout path" -
directory with the specified relative path will be
created in the working
directory and sources from the module will be checked
out into the
directory. For each cvs root you can configure
corresponding branch name.
For example, you can create vcs roots:
checkout path: 'app' cvs: module 'App'
checkout path: 'common' module 'Common'
checkout path: empty module 'Main'

Content of module Main will be checked out into
working directory. In this
directory will be created directories 'common' and
'app'. Content of the
directories will be checked out from the
corresponding modules (using
specified branches).


Got it. I will try and put together such a project for one of our HEAD/branch hybrid builds.


2) Team server does not support "branches" (unlike
cvs support), so, if
there are several projects and the only difference is
branche name you have
to create one project per each branch (or set of
branches).


Okay. That's what I thought.

Thanks.

--Tim

0
Comment actions Permalink

We started using MySQL as the backend. It seems to

work much better. Is this configuration preferred
over hsql?

For environments with dozen of agents and projects
- definetely yes.


Thanks.


I'm still a bit hazy on "Properties" vs.

"environment" variables and how both end up used in
the build.

We'll add some help regarding this issue.
Properties are passed via -Dname=value and
environment,
ell.. as environment variables to the build process.

>

Thanks. I think this is mostly a matter of documentation. Though some more complex example configuration files would be nice. I know you posted your internal build configuration, but that is mostly Perforce and SVN. It would be great if you could extend what you have currently with the builds for the open source projects to include the following:

Building against a branch/tag in CVS/Subversion.
Build with multiple repositories in CVS/Subversion.
Build with different roots.
Multiple builds for a project.
Use of properties to alter the build.
A build that has certain requirements (and have the buildAgent.properties contain that requirement, though commented out with directions on how to enable it).


Pausing projects is a bit difficult as it is a few

levels in. It would be nice to have the projects page
offer both "Run" and "Pause" buttons.

Currently you may pause build configuration either
from build configuration page, or from
dministration interface. Do you mean you nead a
possibility to pause a project with all it's build
configurations?


No. It's just a UI thing where it would be nice to have a view of all the projects/builds and be able to Pause them all from that one screen, just like you can run them all.


More ability to configure the Projects view would

be nice. Things such as filtering out successful
builds and/or broken builds with someone taking
responsibility. A collapse all for builds within a
project. A top level collapse for a project which
shows some metrics for that project.

Please file requests for ideas you find the most
attractive.


http://www.jetbrains.net/jira/browse/TW-529
http://www.jetbrains.net/jira/browse/TW-531
http://www.jetbrains.net/jira/browse/TW-532

--Tim

0
Comment actions Permalink

Tim McNerney wrote:


Thanks. I think this is mostly a matter of documentation. Though some more complex example configuration files would be nice. I know you posted your internal build configuration, but that is mostly Perforce and SVN. It would be great if you could extend what you have currently with the builds for the open source projects to include the following:

Building against a branch/tag in CVS/Subversion.
Build with multiple repositories in CVS/Subversion.
Build with different roots.
Multiple builds for a project.
Use of properties to alter the build.
A build that has certain requirements (and have the buildAgent.properties contain that requirement, though commented out with directions on how to enable it).


Honestly, I don't think there will be a need to provide example configuration files, because
we've implemented web-based interface for configuring the builds.
But we plan to add some online help for the configuration settings and there will be some examples.


>>> Pausing projects is a bit difficult as it is a few
>> levels in. It would be nice to have the projects page
>> offer both "Run" and "Pause" buttons.
>>
>> Currently you may pause build configuration either
>> from build configuration page, or from
>> dministration interface. Do you mean you nead a
>> possibility to pause a project with all it's build
>> configurations?


No. It's just a UI thing where it would be nice to have a view of all the projects/builds and be able to Pause them all from that one screen, just like you can run them all.


Why would you need this? What is a use case when you need to pause several builds at once manually?


>>
>> Please file requests for ideas you find the most
>> attractive.


http://www.jetbrains.net/jira/browse/TW-529
http://www.jetbrains.net/jira/browse/TW-531
http://www.jetbrains.net/jira/browse/TW-532


Thanks for the reports!


--Tim



--
Kirill Maximov
Software Developer
JetBrains, Inc.
http://www.jetbrains.com
"Develop with pleasure!"

0
Comment actions Permalink

Tim McNerney wrote:


Thanks. I think this is mostly a matter of

documentation. Though some more complex example
configuration files would be nice. I know you posted
your internal build configuration, but that is mostly
Perforce and SVN. It would be great if you could
extend what you have currently with the builds for
the open source projects to include the following:


Building against a branch/tag in CVS/Subversion.
Build with multiple repositories in CVS/Subversion.
Build with different roots.
Multiple builds for a project.
Use of properties to alter the build.
A build that has certain requirements (and have the

buildAgent.properties contain that requirement,
though commented out with directions on how to enable
it).

Honestly, I don't think there will be a need to
provide example configuration files, because
we've implemented web-based interface for
configuring the builds.
But we plan to add some online help for the
configuration settings and there will be some
examples.


Perhaps, but at this point, I've been unable to successfully set up a build with multiple repositories or to get a build with a CVS branch working correctly. I'll try again with the next EAP and see if I have more luck. I found the sample config file very helpful at first (necessary, really) and I still find it useful.


>>> Pausing projects is a bit difficult as it is a
few
>> levels in. It would be nice to have the projects
page
>> offer both "Run" and "Pause" buttons.
>>
>> Currently you may pause build configuration either
>> from build configuration page, or from
>> dministration interface. Do you mean you nead a
>> possibility to pause a project with all it's build

>> configurations?


No. It's just a UI thing where it would be nice to

have a view of all the projects/builds and be able to
Pause them all from that one screen, just like you
can run them all.

Why would you need this? What is a use case when
you need to pause several builds at once manually?


It's less about pausing several builds at once and more about having the ability to pause at that level seems more natural. Not a big thing and maybe once I get things in place and working well, it will be less of an issue. I did need it recently because several of the builds had bad configs and there was no way they were going to build correctly and I didn't want them taking up all of the build time.

--Tim

0

Please sign in to leave a comment.