TeamCity Comparison Matrix Errors

I'm a fan of TC, but I think there are some errors in the comparison matrix with CC.

No VCS checkout code in build script - CC offers this. It will do the checkout for you. This is how we use it.

View change log - I don't know what this is supposed to mean, but I can view all the changes for a given build on that build page in CC. It contains the log messages. It is also easy to set up CC to work with external tools for VCS viewing, which gives me an easy way to see diffs in the changes, which is a big advantage of CC over TC

Stop running build - This almost never seems to work for me. But I guess that is a bug and not a lack of the feature.

Browse build results - Unless you have a very specific concept in mind, CC does this.

Browse changes - They are on the build page under modifications.

"You broke the build" notification - You want to see my inbox with "You broke the build" notifications? Plus, CC supports RSS for this. So you should give CC an check here and add an item about RSS and give TC an X and CC a check.

Have some faith in your product. I'm always more impressed by an accurate comparision, including failings, than I am about a company that picks a set of features where it wins in almost every case.

--Tim

11 comments
Comment actions Permalink

Thank you for your comments, Tim. We will check comparison matrix once
more. Of course, we would like it to be correct.

Tim McNerney wrote:

I'm a fan of TC, but I think there are some errors in the comparison matrix with CC.

No VCS checkout code in build script - CC offers this. It will do the checkout for you. This is how we use it.

View change log - I don't know what this is supposed to mean, but I can view all the changes for a given build on that build page in CC. It contains the log messages. It is also easy to set up CC to work with external tools for VCS viewing, which gives me an easy way to see diffs in the changes, which is a big advantage of CC over TC

Stop running build - This almost never seems to work for me. But I guess that is a bug and not a lack of the feature.

Browse build results - Unless you have a very specific concept in mind, CC does this.

Browse changes - They are on the build page under modifications.

"You broke the build" notification - You want to see my inbox with "You broke the build" notifications? Plus, CC supports RSS for this. So you should give CC an check here and add an item about RSS and give TC an X and CC a check.

Have some faith in your product. I'm always more impressed by an accurate comparision, including failings, than I am about a company that picks a set of features where it wins in almost every case.

--Tim

0
Comment actions Permalink

No VCS checkout code in build script - CC offers this. It will do the
checkout for you. This is how we use it.


This must be an outdated information - it used to allow checking out only
the build script file. We'll correct this.

View change log - I don't know what this is supposed to mean, but I
can view all the changes for a given build on that build page in CC.
It contains the log messages.


Open page of some build configuration, find the section "History" and the
link "change log" there. It's another way to browse changes - not from builds,
but from commits. That's useful sometimes.

It is also easy to set up CC to work
with external tools for VCS viewing, which gives me an easy way to see
diffs in the changes, which is a big advantage of CC over TC


Built-in diff functionality will be added in the next version of TC.


Stop running build - This almost never seems to work for me. But I
guess that is a bug and not a lack of the feature.


Of course, this must be a bug.


Browse build results - Unless you have a very specific concept in
mind, CC does this.

Browse changes - They are on the build page under modifications.

"You broke the build" notification - You want to see my inbox with
"You broke the build" notifications? Plus, CC supports RSS for this.
So you should give CC an check here and add an item about RSS and give
TC an X and CC a check.


The above 3 lines are from "IDE integration" section. So they are related
to features available right from the IDE. We could make the layout of the
matrix better, however, to avoid such confusion.


Have some faith in your product. I'm always more impressed by an
accurate comparision, including failings, than I am about a company
that picks a set of features where it wins in almost every case.


We've tried to do an accurate comparision. Unfortunately, neither CruiseControl
nor CruiseControl.NET do not have any kind of "feature lists" on their sites.
So we've just included the features that we would use ourself, consider as
important. That's no surprise that most of the features that we would use
ourself or consider as important are already in our product :-).

Valentin Kipiatkov
Chief Scientist, Vice President of Product Development
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"

I'm a fan of TC, but I think there are some errors in the comparison
matrix with CC.

No VCS checkout code in build script - CC offers this. It will do the
checkout for you. This is how we use it.

View change log - I don't know what this is supposed to mean, but I
can view all the changes for a given build on that build page in CC.
It contains the log messages. It is also easy to set up CC to work
with external tools for VCS viewing, which gives me an easy way to see
diffs in the changes, which is a big advantage of CC over TC

Stop running build - This almost never seems to work for me. But I
guess that is a bug and not a lack of the feature.

Browse build results - Unless you have a very specific concept in
mind, CC does this.

Browse changes - They are on the build page under modifications.

"You broke the build" notification - You want to see my inbox with
"You broke the build" notifications? Plus, CC supports RSS for this.
So you should give CC an check here and add an item about RSS and give
TC an X and CC a check.

Have some faith in your product. I'm always more impressed by an
accurate comparision, including failings, than I am about a company
that picks a set of features where it wins in almost every case.

--Tim



0
Comment actions Permalink

No VCS checkout code in build script - CC offers

this. It will do the

checkout for you. This is how we use it.


This must be an outdated information - it used to
allow checking out only
the build script file. We'll correct this.


That may be one suggested way to use it, but it has allowed for full updates via CC for years now.

View change log - I don't know what this is

supposed to mean, but I

can view all the changes for a given build on that

build page in CC.

It contains the log messages.


Open page of some build configuration, find the
section "History" and the
link "change log" there. It's another way to browse
changes - not from builds,
but from commits. That's useful sometimes.


Okay. So basically the modifications section all the builds on one page. Seems nice mainly for the Pending changes section, but still, not in CC. But again, pretty minor. I'm sure there are plenty of things somewhat different in CC or Luntbuild, etc that aren't in TC that aren't really a huge advantage, but just a different way of looking at things.



Browse build results - Unless you have a very

specific concept in

mind, CC does this.

Browse changes - They are on the build page under

modifications.


"You broke the build" notification - You want to

see my inbox with

"You broke the build" notifications? Plus, CC

supports RSS for this.

So you should give CC an check here and add an item

about RSS and give

TC an X and CC a check.


The above 3 lines are from "IDE integration" section.
So they are related
to features available right from the IDE. We could
make the layout of the
matrix better, however, to avoid such confusion.


Okay. Then this is really padding. If you already have a checkbox for IDE Integration and only TC has it, of course "Notification to IDE" is going to be absent on CC.


Have some faith in your product. I'm always more

impressed by an

accurate comparision, including failings, than I am

about a company

that picks a set of features where it wins in

almost every case.

We've tried to do an accurate comparision.
Unfortunately, neither CruiseControl
nor CruiseControl.NET do not have any kind of
"feature lists" on their sites.
So we've just included the features that we would use
ourself, consider as
important. That's no surprise that most of the
features that we would use
ourself or consider as important are already in our
product :).


Granted. And I appreciate that not only do you offer such a list, but you listen to others to keep it correct. But I would have thought you guys would have spent some time working with the other tools available to learn what they do well and poorly. At least enough to be able to make an accurate list.

The features you have beyond CC are huge. And this is in v1. You're rightly proud of your work. But having looked at dozens of these comparison charts put up by vendors, I have to tell you that I never trust one that doesn't have a lot of X's for the featured product. It demonstrates a lack of knowledge of the other products or an attempt to make the feature product look good. But for me, it just makes the comparison questionable.

--Tim

0
Comment actions Permalink

Oh, and let's not forget the following features which have been huge for us in our use of CC:

  • Free

  • Completely customizable/extendable.


--Tim

0
Comment actions Permalink

Granted. And I appreciate that not only do you offer such a list, but
you listen to others to keep it correct. But I would have thought you
guys would have spent some time working with the other tools available
to learn what they do well and poorly. At least enough to be able to
make an accurate list.


Here we've used 2 tools: CruiseControl (for Java development) and Draco.NET
(for .NET development). We did not include Draco.NET because it seems to
be much less popular than CruiseControl.NET (but I can be wrong in this).

Our Java teams have been using CruiseControl for a few years and if there
were any feature of CruiseControl
that they were missing in TC we would definitely include it into the feature
matrix (but better implement it in TC). Of course, they do not know all the
features of Cruise Control.

As for Draco.NET, we suffered because we missed lot of capabilities in it
(missing in CruiseControl and CruiseControl.NET as well). That led us to
producing a set of internal tools built on top of Draco that added some of
the missing capabilities. Some of the ideas of those tools were used in TC
development.

Valentin Kipiatkov
Chief Scientist, Vice President of Product Development
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"

>>> No VCS checkout code in build script - CC offers
>>>
>> this. It will do the
>>
>>> checkout for you. This is how we use it.
>>>
>> This must be an outdated information - it used to
>> allow checking out only
>> the build script file. We'll correct this.

That may be one suggested way to use it, but it has allowed for full
updates via CC for years now.

>>> View change log - I don't know what this is
>>>
>> supposed to mean, but I
>>
>>> can view all the changes for a given build on that
>>>
>> build page in CC.
>>
>>> It contains the log messages.
>>>
>> Open page of some build configuration, find the
>> section "History" and the
>> link "change log" there. It's another way to browse
>> changes - not from builds,
>> but from commits. That's useful sometimes.

Okay. So basically the modifications section all the builds on one
page. Seems nice mainly for the Pending changes section, but still,
not in CC. But again, pretty minor. I'm sure there are plenty of
things somewhat different in CC or Luntbuild, etc that aren't in TC
that aren't really a huge advantage, but just a different way of
looking at things.

>>> Browse build results - Unless you have a very
>>>
>> specific concept in
>>
>>> mind, CC does this.
>>>
>>> Browse changes - They are on the build page under
>>>
>> modifications.
>>
>>> "You broke the build" notification - You want to
>>>
>> see my inbox with
>>
>>> "You broke the build" notifications? Plus, CC
>>>
>> supports RSS for this.
>>
>>> So you should give CC an check here and add an item
>>>
>> about RSS and give
>>
>>> TC an X and CC a check.
>>>
>> The above 3 lines are from "IDE integration" section.
>> So they are related
>> to features available right from the IDE. We could
>> make the layout of the
>> matrix better, however, to avoid such confusion.

Okay. Then this is really padding. If you already have a checkbox for
IDE Integration and only TC has it, of course "Notification to IDE" is
going to be absent on CC.

>>> Have some faith in your product. I'm always more
>>>
>> impressed by an
>>
>>> accurate comparision, including failings, than I am
>>>
>> about a company
>>
>>> that picks a set of features where it wins in
>>>
>> almost every case.
>>
>> We've tried to do an accurate comparision.
>> Unfortunately, neither CruiseControl
>> nor CruiseControl.NET do not have any kind of
>> "feature lists" on their sites.
>> So we've just included the features that we would use
>> ourself, consider as
>> important. That's no surprise that most of the
>> features that we would use
>> ourself or consider as important are already in our
>> product :-).

Granted. And I appreciate that not only do you offer such a list, but
you listen to others to keep it correct. But I would have thought you
guys would have spent some time working with the other tools available
to learn what they do well and poorly. At least enough to be able to
make an accurate list.

The features you have beyond CC are huge. And this is in v1. You're
rightly proud of your work. But having looked at dozens of these
comparison charts put up by vendors, I have to tell you that I never
trust one that doesn't have a lot of X's for the featured product. It
demonstrates a lack of knowledge of the other products or an attempt
to make the feature product look good. But for me, it just makes the
comparison questionable.

--Tim



0
Comment actions Permalink

Oh, and let's not forget the following features which have been huge
for us in our use of CC:

  • Free


That's not a feature of the product itself.

  • Completely customizable/extendable.


What exactly do you need to customize/extend in TC?

Valentin Kipiatkov
Chief Scientist, Vice President of Product Development
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"

Oh, and let's not forget the following features which have been huge
for us in our use of CC:

  • Free

  • Completely customizable/extendable.

--Tim



0
Comment actions Permalink

As stated in my other posts CC has two features that TC appears to lack at the moment (sorry to keep harping on about this - they are important features for me)...

1. The ability to ignore files for vcn triggered builds.
2. The ability to work with a vcn 'working directory'.

I realise I can write my own build target to perform my build checkout - which will create a working directory. However, this decouples that build configuration from vcn triggering.

Please correct me if I'm wrong about this!

I'll bet there's a lot of CC users out there who manipulate their vcn from within their builds that are going to find this a problem too.

I'm thinking that perhaps TC is based on JetBrains internal use of CC as opposed to it's full feature set? It just doesn't seem as flexible in some areas at the moment, while in others it's way ahead.

Paul

0
Comment actions Permalink
  • Free


That's not a feature of the product itself.


True.

  • Completely customizable/extendable.


What exactly do you need to customize/extend in TC?


Support for other testing frameworks. On the results page in CruiseControl, they have a summary of the build, much as you have viewLog page. It is trivial to add support for TestNG or a custom testing framework so that the results appear as JUnit does.

We do the same with Checkstyle. We want to have checkstyle errors appear on the buildresults page so it is easy to fix when a build fails because of it.

You have code cover results on this page. We currently use Clover. To view the Clover coverage results in a manner similar to in TC on CC, it would be trivial to add it to the buildresults page. In TC, we'd have to put the results in the Artefacts.

If you use Fisheye or WinCVS or the like, it is easy to modify the Changes section of the buildresults file to link to the page so you can quickly see how the file changed. I know you have plans to add this, but I can do it in CC now because I have the source code and the results page is mostly XSLT transforms from the log file, which is pretty easy to work with.

These are just some of the examples of how I've modified our CC to customize it for our build practice which are a big win. TC does lots of stuff right, but in the end, I have to rely on Jetbrains for all the functionality.

This is why I opened:

http://www.jetbrains.net/jira/browse/TW-370

It solves some of the customization issues that I've run into. But generally, the idea is that there is currently no end user extensibility of TC.

--Tim

0
Comment actions Permalink


Unfortunately, neither CruiseControl
nor CruiseControl.NET do not have any kind of
"feature lists" on their sites.
...


Are you familiar with the Continuous Integration Server Feature Matrix?

http://damagecontrol.codehaus.org/ContinuousIntegrationServerFeatureMatrix

I'm actually a bit surprised that someone hasn't added TeamCity already...

Jtf

0
Comment actions Permalink

... sorry dupe post ...

0
Comment actions Permalink

Are you familiar with the Continuous Integration Server Feature
Matrix?

http://damagecontrol.codehaus.org/ContinuousIntegrationServer+Featur
e+Matrix


Yes, we've seen it.

Valentin Kipiatkov
CTO and Chief Scientist
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"

>> Unfortunately, neither CruiseControl
>> nor CruiseControl.NET do not have any kind of
>> "feature lists" on their sites.
>> ...

Are you familiar with the Continuous Integration Server Feature
Matrix?

http://damagecontrol.codehaus.org/ContinuousIntegrationServer+Featur
e+Matrix

I'm actually a bit surprised that someone hasn't added TeamCity
already...

Jtf



0

Please sign in to leave a comment.