Help needed with TeamCity 4.5 / ClearCase integration
Is there a way to speed up the ClearCase integration?
I am trying to setup a TeamCity environment for my development group to evaluate but I am having problems setting up ClearCase integration. I have read numerous posts and so far I have not been able to resolve my issue.
For the evaluation I have loaded the Csla.NET 3-6-3 Framework source and nUnit test set into a VOB named CI_EVAL_PVOB in ClearCase. I would like to demonstrate the tools ability to build the framework and execute the unit tests. I can successfully do this on my development machine using VS2008 and nUnit.
When I execute a build I see a tomcat6 process using 99% of my CPU resources. In trying to debug the issue and I have found that the teamcity-vcs.log shows that it is slooooowly (11h:44m:33s) crawling through the the source tree executing various cleartool commands like 'cleartool lsvtree -obs -all' and 'cleartool lshistory -all -since' which makes me wonder if I have the correct configuration since I can compile and test this code in under 5 minutes outside of TeamCity.
Here are the specifics:
TeamCity (4.5) is installed on a fully patched Windows 2003 server WM image, using all the default installation options.
I have a DynamicView created on that machine @ "m:\TeamCity_CI_EVAL_SQU_DEV"
I created a single project named "CSLA 3.6.x" with a single build configuration called "CSLA 3.6.x Build"
Version Control Settings
VCS Root Name: CSLA 3.6.x Framework
Type of VCS: ClearCase
ClearCase view path: M:\TeamCity_CI_EVAL_SQU_DEV
Relative path within the view: \CI_EVAL_SRC\CSLA 3.6.x\Framework\
Use ClearCase: UCM
Build Runner
Build Runner: sln2008
Solution file path: .\csla-build.sln
Targets: Rebuild
Configuration: Release
Update:
After trying numerous configurations to try and get a reasonable performance out of TeamCity 4.5 and ClearCase I copied the source to a VSS 6.0 repository and detached the ClearCase VCS Root and created a new VSS VCS Root. The source update took 7s and compiled in less than a minute.
The issue appears to be in the ClearCase plugin.
Attachment(s):
CSLA 3.6.x Build3.PNG
CSLA_3.6.x_CSLA_3.6.x_Build_3.log
Please sign in to leave a comment.
Christopher,
The first build in new ClearCase configuration can take much time because of loading whole project sources. In the further builds only changed files are loaded from ClearCase and non-changed files are loaded from the caches, so it takes the less time. But if the caches became incorrect (e.g. if you change your configuration settings or change config spec of corresponding ClearCase view) then TC need to load all sources from ClearCase again.
Hi,
I experienced that the cache become also incorrect when a ClearCase managed element version is removed (on a file on a directory).
The only solution I've found is to clear the cache on TeamCity server and make a manual update of the snapshot view.
Regards,
Olivier.
Olivier,
This problem seems to be fixed. Can you please upgrade your TC to 4.5.3? If it does not resolve the problem can you please provide me TC server debug logs, VCS root settings and your ClearCase cache files?
Maxim,
Thanks for your reply. I did notice that the source update was a little faster on subsequent build requests but only down to 6 hours or so. I still have not been able to get a good build either and with the slow turnaround time it makes things difficult to debug. Did I mention that the source update for the same code loaded into VSS took 7 seconds and the build completed in 41 seconds?
Chris
Christopher,
Can you please provide me your TC server debug logs for investigating the problem?
My original post did include the VCS log and a image of my build configuration page. Have you had a chance to review them? Let me express again that my biggest concern in the slowness of the "Updating Sources" functionality as it relates to ClearCase. In my original post I added the elapsed time to the "Updating Sources" log entry since there was not other indication that it was taking so long.
Attachment(s):
localhost.2009-07-01.log
jakarta_service_20090702.log
jakarta_service_20090701.log
catalina.2009-07-02.log
catalina.2009-07-01.log
Hi,
When I'll upgrade the server to a new TeamCity version (July maybe), I'll test (and send your files if error occurs again).
Thanks for your help.
Regards,
Olivier.
Christopher,
Your original post did include only build log, not VCS log. Please, see http://www.jetbrains.net/confluence/display/TCD4/Reporting+Issues#ReportingIssues-VersionControlDebugLogging for more information about VCS debug logs. Can you please collect VCS debug logs for your long-time build and provide it to me? (I need "teamcity-vcs.log" and "teamcity-clearcase.log" files)
Maxim,
I have attached the teamcity-vcs.log to this message but there is no teamcity-clearcase.log on my system. Please remember that I am now using a temporary VSS repository to continue my evaluation of your product so you will see VSS references in that file. If we can determine what is causing the performance problem with ClearCase I will be happy to go back to the VCS root I created earlier.
Chris
Attachment(s):
teamcity-vcs.log
Christopher,
Unfortunately, I need exactly teamcity-clearcase.log and teamcity-vcs.log with ClearCase log entries. Can you please go back to ClearCase VCS root, run your long-time build and provide me collected logs? Do not forget to enable debug information logging (see the link above).
Christopher,
To get teamcity-clearcase.log you will need to enable the log as described in the doc: http://www.jetbrains.net/confluence/display/TCD4/Reporting+Issues#ReportingIssues-ClearCase
Namely, uncomment Clearcase-related lines in the <TeamCity home>\conf\teamcity-server-log4j.xml file.
Thanks for your help.
I modified the log4j files as specified and reattached the ClearCase VCS Root I had created earlier. Even though my builds have been failing, they are doing so with in a few minutes. So I restored the original config files and have been busy actually working on my primary task again.
Hi Maxim / Olivier,
I have also had this exact problem on 4.5 (when a clearcase version is removed). To get around this, I've found that I've had to create a new clearcase snapshot view, clone the build configuration and create a new VCS root pointing at the new view - very frustrating as I lose all the history.
When you say you're clearing the cache, where exactly in the folder structure is that cache? (I'm running teamcity on windows) If I have this problem again I'd be great if I can avoid the clone / new view exercise.
Thanks in advance.
Nick.
Hi!
By the cache I mean the "caches" folder in the "system" folder from the TeamCity server data folder structure.
You can remove also temporary files and agent work folder content.
Olivier.