TeamCity server takes too much CPU and memory

Hello,

  You get java.lang.OutOfMemoryError: PermGen space error. That means, there is some memory issues with the server.
  Could you please increase JVM memory parameter for PermGen space, like  -XX:MaxPermSize=200m

  Please read more about OutOfMemoryError errors in our docs.

  Hope this helps,
  KIR

Le 26/04/2010 13:24, Kirill Maximov a ecrit :

Hello,

>

   You get java.lang.OutOfMemoryError: PermGen space error. That means, there is some memory issues with the server.
   Could you please increase JVM memory parameter for PermGen space, like  -XX:MaxPermSize=200m

Yep, it's done.

>

   Please read more about OutOfMemoryError errors in http://confluence.jetbrains.net/display/TCD5/Reporting+Issues#ReportingIssues-OutOfMemoryProblems.

>

   Hope this helps,

I hope too. I'll let you know if it happens again.

Gérald

Le 26/04/2010 13:24, Kirill Maximov a ecrit :

Hello,

>

   You get java.lang.OutOfMemoryError: PermGen space error. That means, there is some memory issues with the server.
   Could you please increase JVM memory parameter for PermGen space, like  -XX:MaxPermSize=200m

>

   Please read more about OutOfMemoryError errors in http://confluence.jetbrains.net/display/TCD5/Reporting+Issues#ReportingIssues-OutOfMemoryProblems.

It doesn't help.. Still the same issue.

I've read documentation, and set -XX:+HeadDumpOnOutOfMemoryError
parameter, I hope it will work.

Something funny, or I don't understand the documentation :

     Since TeamCity 5.1 you can dump memory snapshot right from the web UI:
go to the Administration | Server Configuration | Diagnostics tab and
click Dump Memory Snapshot.


How do I get a memory snapshot with webUI as server is taking too much
memory and process and doesn't respond? :)

Gérald

Could you please describe your current memory settings? Do you use 64-bit JVM or 32-bit?

As for memory dump on diagnostic page - of course you won't be able to use it if server does not respond, but still it is useful in some cases when server is given more memory than it should use, and you see the high memory usage.

Le 27/04/2010 19:40, Pavel Sher a ecrit :

Could you please describe your current memory settings? Do you use 64-bit JVM or 32-bit?

At first it was this :
#TEAMCITY_SERVER_MEM_OPTS="-Xmx512m -XX:MaxPermSize=150m"

Then I tried this :

1. Options recommended for dedicated server installation (commented by

default)
#TEAMCITY_SERVER_MEM_OPTS="-Xms750m -Xmx750m -XX:MaxPermSize=200m"

Now it's this :
TEAMCITY_SERVER_MEM_OPTS="-Xmx512m -XX:MaxPermSize=200m
-XX:+HeapDumpOnOutOfMemoryError"

It's 32 bits, Ubuntu 7.10, with 3Gb RAM

As for memory dump on diagnostic page - of course you won't be able to use it if server does not respond, but still it is useful in some cases when server is given more memory than it should use, and you see the high memory usage.

Ok.

Gérald

Ok, once you'll have memory dump please upload it to our ftp according to instructions about out of memory problems, and send name of the file to: teamcity-feedback[at]jetbrains.com

Le 27/04/2010 20:07, Pavel Sher a ecrit :

Ok, once you'll have memory dump please upload it to our ftp according to instructions about out of memory problems, and send name of the file to: teamcity-feedback[at]jetbrains.com

It's happening again (it's happening everyday :( ) but I don't see any
.hprof file in bin directory... Does it dumps itself? How long is it to
dump it?

Gérald

Le 28/04/2010 16:24, Gérald Fauvelle a ecrit :

Le 27/04/2010 20:07, Pavel Sher a ecrit :

>>  Ok, once you'll have memory dump please upload it to our ftp according to instructions about out of memory problems, and send name of the file to: teamcity-feedback[at]jetbrains.com
>

It's happening again (it's happening everyday :( ) but I don't see any
.hprof file in bin directory... Does it dumps itself? How long is it to
dump it?

How can I know if options I give are use by TeamCityserver?

If I do a ps to see the proces, I see this as a command line used to run
TeamCity:

/usr/lib/jvm/java-6-sun/bin/java
-Djava.util.logging.config.file=/usr/local/TeamCity/conf/logging.properties
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Djava.endorsed.dirs=/usr/local/TeamCity/endorsed -classpath
/usr/local/TeamCity/bin/bootstrap.jar
-Dcatalina.base=/usr/local/TeamCity -Dcatalina.home=/usr/local/TeamCity
-Djava.io.tmpdir=/usr/local/TeamCity/temp
org.apache.catalina.startup.Bootstrap start


Gérald

.hprof file must be created in the process working directory. Did you restart server after the error? If not, try to dump memory from the diagnostic tab and upload it to our ftp.

Le 28/04/2010 19:00, Pavel Sher a ecrit :

hprof file must be created in the process working directory.

> Did you restart server after the error?

Yes, I did it again.

> If not, try to dump memory from the diagnostic tab and upload
> it to our ftp.

Yeah, but the problem is that the server doesn't respond at all... It
takes 100% cpu and web interface is not loaded.

Gérald

TEAMCITY_MEM_OPTS are only used if TeamCity is started by our own .sh files, i.e. by teamcity-server.sh or by runAll.sh
If you deploy TeamCity war to already started Tomcat container or if you started TeamCity by startup.sh, then these settings won't be passed to JVM.
In the latter case you need to use JAVA_OPTS environment variable, I guess this is the case here.

Le 28/04/2010 19:36, Pavel Sher a ecrit :

TEAMCITY_MEM_OPTS are only used if TeamCity is started by our own .sh files, i.e. by teamcity-server.sh or by runAll.sh
If you deploy TeamCity war to already started Tomcat container or if you started TeamCity by startup.sh, then these settings won't be passed to JVM.
In the latter case you need to use JAVA_OPTS environment variable, I guess this is the case here.
  

I use teamcity-server.sh. I put some echo $TEAMCITY_MEM_OPTS in the
script, it seems the parameters are defined.

I've restarted the server, I'll come back when it happens again.

Gérald