Keep getting java heap out of space

Hi all,

We have a teamcity server that is using the latest release (Calcutta 4.1 EAP 8725) keep getting the java.lang.OutOfMemoryError

We already change the memory to use 1024. But we still have the error.

TEAMCITY_SERVER_MEM_OPTS -Xms1024m -Xmx1024m -XX:MaxPermSize 384m

We are currently using both the svn and the starteam plugin, we suspect that the problem might come from the starteam plugin but we cannot find the source of the problem.

There does'nt seem to be any information in the log, can someone helps us with this ?

Thanks!

I have attached the log with this.



Attachment(s):
teamcity-server.log
9 comments
Comment actions Permalink

Please take a look here: http://www.jetbrains.net/confluence/display/TCD4/Reporting+Issues#ReportingIssues-OutOfMemoryProblems

I would also suggest to decrease -Xmx to 512m because this should be enough for the server to operate. Also it is not easy to load and analyze big memory snapshots.

0
Comment actions Permalink

Hi,

Is there a special place where i need to put the -XX:+HeapDumpOnOutOfMemoryError in order to use in ? I'm passing it in the TEAMCITY_SERVER_MEM_OPTS from the teamcity-server.sh script, but this does'nt seem to work. The server is now crashing  2 time a day and each time there is no memory dump generate.

Note: i'm using java 1.5.0_11


Thanks!

0
Comment actions Permalink

This option should work starting from java 1.5.0_07, .hprof files are created in the Tomcat working directory (usually it's TeamCity/bin if teamcity-server.sh script is started from this directory).

0
Comment actions Permalink

I've done a search on all my system and there was no .hprof that was generated, is there somewhere in the log where i can see the the arguments was properly passed to the jvm ?

Thanks!

0
Comment actions Permalink

Please try to start teamcity-server.sh script using the following command:
sh -x ./teamcity-server.sh start

You should see that option is passed to JVM. You can also try to update your Java to 1.6, probably this option support is platform dependent. And finally another possible way to obtain snapshot is to install our plugin: http://www.jetbrains.net/confluence/display/TW/Server+Profiling but in this case you will have to watch for memory usage and click on a button in profiler plugin to create memory dump.

0
Comment actions Permalink

Perfect! Using the sh -x teamcity-server.sh i was able to see that the arguments where not passed properly...

I have one more question, do i have to pass the -XX:+HeapDumpOnOutOFMemoryError  before the -server (at the beginning of the argument) or after or it doesn't matter ?


Thanks!

0
Comment actions Permalink

It does not matter

0
Comment actions Permalink

Ok now that the argument are passed properly i did'nt get the java heap out of space error.

Thanks for your help!

But i have another question, have you seen this error before :

java.io.FileNotFoundException: /d/work/TeamCityDATA/config/cae-teamcity-agent-plugins/project-config.xml (Too many open files)?

I have attached the log file.



Attachment(s):
teamcity-log.zip
0
Comment actions Permalink

Since you are using Unix I think the reason of this error is limitation applied by the OS on a number of opened sockets and files. Try to increase number of open files with help of ulimit command. May be this will help you too: http://www.unix.com/unix-advanced-expert-users/12540-too-many-files-open-questions.html

0

Please sign in to leave a comment.