forced ulimit of 4096/4096 causing Gatling tests to fail: java.net.SocketException: Too many open files

Greetings,

I'm using Gatling to run load tests against our REST services, and have run into an unexpected issue, given that the linux agents are configured with /etc/security/limits.conf:

@teamcity soft nofile 8192
@teamcity hard nofile 16384

When I run a simple build script to report ulimit -Hn and ulimit -Sn, it reports 4096 and 4096, respectively.

We even tried using a wildcard, instead of limiting it to the teamcity user.

Is TeamCity over-riding the ulimit somehow? Can I prevent or configure this over-ride?

Thanks!
--sky

6 comments
Comment actions Permalink

Did you update these values on all TC agents? TC doesn't run builds by itself. Only agents do that.

0
Comment actions Permalink

Yes, all build servers are managed using ansible.

0
Comment actions Permalink

what would it print, if you add ulimit -Hn in the beginning of agent.sh script?

0
Comment actions Permalink

Added ulimit -Hn at top of agent.sh...

[root@novb-core15 init.d]# service teamcity-agent start
16384
Starting TeamCity build agent...
Java executable is found in '/usr/lib/jvm/jre-1.7.0-openjdk.x86_64'.
Starting TeamCity Build Agent Launcher...
Agent home directory is /opt/teamcity
Current Java runtime version is 1.7
Lock file: /opt/teamcity/logs/buildAgent.properties.lock
Using no lock
Done [4298], see log at /opt/teamcity/logs/teamcity-agent.log

0
Comment actions Permalink

Apparently, the agent was not being restarted as part of provisioning. Will be adding this to the playbook.

Thanks, all!

0
Comment actions Permalink

TC's command line script is executed in the same environment as build agent starts. If script does something wrong - 99% that it was started in wrong environment.

0

Please sign in to leave a comment.