[erlang-questions] {error,emfile}

[Roberto Ostinelli]

Dear all,
I've built a benchmarking tool and I experience an error when connecting
more than 1024 clients to a SSL server. The call to ssl:connect/3 returns a
{error,emfile} instead of a Socket.

According to inet POSIX Error Codes [1], emfile corresponds to too many
open files.
I'm running from a UBUNTU 14.04 LTS box.

However, I've set my /etc/security/limits.conf with:

*   soft    nofile  999999
*   hard    nofile      999999

and rebooted the machine. I can see it applied:

$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 241197
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 999999
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 241197
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited


I've also set /etc/sysctl.conf with:

# number of file descriptors
fs.file-max=2001000

# extend the ports range
net.ipv4.ip_local_port_range=1024 65535

# increase the max number of receive and send buffer size
net.core.rmem_max=33554432
net.core.wmem_max=33554432

# increase TCP auto-tuning buffer limits settings
net.ipv4.tcp_rmem=4096 16384 33554432
net.ipv4.tcp_wmem=4096 16384 33554432
net.ipv4.tcp_mem=786432 1048576 26777216

# increase the number of memory map areas for the server process
vm.max_map_count=131060

# swap only to avoid an out of memory condition
vm.swappiness=0

# try to always keep 64MB of RAM free
vm.min_free_kbytes=65536

# prevent flooding detection when receiving large number of SYN packets
# use with care as they can detec DoS attacks
# needed with artificial loads while running the benchmark
net.ipv4.tcp_max_tw_buckets=360000
net.core.somaxconn=4096
net.ipv4.tcp_max_syn_backlog=16000
net.ipv4.tcp_syncookies=0
net.ipv4.tcp_tw_recycle=1

# insure keepalive is enabled - use only if needed
net.ipv4.tcp_keepalive_time=900
net.ipv4.tcp_keepalive_intvl=180
net.ipv4.tcp_keepalive_probes=5

And applied them with sudo sysctl -p /etc/sysctl.conf.

Finally, in my Erlang release I've set into vm.args:

+K true
+P 2000000
+Q 1000000
+A 10

When I run my release and attach to the node, I can see:

1> erlang:system_info(check_io).
[{name,erts_poll},
 {primary,epoll},
 {fallback,poll},
 {kernel_poll,epoll},
 {memory_size,113328},
 {total_poll_set_size,1243},
 {fallback_poll_set_size,0},
 {lazy_updates,true},
 {pending_updates,0},
 {batch_updates,false},
 {concurrent_updates,true},
 {max_fds,999999},
 {active_fds,1}]

After all this I cannot connect more than 1024 outgoing HTTPS clients.

Any ideas?

Thank you,
r.



[1] http://erlang.org/doc/man/inet.html#id132028
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20150107/25e88fca/attachment.htm>