[erlang-questions] Yaws and httpc

[Avinash Dhumane]

I have a web client written using "httpc" module, and I have Yaws web  
server with all default configurations except for "listen_backlog", which  
is set to 10000 (ten thousand).

I am not sure how to program error-recovery for a test-client application  
that uses "httpc" module when httpc:request/2 encounters an error as  
follows:

>>>> Error in process <0.17159.0> with exit value:  
>>>> {{badmatch,{error,socket_closed_remotely}},[{qyuki_test,'-work/2-fun-0-',6,[{file,"qyuki_test.erl"},{line,34}]},{lists,map,2,[{file,"lists.erl"},{line,1173}]},{qyuki_test,work,2,[{file,"qyuki_test.erl"},{line,31}]}]}

I presumed I would close the the dynamically created profile [*1] using  
inets:stop/2, and restart it with the same profile-name using  
inets:start/2, and submit the http request again using httpc:request/2.  
But, then, it encounters this error:

>>>> Error in process <0.11323.0> with exit value:  
>>>> {{case_clause,{error,{failed_connect,[{to_address,{"192.168.1.4",80}},{inet,[inet],etimedout}]}}},[{qyuki_test,web_tran,6,[{file,"qyuki_test.erl"},{line,40}]},{qyuki_test,work,2,[{file,"qyuki_test.erl"},{line,31}]}]}

Basically, the test-client application spawns 10000 (ten thousand)  
processes [*2], each creating its own profile with max_sessions = 1 and  
max_keep_alive_length = 1 [*3], and submitting the transactions  
synchronously in a loop with an internal of 20 seconds between each  
submission. The first submission for all processes go through  
successfully; but, from second submission onwards, some of the client  
processes start receiving "socket_closed_remotely" errors as shown above.  
Please advise on how to recover from this error and resume the  
transactions on the client side [*4]

I tried altering Yaws configuration parameter keepalive_timeout from 30000  
to 120000. I also tried keeping an internal (60 seconds) between stopping  
and restarting the client profile upon encountering the socket failures.  
But that didn't alter the error behaviours seen at the client side [*5]

My assumption is that Yaws should not close the socket because the client  
takes care of not keeping the connection idle for more than 20 seconds.

Please advise.

Thanks
Avinash

[*1] The test-application needs to simulate many "web browser" clients, so  
I suppose I keep a separate profile for each client process.
[*2] Both Erlang web client and Yaws (1.92) web server are running on  
R15B01 (erts-5.9.1) and on 32-bit Ubuntu Server 12, and on separate  
machines. ERL_MAX_PORTS, ERL_MAX_ETS_TABLES and "ulimit -n" are set to 64K  
on both client & server machines.
[*3] My assumption is that "httpc" inets service should not open more than  
one connection to web server since test-client is submitting the  
transactions synchronously. But, I am seeing that the number of  
connections to the web server grow more than 16000 even when there are  
only 10000 "httpc" services running in the inets container.
[*4] Yaws logs show no abnormal activities when "httpc" services on the  
client receive sockets failures, except for information on reaching the  
maximum number of files in cache.
[*5] I have raised the number of processes to 500000 using +P only on the  
client side ERTS.