[erlang-questions] Cowboy vs Misultin

[Andy W. Song]

OK, the competition comes again. I finally wrote a client that performs
decently. It's here <https://github.com/awsong/Golang-WebSocket-Client>.

I tested my previous Cowboy first( I changed send_interval from 1000 to
900000 of line 64 of src/websocket_handler.erl in cowboy_examples ). Here is
the version:
*Cowboy_examples*: 343d002 Initial commit with a default and websocket
handler.
*Cowboy*: 5d69825 Add a max_connections transport option, Mon May 9 22:01:31
2011

The command line I use on the client is "./conn -c 10000 -n 10". It'll
create 100K connections, 10K per IP to the server. It takes about 7 and half
minutes to finish creating these connections. During the creation, server
CPU usage is about 110%(dual core), client cpu usage is under 20%.

Server memory usage(Cowboy residence memory) :
Right after Cowboy starts: 525M
After 100K connection established: 1842M
After 5M packets passed: 2824M
After 9M packets passed: 3231M
After 87.7 M packets passed, 5746M
[{total,4155518040},
 {processes,1701318856},
 {processes_used,1701299096},
 {system,2454199184},
 {atom,497457},
 {atom_used,482279},
 {binary,1787851232},
 {code,4041219},
 {ets,41231176}]
There seems a memory leak. If I don't stop, the memory usage will keep going
higher. 40% CPU, 1.5 MB/s traffic.

*Latest Cowboy and Cowboy_examples* (same send_interval change). It takes
about 7 and half minutes to finish creating 100K connections. During the
creation, server CPU usage is about 110%(dual core), client cpu usage is
under 20%. After 158.9M packets passed, 1464M residence memory used, 30%
CPU, 1.5 MB/s traffic, .

*Latest Misultin* from github, with patch (see attachment):
It takes about 45s to finish creating 100K connections, during creation cpu
usage is 120%. After creation, residence memory usage is 2684M. And it goes
down slowly to 1416M after 40M packets are passed, cpu usage is about 50%

Another thing worth to note is that when I hit CTRL-C on the client side,
misultin will quickly destroy the connections and release memory and go back
to normal. But Cowboy will consume almost all the cpu time and stay there
for a long time (I didn't have the patience to measure how long, so just hit
ctrl-c to quick erlang).

Cheers
Andy


On Wed, Jun 1, 2011 at 8:26 PM, Andy W. Song <wsongcn@REDACTED> wrote:

> BTW, the latest version uses less memory too, about 3.2G for 200K
> connections.
>
>
> On Wed, Jun 1, 2011 at 8:23 PM, Andy W. Song <wsongcn@REDACTED> wrote:
>
>> Missed one attachment.
>>
>>
>> On Wed, Jun 1, 2011 at 8:21 PM, Andy W. Song <wsongcn@REDACTED> wrote:
>>
>>> I tested out the latest Misultin with this change, CPU usage is about 2
>>> to 3 percent lower.
>>>
>>> With the attached Misultin package, I can still reproduce the large
>>> packet problem.
>>>
>>> With latest version, Misultin doesn't report any error. But it doesn't
>>> reply to the client either. Seems it just drops the packet. Another attached
>>> file is the tcpdump result.
>>>
>>> I will try Mochiweb later just to compare.
>>>
>>>
>>> On Wed, Jun 1, 2011 at 12:09 AM, Roberto Ostinelli <roberto@REDACTED>wrote:
>>>
>>>>
>>>> On May 31, 2011, at 12:54 PM, Andy W. Song wrote:
>>>>
>>>> Would you mind testing it using my C client? Search "->len" and give
>>>> it a value between 1500 and 2000, hopefully you'll see the result I
>>>> saw.
>>>>
>>>> Regards
>>>> Andy
>>>>
>>>>
>>>> Hi Andy,
>>>>
>>>> will do that.
>>>>
>>>> Meanwhile, I've added an undocumented option, {ws_no_header, true} so
>>>> that headers are not duplicated in the record and should use less
>>>> memory. plus, I've optimized binary code for websockets.
>>>>
>>>> can you please retry the test so we have a matter of comparison?
>>>>
>>>> the only thing you need to change is to add this option in misultin
>>>> initialization, i.e.:
>>>>
>>>>  % start misultin http server
>>>> start(Port) ->
>>>> misultin:start_link([
>>>>  *{ws_no_header, true},*
>>>>  {port, Port}, {loop, fun(Req) -> handle_http(Req, Port) end},
>>>> {ws_loop, fun(Ws) -> handle_websocket(Ws) end}, {ws_autoexit, false}
>>>>  ]).
>>>>
>>>> please let me know...
>>>>
>>>> r.
>>>>
>>>
>>>
>>>
>>> --
>>> ---------------------------------------------------------------
>>> 有志者，事竟成，破釜沉舟，百二秦关终属楚
>>> 苦心人，天不负，卧薪尝胆，三千越甲可吞吴
>>>
>>>
>>
>>
>> --
>> ---------------------------------------------------------------
>> 有志者，事竟成，破釜沉舟，百二秦关终属楚
>> 苦心人，天不负，卧薪尝胆，三千越甲可吞吴
>>
>>
>
>
> --
> ---------------------------------------------------------------
> 有志者，事竟成，破釜沉舟，百二秦关终属楚
> 苦心人，天不负，卧薪尝胆，三千越甲可吞吴
>
>


-- 
---------------------------------------------------------------
有志者，事竟成，破釜沉舟，百二秦关终属楚
苦心人，天不负，卧薪尝胆，三千越甲可吞吴
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110622/8def0647/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: hibernate.patch
Type: application/octet-stream
Size: 4522 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110622/8def0647/attachment.obj>