[erlang-questions] Cowboy vs Misultin
Andy W. Song
wsongcn@REDACTED
Wed Jun 22 17:30:01 CEST 2011
I can roughly feel it after about 15K~20K.
On Wed, Jun 22, 2011 at 11:24 PM, Loïc Hoguin <essen@REDACTED> wrote:
> Odd.
>
> I'll try to run the client you wrote to reproduce whenever I receive the
> hardware I'm waiting for. Not sure what could do this though. Do you
> have an idea of what's the number of connections before it starts
> slowing down?
>
> On 06/22/2011 05:08 PM, Andy W. Song wrote:
> > Thanks for the link. I removed the send_interval call and retested but
> > it didn't make much of a difference. It still takes about 7 and half
> > minutes to create 100K connections. During creation I can see several
> > time outs from client side. And the creation speed gets slower when the
> > number of connections gets higher. Although the average CPU usage
> > doesn't change much.
> >
> > After creation, with 1.5MB/s traffic, the CPU usage is still about the
> > same as before which is 30%.
> >
> > Cheers.
> > Andy
> >
> > On Wed, Jun 22, 2011 at 10:34 PM, Loïc Hoguin <essen@REDACTED
> > <mailto:essen@REDACTED>> wrote:
> >
> > Memory usage sounds a bit high for cowboy_http_req:compact. But maybe
> > not. Good point though it's already in the examples just like
> hibernate.
> > My bad.
> >
> > On the other hand if you have a timer per connection it's definitely
> not
> > going to scale, timers are very slow so having 100k timers isn't
> going
> > to work well. Remove the send_interval and it should work a lot
> better.
> >
> > For future reference:
> >
> http://www.erlang.org/doc/efficiency_guide/commoncaveats.html#id58959
> >
> > Waiting for yet another comparison, have fun. :)
> >
> > On 06/22/2011 04:27 PM, Andy W. Song wrote:
> > > I actually didn't patch. I want to keep my change as small as
> > possible.
> > > I will remove that send_interval and test it again. Sorry I didn't
> pay
> > > much attention about this.
> > >
> > > BTW, how can you tell I used or did not use
> > cowboy_http_req:compact? And
> > > I see hibernate is already the default behavior, isn't it?
> > >
> > > Thanks
> > > Andy
> > >
> > > On Wed, Jun 22, 2011 at 10:06 PM, Loïc Hoguin <essen@REDACTED
> > <mailto:essen@REDACTED>
> > > <mailto:essen@REDACTED <mailto:essen@REDACTED>>> wrote:
> > >
> > > Hello,
> > >
> > > Did you fix the many issues in your cowboy application code I
> > reported
> > > in my previous emails? I see you removed the receive after from
> > > misultin, did you remove it from cowboy too? You could very
> > well just be
> > > measuring Erlang's timer modules bad performance there.
> > >
> > > Also I see you're not using cowboy_http_req:compact and the
> > hibernate
> > > option for cowboy that is doing it properly (unlike your
> patch).
> > >
> > > And again, you should probably come to IRC and ask for advice
> > from the
> > > other people benchmarking it.
> > >
> > > Cheers.
> > >
> > > On 06/22/2011 03:14 PM, Andy W. Song wrote:
> > > > OK, the competition comes again. I finally wrote a client
> that
> > > performs
> > > > decently. It's here
> > > <https://github.com/awsong/Golang-WebSocket-Client>.
> > > >
> > > > I tested my previous Cowboy first( I changed send_interval
> from
> > > 1000 to
> > > > 900000 of line 64 of src/websocket_handler.erl in
> > cowboy_examples ).
> > > > Here is the version:
> > > > *Cowboy_examples*: 343d002 Initial commit with a default and
> > websocket
> > > > handler.
> > > > *Cowboy*: 5d69825 Add a max_connections transport option,
> > Mon May 9
> > > > 22:01:31 2011
> > > >
> > > > The command line I use on the client is "./conn -c 10000 -n
> > 10". It'll
> > > > create 100K connections, 10K per IP to the server. It takes
> > about
> > > 7 and
> > > > half minutes to finish creating these connections. During the
> > > creation,
> > > > server CPU usage is about 110%(dual core), client cpu usage
> is
> > > under 20%.
> > > >
> > > > Server memory usage(Cowboy residence memory) :
> > > > Right after Cowboy starts: 525M
> > > > After 100K connection established: 1842M
> > > > After 5M packets passed: 2824M
> > > > After 9M packets passed: 3231M
> > > > After 87.7 M packets passed, 5746M
> > > > [{total,4155518040 <tel:4155518040> <tel:4155518040
> > <tel:4155518040>> <tel:4155518040 <tel:4155518040>
> > > <tel:4155518040 <tel:4155518040>>>},
> > > > {processes,1701318856},
> > > > {processes_used,1701299096},
> > > > {system,2454199184},
> > > > {atom,497457},
> > > > {atom_used,482279},
> > > > {binary,1787851232},
> > > > {code,4041219},
> > > > {ets,41231176}]
> > > > There seems a memory leak. If I don't stop, the memory usage
> > will keep
> > > > going higher. 40% CPU, 1.5 MB/s traffic.
> > > >
> > > > *Latest Cowboy and Cowboy_examples* (same send_interval
> > change). It
> > > > takes about 7 and half minutes to finish creating 100K
> > connections.
> > > > During the creation, server CPU usage is about 110%(dual
> > core), client
> > > > cpu usage is under 20%. After 158.9M packets passed, 1464M
> > residence
> > > > memory used, 30% CPU, 1.5 MB/s traffic, .
> > > >
> > > > *Latest Misultin* from github, with patch (see attachment):
> > > > It takes about 45s to finish creating 100K connections,
> during
> > > creation
> > > > cpu usage is 120%. After creation, residence memory usage is
> > > 2684M. And
> > > > it goes down slowly to 1416M after 40M packets are passed,
> cpu
> > > usage is
> > > > about 50%
> > > >
> > > > Another thing worth to note is that when I hit CTRL-C on the
> > client
> > > > side, misultin will quickly destroy the connections and
> > release memory
> > > > and go back to normal. But Cowboy will consume almost all
> > the cpu time
> > > > and stay there for a long time (I didn't have the patience
> > to measure
> > > > how long, so just hit ctrl-c to quick erlang).
> > > >
> > > > Cheers
> > > > Andy
> > > >
> > > >
> > > > On Wed, Jun 1, 2011 at 8:26 PM, Andy W. Song
> > <wsongcn@REDACTED <mailto:wsongcn@REDACTED>
> > > <mailto:wsongcn@REDACTED <mailto:wsongcn@REDACTED>>
> > > > <mailto:wsongcn@REDACTED <mailto:wsongcn@REDACTED>
> > <mailto:wsongcn@REDACTED <mailto:wsongcn@REDACTED>>>> wrote:
> > > >
> > > > BTW, the latest version uses less memory too, about 3.2G
> > for 200K
> > > > connections.
> > > >
> > > >
> > > > On Wed, Jun 1, 2011 at 8:23 PM, Andy W. Song
> > > <wsongcn@REDACTED <mailto:wsongcn@REDACTED>
> > <mailto:wsongcn@REDACTED <mailto:wsongcn@REDACTED>>
> > > > <mailto:wsongcn@REDACTED <mailto:wsongcn@REDACTED>
> > <mailto:wsongcn@REDACTED <mailto:wsongcn@REDACTED>>>> wrote:
> > > >
> > > > Missed one attachment.
> > > >
> > > >
> > > > On Wed, Jun 1, 2011 at 8:21 PM, Andy W. Song
> > > <wsongcn@REDACTED <mailto:wsongcn@REDACTED>
> > <mailto:wsongcn@REDACTED <mailto:wsongcn@REDACTED>>
> > > > <mailto:wsongcn@REDACTED <mailto:wsongcn@REDACTED>
> > <mailto:wsongcn@REDACTED <mailto:wsongcn@REDACTED>>>> wrote:
> > > >
> > > > I tested out the latest Misultin with this
> > change, CPU
> > > usage
> > > > is about 2 to 3 percent lower.
> > > >
> > > > With the attached Misultin package, I can still
> > reproduce
> > > > the large packet problem.
> > > >
> > > > With latest version, Misultin doesn't report any
> > > error. But
> > > > it doesn't reply to the client either. Seems it
> > just drops
> > > > the packet. Another attached file is the tcpdump
> > result.
> > > >
> > > > I will try Mochiweb later just to compare.
> > > >
> > > >
> > > > On Wed, Jun 1, 2011 at 12:09 AM, Roberto
> Ostinelli
> > > > <roberto@REDACTED
> > <mailto:roberto@REDACTED> <mailto:roberto@REDACTED
> > <mailto:roberto@REDACTED>>
> > > <mailto:roberto@REDACTED <mailto:roberto@REDACTED>
> > <mailto:roberto@REDACTED <mailto:roberto@REDACTED>>>> wrote:
> > > >
> > > >
> > > > On May 31, 2011, at 12:54 PM, Andy W. Song
> > wrote:
> > > >
> > > >> Would you mind testing it using my C
> > client? Search
> > > >> "->len" and give
> > > >> it a value between 1500 and 2000, hopefully
> > > you'll see
> > > >> the result I
> > > >> saw.
> > > >>
> > > >> Regards
> > > >> Andy
> > > >
> > > > Hi Andy,
> > > >
> > > > will do that.
> > > >
> > > > Meanwhile, I've added an undocumented option,
> > > > {ws_no_header, true} so that headers are not
> > > duplicated
> > > > in the record and should use less memory.
> > plus, I've
> > > > optimized binary code for websockets.
> > > >
> > > > can you please retry the test so we have a
> > matter of
> > > > comparison?
> > > >
> > > > the only thing you need to change is to add
> this
> > > option
> > > > in misultin initialization, i.e.:
> > > >
> > > > % start misultin http server
> > > > start(Port) ->
> > > > misultin:start_link([
> > > > *{ws_no_header, true},*
> > > > {port, Port}, {loop, fun(Req) ->
> > handle_http(Req,
> > > Port)
> > > > end},
> > > > {ws_loop, fun(Ws) -> handle_websocket(Ws)
> end},
> > > > {ws_autoexit, false}
> > > > ]).
> > > >
> > > > please let me know...
> > > >
> > > > r.
> > > >
> > > >
> > > >
> > > >
> > > > --
> > > >
> > > ---------------------------------------------------------------
> > > > 有志者，事竟成，破釜沉舟，百二秦关终属楚
> > > > 苦心人，天不负，卧薪尝胆，三千越甲可吞吴
> > > >
> > > >
> > > >
> > > >
> > > > --
> > > >
> > > ---------------------------------------------------------------
> > > > 有志者，事竟成，破釜沉舟，百二秦关终属楚
> > > > 苦心人，天不负，卧薪尝胆，三千越甲可吞吴
> > > >
> > > >
> > > >
> > > >
> > > > --
> > > >
> > ---------------------------------------------------------------
> > > > 有志者，事竟成，破釜沉舟，百二秦关终属楚
> > > > 苦心人，天不负，卧薪尝胆，三千越甲可吞吴
> > > >
> > > >
> > > >
> > > >
> > > > --
> > > >
> ---------------------------------------------------------------
> > > > 有志者，事竟成，破釜沉舟，百二秦关终属楚
> > > > 苦心人，天不负，卧薪尝胆，三千越甲可吞吴
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > erlang-questions mailing list
> > > > erlang-questions@REDACTED
> > <mailto:erlang-questions@REDACTED>
> > <mailto:erlang-questions@REDACTED
> > <mailto:erlang-questions@REDACTED>>
> > > > http://erlang.org/mailman/listinfo/erlang-questions
> > >
> > >
> > > --
> > > Loïc Hoguin
> > > Dev:Extend
> > >
> > >
> > >
> > >
> > > --
> > > ---------------------------------------------------------------
> > > 有志者，事竟成，破釜沉舟，百二秦关终属楚
> > > 苦心人，天不负，卧薪尝胆，三千越甲可吞吴
> > >
> >
> >
> > --
> > Loïc Hoguin
> > Dev:Extend
> >
> >
> >
> >
> > --
> > ---------------------------------------------------------------
> > 有志者，事竟成，破釜沉舟，百二秦关终属楚
> > 苦心人，天不负，卧薪尝胆，三千越甲可吞吴
> >
>
>
> --
> Loïc Hoguin
> Dev:Extend
>
--
---------------------------------------------------------------
有志者，事竟成，破釜沉舟，百二秦关终属楚
苦心人，天不负，卧薪尝胆，三千越甲可吞吴
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110622/ca65a984/attachment.htm>
More information about the erlang-questions
mailing list