[erlang-questions] Cowboy vs Misultin
Loïc Hoguin
essen@REDACTED
Wed Jun 22 17:24:58 CEST 2011
Odd.
I'll try to run the client you wrote to reproduce whenever I receive the
hardware I'm waiting for. Not sure what could do this though. Do you
have an idea of what's the number of connections before it starts
slowing down?
On 06/22/2011 05:08 PM, Andy W. Song wrote:
> Thanks for the link. I removed the send_interval call and retested but
> it didn't make much of a difference. It still takes about 7 and half
> minutes to create 100K connections. During creation I can see several
> time outs from client side. And the creation speed gets slower when the
> number of connections gets higher. Although the average CPU usage
> doesn't change much.
>
> After creation, with 1.5MB/s traffic, the CPU usage is still about the
> same as before which is 30%.
>
> Cheers.
> Andy
>
> On Wed, Jun 22, 2011 at 10:34 PM, Loïc Hoguin <essen@REDACTED
> <mailto:essen@REDACTED>> wrote:
>
> Memory usage sounds a bit high for cowboy_http_req:compact. But maybe
> not. Good point though it's already in the examples just like hibernate.
> My bad.
>
> On the other hand if you have a timer per connection it's definitely not
> going to scale, timers are very slow so having 100k timers isn't going
> to work well. Remove the send_interval and it should work a lot better.
>
> For future reference:
> http://www.erlang.org/doc/efficiency_guide/commoncaveats.html#id58959
>
> Waiting for yet another comparison, have fun. :)
>
> On 06/22/2011 04:27 PM, Andy W. Song wrote:
> > I actually didn't patch. I want to keep my change as small as
> possible.
> > I will remove that send_interval and test it again. Sorry I didn't pay
> > much attention about this.
> >
> > BTW, how can you tell I used or did not use
> cowboy_http_req:compact? And
> > I see hibernate is already the default behavior, isn't it?
> >
> > Thanks
> > Andy
> >
> > On Wed, Jun 22, 2011 at 10:06 PM, Loïc Hoguin <essen@REDACTED
> <mailto:essen@REDACTED>
> > <mailto:essen@REDACTED <mailto:essen@REDACTED>>> wrote:
> >
> > Hello,
> >
> > Did you fix the many issues in your cowboy application code I
> reported
> > in my previous emails? I see you removed the receive after from
> > misultin, did you remove it from cowboy too? You could very
> well just be
> > measuring Erlang's timer modules bad performance there.
> >
> > Also I see you're not using cowboy_http_req:compact and the
> hibernate
> > option for cowboy that is doing it properly (unlike your patch).
> >
> > And again, you should probably come to IRC and ask for advice
> from the
> > other people benchmarking it.
> >
> > Cheers.
> >
> > On 06/22/2011 03:14 PM, Andy W. Song wrote:
> > > OK, the competition comes again. I finally wrote a client that
> > performs
> > > decently. It's here
> > <https://github.com/awsong/Golang-WebSocket-Client>.
> > >
> > > I tested my previous Cowboy first( I changed send_interval from
> > 1000 to
> > > 900000 of line 64 of src/websocket_handler.erl in
> cowboy_examples ).
> > > Here is the version:
> > > *Cowboy_examples*: 343d002 Initial commit with a default and
> websocket
> > > handler.
> > > *Cowboy*: 5d69825 Add a max_connections transport option,
> Mon May 9
> > > 22:01:31 2011
> > >
> > > The command line I use on the client is "./conn -c 10000 -n
> 10". It'll
> > > create 100K connections, 10K per IP to the server. It takes
> about
> > 7 and
> > > half minutes to finish creating these connections. During the
> > creation,
> > > server CPU usage is about 110%(dual core), client cpu usage is
> > under 20%.
> > >
> > > Server memory usage(Cowboy residence memory) :
> > > Right after Cowboy starts: 525M
> > > After 100K connection established: 1842M
> > > After 5M packets passed: 2824M
> > > After 9M packets passed: 3231M
> > > After 87.7 M packets passed, 5746M
> > > [{total,4155518040 <tel:4155518040> <tel:4155518040
> <tel:4155518040>> <tel:4155518040 <tel:4155518040>
> > <tel:4155518040 <tel:4155518040>>>},
> > > {processes,1701318856},
> > > {processes_used,1701299096},
> > > {system,2454199184},
> > > {atom,497457},
> > > {atom_used,482279},
> > > {binary,1787851232},
> > > {code,4041219},
> > > {ets,41231176}]
> > > There seems a memory leak. If I don't stop, the memory usage
> will keep
> > > going higher. 40% CPU, 1.5 MB/s traffic.
> > >
> > > *Latest Cowboy and Cowboy_examples* (same send_interval
> change). It
> > > takes about 7 and half minutes to finish creating 100K
> connections.
> > > During the creation, server CPU usage is about 110%(dual
> core), client
> > > cpu usage is under 20%. After 158.9M packets passed, 1464M
> residence
> > > memory used, 30% CPU, 1.5 MB/s traffic, .
> > >
> > > *Latest Misultin* from github, with patch (see attachment):
> > > It takes about 45s to finish creating 100K connections, during
> > creation
> > > cpu usage is 120%. After creation, residence memory usage is
> > 2684M. And
> > > it goes down slowly to 1416M after 40M packets are passed, cpu
> > usage is
> > > about 50%
> > >
> > > Another thing worth to note is that when I hit CTRL-C on the
> client
> > > side, misultin will quickly destroy the connections and
> release memory
> > > and go back to normal. But Cowboy will consume almost all
> the cpu time
> > > and stay there for a long time (I didn't have the patience
> to measure
> > > how long, so just hit ctrl-c to quick erlang).
> > >
> > > Cheers
> > > Andy
> > >
> > >
> > > On Wed, Jun 1, 2011 at 8:26 PM, Andy W. Song
> <wsongcn@REDACTED <mailto:wsongcn@REDACTED>
> > <mailto:wsongcn@REDACTED <mailto:wsongcn@REDACTED>>
> > > <mailto:wsongcn@REDACTED <mailto:wsongcn@REDACTED>
> <mailto:wsongcn@REDACTED <mailto:wsongcn@REDACTED>>>> wrote:
> > >
> > > BTW, the latest version uses less memory too, about 3.2G
> for 200K
> > > connections.
> > >
> > >
> > > On Wed, Jun 1, 2011 at 8:23 PM, Andy W. Song
> > <wsongcn@REDACTED <mailto:wsongcn@REDACTED>
> <mailto:wsongcn@REDACTED <mailto:wsongcn@REDACTED>>
> > > <mailto:wsongcn@REDACTED <mailto:wsongcn@REDACTED>
> <mailto:wsongcn@REDACTED <mailto:wsongcn@REDACTED>>>> wrote:
> > >
> > > Missed one attachment.
> > >
> > >
> > > On Wed, Jun 1, 2011 at 8:21 PM, Andy W. Song
> > <wsongcn@REDACTED <mailto:wsongcn@REDACTED>
> <mailto:wsongcn@REDACTED <mailto:wsongcn@REDACTED>>
> > > <mailto:wsongcn@REDACTED <mailto:wsongcn@REDACTED>
> <mailto:wsongcn@REDACTED <mailto:wsongcn@REDACTED>>>> wrote:
> > >
> > > I tested out the latest Misultin with this
> change, CPU
> > usage
> > > is about 2 to 3 percent lower.
> > >
> > > With the attached Misultin package, I can still
> reproduce
> > > the large packet problem.
> > >
> > > With latest version, Misultin doesn't report any
> > error. But
> > > it doesn't reply to the client either. Seems it
> just drops
> > > the packet. Another attached file is the tcpdump
> result.
> > >
> > > I will try Mochiweb later just to compare.
> > >
> > >
> > > On Wed, Jun 1, 2011 at 12:09 AM, Roberto Ostinelli
> > > <roberto@REDACTED
> <mailto:roberto@REDACTED> <mailto:roberto@REDACTED
> <mailto:roberto@REDACTED>>
> > <mailto:roberto@REDACTED <mailto:roberto@REDACTED>
> <mailto:roberto@REDACTED <mailto:roberto@REDACTED>>>> wrote:
> > >
> > >
> > > On May 31, 2011, at 12:54 PM, Andy W. Song
> wrote:
> > >
> > >> Would you mind testing it using my C
> client? Search
> > >> "->len" and give
> > >> it a value between 1500 and 2000, hopefully
> > you'll see
> > >> the result I
> > >> saw.
> > >>
> > >> Regards
> > >> Andy
> > >
> > > Hi Andy,
> > >
> > > will do that.
> > >
> > > Meanwhile, I've added an undocumented option,
> > > {ws_no_header, true} so that headers are not
> > duplicated
> > > in the record and should use less memory.
> plus, I've
> > > optimized binary code for websockets.
> > >
> > > can you please retry the test so we have a
> matter of
> > > comparison?
> > >
> > > the only thing you need to change is to add this
> > option
> > > in misultin initialization, i.e.:
> > >
> > > % start misultin http server
> > > start(Port) ->
> > > misultin:start_link([
> > > *{ws_no_header, true},*
> > > {port, Port}, {loop, fun(Req) ->
> handle_http(Req,
> > Port)
> > > end},
> > > {ws_loop, fun(Ws) -> handle_websocket(Ws) end},
> > > {ws_autoexit, false}
> > > ]).
> > >
> > > please let me know...
> > >
> > > r.
> > >
> > >
> > >
> > >
> > > --
> > >
> > ---------------------------------------------------------------
> > > 有志者，事竟成，破釜沉舟，百二秦关终属楚
> > > 苦心人，天不负，卧薪尝胆，三千越甲可吞吴
> > >
> > >
> > >
> > >
> > > --
> > >
> > ---------------------------------------------------------------
> > > 有志者，事竟成，破釜沉舟，百二秦关终属楚
> > > 苦心人，天不负，卧薪尝胆，三千越甲可吞吴
> > >
> > >
> > >
> > >
> > > --
> > >
> ---------------------------------------------------------------
> > > 有志者，事竟成，破釜沉舟，百二秦关终属楚
> > > 苦心人，天不负，卧薪尝胆，三千越甲可吞吴
> > >
> > >
> > >
> > >
> > > --
> > > ---------------------------------------------------------------
> > > 有志者，事竟成，破釜沉舟，百二秦关终属楚
> > > 苦心人，天不负，卧薪尝胆，三千越甲可吞吴
> > >
> > >
> > >
> > > _______________________________________________
> > > erlang-questions mailing list
> > > erlang-questions@REDACTED
> <mailto:erlang-questions@REDACTED>
> <mailto:erlang-questions@REDACTED
> <mailto:erlang-questions@REDACTED>>
> > > http://erlang.org/mailman/listinfo/erlang-questions
> >
> >
> > --
> > Loïc Hoguin
> > Dev:Extend
> >
> >
> >
> >
> > --
> > ---------------------------------------------------------------
> > 有志者，事竟成，破釜沉舟，百二秦关终属楚
> > 苦心人，天不负，卧薪尝胆，三千越甲可吞吴
> >
>
>
> --
> Loïc Hoguin
> Dev:Extend
>
>
>
>
> --
> ---------------------------------------------------------------
> 有志者，事竟成，破釜沉舟，百二秦关终属楚
> 苦心人，天不负，卧薪尝胆，三千越甲可吞吴
>
--
Loïc Hoguin
Dev:Extend
More information about the erlang-questions
mailing list