<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">I dont think so!<div><br><div><div>When I tested Wei's code teh first time, I suppressed all <b>io:format</b> statements and made</div><div>the inner loop lot simpler. Something like that:</div><div><br></div><div>[...]</div><div><div>loop(S,B) -></div><div> inet:setopts(S, [{active, once}]),</div><div> receive</div><div> {tcp, S, _Data} -></div><div> Response = <<"HTTP/1.1 200 OK\r\nContent-Length:</div><div>12\r\nConnection: Keep-Alive\r\n\r\nhello world!">>,</div><div> gen_tcp:send(S, Response),</div><div> loop(S, <<>>)</div><div>[...]</div></div><div><br></div><div>didn't change anything in my case.</div><div><br></div><div>Hope this help!</div><div><br></div><div><div>Regards,</div><div>Zabrane</div></div><div><br></div><div><div><div>On Jul 10, 2012, at 11:14 PM, erlang wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>Hi, all (first time)<br>on start - i'm sorry for my englisch language.<br>i think that the problem is on all lines where you use "io:format"<br>In my system when i start wrote in erlang I used many times io:format for debug. It wos big bottleneck.<br><br><br>JanM<br><br><br>W dniu 2012-07-09 11:01, Wei Cao pisze:<br><blockquote type="cite">Hi, all<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">We wrote a proxy server in Erlang, the proxy's logic is rather simple,<br></blockquote><blockquote type="cite">it listens on some TCP port, establishes new connection from user<br></blockquote><blockquote type="cite">client, forward packets back and forth between the client and backend<br></blockquote><blockquote type="cite">server after authentication until connection is closed.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">It's very easy to write such a proxy in Erlang, fork a process for<br></blockquote><blockquote type="cite">each new user connection and connect to the backend server in the same<br></blockquote><blockquote type="cite">process, the process works like a pipe, sockets from both side is set<br></blockquote><blockquote type="cite">to the active once mode, whenever a tcp packet is received from one<br></blockquote><blockquote type="cite">socket, the packet will be sent to other socket. (A simplified version<br></blockquote><blockquote type="cite">of proxy code is attached at the end of the mail)<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">However, the performance is not quite satisfying, the proxy can handle<br></blockquote><blockquote type="cite">maximum only 40k requests on our 16 core machine(Intel Xeon L5630,<br></blockquote><blockquote type="cite">2.13GHz) with heavy stress(100 concurrent clients). We then analyzed<br></blockquote><blockquote type="cite">the behavior of beam.smp use tools like tcprstat, mpstat, perf, and<br></blockquote><blockquote type="cite">SystemTap.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">tcprstat shows QPS is about 40k, have average 1.7 millisecond latency<br></blockquote><blockquote type="cite">for each request.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">timestamp<span class="Apple-tab-span" style="white-space:pre"> </span>count<span class="Apple-tab-span" style="white-space:pre"> </span>max<span class="Apple-tab-span" style="white-space:pre"> </span>min<span class="Apple-tab-span" style="white-space:pre"> </span>avg<span class="Apple-tab-span" style="white-space:pre"> </span>med<span class="Apple-tab-span" style="white-space:pre"> </span>stddev<span class="Apple-tab-span" style="white-space:pre"> </span>95_max<span class="Apple-tab-span" style="white-space:pre"> </span>95_avg<span class="Apple-tab-span" style="white-space:pre"> </span>95_std<span class="Apple-tab-span" style="white-space:pre"> </span>99_max<span class="Apple-tab-span" style="white-space:pre"> </span>99_avg<span class="Apple-tab-span" style="white-space:pre"> </span>99_std<br></blockquote><blockquote type="cite">1341813326<span class="Apple-tab-span" style="white-space:pre"> </span>39416<span class="Apple-tab-span" style="white-space:pre"> </span>17873<span class="Apple-tab-span" style="white-space:pre"> </span>953<span class="Apple-tab-span" style="white-space:pre"> </span>1718<span class="Apple-tab-span" style="white-space:pre"> </span>1519<span class="Apple-tab-span" style="white-space:pre"> </span>724<span class="Apple-tab-span" style="white-space:pre"> </span>2919<span class="Apple-tab-span" style="white-space:pre"> </span>1609<span class="Apple-tab-span" style="white-space:pre"> </span>340<span class="Apple-tab-span" style="white-space:pre"> </span>3813<span class="Apple-tab-span" style="white-space:pre"> </span>1674<span class="Apple-tab-span" style="white-space:pre"> </span>462<br></blockquote><blockquote type="cite">1341813327<span class="Apple-tab-span" style="white-space:pre"> </span>40528<span class="Apple-tab-span" style="white-space:pre"> </span>9275<span class="Apple-tab-span" style="white-space:pre"> </span>884<span class="Apple-tab-span" style="white-space:pre"> </span>1645<span class="Apple-tab-span" style="white-space:pre"> </span>1493<span class="Apple-tab-span" style="white-space:pre"> </span>491<span class="Apple-tab-span" style="white-space:pre"> </span>2777<span class="Apple-tab-span" style="white-space:pre"> </span>1559<span class="Apple-tab-span" style="white-space:pre"> </span>290<span class="Apple-tab-span" style="white-space:pre"> </span>3508<span class="Apple-tab-span" style="white-space:pre"> </span>1619<span class="Apple-tab-span" style="white-space:pre"> </span>409<br></blockquote><blockquote type="cite">1341813328<span class="Apple-tab-span" style="white-space:pre"> </span>40009<span class="Apple-tab-span" style="white-space:pre"> </span>18105<span class="Apple-tab-span" style="white-space:pre"> </span>925<span class="Apple-tab-span" style="white-space:pre"> </span>1694<span class="Apple-tab-span" style="white-space:pre"> </span>1507<span class="Apple-tab-span" style="white-space:pre"> </span>714<span class="Apple-tab-span" style="white-space:pre"> </span>2868<span class="Apple-tab-span" style="white-space:pre"> </span>1586<span class="Apple-tab-span" style="white-space:pre"> </span>328<span class="Apple-tab-span" style="white-space:pre"> </span>3753<span class="Apple-tab-span" style="white-space:pre"> </span>1650<span class="Apple-tab-span" style="white-space:pre"> </span>450<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">mpstat shows 30% CPU is idle,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">03:30:19 PM CPU %usr %nice %sys %iowait %irq %soft<br></blockquote><blockquote type="cite">%steal %guest %idle<br></blockquote><blockquote type="cite">03:30:20 PM all 38.69 0.00 21.92 0.00 0.06 7.52<br></blockquote><blockquote type="cite">0.00 0.00 31.80<br></blockquote><blockquote type="cite">03:30:21 PM all 37.56 0.00 21.99 0.00 0.00 7.50<br></blockquote><blockquote type="cite">0.00 0.00 32.95<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">and perf top shows, much time is wasted in scheduler_wait, in spin wait I guess.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">9320.00 19.8% scheduler_wait<br></blockquote><blockquote type="cite">/home/mingsong.cw/erlangr16b/lib/erlang/erts-5.10/bin/beam.smp<br></blockquote><blockquote type="cite">1813.00 3.9% process_main<br></blockquote><blockquote type="cite">/home/mingsong.cw/erlangr16b/lib/erlang/erts-5.10/bin/beam.smp<br></blockquote><blockquote type="cite">1379.00 2.9% _spin_lock<br></blockquote><blockquote type="cite">/usr/lib/debug/lib/modules/2.6.32-131.21.1.tb477.el6.x86_64/vmlinux<br></blockquote><blockquote type="cite">1201.00 2.6% schedule<br></blockquote><blockquote type="cite">/home/mingsong.cw/erlangr16b/lib/erlang/erts-5.10/bin/beam.smp<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">It seems the performance may be associated with scheduler_wait() and<br></blockquote><blockquote type="cite">erts_check_io(), with a SystemTap script(attached at the end of this<br></blockquote><blockquote type="cite">mail), we can find out how many times the system call epoll_wait is<br></blockquote><blockquote type="cite">invoked by beam.smp and each time, how many revents it gets.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"> cpu process times<br></blockquote><blockquote type="cite">revents min max avg timeouts<br></blockquote><blockquote type="cite"> all<br></blockquote><blockquote type="cite">1754 128042 - - 73 3<br></blockquote><blockquote type="cite">[14] beam.smp 151<br></blockquote><blockquote type="cite">14127 82 97 93 0<br></blockquote><blockquote type="cite">[ 5] beam.smp 142<br></blockquote><blockquote type="cite">13291 83 97 93 0<br></blockquote><blockquote type="cite">[13] beam.smp 127<br></blockquote><blockquote type="cite">11948 86 96 94 0<br></blockquote><blockquote type="cite">[ 6] beam.smp 127<br></blockquote><blockquote type="cite">11836 81 96 93 0<br></blockquote><blockquote type="cite">[ 4] beam.smp 121<br></blockquote><blockquote type="cite">11323 81 96 93 0<br></blockquote><blockquote type="cite">[15] beam.smp 117<br></blockquote><blockquote type="cite">10935 83 96 93 0<br></blockquote><blockquote type="cite">[12] beam.smp 486<br></blockquote><blockquote type="cite">10128 0 96 20 2<br></blockquote><blockquote type="cite">[ 1] beam.smp 71<br></blockquote><blockquote type="cite">6549 71 100 92 0<br></blockquote><blockquote type="cite">[ 2] beam.smp 62<br></blockquote><blockquote type="cite">5695 82 96 91 0<br></blockquote><blockquote type="cite">[ 7] beam.smp 55<br></blockquote><blockquote type="cite">5102 81 95 92 0<br></blockquote><blockquote type="cite">[11] beam.smp 52<br></blockquote><blockquote type="cite">4822 85 95 92 0<br></blockquote><blockquote type="cite">[ 9] beam.smp 52<br></blockquote><blockquote type="cite">4799 85 95 92 0<br></blockquote><blockquote type="cite">[ 8] beam.smp 51<br></blockquote><blockquote type="cite">4680 78 95 91 0<br></blockquote><blockquote type="cite">[10] beam.smp 49<br></blockquote><blockquote type="cite">4508 85 97 92 0<br></blockquote><blockquote type="cite">[ 3] beam.smp 46<br></blockquote><blockquote type="cite">4211 81 95 91 0<br></blockquote><blockquote type="cite">[ 0] beam.smp 44<br></blockquote><blockquote type="cite">4088 83 95 92 0<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">The resuls shows, epoll_wait is invoked 1754 times each second, and<br></blockquote><blockquote type="cite">get 73 io events in average. This is unacceptable for writing high<br></blockquote><blockquote type="cite">performance server. Because if epoll_wait is invoked no more than 2k<br></blockquote><blockquote type="cite">times per second, then read/write a packet would cost more than 500ms,<br></blockquote><blockquote type="cite">which causes long delay and affects the throughput finally.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">The problem relies on there is only one global pollset in system wide,<br></blockquote><blockquote type="cite">so at a time there is no more than one scheduler can call<br></blockquote><blockquote type="cite">erts_check_io() to obtain pending io tasks from underlying pollset,<br></blockquote><blockquote type="cite">and no scheduler can call erts_check_io() before all pending io<br></blockquote><blockquote type="cite">tasks're processed, so for IO bounded application, it's very likely<br></blockquote><blockquote type="cite">that a scheduler finish its own job, but must wait idly for other<br></blockquote><blockquote type="cite">schedulers to complete theirs.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Hence, we develops a patch to slove this problem, by having a pollset<br></blockquote><blockquote type="cite">for each scheduler, so that each scheduler can invoke erts_check_io()<br></blockquote><blockquote type="cite">on its own pollset concurrently. After a scheduler complete its tasks,<br></blockquote><blockquote type="cite">it can invoke erts_check_io() immediately no matter what state other<br></blockquote><blockquote type="cite">schedulers're in. This patch also handles port migration situation,<br></blockquote><blockquote type="cite">all used file descriptors in each port're recorded, when a port is<br></blockquote><blockquote type="cite">migrated, these<br></blockquote><blockquote type="cite">fd 're removed from original scheduler's pollset, and added to new scheduler's.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Bind port to scheduler together with process is also helpful to<br></blockquote><blockquote type="cite">performance, it reduces the cost of thread switches and<br></blockquote><blockquote type="cite">synchronization, and bound port won't be migrated between schedulers.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">After apply the two patches, with the same pressure(100 concurrent<br></blockquote><blockquote type="cite">clients),epoll_wait is invoked 49332 times per second, and get 3<br></blockquote><blockquote type="cite">revents each time in average, that is to say, our server responds<br></blockquote><blockquote type="cite">quicker and become more realtime.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"> cpu process times<br></blockquote><blockquote type="cite">revents min max avg timeouts<br></blockquote><blockquote type="cite"> all<br></blockquote><blockquote type="cite">49332 217186 - - 4 3<br></blockquote><blockquote type="cite">[ 2] beam.smp 3219<br></blockquote><blockquote type="cite">16050 2 7 4 0<br></blockquote><blockquote type="cite">[11] beam.smp 4275<br></blockquote><blockquote type="cite">16033 1 6 3 0<br></blockquote><blockquote type="cite">[ 8] beam.smp 4240<br></blockquote><blockquote type="cite">15992 1 6 3 0<br></blockquote><blockquote type="cite">[ 9] beam.smp 4316<br></blockquote><blockquote type="cite">15964 0 6 3 2<br></blockquote><blockquote type="cite">[10] beam.smp 4139<br></blockquote><blockquote type="cite">15851 1 6 3 0<br></blockquote><blockquote type="cite">[ 3] beam.smp 4256<br></blockquote><blockquote type="cite">15816 1 6 3 0<br></blockquote><blockquote type="cite">[ 1] beam.smp 3107<br></blockquote><blockquote type="cite">15800 2 7 5 0<br></blockquote><blockquote type="cite">[ 0] beam.smp 3727<br></blockquote><blockquote type="cite">15259 1 6 4 0<br></blockquote><blockquote type="cite">[ 7] beam.smp 2810<br></blockquote><blockquote type="cite">14722 3 7 5 0<br></blockquote><blockquote type="cite">[13] beam.smp 1981<br></blockquote><blockquote type="cite">11499 4 7 5 0<br></blockquote><blockquote type="cite">[15] beam.smp 2285<br></blockquote><blockquote type="cite">10942 3 6 4 0<br></blockquote><blockquote type="cite">[14] beam.smp 2258<br></blockquote><blockquote type="cite">10866 3 6 4 0<br></blockquote><blockquote type="cite">[ 4] beam.smp 2246<br></blockquote><blockquote type="cite">10849 3 6 4 0<br></blockquote><blockquote type="cite">[ 6] beam.smp 2206<br></blockquote><blockquote type="cite">10730 3 6 4 0<br></blockquote><blockquote type="cite">[12] beam.smp 2173<br></blockquote><blockquote type="cite">10573 3 6 4 0<br></blockquote><blockquote type="cite">[ 5] beam.smp 2093<br></blockquote><blockquote type="cite">10240 3 6 4 0<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">scheduler_wait no longer take so much time now,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"> 169.00 6.2% process_main beam.smp<br></blockquote><blockquote type="cite"> 55.00 2.0% _spin_lock [kernel]<br></blockquote><blockquote type="cite"> 45.00 1.7% driver_deliver_term beam.smp<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">so is idle CPU time<br></blockquote><blockquote type="cite">04:30:44 PM CPU %usr %nice %sys %iowait %irq %soft<br></blockquote><blockquote type="cite">%steal %guest %idle<br></blockquote><blockquote type="cite">04:30:45 PM all 60.34 0.00 21.44 0.00 0.06 16.45<br></blockquote><blockquote type="cite">0.00 0.00 1.71<br></blockquote><blockquote type="cite">04:30:46 PM all 60.99 0.00 21.22 0.00 0.00 16.26<br></blockquote><blockquote type="cite">0.00 0.00 1.52<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">and tcprstat shows, QPS is getting 100K, latency is less than 1 millisecond<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">timestamp<span class="Apple-tab-span" style="white-space:pre"> </span>count<span class="Apple-tab-span" style="white-space:pre"> </span>max<span class="Apple-tab-span" style="white-space:pre"> </span>min<span class="Apple-tab-span" style="white-space:pre"> </span>avg<span class="Apple-tab-span" style="white-space:pre"> </span>med<span class="Apple-tab-span" style="white-space:pre"> </span>stddev<span class="Apple-tab-span" style="white-space:pre"> </span>95_max<span class="Apple-tab-span" style="white-space:pre"> </span>95_avg<span class="Apple-tab-span" style="white-space:pre"> </span>95_std<span class="Apple-tab-span" style="white-space:pre"> </span>99_max<span class="Apple-tab-span" style="white-space:pre"> </span>99_avg<span class="Apple-tab-span" style="white-space:pre"> </span>99_std<br></blockquote><blockquote type="cite">1341822689<span class="Apple-tab-span" style="white-space:pre"> </span>96078<span class="Apple-tab-span" style="white-space:pre"> </span>11592<span class="Apple-tab-span" style="white-space:pre"> </span>314<span class="Apple-tab-span" style="white-space:pre"> </span>910<span class="Apple-tab-span" style="white-space:pre"> </span>817<span class="Apple-tab-span" style="white-space:pre"> </span>311<span class="Apple-tab-span" style="white-space:pre"> </span>1447<span class="Apple-tab-span" style="white-space:pre"> </span>869<span class="Apple-tab-span" style="white-space:pre"> </span>228<span class="Apple-tab-span" style="white-space:pre"> </span>1777<span class="Apple-tab-span" style="white-space:pre"> </span>897<span class="Apple-tab-span" style="white-space:pre"> </span>263<br></blockquote><blockquote type="cite">1341822690<span class="Apple-tab-span" style="white-space:pre"> </span>100651<span class="Apple-tab-span" style="white-space:pre"> </span>24245<span class="Apple-tab-span" style="white-space:pre"> </span>209<span class="Apple-tab-span" style="white-space:pre"> </span>914<span class="Apple-tab-span" style="white-space:pre"> </span>819<span class="Apple-tab-span" style="white-space:pre"> </span>381<span class="Apple-tab-span" style="white-space:pre"> </span>1458<span class="Apple-tab-span" style="white-space:pre"> </span>870<span class="Apple-tab-span" style="white-space:pre"> </span>229<span class="Apple-tab-span" style="white-space:pre"> </span>1800<span class="Apple-tab-span" style="white-space:pre"> </span>899<span class="Apple-tab-span" style="white-space:pre"> </span>263<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I also write a extreamly simple keep-alive http server(attached at the<br></blockquote><blockquote type="cite">end of mail), to compare performance before and after applying the<br></blockquote><blockquote type="cite">patches, mearused with apache ab tool(ab -n 1000000 -c 100 -k ), a 30%<br></blockquote><blockquote type="cite">performance gain can be get.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">before<br></blockquote><blockquote type="cite">Requests per second: 103671.70 [#/sec] (mean)<br></blockquote><blockquote type="cite">Time per request: 0.965 [ms] (mean)<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">after<br></blockquote><blockquote type="cite">Requests per second: 133701.24 [#/sec] (mean)<br></blockquote><blockquote type="cite">Time per request: 0.748 [ms] (mean)<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Patches can be found at github, compile with<br></blockquote><blockquote type="cite">./configure CFLAGS=-DERTS_POLLSET_PER_SCHEDULER<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Pollset per scheduler:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">git fetch <a href="git://github.com/weicao/otp.git">git://github.com/weicao/otp.git</a> pollset_per_scheduler<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><a href="https://github.com/weicao/otp/compare/weicao:master...weicao:pollset_per_scheduler">https://github.com/weicao/otp/compare/weicao:master...weicao:pollset_per_scheduler</a><br></blockquote><blockquote type="cite"><a href="https://github.com/weicao/otp/compare/weicao:master...weicao:pollset_per_scheduler.patch">https://github.com/weicao/otp/compare/weicao:master...weicao:pollset_per_scheduler.patch</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Bind port to scheduler:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">git fetch <a href="git://github.com/weicao/otp.git">git://github.com/weicao/otp.git</a> bind_port_to_scheduler<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><a href="https://github.com/weicao/otp/compare/weicao:pollset_per_scheduler...weicao:bind_port_to_scheduler">https://github.com/weicao/otp/compare/weicao:pollset_per_scheduler...weicao:bind_port_to_scheduler</a><br></blockquote><blockquote type="cite"><a href="https://github.com/weicao/otp/compare/weicao:pollset_per_scheduler...weicao:bind_port_to_scheduler.patch">https://github.com/weicao/otp/compare/weicao:pollset_per_scheduler...weicao:bind_port_to_scheduler.patch</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Appendix:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">-----------------------------------<br></blockquote><blockquote type="cite">proxy.erl<br></blockquote><blockquote type="cite">------------------------------------<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">-module(proxy).<br></blockquote><blockquote type="cite">-compile(export_all).<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">-define(RECBUF_SIZE, 8192).<br></blockquote><blockquote type="cite">-define(ACCEPT_TIMEOUT, 2000).<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">start([MyPortAtom, DestIpAtom, DestPortAtom]) -><br></blockquote><blockquote type="cite"> {MyPort, []} = string:to_integer(atom_to_list(MyPortAtom)),<br></blockquote><blockquote type="cite"> DestIp = atom_to_list(DestIpAtom),<br></blockquote><blockquote type="cite"> {DestPort, []} = string:to_integer(atom_to_list(DestPortAtom)),<br></blockquote><blockquote type="cite"> listen(MyPort, DestIp, DestPort),<br></blockquote><blockquote type="cite"> receive Any -><br></blockquote><blockquote type="cite"> io:format("recv ~p~n", [Any])<br></blockquote><blockquote type="cite"> end.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">listen(MyPort, DestIp, DestPort) -><br></blockquote><blockquote type="cite"> io:format("start proxy on [local] 0.0.0.0:~p -> [remote] ~p:~p~n",<br></blockquote><blockquote type="cite">[MyPort, DestIp, DestPort]),<br></blockquote><blockquote type="cite"> case gen_tcp:listen(MyPort,<br></blockquote><blockquote type="cite"> [inet,<br></blockquote><blockquote type="cite"> {ip, {0,0,0,0}},<br></blockquote><blockquote type="cite"> binary,<br></blockquote><blockquote type="cite"> {reuseaddr, true},<br></blockquote><blockquote type="cite"> {recbuf, ?RECBUF_SIZE},<br></blockquote><blockquote type="cite"> {active, false},<br></blockquote><blockquote type="cite"> {nodelay, true}<br></blockquote><blockquote type="cite"> ]) of<br></blockquote><blockquote type="cite"> {ok, Listen} -><br></blockquote><blockquote type="cite"> N = erlang:system_info(schedulers),<br></blockquote><blockquote type="cite"> lists:foreach(fun(I) -> accept(Listen, DestIp, DestPort,<br></blockquote><blockquote type="cite">I) end, lists:seq(1,N));<br></blockquote><blockquote type="cite"> {error, Reason} -><br></blockquote><blockquote type="cite"> io:format("error listen ~p~n", [Reason])<br></blockquote><blockquote type="cite"> end.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">accept(Listen, DestIp, DestPort, I) -><br></blockquote><blockquote type="cite"> spawn_opt(?MODULE, loop, [Listen, DestIp, DestPort, I], [{scheduler, I}]).<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">loop(Listen, DestIp, DestPort, I) -><br></blockquote><blockquote type="cite"> case gen_tcp:accept(Listen, ?ACCEPT_TIMEOUT) of<br></blockquote><blockquote type="cite"> {ok, S1} -><br></blockquote><blockquote type="cite"> accept(Listen, DestIp, DestPort, I),<br></blockquote><blockquote type="cite"> case catch gen_tcp:connect(DestIp, DestPort,<br></blockquote><blockquote type="cite"> [inet, binary, {active, false},<br></blockquote><blockquote type="cite"> {reuseaddr, true}, {nodelay, true}]) of<br></blockquote><blockquote type="cite"> {ok, S2} -><br></blockquote><blockquote type="cite"> io:format("new connection~n"),<br></blockquote><blockquote type="cite"> loop1(S1, S2);<br></blockquote><blockquote type="cite"> {error, Reason} -><br></blockquote><blockquote type="cite"> io:format("error connect ~p~n", [Reason])<br></blockquote><blockquote type="cite"> end;<br></blockquote><blockquote type="cite"> {error, timeout} -><br></blockquote><blockquote type="cite"> loop(Listen, DestIp, DestPort, I);<br></blockquote><blockquote type="cite"> Error -><br></blockquote><blockquote type="cite"> io:format("error accept ~p~n", [Error]),<br></blockquote><blockquote type="cite"> accept(Listen, DestIp, DestPort, I)<br></blockquote><blockquote type="cite"> end.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">loop1(S1, S2) -><br></blockquote><blockquote type="cite"> active(S1, S2),<br></blockquote><blockquote type="cite"> receive<br></blockquote><blockquote type="cite"> {tcp, S1, Data} -><br></blockquote><blockquote type="cite"> gen_tcp:send(S2, Data),<br></blockquote><blockquote type="cite"> loop1(S1, S2);<br></blockquote><blockquote type="cite"> {tcp, S2, Data} -><br></blockquote><blockquote type="cite"> gen_tcp:send(S1, Data),<br></blockquote><blockquote type="cite"> loop1(S1, S2);<br></blockquote><blockquote type="cite"> {tcp_closed, S1} -><br></blockquote><blockquote type="cite"> io:format("S1 close~n"),<br></blockquote><blockquote type="cite"> gen_tcp:close(S1),<br></blockquote><blockquote type="cite"> gen_tcp:close(S2);<br></blockquote><blockquote type="cite"> {tcp_closed, S2} -><br></blockquote><blockquote type="cite"> io:format("S2 close~n"),<br></blockquote><blockquote type="cite"> gen_tcp:close(S1),<br></blockquote><blockquote type="cite"> gen_tcp:close(S2)<br></blockquote><blockquote type="cite"> end.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">active(S1,S2)-><br></blockquote><blockquote type="cite"> inet:setopts(S1, [{active, once}]),<br></blockquote><blockquote type="cite"> inet:setopts(S2, [{active, once}]).<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">-----------------------------------<br></blockquote><blockquote type="cite">epollwait.stp<br></blockquote><blockquote type="cite">------------------------------------<br></blockquote><blockquote type="cite">#! /usr/bin/env stap<br></blockquote><blockquote type="cite">#<br></blockquote><blockquote type="cite">#<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">global epoll_timeout_flag, epoll_count, epoll_min, epoll_max,<br></blockquote><blockquote type="cite">epoll_times, epoll_timeouts<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">probe syscall.epoll_wait {<br></blockquote><blockquote type="cite"> if(timeout > 0) {<br></blockquote><blockquote type="cite"> epoll_timeout_flag[pid()] = 1<br></blockquote><blockquote type="cite"> }<br></blockquote><blockquote type="cite">}<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">probe syscall.epoll_wait.return {<br></blockquote><blockquote type="cite"> c = cpu()<br></blockquote><blockquote type="cite"> p = execname()<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"> epoll_times[c,p] ++<br></blockquote><blockquote type="cite"> epoll_count[c,p] += $return<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"> if($return == 0 && pid() in epoll_timeout_flag) {<br></blockquote><blockquote type="cite"> epoll_timeouts[c,p] ++<br></blockquote><blockquote type="cite"> delete epoll_timeout_flag[pid()]<br></blockquote><blockquote type="cite"> }<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"> if(!([c, p] in epoll_min)) {<br></blockquote><blockquote type="cite"> epoll_min[c,p] = $return<br></blockquote><blockquote type="cite"> } else if($return < epoll_min[c,p]) {<br></blockquote><blockquote type="cite"> epoll_min[c,p] = $return<br></blockquote><blockquote type="cite"> }<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"> if($return > epoll_max[c,p]) {<br></blockquote><blockquote type="cite"> epoll_max[c,p] = $return<br></blockquote><blockquote type="cite"> }<br></blockquote><blockquote type="cite">}<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">probe timer.s($1) {<br></blockquote><blockquote type="cite"> printf ("%4s %45s %10s %10s %10s %10s %10s %10s\n", "cpu",<br></blockquote><blockquote type="cite">"process", "times", "revents", "min", "max", "avg", "timeouts" )<br></blockquote><blockquote type="cite"> foreach ([cpu, process] in epoll_count- limit 20) {<br></blockquote><blockquote type="cite"> all_epoll_times += epoll_times[cpu,process]<br></blockquote><blockquote type="cite"> all_epoll_count += epoll_count[cpu,process]<br></blockquote><blockquote type="cite"> all_epoll_timeouts += epoll_timeouts[cpu,process]<br></blockquote><blockquote type="cite"> }<br></blockquote><blockquote type="cite"> printf ("%4s %45s %10d %10d %10s %10s %10d %10d\n",<br></blockquote><blockquote type="cite"> "all", "", all_epoll_times, all_epoll_count, "-", "-",<br></blockquote><blockquote type="cite"> all_epoll_count == 0? 0:all_epoll_count/all_epoll_times, all_epoll_timeouts)<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"> foreach ([cpu, process] in epoll_count- limit 20) {<br></blockquote><blockquote type="cite"> printf ("[%2d] %45s %10d %10d %10d %10d %10d %10d\n",<br></blockquote><blockquote type="cite"> cpu, process, epoll_times[cpu, process], epoll_count[cpu, process],<br></blockquote><blockquote type="cite"> epoll_min[cpu, process], epoll_max[cpu, process],<br></blockquote><blockquote type="cite"> epoll_count[cpu,process]/epoll_times[cpu,process],<br></blockquote><blockquote type="cite"> epoll_timeouts[cpu,process])<br></blockquote><blockquote type="cite"> }<br></blockquote><blockquote type="cite"> delete epoll_count<br></blockquote><blockquote type="cite"> delete epoll_min<br></blockquote><blockquote type="cite"> delete epoll_max<br></blockquote><blockquote type="cite"> delete epoll_times<br></blockquote><blockquote type="cite"> delete epoll_timeouts<br></blockquote><blockquote type="cite"> printf ("--------------------------------------------------------------------------\n\n"<br></blockquote><blockquote type="cite">)<br></blockquote><blockquote type="cite">}<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">------------------------------------------------<br></blockquote><blockquote type="cite">ehttpd.erl<br></blockquote><blockquote type="cite">-------------------------------------------------<br></blockquote><blockquote type="cite">-module(ehttpd).<br></blockquote><blockquote type="cite">-compile(export_all).<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">start() -><br></blockquote><blockquote type="cite"> start(8888).<br></blockquote><blockquote type="cite">start(Port) -><br></blockquote><blockquote type="cite"> N = erlang:system_info(schedulers),<br></blockquote><blockquote type="cite"> listen(Port, N),<br></blockquote><blockquote type="cite"> io:format("ehttpd ready with ~b schedulers on port ~b~n", [N, Port]),<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"> register(?MODULE, self()),<br></blockquote><blockquote type="cite"> receive Any -> io:format("~p~n", [Any]) end. %% to stop: ehttpd!stop.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">listen(Port, N) -><br></blockquote><blockquote type="cite"> Opts = [inet,<br></blockquote><blockquote type="cite"> binary,<br></blockquote><blockquote type="cite"> {active, false},<br></blockquote><blockquote type="cite"> {recbuf, 8192},<br></blockquote><blockquote type="cite"> {nodelay,true},<br></blockquote><blockquote type="cite"> {reuseaddr, true}],<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"> {ok, S} = gen_tcp:listen(Port, Opts),<br></blockquote><blockquote type="cite"> lists:foreach(fun(I)-> spawn_opt(?MODULE, accept, [S, I],<br></blockquote><blockquote type="cite">[{scheduler, I}]) end, lists:seq(1, N)).<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">accept(S, I) -><br></blockquote><blockquote type="cite"> case gen_tcp:accept(S) of<br></blockquote><blockquote type="cite"> {ok, Socket} -><br></blockquote><blockquote type="cite"> spawn_opt(?MODULE, accept, [S, I],[{scheduler,I}] ),<br></blockquote><blockquote type="cite"> io:format("new connection @~p~n", [I]),<br></blockquote><blockquote type="cite"> loop(Socket,<<>>);<br></blockquote><blockquote type="cite"> Error -> erlang:error(Error)<br></blockquote><blockquote type="cite"> end.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">loop(S,B) -><br></blockquote><blockquote type="cite"> inet:setopts(S, [{active, once}]),<br></blockquote><blockquote type="cite"> receive<br></blockquote><blockquote type="cite"> {tcp, S, Data} -><br></blockquote><blockquote type="cite"> B1 = <<B/binary, Data/binary>>,<br></blockquote><blockquote type="cite"> case binary:part(B1,{byte_size(B1), -4}) of<br></blockquote><blockquote type="cite"> <<"\r\n\r\n">> -><br></blockquote><blockquote type="cite"> Response = <<"HTTP/1.1 200 OK\r\nContent-Length:<br></blockquote><blockquote type="cite">12\r\nConnection: Keep-Alive\r\n\r\nhello world!">>,<br></blockquote><blockquote type="cite"> gen_tcp:send(S, Response),<br></blockquote><blockquote type="cite"> loop(S, <<>>);<br></blockquote><blockquote type="cite"> _ -><br></blockquote><blockquote type="cite"> loop(S, B1)<br></blockquote><blockquote type="cite"> end;<br></blockquote><blockquote type="cite"> {tcp_closed, S} -><br></blockquote><blockquote type="cite"> io:format("connection closed forced~n"),<br></blockquote><blockquote type="cite"> gen_tcp:close(S);<br></blockquote><blockquote type="cite"> Error -><br></blockquote><blockquote type="cite"> io:format("unexpected message~p~n", [Error]),<br></blockquote><blockquote type="cite"> Error<br></blockquote><blockquote type="cite"> end.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><br><br>_______________________________________________<br>erlang-questions mailing list<br><a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>http://erlang.org/mailman/listinfo/erlang-questions<br></div></blockquote></div><br><div apple-content-edited="true">
<div><br></div>
</div>
<br></div></div></div></body></html>