<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">I dont think so!<div><br><div><div>When I tested Wei's code teh first time, I suppressed all <b>io:format</b> statements and made</div><div>the inner loop lot simpler. Something like that:</div><div><br></div><div>[...]</div><div><div>loop(S,B) -></div><div>    inet:setopts(S, [{active, once}]),</div><div>    receive</div><div>        {tcp, S, _Data} -></div><div>            Response = <<"HTTP/1.1 200 OK\r\nContent-Length:</div><div>12\r\nConnection: Keep-Alive\r\n\r\nhello world!">>,</div><div>                    gen_tcp:send(S, Response),</div><div>                    loop(S, <<>>)</div><div>[...]</div></div><div><br></div><div>didn't change anything in my case.</div><div><br></div><div>Hope this help!</div><div><br></div><div><div>Regards,</div><div>Zabrane</div></div><div><br></div><div><div><div>On Jul 10, 2012, at 11:14 PM, erlang wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>Hi, all (first time)<br>on start - i'm sorry for my englisch language.<br>i think that the problem is on all lines where you use "io:format"<br>In my system when i start wrote in erlang I used many times io:format for debug. It wos big bottleneck.<br><br><br>JanM<br><br><br>W dniu 2012-07-09 11:01, Wei Cao pisze:<br><blockquote type="cite">Hi, all<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">We wrote a proxy server in Erlang, the proxy's logic is rather simple,<br></blockquote><blockquote type="cite">it listens on some TCP port, establishes new connection from user<br></blockquote><blockquote type="cite">client, forward packets back and forth between the client and backend<br></blockquote><blockquote type="cite">server after authentication until connection is closed.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">It's very easy to write such a proxy in Erlang, fork a process for<br></blockquote><blockquote type="cite">each new user connection and connect to the backend server in the same<br></blockquote><blockquote type="cite">process,  the process works like a pipe, sockets from both side is set<br></blockquote><blockquote type="cite">to the active once mode, whenever a tcp packet is received from one<br></blockquote><blockquote type="cite">socket, the packet will be sent to other socket. (A simplified version<br></blockquote><blockquote type="cite">of proxy code is attached at the end of the mail)<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">However, the performance is not quite satisfying, the proxy can handle<br></blockquote><blockquote type="cite">maximum only 40k requests on our 16 core machine(Intel Xeon L5630,<br></blockquote><blockquote type="cite">2.13GHz) with heavy stress(100 concurrent clients). We then analyzed<br></blockquote><blockquote type="cite">the behavior of beam.smp use tools like tcprstat, mpstat, perf, and<br></blockquote><blockquote type="cite">SystemTap.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">tcprstat shows QPS is about 40k, have average 1.7 millisecond latency<br></blockquote><blockquote type="cite">for each request.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">timestamp<span class="Apple-tab-span" style="white-space:pre">     </span>count<span class="Apple-tab-span" style="white-space:pre">       </span>max<span class="Apple-tab-span" style="white-space:pre"> </span>min<span class="Apple-tab-span" style="white-space:pre"> </span>avg<span class="Apple-tab-span" style="white-space:pre"> </span>med<span class="Apple-tab-span" style="white-space:pre"> </span>stddev<span class="Apple-tab-span" style="white-space:pre">      </span>95_max<span class="Apple-tab-span" style="white-space:pre">      </span>95_avg<span class="Apple-tab-span" style="white-space:pre">      </span>95_std<span class="Apple-tab-span" style="white-space:pre">      </span>99_max<span class="Apple-tab-span" style="white-space:pre">      </span>99_avg<span class="Apple-tab-span" style="white-space:pre">      </span>99_std<br></blockquote><blockquote type="cite">1341813326<span class="Apple-tab-span" style="white-space:pre">       </span>39416<span class="Apple-tab-span" style="white-space:pre">       </span>17873<span class="Apple-tab-span" style="white-space:pre">       </span>953<span class="Apple-tab-span" style="white-space:pre"> </span>1718<span class="Apple-tab-span" style="white-space:pre">        </span>1519<span class="Apple-tab-span" style="white-space:pre">        </span>724<span class="Apple-tab-span" style="white-space:pre"> </span>2919<span class="Apple-tab-span" style="white-space:pre">        </span>1609<span class="Apple-tab-span" style="white-space:pre">        </span>340<span class="Apple-tab-span" style="white-space:pre"> </span>3813<span class="Apple-tab-span" style="white-space:pre">        </span>1674<span class="Apple-tab-span" style="white-space:pre">        </span>462<br></blockquote><blockquote type="cite">1341813327<span class="Apple-tab-span" style="white-space:pre">  </span>40528<span class="Apple-tab-span" style="white-space:pre">       </span>9275<span class="Apple-tab-span" style="white-space:pre">        </span>884<span class="Apple-tab-span" style="white-space:pre"> </span>1645<span class="Apple-tab-span" style="white-space:pre">        </span>1493<span class="Apple-tab-span" style="white-space:pre">        </span>491<span class="Apple-tab-span" style="white-space:pre"> </span>2777<span class="Apple-tab-span" style="white-space:pre">        </span>1559<span class="Apple-tab-span" style="white-space:pre">        </span>290<span class="Apple-tab-span" style="white-space:pre"> </span>3508<span class="Apple-tab-span" style="white-space:pre">        </span>1619<span class="Apple-tab-span" style="white-space:pre">        </span>409<br></blockquote><blockquote type="cite">1341813328<span class="Apple-tab-span" style="white-space:pre">  </span>40009<span class="Apple-tab-span" style="white-space:pre">       </span>18105<span class="Apple-tab-span" style="white-space:pre">       </span>925<span class="Apple-tab-span" style="white-space:pre"> </span>1694<span class="Apple-tab-span" style="white-space:pre">        </span>1507<span class="Apple-tab-span" style="white-space:pre">        </span>714<span class="Apple-tab-span" style="white-space:pre"> </span>2868<span class="Apple-tab-span" style="white-space:pre">        </span>1586<span class="Apple-tab-span" style="white-space:pre">        </span>328<span class="Apple-tab-span" style="white-space:pre"> </span>3753<span class="Apple-tab-span" style="white-space:pre">        </span>1650<span class="Apple-tab-span" style="white-space:pre">        </span>450<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">mpstat shows 30% CPU is idle,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">03:30:19 PM  CPU    %usr   %nice    %sys %iowait    %irq   %soft<br></blockquote><blockquote type="cite">%steal  %guest   %idle<br></blockquote><blockquote type="cite">03:30:20 PM  all   38.69    0.00   21.92    0.00    0.06    7.52<br></blockquote><blockquote type="cite">0.00    0.00   31.80<br></blockquote><blockquote type="cite">03:30:21 PM  all   37.56    0.00   21.99    0.00    0.00    7.50<br></blockquote><blockquote type="cite">0.00    0.00   32.95<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">and perf top shows, much time is wasted in scheduler_wait, in spin wait I guess.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">9320.00 19.8% scheduler_wait<br></blockquote><blockquote type="cite">/home/mingsong.cw/erlangr16b/lib/erlang/erts-5.10/bin/beam.smp<br></blockquote><blockquote type="cite">1813.00  3.9% process_main<br></blockquote><blockquote type="cite">/home/mingsong.cw/erlangr16b/lib/erlang/erts-5.10/bin/beam.smp<br></blockquote><blockquote type="cite">1379.00  2.9% _spin_lock<br></blockquote><blockquote type="cite">/usr/lib/debug/lib/modules/2.6.32-131.21.1.tb477.el6.x86_64/vmlinux<br></blockquote><blockquote type="cite">1201.00  2.6% schedule<br></blockquote><blockquote type="cite">/home/mingsong.cw/erlangr16b/lib/erlang/erts-5.10/bin/beam.smp<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">It seems the performance may be associated with scheduler_wait() and<br></blockquote><blockquote type="cite">erts_check_io(), with a SystemTap script(attached at the end of this<br></blockquote><blockquote type="cite">mail), we can find out how many times the system call epoll_wait is<br></blockquote><blockquote type="cite">invoked by beam.smp and each time, how many revents it gets.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">  cpu                                       process      times<br></blockquote><blockquote type="cite">revents        min        max        avg   timeouts<br></blockquote><blockquote type="cite">  all<br></blockquote><blockquote type="cite">1754     128042          -          -         73          3<br></blockquote><blockquote type="cite">[14]                                      beam.smp        151<br></blockquote><blockquote type="cite">14127         82         97         93          0<br></blockquote><blockquote type="cite">[ 5]                                      beam.smp        142<br></blockquote><blockquote type="cite">13291         83         97         93          0<br></blockquote><blockquote type="cite">[13]                                      beam.smp        127<br></blockquote><blockquote type="cite">11948         86         96         94          0<br></blockquote><blockquote type="cite">[ 6]                                      beam.smp        127<br></blockquote><blockquote type="cite">11836         81         96         93          0<br></blockquote><blockquote type="cite">[ 4]                                      beam.smp        121<br></blockquote><blockquote type="cite">11323         81         96         93          0<br></blockquote><blockquote type="cite">[15]                                      beam.smp        117<br></blockquote><blockquote type="cite">10935         83         96         93          0<br></blockquote><blockquote type="cite">[12]                                      beam.smp        486<br></blockquote><blockquote type="cite">10128          0         96         20          2<br></blockquote><blockquote type="cite">[ 1]                                      beam.smp         71<br></blockquote><blockquote type="cite">6549         71        100         92          0<br></blockquote><blockquote type="cite">[ 2]                                      beam.smp         62<br></blockquote><blockquote type="cite">5695         82         96         91          0<br></blockquote><blockquote type="cite">[ 7]                                      beam.smp         55<br></blockquote><blockquote type="cite">5102         81         95         92          0<br></blockquote><blockquote type="cite">[11]                                      beam.smp         52<br></blockquote><blockquote type="cite">4822         85         95         92          0<br></blockquote><blockquote type="cite">[ 9]                                      beam.smp         52<br></blockquote><blockquote type="cite">4799         85         95         92          0<br></blockquote><blockquote type="cite">[ 8]                                      beam.smp         51<br></blockquote><blockquote type="cite">4680         78         95         91          0<br></blockquote><blockquote type="cite">[10]                                      beam.smp         49<br></blockquote><blockquote type="cite">4508         85         97         92          0<br></blockquote><blockquote type="cite">[ 3]                                      beam.smp         46<br></blockquote><blockquote type="cite">4211         81         95         91          0<br></blockquote><blockquote type="cite">[ 0]                                      beam.smp         44<br></blockquote><blockquote type="cite">4088         83         95         92          0<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">The resuls shows, epoll_wait is invoked 1754 times each second, and<br></blockquote><blockquote type="cite">get 73 io events in average. This is unacceptable for writing high<br></blockquote><blockquote type="cite">performance server. Because if epoll_wait is invoked no more than 2k<br></blockquote><blockquote type="cite">times per second, then read/write a packet would cost more than 500ms,<br></blockquote><blockquote type="cite">which causes long delay and affects the throughput finally.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">The problem relies on there is only one global pollset in system wide,<br></blockquote><blockquote type="cite">so at a time there is no more than one scheduler can call<br></blockquote><blockquote type="cite">erts_check_io() to obtain pending io tasks from underlying pollset,<br></blockquote><blockquote type="cite">and no scheduler can call erts_check_io() before all pending io<br></blockquote><blockquote type="cite">tasks're processed, so for IO bounded application, it's very likely<br></blockquote><blockquote type="cite">that a scheduler finish its own job, but must wait idly for other<br></blockquote><blockquote type="cite">schedulers to complete theirs.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Hence, we develops a patch to slove this problem, by having a pollset<br></blockquote><blockquote type="cite">for each scheduler, so that each scheduler can invoke erts_check_io()<br></blockquote><blockquote type="cite">on its own pollset concurrently. After a scheduler complete its tasks,<br></blockquote><blockquote type="cite">it can invoke erts_check_io() immediately no matter what state other<br></blockquote><blockquote type="cite">schedulers're in. This patch also handles port migration situation,<br></blockquote><blockquote type="cite">all used file descriptors in each port're recorded, when a port is<br></blockquote><blockquote type="cite">migrated, these<br></blockquote><blockquote type="cite">fd 're removed from original scheduler's pollset, and added to new scheduler's.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Bind port to scheduler together with process is also helpful to<br></blockquote><blockquote type="cite">performance, it reduces the cost of thread switches and<br></blockquote><blockquote type="cite">synchronization, and bound port won't be migrated between schedulers.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">After apply the two patches, with the same pressure(100 concurrent<br></blockquote><blockquote type="cite">clients),epoll_wait is invoked 49332  times per second, and get 3<br></blockquote><blockquote type="cite">revents each time in average, that is to say, our server responds<br></blockquote><blockquote type="cite">quicker and become more realtime.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">  cpu                                       process      times<br></blockquote><blockquote type="cite">revents        min        max        avg   timeouts<br></blockquote><blockquote type="cite">  all<br></blockquote><blockquote type="cite">49332     217186          -          -          4          3<br></blockquote><blockquote type="cite">[ 2]                                      beam.smp       3219<br></blockquote><blockquote type="cite">16050          2          7          4          0<br></blockquote><blockquote type="cite">[11]                                      beam.smp       4275<br></blockquote><blockquote type="cite">16033          1          6          3          0<br></blockquote><blockquote type="cite">[ 8]                                      beam.smp       4240<br></blockquote><blockquote type="cite">15992          1          6          3          0<br></blockquote><blockquote type="cite">[ 9]                                      beam.smp       4316<br></blockquote><blockquote type="cite">15964          0          6          3          2<br></blockquote><blockquote type="cite">[10]                                      beam.smp       4139<br></blockquote><blockquote type="cite">15851          1          6          3          0<br></blockquote><blockquote type="cite">[ 3]                                      beam.smp       4256<br></blockquote><blockquote type="cite">15816          1          6          3          0<br></blockquote><blockquote type="cite">[ 1]                                      beam.smp       3107<br></blockquote><blockquote type="cite">15800          2          7          5          0<br></blockquote><blockquote type="cite">[ 0]                                      beam.smp       3727<br></blockquote><blockquote type="cite">15259          1          6          4          0<br></blockquote><blockquote type="cite">[ 7]                                      beam.smp       2810<br></blockquote><blockquote type="cite">14722          3          7          5          0<br></blockquote><blockquote type="cite">[13]                                      beam.smp       1981<br></blockquote><blockquote type="cite">11499          4          7          5          0<br></blockquote><blockquote type="cite">[15]                                      beam.smp       2285<br></blockquote><blockquote type="cite">10942          3          6          4          0<br></blockquote><blockquote type="cite">[14]                                      beam.smp       2258<br></blockquote><blockquote type="cite">10866          3          6          4          0<br></blockquote><blockquote type="cite">[ 4]                                      beam.smp       2246<br></blockquote><blockquote type="cite">10849          3          6          4          0<br></blockquote><blockquote type="cite">[ 6]                                      beam.smp       2206<br></blockquote><blockquote type="cite">10730          3          6          4          0<br></blockquote><blockquote type="cite">[12]                                      beam.smp       2173<br></blockquote><blockquote type="cite">10573          3          6          4          0<br></blockquote><blockquote type="cite">[ 5]                                      beam.smp       2093<br></blockquote><blockquote type="cite">10240          3          6          4          0<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">scheduler_wait no longer take so much time now,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">               169.00  6.2% process_main                    beam.smp<br></blockquote><blockquote type="cite">                55.00  2.0% _spin_lock                      [kernel]<br></blockquote><blockquote type="cite">                45.00  1.7% driver_deliver_term             beam.smp<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">so is idle CPU time<br></blockquote><blockquote type="cite">04:30:44 PM  CPU    %usr   %nice    %sys %iowait    %irq   %soft<br></blockquote><blockquote type="cite">%steal  %guest   %idle<br></blockquote><blockquote type="cite">04:30:45 PM  all   60.34    0.00   21.44    0.00    0.06   16.45<br></blockquote><blockquote type="cite">0.00    0.00    1.71<br></blockquote><blockquote type="cite">04:30:46 PM  all   60.99    0.00   21.22    0.00    0.00   16.26<br></blockquote><blockquote type="cite">0.00    0.00    1.52<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">and tcprstat shows, QPS is getting 100K, latency is less than 1 millisecond<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">timestamp<span class="Apple-tab-span" style="white-space:pre">     </span>count<span class="Apple-tab-span" style="white-space:pre">       </span>max<span class="Apple-tab-span" style="white-space:pre"> </span>min<span class="Apple-tab-span" style="white-space:pre"> </span>avg<span class="Apple-tab-span" style="white-space:pre"> </span>med<span class="Apple-tab-span" style="white-space:pre"> </span>stddev<span class="Apple-tab-span" style="white-space:pre">      </span>95_max<span class="Apple-tab-span" style="white-space:pre">      </span>95_avg<span class="Apple-tab-span" style="white-space:pre">      </span>95_std<span class="Apple-tab-span" style="white-space:pre">      </span>99_max<span class="Apple-tab-span" style="white-space:pre">      </span>99_avg<span class="Apple-tab-span" style="white-space:pre">      </span>99_std<br></blockquote><blockquote type="cite">1341822689<span class="Apple-tab-span" style="white-space:pre">       </span>96078<span class="Apple-tab-span" style="white-space:pre">       </span>11592<span class="Apple-tab-span" style="white-space:pre">       </span>314<span class="Apple-tab-span" style="white-space:pre"> </span>910<span class="Apple-tab-span" style="white-space:pre"> </span>817<span class="Apple-tab-span" style="white-space:pre"> </span>311<span class="Apple-tab-span" style="white-space:pre"> </span>1447<span class="Apple-tab-span" style="white-space:pre">        </span>869<span class="Apple-tab-span" style="white-space:pre"> </span>228<span class="Apple-tab-span" style="white-space:pre"> </span>1777<span class="Apple-tab-span" style="white-space:pre">        </span>897<span class="Apple-tab-span" style="white-space:pre"> </span>263<br></blockquote><blockquote type="cite">1341822690<span class="Apple-tab-span" style="white-space:pre">  </span>100651<span class="Apple-tab-span" style="white-space:pre">      </span>24245<span class="Apple-tab-span" style="white-space:pre">       </span>209<span class="Apple-tab-span" style="white-space:pre"> </span>914<span class="Apple-tab-span" style="white-space:pre"> </span>819<span class="Apple-tab-span" style="white-space:pre"> </span>381<span class="Apple-tab-span" style="white-space:pre"> </span>1458<span class="Apple-tab-span" style="white-space:pre">        </span>870<span class="Apple-tab-span" style="white-space:pre"> </span>229<span class="Apple-tab-span" style="white-space:pre"> </span>1800<span class="Apple-tab-span" style="white-space:pre">        </span>899<span class="Apple-tab-span" style="white-space:pre"> </span>263<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I also write a extreamly simple keep-alive http server(attached at the<br></blockquote><blockquote type="cite">end of mail), to compare performance before and after applying the<br></blockquote><blockquote type="cite">patches, mearused with apache ab tool(ab -n 1000000 -c 100 -k ), a 30%<br></blockquote><blockquote type="cite">performance gain can be get.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">before<br></blockquote><blockquote type="cite">Requests per second:    103671.70 [#/sec] (mean)<br></blockquote><blockquote type="cite">Time per request:       0.965 [ms] (mean)<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">after<br></blockquote><blockquote type="cite">Requests per second:    133701.24 [#/sec] (mean)<br></blockquote><blockquote type="cite">Time per request:       0.748 [ms] (mean)<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Patches can be found at github, compile with<br></blockquote><blockquote type="cite">./configure CFLAGS=-DERTS_POLLSET_PER_SCHEDULER<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Pollset per scheduler:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">git fetch <a href="git://github.com/weicao/otp.git">git://github.com/weicao/otp.git</a> pollset_per_scheduler<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><a href="https://github.com/weicao/otp/compare/weicao:master...weicao:pollset_per_scheduler">https://github.com/weicao/otp/compare/weicao:master...weicao:pollset_per_scheduler</a><br></blockquote><blockquote type="cite"><a href="https://github.com/weicao/otp/compare/weicao:master...weicao:pollset_per_scheduler.patch">https://github.com/weicao/otp/compare/weicao:master...weicao:pollset_per_scheduler.patch</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Bind port to scheduler:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">git fetch <a href="git://github.com/weicao/otp.git">git://github.com/weicao/otp.git</a> bind_port_to_scheduler<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><a href="https://github.com/weicao/otp/compare/weicao:pollset_per_scheduler...weicao:bind_port_to_scheduler">https://github.com/weicao/otp/compare/weicao:pollset_per_scheduler...weicao:bind_port_to_scheduler</a><br></blockquote><blockquote type="cite"><a href="https://github.com/weicao/otp/compare/weicao:pollset_per_scheduler...weicao:bind_port_to_scheduler.patch">https://github.com/weicao/otp/compare/weicao:pollset_per_scheduler...weicao:bind_port_to_scheduler.patch</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Appendix:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">-----------------------------------<br></blockquote><blockquote type="cite">proxy.erl<br></blockquote><blockquote type="cite">------------------------------------<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">-module(proxy).<br></blockquote><blockquote type="cite">-compile(export_all).<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">-define(RECBUF_SIZE, 8192).<br></blockquote><blockquote type="cite">-define(ACCEPT_TIMEOUT, 2000).<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">start([MyPortAtom, DestIpAtom, DestPortAtom]) -><br></blockquote><blockquote type="cite">     {MyPort, []} =  string:to_integer(atom_to_list(MyPortAtom)),<br></blockquote><blockquote type="cite">     DestIp = atom_to_list(DestIpAtom),<br></blockquote><blockquote type="cite">     {DestPort, []} = string:to_integer(atom_to_list(DestPortAtom)),<br></blockquote><blockquote type="cite">     listen(MyPort, DestIp, DestPort),<br></blockquote><blockquote type="cite">     receive Any -><br></blockquote><blockquote type="cite">         io:format("recv ~p~n", [Any])<br></blockquote><blockquote type="cite">     end.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">listen(MyPort, DestIp, DestPort) -><br></blockquote><blockquote type="cite">     io:format("start proxy on [local] 0.0.0.0:~p -> [remote] ~p:~p~n",<br></blockquote><blockquote type="cite">[MyPort, DestIp, DestPort]),<br></blockquote><blockquote type="cite">     case gen_tcp:listen(MyPort,<br></blockquote><blockquote type="cite">            [inet,<br></blockquote><blockquote type="cite">             {ip, {0,0,0,0}},<br></blockquote><blockquote type="cite">             binary,<br></blockquote><blockquote type="cite">             {reuseaddr, true},<br></blockquote><blockquote type="cite">             {recbuf, ?RECBUF_SIZE},<br></blockquote><blockquote type="cite">             {active, false},<br></blockquote><blockquote type="cite">             {nodelay, true}<br></blockquote><blockquote type="cite">            ]) of<br></blockquote><blockquote type="cite">         {ok, Listen} -><br></blockquote><blockquote type="cite">             N = erlang:system_info(schedulers),<br></blockquote><blockquote type="cite">             lists:foreach(fun(I) -> accept(Listen, DestIp, DestPort,<br></blockquote><blockquote type="cite">I) end, lists:seq(1,N));<br></blockquote><blockquote type="cite">         {error, Reason} -><br></blockquote><blockquote type="cite">             io:format("error listen ~p~n", [Reason])<br></blockquote><blockquote type="cite">     end.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">accept(Listen, DestIp, DestPort, I) -><br></blockquote><blockquote type="cite">     spawn_opt(?MODULE, loop, [Listen, DestIp, DestPort, I], [{scheduler, I}]).<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">loop(Listen, DestIp, DestPort, I) -><br></blockquote><blockquote type="cite">     case gen_tcp:accept(Listen, ?ACCEPT_TIMEOUT) of<br></blockquote><blockquote type="cite">         {ok, S1} -><br></blockquote><blockquote type="cite">             accept(Listen, DestIp, DestPort, I),<br></blockquote><blockquote type="cite">             case catch gen_tcp:connect(DestIp, DestPort,<br></blockquote><blockquote type="cite">                            [inet, binary, {active, false},<br></blockquote><blockquote type="cite">                             {reuseaddr, true}, {nodelay, true}]) of<br></blockquote><blockquote type="cite">                 {ok, S2} -><br></blockquote><blockquote type="cite">                     io:format("new connection~n"),<br></blockquote><blockquote type="cite">                     loop1(S1, S2);<br></blockquote><blockquote type="cite">                 {error, Reason} -><br></blockquote><blockquote type="cite">                     io:format("error connect ~p~n", [Reason])<br></blockquote><blockquote type="cite">             end;<br></blockquote><blockquote type="cite">         {error, timeout} -><br></blockquote><blockquote type="cite">             loop(Listen, DestIp, DestPort, I);<br></blockquote><blockquote type="cite">         Error -><br></blockquote><blockquote type="cite">             io:format("error accept ~p~n", [Error]),<br></blockquote><blockquote type="cite">             accept(Listen, DestIp, DestPort, I)<br></blockquote><blockquote type="cite">     end.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">loop1(S1, S2) -><br></blockquote><blockquote type="cite">     active(S1, S2),<br></blockquote><blockquote type="cite">     receive<br></blockquote><blockquote type="cite">         {tcp, S1, Data} -><br></blockquote><blockquote type="cite">             gen_tcp:send(S2, Data),<br></blockquote><blockquote type="cite">             loop1(S1, S2);<br></blockquote><blockquote type="cite">         {tcp, S2, Data} -><br></blockquote><blockquote type="cite">             gen_tcp:send(S1, Data),<br></blockquote><blockquote type="cite">             loop1(S1, S2);<br></blockquote><blockquote type="cite">         {tcp_closed, S1} -><br></blockquote><blockquote type="cite">             io:format("S1 close~n"),<br></blockquote><blockquote type="cite">             gen_tcp:close(S1),<br></blockquote><blockquote type="cite">             gen_tcp:close(S2);<br></blockquote><blockquote type="cite">         {tcp_closed, S2} -><br></blockquote><blockquote type="cite">             io:format("S2 close~n"),<br></blockquote><blockquote type="cite">             gen_tcp:close(S1),<br></blockquote><blockquote type="cite">             gen_tcp:close(S2)<br></blockquote><blockquote type="cite">     end.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">active(S1,S2)-><br></blockquote><blockquote type="cite">     inet:setopts(S1, [{active, once}]),<br></blockquote><blockquote type="cite">     inet:setopts(S2, [{active, once}]).<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">-----------------------------------<br></blockquote><blockquote type="cite">epollwait.stp<br></blockquote><blockquote type="cite">------------------------------------<br></blockquote><blockquote type="cite">#! /usr/bin/env stap<br></blockquote><blockquote type="cite">#<br></blockquote><blockquote type="cite">#<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">global epoll_timeout_flag, epoll_count, epoll_min, epoll_max,<br></blockquote><blockquote type="cite">epoll_times, epoll_timeouts<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">probe  syscall.epoll_wait {<br></blockquote><blockquote type="cite">     if(timeout > 0) {<br></blockquote><blockquote type="cite">         epoll_timeout_flag[pid()] = 1<br></blockquote><blockquote type="cite">     }<br></blockquote><blockquote type="cite">}<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">probe  syscall.epoll_wait.return {<br></blockquote><blockquote type="cite">   c = cpu()<br></blockquote><blockquote type="cite">   p = execname()<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">   epoll_times[c,p] ++<br></blockquote><blockquote type="cite">   epoll_count[c,p] += $return<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">   if($return == 0 && pid() in epoll_timeout_flag) {<br></blockquote><blockquote type="cite">     epoll_timeouts[c,p] ++<br></blockquote><blockquote type="cite">     delete epoll_timeout_flag[pid()]<br></blockquote><blockquote type="cite">   }<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">   if(!([c, p] in epoll_min)) {<br></blockquote><blockquote type="cite">     epoll_min[c,p] = $return<br></blockquote><blockquote type="cite">   } else if($return < epoll_min[c,p]) {<br></blockquote><blockquote type="cite">     epoll_min[c,p] = $return<br></blockquote><blockquote type="cite">   }<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">   if($return > epoll_max[c,p]) {<br></blockquote><blockquote type="cite">     epoll_max[c,p] = $return<br></blockquote><blockquote type="cite">   }<br></blockquote><blockquote type="cite">}<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">probe timer.s($1) {<br></blockquote><blockquote type="cite">   printf ("%4s %45s %10s %10s %10s %10s %10s %10s\n", "cpu",<br></blockquote><blockquote type="cite">"process", "times", "revents", "min", "max", "avg", "timeouts" )<br></blockquote><blockquote type="cite">   foreach ([cpu, process] in epoll_count- limit 20) {<br></blockquote><blockquote type="cite">       all_epoll_times += epoll_times[cpu,process]<br></blockquote><blockquote type="cite">       all_epoll_count += epoll_count[cpu,process]<br></blockquote><blockquote type="cite">       all_epoll_timeouts += epoll_timeouts[cpu,process]<br></blockquote><blockquote type="cite">   }<br></blockquote><blockquote type="cite">   printf ("%4s %45s %10d %10d %10s %10s %10d %10d\n",<br></blockquote><blockquote type="cite">     "all", "", all_epoll_times, all_epoll_count, "-", "-",<br></blockquote><blockquote type="cite">     all_epoll_count == 0? 0:all_epoll_count/all_epoll_times, all_epoll_timeouts)<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">   foreach ([cpu, process] in epoll_count- limit 20) {<br></blockquote><blockquote type="cite">      printf ("[%2d] %45s %10d %10d %10d %10d %10d %10d\n",<br></blockquote><blockquote type="cite">         cpu, process, epoll_times[cpu, process], epoll_count[cpu, process],<br></blockquote><blockquote type="cite">         epoll_min[cpu, process], epoll_max[cpu, process],<br></blockquote><blockquote type="cite">         epoll_count[cpu,process]/epoll_times[cpu,process],<br></blockquote><blockquote type="cite">         epoll_timeouts[cpu,process])<br></blockquote><blockquote type="cite">   }<br></blockquote><blockquote type="cite">   delete epoll_count<br></blockquote><blockquote type="cite">   delete epoll_min<br></blockquote><blockquote type="cite">   delete epoll_max<br></blockquote><blockquote type="cite">   delete epoll_times<br></blockquote><blockquote type="cite">   delete epoll_timeouts<br></blockquote><blockquote type="cite">   printf ("--------------------------------------------------------------------------\n\n"<br></blockquote><blockquote type="cite">)<br></blockquote><blockquote type="cite">}<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">------------------------------------------------<br></blockquote><blockquote type="cite">ehttpd.erl<br></blockquote><blockquote type="cite">-------------------------------------------------<br></blockquote><blockquote type="cite">-module(ehttpd).<br></blockquote><blockquote type="cite">-compile(export_all).<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">start() -><br></blockquote><blockquote type="cite">     start(8888).<br></blockquote><blockquote type="cite">start(Port) -><br></blockquote><blockquote type="cite">     N = erlang:system_info(schedulers),<br></blockquote><blockquote type="cite">     listen(Port, N),<br></blockquote><blockquote type="cite">     io:format("ehttpd ready with ~b schedulers on port ~b~n", [N, Port]),<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">     register(?MODULE, self()),<br></blockquote><blockquote type="cite">     receive Any -> io:format("~p~n", [Any]) end.  %% to stop: ehttpd!stop.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">listen(Port, N) -><br></blockquote><blockquote type="cite">     Opts = [inet,<br></blockquote><blockquote type="cite">             binary,<br></blockquote><blockquote type="cite">             {active, false},<br></blockquote><blockquote type="cite">             {recbuf, 8192},<br></blockquote><blockquote type="cite">             {nodelay,true},<br></blockquote><blockquote type="cite">             {reuseaddr, true}],<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">     {ok, S} = gen_tcp:listen(Port, Opts),<br></blockquote><blockquote type="cite">     lists:foreach(fun(I)-> spawn_opt(?MODULE, accept, [S, I],<br></blockquote><blockquote type="cite">[{scheduler, I}]) end, lists:seq(1, N)).<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">accept(S, I) -><br></blockquote><blockquote type="cite">     case gen_tcp:accept(S) of<br></blockquote><blockquote type="cite">         {ok, Socket} -><br></blockquote><blockquote type="cite">             spawn_opt(?MODULE, accept, [S, I],[{scheduler,I}] ),<br></blockquote><blockquote type="cite">             io:format("new connection @~p~n", [I]),<br></blockquote><blockquote type="cite">             loop(Socket,<<>>);<br></blockquote><blockquote type="cite">         Error    -> erlang:error(Error)<br></blockquote><blockquote type="cite">     end.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">loop(S,B) -><br></blockquote><blockquote type="cite">     inet:setopts(S, [{active, once}]),<br></blockquote><blockquote type="cite">     receive<br></blockquote><blockquote type="cite">         {tcp, S, Data} -><br></blockquote><blockquote type="cite">             B1 = <<B/binary, Data/binary>>,<br></blockquote><blockquote type="cite">             case binary:part(B1,{byte_size(B1), -4}) of<br></blockquote><blockquote type="cite">                 <<"\r\n\r\n">> -><br></blockquote><blockquote type="cite">                     Response = <<"HTTP/1.1 200 OK\r\nContent-Length:<br></blockquote><blockquote type="cite">12\r\nConnection: Keep-Alive\r\n\r\nhello world!">>,<br></blockquote><blockquote type="cite">                     gen_tcp:send(S, Response),<br></blockquote><blockquote type="cite">                     loop(S, <<>>);<br></blockquote><blockquote type="cite">                 _ -><br></blockquote><blockquote type="cite">                     loop(S, B1)<br></blockquote><blockquote type="cite">             end;<br></blockquote><blockquote type="cite">         {tcp_closed, S} -><br></blockquote><blockquote type="cite">             io:format("connection closed forced~n"),<br></blockquote><blockquote type="cite">             gen_tcp:close(S);<br></blockquote><blockquote type="cite">         Error -><br></blockquote><blockquote type="cite">             io:format("unexpected message~p~n", [Error]),<br></blockquote><blockquote type="cite">             Error<br></blockquote><blockquote type="cite">     end.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><br><br>_______________________________________________<br>erlang-questions mailing list<br><a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>http://erlang.org/mailman/listinfo/erlang-questions<br></div></blockquote></div><br><div apple-content-edited="true">
<div><br></div>
</div>
<br></div></div></div></body></html>