<html>
<head>
<meta http-equiv=Content-Type content="text/html; charset=iso-8859-1">
<meta name=Generator content="Microsoft Word 11 (filtered)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:Arial;}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:blue;
text-decoration:underline;}
span.Typed
{font-family:"Courier New";}
span.tty
{font-family:"Courier New";}
span.Name
{font-style:italic;}
span.Variable
{font-family:"Times New Roman";
font-style:italic;}
span.EmailStyle21
{font-family:Arial;
color:navy;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
{page:Section1;}
-->
</style>
</head>
<body lang=EN-US link=blue vlink=blue>
<div class=Section1>
<p class=MsoNormal><span style='color:navy'>I tried some alternative block
sizes (using the blksize option). I found that from 1 to somewhere
around––maybe a bit short of––1000 bytes, the test was
able to send about 300,000 blocks in 10 seconds regardless of size. (That
means, 0.03 MB/sec for block size of 1, 0.3 MB/sec for block size of 10, 3
MB/sec for block size of 100, etc.) I suspect the system was CPU bound at
those levels.</span></p>
<p class=MsoNormal><span style='color:navy'> </span></p>
<p class=MsoNormal><span style='color:navy'>Above 1000, the number of blocks
sent seemed to decrease, though this was more than offset by the increased size
of the blocks. Above about 10,000 byte blocks (may have been less, I didn’t
check any value between 4,000 and 10,000), however, performance peaked and
block size no longer mattered: it always sent between 70 and 80 MB/sec. My
machine is clearly slower than Edwin’s…</span></p>
<p class=MsoNormal><span style='color:navy'> </span></p>
<p class=MsoNormal><span style='color:navy'>DBM</span></p>
<p class=MsoNormal><span style='color:navy'> </span></p>
<div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'>
<div>
<div class=MsoNormal align=center style='text-align:center'>
<hr size=2 width="100%" align=center tabindex=-1>
</div>
<p class=MsoNormal><b><span style='font-family:Tahoma'>From:</span></b><span
style='font-family:Tahoma'> erlang-questions-bounces@erlang.org
[mailto:erlang-questions-bounces@erlang.org] <b>On Behalf Of </b>Rapsey<br>
<b>Sent:</b> Tuesday, June 24, 2008 14:01<br>
<b>To:</b> erlang-questions@erlang.org<br>
<b>Subject:</b> Re: [erlang-questions] why is gen_tcp:send slow?</span></p>
</div>
<p class=MsoNormal> </p>
<p class=MsoNormal style='margin-bottom:12.0pt'>You're using very large
packets. I think the results would be much more telling if the packets would be
a few kB at most. That is closer to most real life situations. <br>
<br>
<br>
Sergej</p>
<div>
<p class=MsoNormal>On Tue, Jun 24, 2008 at 8:43 PM, Edwin Fine <<a
href="mailto:erlang-questions_efine@usa.net">erlang-questions_efine@usa.net</a>>
wrote:</p>
<p class=MsoNormal style='margin-bottom:12.0pt'>I wrote a small benchmark in
Erlang to see how fast I could get socket communications to go. All the
benchmark does is pump the same buffer to a socket for (by default) 10 seconds.
It uses {active, once} each time, just like you do. <br>
<br>
Server TCP options:<br>
{active, once},<br>
{reuseaddr, true},<br>
{packet, 0},<br>
{packet_size, 65536},<br>
{recbuf, 1000000}<br>
<br>
Client TCP options:<br>
{packet, raw},<br>
{packet_size, 65536},<br>
{sndbuf, 1024 * 1024},<br>
{send_timeout, 3000}<br>
<br>
Here are some results using Erlang R12B-3 (erl +K true in the Linux version):<br>
<br>
Linux (Ubuntu 8.10 x86_64, Intel Core 2 Q6600, 8 GB):<br>
- Using localhost (<a href="http://127.0.0.1" target="_blank">127.0.0.1</a>):
7474.14 MB in 10.01 secs (746.66 MB/sec)<br>
- Using 192.168.x.x IP address: 8064.94 MB in 10.00 secs (806.22 MB/sec) [Don't
ask me why it's faster than using loopback, I repeated the tests and got the
same result]<br>
<br>
Windows XP SP3 (32 bits), Intel Core 2 Duo E6600:<br>
- Using loopback: 2166.97 MB in 10.02 secs (216.35 MB/sec)<br>
- Using 192.168.x.x IP address: 2140.72 MB in 10.02 secs (213.75 MB/sec)<br>
- On Gigabit Ethernet to the Q6600 Linux box: 1063.61 MB in 10.02 secs (106.17
MB/sec) using non-jumbo frames. I don't think my router supports jumbo frames.<br>
<br>
There's undoubtedly a huge discrepancy between the two systems, whether because
of kernel poll in Linux, or that it's 64 bits, or unoptimized Windows TCP/IP
flags, I don't know. I don't believe it's the number of CPUs (there's only 1
process sending and one receiving), or the CPU speed (they are both 2.4 GHz
Core 2s).<br>
<br>
Maybe some Erlang TCP/IP gurus could comment.<br>
<br>
I've attached the code for interest. It's not supposed to be production
quality, so please don't beat me up :) although I am always open to suggestions
for improvement. If you do improve it, I'd like to see what you've done. Maybe
there is another simple Erlang tcp benchmark program out there (i.e. not
Tsung), but I couldn't find one in a cursory Google search.<br>
<br>
To run:<br>
<br>
VM1:<br>
<br>
tb_server:start(Port, Opts).<br>
tb_server:stop() to stop.<br>
<br>
Port = integer()<br>
Opts = []|[opt()]<br>
opt() = {atom(), term()} (Accepts inet setopts options, too)<br>
<br>
The server prints out the transfer rate (for simplicity).<br>
<br>
VM2:<br>
tb_client(Host, Port, Opts).<br>
<br>
Host = atom()|string() hostname or IP address<br>
Port, Opts as in tb_server<br>
<br>
Runs for 10 seconds, sending a 64K buffer as fast as possible to Host/Port.<br>
You can change this to 20 seconds (e.g.) by adding the tupls {time_limit,
20000} to Opts.<br>
You can change buffer size by adding the tuple {blksize, Bytes} to Opts.</p>
<div>
<p class=MsoNormal>2008/6/20 Rapsey <<a href="mailto:rapsey@gmail.com"
target="_blank">rapsey@gmail.com</a>>:</p>
<div>
<div>
<p class=MsoNormal>All data goes through nginx which acts as a proxy. Its CPU
consumption is never over 1%.<br>
<br>
<br>
Sergej</p>
<div>
<div>
<p class=MsoNormal style='margin-bottom:12.0pt'> </p>
<div>
<p class=MsoNormal>On Thu, Jun 19, 2008 at 9:35 PM, Javier París Fernández <<a
href="mailto:javierparis@udc.es" target="_blank">javierparis@udc.es</a>>
wrote:</p>
<p class=MsoNormal><br>
El 19/06/2008, a las 20:06, Rapsey escribió:</p>
<div>
<blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;
margin-left:4.8pt;margin-right:0in'>
<p class=MsoNormal style='margin-bottom:12.0pt'> </p>
<p class=MsoNormal>It loops from another module, that way I can update the code
at any time without disrupting anything.<br>
The packets are generally a few hundred bytes big, except keyframes which tend
to be in the kB range. I haven't tried looking with wireshark. Still it
seems a bit odd that a large CPU consumption would be the symptom. The traffic
is strictly one way. Either someone is sending the stream or receiving it.<br>
The transmit could of course be written with a passive receive, but the code
would be significantly uglier. I'm sure someone here knows if setting {active,
once} every packet is CPU intensive or not.<br>
It seems the workings of gen_tcp is quite platform dependent. If I run the code
in windows, sending more than 128 bytes per gen_tcp call significantly
decreases network output.<br>
Oh and I forgot to mention I use R12B-3.</p>
</blockquote>
<p class=MsoNormal> </p>
</div>
<p class=MsoNormal>Hi,<br>
<br>
Without being an expert.<br>
<br>
200-300 mb/s in small (hundreds of bytes) packets means a *lot* of system
calls if you are doing a gen_tcp:send for each one. If you buffer 3 packets,
you are reducing that by a factor of 3 :). I'd try to do an small test doing
the same thing in C and compare the results. I think it will also eat a lot of
CPU.<br>
<br>
About the proxy CPU... I'm a bit lost about it, but speculating wildly it is
possible that the time spent doing the system calls that gen_tcp is doing is
added to the proxy CPU process.<br>
<br>
Regards.</p>
</div>
<p class=MsoNormal> </p>
</div>
</div>
<p class=MsoNormal> </p>
</div>
</div>
<div>
<p class=MsoNormal>_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
<a href="http://www.erlang.org/mailman/listinfo/erlang-questions"
target="_blank">http://www.erlang.org/mailman/listinfo/erlang-questions</a></p>
</div>
</div>
<p class=MsoNormal> </p>
</div>
<p class=MsoNormal> </p>
</div>
</div>
</body>
</html>