<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">On 06/23/2014 08:29 AM, Max Lapshin
wrote:<br>
</div>
<blockquote
cite="mid:CAMxVRxBf3FVNn5FFMQ2DBRsj5-Fnt+hmn5=YF5OfV7Ckei3_eQ@mail.gmail.com"
type="cite">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Mon, Jun 23, 2014 at 6:09 PM,
Robert Raschke <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:rtrlists@googlemail.com" target="_blank">rtrlists@googlemail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<p>Hi Max,</p>
<p>since you are already using separate nodes running
NIFs, there's probably not much worth in converting to a
C node or port. So this is just for posterity.</p>
</blockquote>
<div><br>
</div>
<div>Wrong. It will be a very big work to convert to C node.</div>
<div><br>
</div>
<div>Right now my data flow is following: </div>
<div><br>
</div>
<div>1) 5-8 mbit of data are transferred to separate node</div>
<div>2) they pass through complicated code on erlang that
prepares them for decoding</div>
<div>3) then original frame goes to decoder</div>
<div>4) decoder outputs from 100 to 1000 megabits of data</div>
<div>5) then they again go to tricky logic</div>
<div>6) after this huge amount of raw data gets into
encoders</div>
<div>7) and again through erlang it gets back, being around
2-4 mbit/s</div>
<div><br>
</div>
<div><br>
</div>
<div>So erlang has to copy less than 10 mbit here and there.
All serious payload is zerocopies inside one node.</div>
<div><br>
</div>
<div>If I change to driver, then I will have to write much
more boilerplate demarshalling code.</div>
<div><br>
</div>
<div>If I change to C node or port then I will have to
rewrite big amount of working erlang code in C.</div>
<div><br>
</div>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
erlang-questions mailing list
<a class="moz-txt-link-abbreviated" href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a>
<a class="moz-txt-link-freetext" href="http://erlang.org/mailman/listinfo/erlang-questions">http://erlang.org/mailman/listinfo/erlang-questions</a>
</pre>
</blockquote>
<tt>If you want, you could use </tt><a class="moz-txt-link-freetext" href="https://github.com/okeuday/GEPD/">https://github.com/okeuday/GEPD/</a>
to be able to switch between an Erlang port and an Erlang port
driver (where the port driver can use the async thread pool). The
code is done with the preprocessor to avoid overhead. It would be
possible to add NIF generation that uses the dirty scheduler for
async calls.<br>
<br>
Since you are dealing with higher throughput though, it seems like
you should want more than 1 connection to the Erlang VM and CloudI
would help you there (<a class="moz-txt-link-freetext" href="http://cloudi.org">http://cloudi.org</a>), since otherwise the single
connection (cnode or port) will always be a bottleneck (making
multiple connections for higher throughput isn't really a way of
solving the problem, just a way of ignoring it).<br>
<br>
With a port driver and port level locking you can have multiple
ports used concurrently if your C/C++ code is perfect. So, that is
an approach if you are able to have perfect source code for
integration.<br>
</body>
</html>