<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.DefaultFontHxMailStyle
{mso-style-name:"Default Font HxMail Style";
font-family:"Calibri",sans-serif;
color:windowtext;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style></head><body lang=EN-IN link=blue vlink="#954F72" style='word-wrap:break-word'><div class=WordSection1><p class=MsoNormal><span class=DefaultFontHxMailStyle>Indeed! <o:p></o:p></span></p><p class=MsoNormal><span class=DefaultFontHxMailStyle><o:p> </o:p></span></p><p class=MsoNormal><span class=DefaultFontHxMailStyle>As Jesper too pointed out, a sub-10-microsecond level turnaround time seems impossible to achieve in Erlang; the best I got is 60 microseconds, and average is in the order of hundreds of microseconds (almost close to a millisecond)<o:p></o:p></span></p><p class=MsoNormal><span class=DefaultFontHxMailStyle><o:p> </o:p></span></p><p class=MsoNormal><span class=DefaultFontHxMailStyle>I am taking the “tick-to-order” processing (in stock-market parlance) out of Erlang and into Rust to meet this ultra-low latency requirement.<o:p></o:p></span></p><p class=MsoNormal><span class=DefaultFontHxMailStyle><o:p> </o:p></span></p><p class=MsoNormal><span class=DefaultFontHxMailStyle>Thanks all for confirming the overall approach.<o:p></o:p></span></p><p class=MsoNormal><span class=DefaultFontHxMailStyle><span style='font-size:14.0pt'><o:p> </o:p></span></span></p><p class=MsoNormal><span class=DefaultFontHxMailStyle><span style='font-size:14.0pt'><o:p> </o:p></span></span></p><div style='mso-element:para-border-div;border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal style='border:none;padding:0cm'><b>From: </b><a href="mailto:max.lapshin@gmail.com">Max Lapshin</a><br><b>Sent: </b>07 April 2021 06:32 PM<br><b>To: </b><a href="mailto:trigunatit@gmail.com">Avinash Dhumane</a><br><b>Cc: </b><a href="mailto:lukasz@niemier.pl">Łukasz Niemier</a>; <a href="mailto:erlang-questions@erlang.org">Erlang Questions</a><br><b>Subject: </b>Re: gen_udp:recv()</p></div><p class=MsoNormal><span class=DefaultFontHxMailStyle><span style='font-size:14.0pt'><o:p> </o:p></span></span></p><p class=MsoNormal>I suppose that you really need the same as me.</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>You need to receive _one_ big binary with an extra field containing</p><p class=MsoNormal>messages boundaries and other meta info.</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>On Wed, Apr 7, 2021 at 1:25 PM Avinash Dhumane <trigunatit@gmail.com> wrote:</p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> Our case is similar. The stock exchange disseminates the market movements (called, ticks) as stream of variable length messages (datagrams or packets), each less than 50 bytes, and each stamped with sequence number, over multicast (udp) streams, each stream replicated so that if a packet is missed (dropped) – due to the nature of underlying udp – it may be received on the replicated stream (before fallback on a tcp-based recovery server).</p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> The message rate (without replication) exceeds 100K per second. Issuing as many calls, at Erlang-application level, viz. gen_udp:recv, is impractical, though I suppose, at underlying OS system-call level, each datagram must be received using a separate call. Therefore, I believed, that Erlang might provide an abstraction, which packs a number of datagrams (packets / messages) arrived so far and deliver to the application, for further handling. One-to-one call from app to Erlang, and further from Erlang to OS, for each datagram, would seem too much taxing.</p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> From: Max Lapshin</p><p class=MsoNormal>> Sent: 06 April 2021 10:20 PM</p><p class=MsoNormal>> To: Łukasz Niemier</p><p class=MsoNormal>> Cc: Erlang Questions</p><p class=MsoNormal>> Subject: Re: gen_udp:recv()</p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> Exactly.</p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> We receive MPEG-TS _stream_ in multicast messages and it is very</p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> painful to receive so many messages (it can be around 90K per second</p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> totally or 300-900 per one stream per second) one by one. Reducing</p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> this about by 10 decreases CPU usage a lot.</p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> On Tue, Apr 6, 2021 at 6:31 PM Łukasz Niemier <lukasz@niemier.pl> wrote:</p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> ></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> > > Well, I like the idea to receive 20-200 UDP messages in a single</p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> > > binary with separate structure holding mapping of this data.</p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> ></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> > Then it sounds like you want stream protocol instead of packet protocol.</p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> > The whole idea of packet protocol is to receive messages in packets.</p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> ></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> > --</p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> ></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> > Łukasz Niemier</p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> > lukasz@niemier.pl</p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>> ></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal>><o:p> </o:p></p><p class=MsoNormal><span class=DefaultFontHxMailStyle><span style='font-size:14.0pt'><o:p> </o:p></span></span></p></div></body></html>