<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On 04 Feb 2020, at 16:53, Alexander Petrovsky <<a href="mailto:askjuise@gmail.com" class="">askjuise@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div dir="ltr" class=""><br class=""></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">вт, 4 февр. 2020 г. в 16:52, Valentin Micic <<a href="mailto:v@micic.co.za" class="">v@micic.co.za</a>>:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="overflow-wrap: break-word;" class=""><div style="overflow-wrap: break-word;" class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On 04 Feb 2020, at 10:31, Alexander Petrovsky <<a href="mailto:askjuise@gmail.com" target="_blank" class="">askjuise@gmail.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class=""><div dir="ltr" class="">Hello!<div class=""><br class=""></div><div class="">The first of all, my erlang version OTP-21.3 (without any minor patches)</div><div class=""><br class=""></div><div class="">I've got the strange floating problem with UDP to receive, which I couldn't debug by myself. Let me describe briefly what I have: There is UDP socket which used for send packets to many peers and receives responses from them (rate ~1pps):</div><div class=""><br class=""></div><div class="">send request: SIP options</div><div class="">receive a response: 100</div><div class="">receive a response: 200</div><div class=""><div class=""><br class=""></div><div class="">In erlang it's implemented like gen_server which receives messages, the sockets options are: <font face="monospace" class="">binary, {active, once}</font>.</div><div class=""><br class=""></div><div class="">Every time, when the message is received from the mailbox, code try to do some job, after that try to handle message bursts by calling in the loop (by example 5 times):</div><div class=""><code class=""><span class="">gen_udp</span><span class="">:</span><span class="">recv</span><span class="">(</span><span class="">Socket</span><span class="">,</span> <span class="">0</span><span class="">,</span> <span class="">0</span><span class="">), <font face="arial, sans-serif" class="">bursts and after that set the socket option </font></span></code>{active, once} <span style="font-family:arial,sans-serif" class="">back.</span></div><div class=""><br class=""></div><div class="">So, here I'm observing floating strange behavior, from time to time the second!! and always the second!! response is absent. I see this UDP missing packets in tcpdump, I don't see any drops in UDP stack at all.</div><div class=""><br class=""></div><div class="">When I'm trying to add some logs into my erlang code, the problem disappears, when I'm run <font face="monospace" class="">strace</font> to <font face="monospace" class="">recvfrom</font> syscall the problem disappears... I't try to debug VM with <font face="monospace" class="">gdb</font> and make some breakpoints on <font face="monospace" class="">sock_recvfrom</font> in <font face="monospace" class="">inet_drv.c</font><font face="arial, sans-serif" class=""> (it's OTP-<font style="background-color:rgb(255,255,255)" class=""><span style="white-space:pre-wrap" class="">21.</span><span style="box-sizing:border-box;border-radius:0.2em;white-space:pre-wrap" class="">3), but </span></font></font>the problem disappears (((</div><div class=""><br class=""></div><div class="">I can't find the problem root cause, is it in my code or is it in VM? Maybe someone could help me with that? </div><div class=""><br class=""></div>-- <br class=""><div dir="ltr" class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class="">Петровский Александр / Alexander Petrovsky,<br class=""><br class="">Skype: askjuise<br class=""><div class="">Phone: +7 931 9877991<div class=""><br class=""></div></div></div></div></div></div></div></div></div>
</div></blockquote></div><div class=""><br class=""></div><div class="">So, I suppose this is related to SIP communication, and I’ll go out on a limb here…</div><div class=""><br class=""></div><div class="">Considering that SIP message, when transported using UDP, must fit into a single UDP message, then, your statement that second and only second response packet is missing, may imply that you are sending response using two UDP packets.</div><div class=""><br class=""></div><div class="">(A) AFAIK, SIP over UDP does not have any continuation, hence, must fit into a single packet.</div><div class="">If you (by any chance) send SIP message over multiple UDP packets, this shouldl not work — saying this just in case.</div><div class=""><br class=""></div><div class="">(B) OTOH, if tcpdump reveals UDP fragmentation (where a single UDP datagram is fragmented over multiple IP packets), then maybe your UDP buffer it to small to assemble the whole message, in which case message might be truncated, and your SIP parser may see it as the part of the message (second!! and always second!!), that’s gone missing.</div><div class=""><br class=""></div><div class="">(C) Finally, when your server processing is too slow for the size of the buffer, then, there’s a chance that receive buffer may get full, in which case it would drop the packet that cannot be added to the receive buffer.</div><div class=""><br class=""></div><div class="">However, judging from your “second!! and always second!!” statement, I’d say that (A) or (B), are more likely, barring something completely different (e.g some other bug, perhaps ;-))</div><div class=""><br class=""></div><div class="">Kind regards</div><div class=""><br class=""></div><div class="">V/</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">.</div><div class=""><br class=""></div><div class=""><br class=""></div></div></div></blockquote></div><br clear="all" class=""><div class="">Accordingly to tcpdump the size on packets from 300 to 400 bytes... also, I've double-checked, there are no fragmented packets at all, my <font face="monospace" class="">ss -anump</font> for beam shows:</div><div class=""><br class=""></div><div class=""><font face="monospace" class="">skmem:(r0,rb16384,t0,tb212992,f4096,w0,o0,bl0,d0)</font><br class=""></div><div class=""><br class=""></div><div class="">where:</div><div class=""> - <font face="monospace" class="">rb16384</font> - rx buffer;</div><div class=""> - <font face="monospace" class="">tb212992</font> - tx buffer;</div><div class=""> - <font face="monospace" class="">d0</font><font face="arial, sans-serif" class=""> - drop counter.</font></div><div class=""><br class=""></div><div class="">So, I guess it's not an (A) nor (B). My internal erlang buffer <strong style="color:rgb(26,26,26);font-family:sans-serif;font-size:16px;background-color:rgb(254,254,254)" class=""><span class="gmail-code" style="font-family:mono,Courier,monospace;font-weight:normal;background-color:rgb(243,243,243)">{buffer, </span></strong><strong style="color:rgb(26,26,26);font-family:sans-serif;font-size:16px;background-color:rgb(254,254,254)" class=""><span class="gmail-code" style="font-family:mono,Courier,monospace;font-weight:normal;background-color:rgb(243,243,243)">Size}</span></strong> (<a href="https://erlang.org/doc/man/inet.html" class="">https://erlang.org/doc/man/inet.html</a>) is 8192. It's could be (C), but the incoming packet rate is really low. Also, it's not a SIP parser for messages received over UDP, due to I don't see and packets in my receive process...</div><div class=""><br class=""></div>-- <br class=""><div dir="ltr" class="gmail_signature"><div dir="ltr" class=""><div class=""><div dir="ltr" class="">Петровский Александр / Alexander Petrovsky,<br class=""><br class="">Skype: askjuise<br class=""><div class="">Phone: +7 931 9877991<div class=""><br class=""></div></div></div></div></div></div></div>
</div></blockquote><br class=""></div><div>We (and by that I include myself as well) have been using {active, once}, followed by gen_udp:recv/3 construct quite often, and I do not recall ever loosing a packet when rates were less than, say, 80,000 UDP messages/second — provided that the server processing is fast enough (but that's a different conversation).</div><div><br class=""></div><div>Transition between {active, once} and gen_udp:recv/3 will *never* loose any packets, as the packet will be in the UDP buffer until recalled by gen_udp:recv or another {active, once} </div><div><br class=""></div><div>Maybe you should consider a situation when the second packet is *not* in the buffer yet (e.g. enroute). In that case gen_udp:recv(sock, 0, 0 ) will fail with {error, timeout}, and you’ll pick up the “second” UDP message only next time when you call {active once}.</div><div><br class=""></div><div>So, why don’t you go and change this timeout value to something other than zero, say, 1000 (one second) just to see if the behaviour changes in any way. Then you could reduce it to something smaller… say 10, or even 5 ms.</div><div><br class=""></div><div><br class=""></div><div>V/</div><div><br class=""></div></body></html>