The missing UDP packets

Alexander Petrovsky askjuise@REDACTED
Tue Feb 4 15:44:46 CET 2020


вт, 4 февр. 2020 г. в 16:46, Andreas Schultz <andreas.schultz@REDACTED
>:

> Am Di., 4. Feb. 2020 um 14:35 Uhr schrieb Alexander Petrovsky <
> askjuise@REDACTED>:
>
>> Paul, thank you for your quick response, it seems like I'm was not
>> accurate enough in my previous letter )
>>
>> I do not wonder about drops, I know that drops could occur in buffers in
>> network interface, in kernel buffers and also in buffers in VM. In my case
>> there are not drops in kernel space accordingly to netstat -S, ss -anump
>> , cat /proc/net/snmp | grep Udp, cat /proc/net/udp; also it could be the
>> bug in my code, which I can't find by some reason ))
>>
>> Please, pay attention that after I'm trying to debug this behavior with
>> strace/gdb the problem has gone. I'm interesting, how could I find/observer
>> is there any drops in VM buffers or so... ?
>>
>
> One thing you could try is to use systemtap instead of strace/gdb to
> monitor the UDP socket. It has much less impact on runtime behavior and
> might provide a clue as to whether the data is actually read from socket.
>
> Also, have you tried using the new socket module instead of gen_udp?
>
> Andreas
>
>
>> вт, 4 февр. 2020 г. в 16:07, Paul Peregud <paulperegud@REDACTED>:
>>
>>> AFAIK UDP packet can be dropped by anyone. By intermediate nodes and
>>> by your kernel. I would just assume that they can be dropped by your
>>> BEAM VM too. I also don't think that "it's always the second packet"
>>> behavior is strange. Play around with kernel buffers sizes and you
>>> will likely observe some change.
>>>
>>> Assumption that UDP packet that was sent will be delivered is wrong.
>>>
>>> On Tue, Feb 4, 2020 at 8:32 AM Alexander Petrovsky <askjuise@REDACTED>
>>> wrote:
>>> >
>>> > Hello!
>>> >
>>> > The first of all, my erlang version OTP-21.3 (without any minor
>>> patches)
>>> >
>>> > I've got the strange floating problem with UDP to receive, which I
>>> couldn't debug by myself. Let me describe briefly what I have: There is UDP
>>> socket which used for send packets to many peers and receives responses
>>> from them (rate ~1pps):
>>> >
>>> > send request: SIP options
>>> > receive a response: 100
>>> > receive a response: 200
>>> >
>>> > In erlang it's implemented like gen_server which receives messages,
>>> the sockets options are: binary, {active, once}.
>>> >
>>> > Every time, when the message is received from the mailbox, code try to
>>> do some job, after that try to handle message bursts by calling in the loop
>>> (by example 5 times):
>>> > gen_udp:recv(Socket, 0, 0), bursts and after that set the socket
>>> option {active, once} back.
>>> >
>>> > So, here I'm observing floating strange behavior, from time to time
>>> the second!! and always the second!! response is absent. I see this UDP
>>> missing packets in tcpdump, I don't see any drops in UDP stack at all.
>>> >
>>> > When I'm trying to add some logs into my erlang code, the problem
>>> disappears, when I'm run strace to recvfrom syscall the problem
>>> disappears... I't try to debug VM with gdb and make some breakpoints on
>>> sock_recvfrom in inet_drv.c (it's OTP-21.3), but the problem disappears (((
>>> >
>>> > I can't find the problem root cause, is it in my code or is it in VM?
>>> Maybe someone could help me with that?
>>> >
>>> > --
>>> > Петровский Александр / Alexander Petrovsky,
>>> >
>>> > Skype: askjuise
>>> > Phone: +7 931 9877991
>>> >
>>>
>>>
>>> --
>>> Best regards,
>>> Paul Peregud
>>> +48602112091
>>>
>>
>>
>> --
>> Петровский Александр / Alexander Petrovsky,
>>
>> Skype: askjuise
>> Phone: +7 931 9877991
>>
>>
>
> --
>
> Andreas Schultz
>
> --
>
> Principal Engineer
>
> t: +49 391 819099-224
>
> ------------------------------- enabling your networks
> -----------------------------
>
> Travelping GmbH
> Roentgenstraße 13
> 39108 Magdeburg
> Germany
>
> t: +49 391 819099-0
> f: +49 391 819099-299
>
> e: info@REDACTED
> w: https://www.travelping.com/
> Company registration: Amtsgericht Stendal
> Geschaeftsfuehrer: Holger Winkelmann
> Reg. No.: HRB 10578
> VAT ID: DE236673780
>


> One thing you could try is to use systemtap instead of strace/gdb to
monitor the UDP socket. It has much less impact on runtime behavior and
might provide a clue as to whether the data is actually read from the
socket.

Good point about systemtap, I will try it, thanks...

> Also, have you tried using the new socket module instead of gen_udp?

Nope, and in my mind it's still in beta and guesses it's doest for
production.



-- 
Петровский Александр / Alexander Petrovsky,

Skype: askjuise
Phone: +7 931 9877991
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20200204/90b8beca/attachment.htm>


More information about the erlang-questions mailing list