[erlang-questions] Troubling gen_tcp.send/3 performance

[Matthew Shapiro]

That's super interesting information, and once the port's buffer fills up
once I can totally see now how it could get into cascading performance
issues, since once the message queue is backed up with A/V data messages
from the publisher's process every gen_tcp:send/3 call now has to do a
selective receive past all of them to find the inet_reply message.

Thanks!

On Sun, Dec 4, 2016 at 9:13 PM, Fred Hebert <mononcqc@REDACTED> wrote:

> On 12/04, Matthew Shapiro wrote:
>
>> Yeah I think I can safely say this is something to do with how VLC is
>> functioning.  Even remotely I'm getting stuck in send, though not as bad,
>> and looking at what's going on in the difference between VLC and Ffmpeg I
>> believe my timestamps are being unexpected and both are reacting weirdly
>> too it, with VLC trying to slow the feed down for whatever reason and that
>> may be causing it to be slow to do a TCP read.  Shot in the dark but at
>> least I know now that it's not an BEAM issue (and my time on this hasn't
>> been to waste).
>>
>> So sorry for the thread when Erlang/Beam isn't at fault in the end :)
>>
>>
> It is and it is not. One of the interesting things with the TCP stack is
> that it sends a signal through ports, and if the send queue is full, the
> whole thing blocks; it however blocks with live data in there, and there's
> no good way to give it up.
>
> The reason is blocks that the TCP work is done over a port program,
> meaning message passing takes place:
>
>    Erlang Process               TCP port program
>         |                               |
>         |----------- Data ------------->|
>         |                               |
>         |<--- {inet_reply,S,Status} ----|
>
> This is roughly what takes place within the thing. This is done through
> calling erlang:port_command/2-3. The thing though is that if the buffer of
> the port program is full, you get descheduled until there's place.
>
> There's a cheat though, one we at Heroku have used in Logplex. The trick
> is to bypass the stack in critical 'must never block' circumstances and
> call port_command yourself, but by passing in the option list '[nosuspend]'.
>
> When you do this and the port program representing the TCP socket is busy,
> you'll get a 'false' return value indicating it failed because it was full
> (you get 'true' when it works):
>
>    case erlang:port_command(Socket, Data, [nosuspend]) of
>        false ->
>            handle_busy(...);
>        true ->
>            receive
>                {inet_reply, Socket, Status} -> % Status == send return
>                    Status
>            end
>    end
>
> This will let you do things like figure out the consumer side (in this
> case VLC) is too slow, and possibly give up sending a few frames and send
> newer ones later.
>
> Control flow is fun!
>
> Regards,
> Fred.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20161205/1b5a3989/attachment.htm>