[erlang-questions] Difference in 'close' for gen_tcp recv vs. active

[Fred Hebert]

Linux 2.6.32-359-ec2 x86_64 on one of the instances on which it can be
triggered.

The issue doesn't seem to be reproducible when everything takes place on
localhost. I'm trying to play around with other instances in similar
security groups to reproduce it using only Erlang stuff.

I'll comb over the sysctl stuff for a culprit.

Regards,
Fred.

On 04/08, Garret Smith wrote:
> What OS are you running, and has the TCP stack been tuned/optimized with
> sysctl's?
> 
> Does the problem happen all the time, or only sometimes?
> 
> In my quick test before posting, gen_tcp:send() returned {error, closed}
> soon after my client disconnected without any recv's on the socket.  I was
> sending single bytes at a time with a 1-second delay, so I'm sure no
> buffers were overflowing to trigger the disconnect.  It acted pretty much
> the same in about 5 attempts, so I'm really curious why we see different
> behavior.
> 
> Tangential, but possibly interesting, I had to comment out 2 of the socket
> options in your list.  I got a badarg from inet_tcp:listen with the
> 'broadcast' and 'dontroute' options.
> 
> -G
> 
> 
> 
> On Tue, Apr 8, 2014 at 5:49 PM, Fred Hebert <mononcqc@REDACTED> wrote:
> 
> > The difference here is that we have a peer that actively closes the
> > connection and the packets are readable and detected as far as tcpdumps
> > go. Things like FINs or RSTs. They're making it to our host, they're
> > just entirely ignored unless we're willing to read undefined amounts of
> > data from the buffer for the stack to apparently get access to them.
> > 99.99% of the time that's gonna be 0 bytes because of HTTP clients' rare
> > tendency to pipeline requests.
> >
> > I'm just extremely annoyed having to figure out a way to add in a buffer
> > to carry possibly unlimited amounts of data to read from a socket so it
> > can figure out if a FIN or RST was sent recently, but I guess that's the
> > way we'll have to do it.
> >
> > Oh well.
> >
> > On 04/08, Garret Smith wrote:
> > > Basically, this has everything to do with TCP sockets (and the OS
> > > implementation) and very little to do with Erlang.  I do get {error,
> > > closed} from gen_tcp:send when the client disconnects.  I'm running
> > > R16B03-1 on FreeBSD 10.
> > >
> > > Here are some decent descriptions of the problem:
> > >
> > >
> > http://www.linuxquestions.org/questions/programming-9/how-could-server-detect-closed-client-socket-using-tcp-and-c-824615/
> > >
> > > The second answer in the page above has some good links too, pulled out
> > for
> > > easy reference:
> > >
> > >
> > http://stackoverflow.com/questions/722240/instantly-detect-client-disconnection-from-server-socket
> > >
> > http://www.softlab.ntua.gr/facilities/documentation/unix/unix-socket-faq/unix-socket-faq-2.html#ss2.8
> > >
> > > -Garret
> > >
> > >
> > > On Tue, Apr 8, 2014 at 1:40 PM, Fred Hebert <mononcqc@REDACTED> wrote:
> > >
> > > > Hi there,
> > > >
> > > > Happy fun case we're hitting on nodes using Erlang right now. We have
> > an
> > > > HTTP proxy that does direct data streaming from a server to a client.
> > > > This is done through a series of recv from the server, and of sends to
> > > > the client. The TCP sockets involved are all in passive mode.
> > > >
> > > > The problem with this approach is that because this is a purely
> > > > unidirectional stream until the server says so or the client quits,
> > we'd
> > > > like to detect both of these events.
> > > >
> > > > The server quitting or being done is easy enough, but the client
> > > > quitting cannot be detected, apparently.
> > > >
> > > > It turns out that gen_tcp:send/2 always returns 'ok' even if the
> > > > connection has been closed by the peer before. Erlang/OTP just won't
> > > > acknowledge the fact unless someone tries to read from the socket,
> > > > either through gen_tcp:recv or by using inet:setopts(Port, [{active,
> > > > once}]), at which point {error, closed} starts being returned by
> > > > gen_tcp:send/2. This happens, even if `{exit_on_close, true}` is
> > > > specified as an option.
> > > >
> > > > The question I have here is why is this behavior different for send
> > than
> > > > recv. It seems that `gen_tcp:send` will happily wait for hours
> > > > pretending to send data (no matter the timeout values used are), even
> > > > once the connection has been closed by the other peer.
> > > >
> > > > Is there any way for me to detect that a connection has been closed
> > > > without possibly having to poll `recv` on each packet I try to send
> > (and
> > > > then may need to buffer all that data, which I'd prefer to avoid) or
> > > > changing the entire app's workflow to be active?
> > > >
> > > > In case, here are the socket options we use:
> > > >
> > > >     [{active,false},
> > > >      {broadcast,false},
> > > >      {buffer,1460},
> > > >      {delay_send,false},
> > > >      {dontroute,false},
> > > >      {exit_on_close,true},
> > > >      {header,0},
> > > >      {high_watermark,8192},
> > > >      {keepalive,false},
> > > >      {linger,{false,0}},
> > > >      {low_watermark,4096},
> > > >      {mode,binary},
> > > >      {nodelay,true},
> > > >      {packet,0},
> > > >      {packet_size,0},
> > > >      {priority,0},
> > > >      {recbuf,87380},
> > > >      {reuseaddr,true},
> > > >      {send_timeout,infinity},
> > > >      {sndbuf,65536}]
> > > >
> > > > Regards,
> > > > Fred.
> > > > _______________________________________________
> > > > erlang-questions mailing list
> > > > erlang-questions@REDACTED
> > > > http://erlang.org/mailman/listinfo/erlang-questions
> > > >
> >