[erlang-questions] gen_sctp throughput mystery

[Raimo Niskanen]

On Thu, Mar 08, 2012 at 04:23:09PM +0100, Johannes Govaerts wrote:
> Hi, 
> 
> Today I stumbled also across this issue. It turned out that you get
> this behavior when not specifying the recbuf option at the server. The
> default recbuf is only 1500 bytes. So when the client sends a package
> he waits for acknowledgment before sending a another package; and at
> the other side the server receives only one package and will only
> acknowledge when sacktimeout is triggered because he never receives a
> second package...
> 
> When using a higher recbuf value there is no throughput issue anymore.
> 
> Regards,
> Johannes

That is great news (that there is an explanation).

So the combination of default recbuf size of 1500 bytes in gen_sctp,
the (Linux ?) kernel default net.sctp.sack_timeout = 200 (RFC recommended),
and sending packets larger than recbuf size choaks the throughput to
1000/net.sctp.scack_timeout packets per second...

What a gotcha!

> 
> On Tue, 6 Mar 2012 11:46:34 +0100
> ext Raimo Niskanen <raimo+erlang-questions@REDACTED> wrote:
> 
> > On Sun, Mar 04, 2012 at 01:19:00PM +1100, Anthony Shipman wrote:
> > > On Sun, 4 Mar 2012 12:43:49 pm Anthony Shipman wrote:
> > > > > The now() value shows the packets being written to the socket
> > > > > at 11ms intervals on the client but they are received at strict
> > > > > 200ms intervals on the server. The server is just doing a
> > > > > simple receive loop.
> > > > >
> > > > > Is there something in SCTP that limits the rate on an SCTP
> > > > > stream?
> > > > 
> > > > I forgot to mention that the behaviour is the same for
> > > > sctp_nodelay=true.
> > > > 
> > > 
> > > Replying to my own question. It's the net.sctp.sack_timeout = 200
> > > kernel parameter. But I don't understand why it should limit the
> > > server to 5 packets per second over a localhost link.
> > 
> > Great finding if true... Can you elaborate? What happens when you
> > tweak this parameter? Note that 1 / 200ms is exactly 5 per second.
> > 
> > I found this link:
> >   http://sandarenu.blogspot.com/2009/11/reducing-time-between-sctp-message.html
> > And a quote from it it says:
> >   That 200ms is a configuration based on the SCTP RFC. According to
> > that an acknowledgment(SACK) should be generated for at least every
> > second packet received, and SHOULD be generated within 200 ms of the
> > arrival of any unacknowledged DATA chunk.
> > So, might it be so that if you send just one data chunk there will be
> > a 200 ms delay before an ack?
> > 
> > I have focused on the SCTP socket API, not dug into the RFC:s...
> > 
> > > 
> > > -- 
> > > Anthony Shipman                    Mamas don't let your babies 
> > > als@REDACTED                   grow up to be outsourced.
> > > _______________________________________________
> > > erlang-questions mailing list
> > > erlang-questions@REDACTED
> > > http://erlang.org/mailman/listinfo/erlang-questions
> > 
> 
> 
> -- 
> Johannes Govaerts
> Nokia Siemens Networks 
> phone: +32 473 36 65 28 
> mailto: johannes.govaerts@REDACTED
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-- 

/ Raimo Niskanen, Erlang/OTP, Ericsson AB