Latency with tcp active mode

[Erik Rigtorp]

Hi!

I'm working on an application where latency is very important. It
listens to messages on some TCP sockets, processes them and sends them
out on some TCP sockets. The problem is that I can measure a huge
round trip latency between sending a message to the application and
receiving the response. After substracting the network round trip i
can see insignificant delays to up to 400ms delays. Nagling is of
course disabled. Apperently there is some significant delay between
the NIC and until i receive the message in an erlang process.

I'm using active mode for the sockets but I've noticed that in
RabbitMQ they use prim_inet:async_recv() to do async tcp receive.
Anyone know why they do that? I've been carefully studying the code in
inet_drv.c and I can't see much difference between the code paths for
using prim_inet:async_recv() and active mode.

Where might this delay come from?

I might try to instrument the runtime with dtrace.

Erik Rigtorp