TCP Flow Control

[Evans, Matthew]

Hi,

I have a situation where I've implemented an Erlang TCP client (HTTP) application, where multiple instances run in a distributed Erlang mesh spread over several nodes within a chassis (actually it is implemented with the httpc module).

The issue we have is that the data link between the different nodes that provides TCP traffic for this service has limited capacity (but the main link into the chassis that hosts the nodes has a massive capacity), and I don't want to swamp that link with lots of TCP traffic. We have no control over the server/producer application.

Normally one could have the consumer processes read from the socket slowly. This should, in theory, cause TCP flow control to kick in and regulate the producer speed (the server).

My understanding of Erlang however is that gen_tcp will read from the socket as fast as it can, and dispatch the data to the calling Erlang process (in effect filling up its message queue if it is reading too slowly). Are there any options that can be set to override this behavior, or provide some sort of back pressure to gen_tcp causing it to read from the socket slowly?

I know I am using httpc to provide my application logic, so would probably need to implement a patch within that module if needed too.

Any suggestions would be appreciated.

Many thanks

Matt