[erlang-questions] Rabbit under small load

[Jesper Louis Andersen]

On May 24, 2013, at 10:28 AM, Max Bourinov <bourinov@REDACTED> wrote:

> Hello Dear Erlangers!
> 
> I have about 3000 erlang-processes which should send messages to AMQP-queue of destination and receive replies from it.
> 
> The message rate at the moment is about 100 M/s
> Average message size: 4 Kb
> 

Is that 100 * 1000 * 1000 messages per second or 100 messages per second?

RabbitMQ should easily be able to handle the latter load without much trouble. I have built a gen_server as a middle-man for RabbitMQ which handles the request/reply toward RMQ more or less like your approach and I can easily do 6000 messages per second with this setup and very little to no tuning. I don't know where the limit to that solution is, but there is a one-core bottleneck in the gen_server.

100 million messages is a different question however. Given the average message size, you are at 400 megabytes per second and hence you need 4 Gigabit of bandwidth to even hope hitting target. Apart from that, I don't think RMQ can handle that amount of load. You may need something else entirely to handle that.

Jesper Louis Andersen
  Erlang Solutions Ltd., Copenhagen