[erlang-questions] Erlang - supervised gen_server to drain off a queue

[JD Bothma]

I'd suggest subscribing to the queue with a 1-message buffer using the
rabbitmq erlang client. I haven't had time to look into how to properly
close the channel if your gen server crashes but I think a good enough
approach is to link your gen_server to the connection pid and the channel
pid - if anything crashes the gen_server can reconnect when it gets
restarted by its supervisor. no polling for messages, you deal with them as
they come in.

So you your application top supervisor can supervise one worker - your
gen_server. Simple as that. There are very few situations leading to the
Erlang VM crashing, especially with such a simple system. One thing to
watch out for is running out of memory. I think a common way of getting
into that situation is having messages building up in the process message
queue, e.g. if you receive rabbitMQ messages without requiring an ack
before getting the next message, and if your gen_server can't keep up. It
would really take many many messages for this to get out of hand, but do
some napkin maths and be aware of it.

More about rabbitmq connection cleanup that I havent had time to look into
properly...
http://grokbase.com/t/rabbitmq/rabbitmq-discuss/116e7des0q/linking-to-a-channel-process-erlang-client
RabbitMQ erlang client including subscribing to a queue with one message
buffered http://www.rabbitmq.com/erlang-client-user-guide.html


On 29 December 2012 08:35, hyperboreean <hyperboreean@REDACTED> wrote:

> Hi guys,
>
> I'm just trying to bring Erlang into my company - it's a hard road, but
> we might eventually use it for at least some of our projects. I want to
> get started by changing a small piece of code that reads an AMQP queue
> and appends the contents of each message to a file. To be more specific,
> there are multiple producers for that queue (which produce error
> messages, if any) and there's this consumer which gets the error
> messages from the queue and updates a file with them. Now, this piece of
> software has always been problematic for us, for one reason or another -
> it's not reliable and when it fails it might bring the AMQP broker to
> its knees because of the number of messages that are not consumed.
>
> I'd like to change this code with some Erlang code - maybe with a
> supervision tree so that if/when the server fails, it gets automatically
> restarted. I was thinking of trying a gen_server, but then I figured out
> that there's no client of the server, the code just has to constantly
> pull data out of that queue.
>
> Any good hints on how to get started on this ? What's the OTP way of
> doing this kind of things?
>
> Thank you!
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20121231/9e96f89b/attachment.htm>