[erlang-questions] process per task or not ?
akonsu
akonsu@REDACTED
Fri Nov 15 19:34:37 CET 2013
Hello,
I am looking for an advice on how to architect my system. I am just a
beginner...
I have a process that receives messages and this process needs to parse
each message, extract some information from the parsed message, and then
send this information to the subscribers.
As I read somewhere, in a situation like this, people usually spawn a
worker process per message, and let this worker do the work, and then send
the results to the publishers.
I have an overwhelming amount of incoming messages (these are tweets that I
get from a twitter stream), and I am not experienced enough to be able to
tell whether this worker-per-task approach is the right way to do it. I am
afraid that my application will be flooded with workers and eventually can
collapse.
I was thinking that maybe I can use a worker process pool? A process pool
seems unusual in an Erlang application, does it not? Although the "Learn
yourself Erlang..." does have a chapter on building a pool like this.
Or maybe rate-limit the number of incoming messages?
I would really appreciate an advice or two.
Thank you
Konstantin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20131115/7e95e0e4/attachment.htm>
More information about the erlang-questions
mailing list