Local node worker process pool

Ciprian Dorin, Craciun ciprian.craciun@REDACTED
Tue Jan 12 14:00:57 CET 2010


    Hello all!

    I have the following simple use-case, and I'm not sure how to
efficiently implement it: I have some processes that need to send
updates to some statistics engine, but I don't want to wait (or
receive) the reply. (By processes I mean gen_server implementations,
and by update I mean `gen_server:call` like messages) Thus I have the
following options:

    a) send the message just like `gen_server:call` would have done,
and then be prepared to ignore the reply; drawbacks:
      * I would have to hard-code the gen_server call protocol, thus
maybe (highly improbable) breaking the backward / forward portability;
      * I have to ignore all the replies, which increases the source
code and might fill the message queue;

    b) spawn a process in the background that shall send the request
(and wait for the reply); drawbacks:
      * it wight overload the virtual machine with too many processes;

    c) implement a `pool` like system that allows me to set the number
of workers, and then enqueue the requests in a round-robin fashion to
the workers; drawbacks:
      * if there are too many requests they are going to pile up in
the queue (actual instance of the `queue` module);
     advantages:
       * as the requests are "on best effort" it allows the virtual
machine to carry on with the important work, and only a portion of the
time (proportional with the number of workers) to be allocated to
these requests;

    d) use the existing `pool` module, but which is designed for
distributed Erlang nodes;


    Also the same problem (which leads only to solutions b and c),
would be asynchronous function calls: I have a function, which takes
some time to compute something, and I don't want to wait for it to
finish, and also it's not that important so I don't want to dedicate
an entire process to it.

    Any ideas? Any already existing solutions?

    Thanks,
    Ciprian.


More information about the erlang-questions mailing list