[erlang-questions] Server which spawns an additional server for each call

[Dave Challis]

Hi Joe,
Fantastic, that's pretty much exactly what I was after.  I didn't 
realise I could send a {noreply, X} from a handle_call, and defer the 
actual reply from a spawned process.

The pure erlang version makes it clear what's going on too - gen_server 
is still a bit of a black box to me I'm afraid.

The gen_server I'm using is a wrapper around a C utility which does some 
parsing (using erlang's port mechanism), and then returns the parsed 
data to the original function caller.

I figured that by having the port initialised when the gen_server 
process is started, it could respond to client requests right away, 
rather than being spawned upon request.

I was also thinking that by having the C program and port set up at 
server start time, then any startup errors from it would be caught then, 
rather than everything appearing to be ok until a client request was made.

Thanks,
Dave

On 19/04/11 16:11, Joe Armstrong wrote:
> It's not entirely clear to me what you want to do. I never understand
> why people use gen_servers
> for everything, pure Erlang is often easier :-)
>
> I assume you want to delegate the response in the server (ie get some
> other process than the
> gen server to do the work, since this takes a long time)
>
> One way to do this is like this:
>
> Write a client stub like this:
>
> foo(X) ->
>      gen_server:call(?Mod, {foo, X}).
>
>
> Write a gen_server handle call method like this:
>
> handle_call({foo, X}, From, State) ->
>      State1 = func1(X, State),
>      State2 = func2(X, State),
>      spawn_link(fun() -> do_something(State1, X, From) end),
>      {noreply, State2}.
>
> do_something(State, X, From) ->
>      Reply = func3(X, State),
>      gen_server:reply(From, Reply).
>
> here I've assumed func1 returns State1 and that State1 is some subset of
> State
> needed in your computation. State2 (returned by func2) is the
> continuation state of the server.
> ie the state the server will have after it has received the foo request,
> but before the delegated function
> has replied. You'll have to write func1 and func2 yourself.
>
> do_something is the delegated function that might take a long time. The
> four lines of
> code which define handle_call should return quickly, ie don't do much
> here, do the work in
> do_something. handle call returns {noreply, State2} which means "don't
> reply to the client
> but continue with state State2.
>
> do_something works in parallel with the server and when it has finished
> calls gen_server:reply/2
> and at this point the client stub routine  foo/1 will return.
>
> This is only one way of doing this. You could do the complex calculation
> inside the client
> and request the data you need from the server, the server can spawn a
> deligate (as above).
> You can use a shared ets table and so on.
>
> if you were to do this in pure erlang it might be easier to see what's
> going on
>
> Define promise and yield thusly:
>
> promise(Fun) ->
>        S = self(),                             %% this self() is
> evaluated *inside* the current function
>        spawn(fun() ->
>                       S ! {self(), Fun()}    %% this self() is evaluated
> *inside* the spawned function
>                   end).
>
> yield(Promise) ->
>       receive
>            {Promise, Result} -> Result
>       end.
>
> promise takes a Fun argument and returns a promise. The promise is a
> promise to
> compute the value. The promise can be redeemed by calling yield. So we
> can write code like this:
>
> P = promise(fun() -> fib(40) end),
> .... do some other stuff that takes a while ...
> Val = yield(Promise)
>
> So we compute fib(40) which takes a long time in parallel with some
> other stuff.
>
>
> The gen_server stuff above has just hidden what is essentially a promise
> and yield and
> a bit of state trickery in a framework module - nothing tricky about
> about it at all.
>
> Given promise and yield and a dash of list comprehensions we can write
> fun stuff like:
>
> parmap(F, L) ->
>       Promises = [promise(fun() -> F(I) end || I <- L],
>       [yield(I) || I <- Promises].
>
> which is a parallel mapping function.
>
> Hope this helps
>
> /Joe
>
>
>
> On Tue, Apr 19, 2011 at 10:58 AM, Dave Challis <dsc@REDACTED
> <mailto:dsc@REDACTED>> wrote:
>
>     Hi,
>     I'm trying to work out the structure for a small bit of
>     functionality in erlang.  I've got a feeling there's an obvious
>     solution, but I'm not experienced enough with the language sure what
>     pattern to best follow...
>
>     So far, I've got a server which implements the gen_server behaviour,
>     a supervisor for it, and a module containing an API for interacting
>     with it (using gen_server:call mostly).
>
>     The server does a lot of work though, so blocks for a while until
>     the call is finished.
>
>     What I'd like to do, is create a new server process each time
>     gen_server:call is invoked by a client, with each server terminating
>     when it's done processing (but always leaving 1 server running).
>
>     This means that clients using the API will have their request
>     processes straight away, without having to wait for all the other
>     calls to the server to finish.
>
>     I can't quite figure out where to place the logic for doing this though.
>
>     * Should the API module ask the supervisor to spawn a new child,
>     then send the client's call to this?
>     * Should API calls go to the supervisor, and have it decide whether
>     to spawn new servers or send the call to an existing one?
>     * Or should each server take care of telling the supervisor to spawn
>     a new child, and pass the call request to the newly spawned one?
>
>     Is this a sensible approach in general, or is there an obvious
>     pattern or some functionality I've missed?
>
>     Thanks,
>
>     --
>     Dave Challis
>     dsc@REDACTED <mailto:dsc@REDACTED>
>     _______________________________________________
>     erlang-questions mailing list
>     erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
>     http://erlang.org/mailman/listinfo/erlang-questions
>
>


-- 
Dave Challis
dsc@REDACTED