[erlang-questions] Thoughts about server abstraction and process management

[zambal]

Dear Community,

Lately I have been thinking about different server abstractions in
Erlang that could complement gen_server and it's cousins, because I
regularly stumble up on these two issues with gen_server:

1. gen_server makes it easy to create robust servers, but it still
needs quite some boiler plate code to get it working. Mostly because
of code like:

   do_something(X) ->
     gen_server:call(?MODULE, {do_something, X}).

   Although the code above is not strictly needed when coding a
gen_server, I'll find myself writing this kind of code a lot.

2. gen_servers don't seem to be suited very well for situations where
you need to easily create multiple instances of a particular server.

I should add to the above that I don't work professionally with Erlang
(my workweek languages are C#, C++ and recently Objective C too) and
that my knowledge of OTP is limited, so I realize that my concerns
with gen_server might be because of lack of experience. If so, please
let me know :-)

However, assuming that my concerns are valid, lets continue with just
the minimal code needed to get an alternative generic server running
without worrying about error handling, timeouts, etc.:

First, the generic server code

%% begin yasa.erl (yet another server abstraction)
-module(yasa).
-export([new/2, client/1]).
-export([behaviour_info/1]).

behaviour_info(callbacks) ->
    [{init, 1}, {handler, 2}, {terminate, 1}];
behaviour_info(_Other) ->
    undefined.

new(Mod, State) ->
    client(spawn(fun()-> server(Mod, Mod:init(State)) end)).

client(Pid) ->
    fun(X) ->
        Pid ! {self(), X},
        receive {Pid, Reply} -> Reply end
    end.

server(Mod, State) ->
    receive
        {Pid, {stop}} ->
            Reply = Mod:terminate(State),
            Pid ! {self(), Reply};
        {Pid, X} ->
            {Reply, NewState} = Mod:handler(X, State),
            Pid ! {self(), Reply},
            server(Mod, NewState);
        _ ->
            server(Mod, State)
    end.
%% end yasa.erl

And here's an example using this generic server:

%% begin stack.erl
-module(stack).
-export([new/0, init/1, handler/2, terminate/1]).

-behaviour(yasa).

new() ->
    yasa:new(?MODULE, []).

init(S) ->
    io:format("stack server ~p started.~n", [self()]),
    S.

handler({push, X}, S) ->
    {ok, [X|S]};

handler({pop}, [X|Xs]) ->
    {X, Xs};

handler({peek}, [X|Xs]) ->
    {X, [X|Xs]};

handler({map_to, AnotherStack, Fun}, [X|Xs]) ->
    R = AnotherStack({push, Fun(X)}),
    {R, Xs};

handler({size}, S) ->
    {length(S), S}.

terminate({_}) ->
    goodbye.
%% end stack.erl

Now you can use this stack server like this:

Eshell V5.6.5  (abort with ^G)
1> S1 = stack:new().
stack server <0.39.0> started.
#Fun<yasa.1.69976941>
2> S2 = stack:new().
stack server <0.41.0> started.
#Fun<yasa.1.69976941>
3> S1({push, 1}).
ok
4> S2({push, 10}).
ok
6> S2({peek}).
10
7> S1({map_to, S2, fun(X)-> X * 42 end}).
ok
8> S1({size}).
0
10> S2({peek}).
42
11> S2({size}).
2
12> S1({stop}).
goodbye

The juicy bit in all this code here is the client/1 function in the
yasa module. It returns a fun with the Pid of the spawned process
encapsulated in it and it sends whatever term you pass to the spawned
process. In my example, the fun's argument is always a tuple, even if
the argument is a single atom, but that's just a personal preference
in order to keep syntax consistent.

I realize the resulting syntax looks a lot like invoking a method on
an object in a OO language, but that's as far as I'm concerned just a
side effect of having a compact syntax for sending messages to a
particular process and waiting for a reply.

I'm pretty happy with the result of all this and I can see myself
using this solution in small Erlang projects, however I'm very curious
what other people think of this generic server solution.

To (hopefully) start off with some discussion, I'd like to add my own
main concern with this solution:

It's quite easy for processes to become orphaned the same way as it's
easy to leak memory in languages with manual memory management. All
you have to do is start a new server in a function without returning/
storing the server's client function or stopping the server. As soon
as the function goes out of scope, there's no (sane) way to have
access to this server anymore, just like you can't access memory
anymore in a C program if a function allocates memory without freeing
it or storing the pointer somewhere.

Of course this problem is not unique to my generic server solution in
Erlang, but I guess it's easier to make this error, since you normally
don't have to think about clearing things up after having used a fun.
So I was wondering if there ever have been any ideas about process
management like how memory is managed in garbage collected languages?
I don't know anything about the internals of garbage collectors, but I
can imagine that something like reference counted Pids could work: as
soon as the last reference to a Pid in an Erlang node is cleared by
the garbage collector and there's still a process active with that
Pid, it should be pretty safe to kill it.

So let me summarize this for my standards unusually long e-mail with
the question I had in my mind before starting to type all this:

what do you think of my generic server solution and do you think my
concerns about process management are valid?

thanks for your time,
vincent