OTP Performance

[Joe Armstrong (TN/EAB)]

Hi Matthew,

I'll try and answer your questions:

Before I answer I have some comments on your program.

Neither of the measurements mean anything.

If you use cast in this way you need some proof that you won't overflow
the message buffers.

If the producer is faster than the consumer then your program will
eventually crash.

By casting 10M message you probably are just filling the message queue,
so
what you have measured is the time to fill a queue, you should really
send a stop message. ie say flood(0, Target) -> Target ! stop.
Then the stop time is the time when the receiver receives the stop
message.

Try this and report back :-)

 Now the questions:

1) I don't know the answer. 

   I don't know if does hurt performance - a system is more than just a 
loop with a cast in it. 

Now I guess there are two kinds of apps - ones that *need* things like
replicated data bases (for fault tolerance) code change on-the-fly
(because the site can never go down) etc. By the time you have added all
the 
other stuff you need (supervisor strategies, replicated data bases on 
different nodes) then the performance difference might not be as great
as 
you think they are going to be.

For small projects without complex requirements for fault-tolerance
then a non-OTP solution might be faster. The answer is "measure and see"

The test of a good design is a modular structure that will allow you to
run with and
without OTP, so that you can make such measurements.

As far as I am aware all the big commercial projects (ie with turnovers
over
100 M$) use OTP and the performance is adequate.

I wrote some comments in  

  http://www.erlang.org/ml-archive/erlang-questions/200605/msg00375.html

Which I'll quote here

> The OTP behaviours are not magic bullets, they are just libraries of
> erlang code for performing repetitious tasks in a consistent manner.

> The main benefit of using (say) gen_servers is organisational - if you
> have a large team of programmers (say a few hundred) and they are all
writing
> client-servers, then it might be a good idea if they all go about this

>  the same way.

> The OTP libraries were written to the standardise the way servers,
etc.-
> were written, this was so that one programmer in a large organisation
could
> understand the code of another programmer in the same organisation.

> There are, of course, no such benefits in a small one-person project.

> Writing a client-server in Erlang is really easy. You need to
understand
> send receive and spawn. Making it fault tolerant is easy (you need to
> understand spawn_link, links, and exit signals). 

> You can have 95% of all the fun by understanding how to roll-your own
> client-servers using spawn, send, receive etc.

> gen_server provides one commonly used architectural pattern, in a
> context where it is suitable for large programmer teams.


2) You have to measure.

Personally I use lot's of processes, each doing very little,
I use binaries for large "string" data.

Using lots of processes will pay back on the multi-core CPUs
so it's the kind of future proof way of doing things.

3) Yes - mostly. Binaries can end up on a shared heap, and 
indeed a shared heap might be used internally. But you're not supposed
to worry about this.

You should write your program without thinking about whether things get
copied
or not. Then measure. Then if you program is too slow ask this group
why.

If you want to send an entire list then I imagine sending them item by
item
would be far slower than just sending the list. Anyway the code would be
far
shorter.

	Pid ! L

vs
	Pid ! start, foreach(fun(I) -> Pid ! I end, L), Pid ! stop
	and some weird code at the receiver to receive the list

      This would be a total mess, the list either gets send in the wrong
order
	since we want to build it tail-first or need reversing

Note that list processing is very efficient and highly optimised.

/Joe


> -----Original Message-----
> From: owner-erlang-questions@REDACTED 
> [mailto:owner-erlang-questions@REDACTED] On Behalf Of 
> Matthew Sackman
> Sent: den 20 augusti 2006 13:11
> To: erlang-questions@REDACTED
> Subject: OTP Performance
> 
> Hi,
> 
> Whilst I think I'm very familiar with the documentation or 
> erlang.org, this has probably been raised for, so feel free 
> to point me in the direction of the answers...
> 
> Exhibit 1:
> -module(test).
> 
> -export([flood/1, flood/2, send/1, receiver/0]).
> 
> flood(Count) ->
>     Target = spawn_link(test, receiver, []),
>     Start = now(),
>     flood(Count, Target),
>     End = now(),
>     io:format("~w~n", [timer:now_diff(End, Start)]),
>     Target ! stop,
>     ok.
> 
> flood(0, _Target) ->
>     ok;
> 
> flood(N, Target) ->
>     send(Target),
>     flood(N-1, Target).
> 
> send(Target) ->
>     Target ! ok.
> 
> receiver() ->
>     receive
>      stop ->
>        exit(normal);
>      _Else ->
>        receiver()
>     end.
> 
> and I get about 3.5 seconds for 10,000,000 messages.
> Exhibit 2:
> -module(test).
> 
> -behavior(gen_server).
> 
> -export([start_link/0, init/1, handle_cast/2, terminate/2]).
> -export([flood/1, flood/2, send/1]).
> 
> flood(Count) ->
>     {ok, Target} = start_link(),
>     Start = now(),
>     flood(Count, Target),
>     End = now(),
>     io:format("~w~n", [timer:now_diff(End, Start)]),
>     gen_server:cast(Target, stop),
>     ok.
> 
> flood(0, _Target) ->
>     ok;
> 
> flood(N, Target) ->
>     send(Target),
>     flood(N-1, Target).
> 
> send(Target) ->
>     gen_server:cast(Target, ok).
> 
> start_link() ->
>     gen_server:start_link(test, {}, []).
> 
> init({}) ->
>     {ok, {}}.
> 
> handle_cast(stop, State) ->
>     {stop, normal, State};
> handle_cast(_Any, State) ->
>     {noreply, State}.
> 
> terminate(normal, _State) ->
>     ok;
> terminate(Code, State) ->
>     io:format("Terminating ~w ~w~n", [Code, State]).
> 
> And it's now about 17 seconds for 10,000,000 messages. Now, I 
> know that gen_servers invokes functions via the Implicit 
> Apply mechanism (eg Mod:handle_cast), and that is expensive. 
> So in many ways, these results aren't surprising.
> 
> So my question is, what does this mean for applications that 
> need to achieve a very high throughput of messages a second?
> 
> 1) Is there a body of applications where OTP has been avoided 
> because it hurts performance?
> 
> 2) Are there other solutions that allow you to use OTP and 
> maintain high message throughput? For example, is it 
> generally considered to have a dozen or so processes that 
> each do a little, and chain messages between them, or have 
> much fewer processes that each do a more sizeable chunk of 
> the work and thus reduce messaging costs?
> 
> 3) Does Erlang implement copy on write for communication? If 
> so, does that mean that batching messages can be a win? E.g. 
> rather than sending the items of a list, send the list itself 
> - does this actually reduce the work done by Erlang (I 
> realise this is a different matter from the OTP-related questions)?
> 
> Many thanks,
> 
> Matthew
> --
> Matthew Sackman
> 
> BOFH excuse #135:
> You put the disk in upside down.
>