[erlang-questions] Re: how to scale into the cloud using process? example computing simple average
Sat May 30 10:48:52 CEST 2009
----- "Kid Erlang" <> wrote:
> hi everyone. i am still confused? is there a good book on
> ERLANG to read on how to do multi process cloud scaling?
> i do not understand your answers
Apologies for the somewhat existential rants.
You should read Joe Armstrong's book, which explains a lot
of things, including how to scale with Distributed Erlang
If you use Distributed Erlang, that is, many Erlang nodes
connected in the most natural Erlang way, you will simply
use processes and message passing. When using sockets, you
can write a module similar to the rpc module, but which
calls term_to_binary(Data) before sending the request to
the socket, and binary_to_term(Bin) on the response coming
back (and vice versa on the other end). It is reasonably
straightforward. Joe's book has a chapter (ch 14) on
socket programming, which pretty much spells out how to
One of the new problems with clouds is how host names can
come and go, making it difficult to have the kind of static
configuration of the members as one usually has in a cluster
(where one usually has full control over the IP addresses,
host names, etc.)
Joel Reymont used to have a blog article about setting up
mnesia for EC2, which AFAIR addressed these issues, but
the article seems to have rotted away.
I found this:
> i want to scale into the cloud to generate bigger sets of
> numbers and be able to run multiple copies of my scripts which
> already run in a collection.
In your example, you are basically doing addition. You should
be aware that this form of work cannot be sped up even by
parallelizing on multicore. Just running through the list and
spawning a process (or even sending a message) is more work than
just performing the addition.
I showed an example of that in my Erlang Factory presentation:
(slide 9 shows some simple benchmark figures on two different
types of jobs being parallelized.)
CTO, Erlang Training & Consulting Ltd.
More information about the erlang-questions