[erlang-questions] how to scale into the cloud using process? example computing simple average

[Joe Armstrong]

On Tue, May 26, 2009 at 8:01 AM, Kid Erlang <kiderlang@REDACTED> wrote:
> On Tue, May 19, 2009 at 2:26 AM, Bengt Kleberg <bengt.kleberg@REDACTED>
> wrote:
>>
>> The return value of spawn/1 is the (Erlang) process identity. If you
>> want data from a process you have to send it (using !) and receive it
>> (using receive).
>
> okay I have rewrite my project to use receive with spawn, but I still do not
> get how to handle errors?  especially in the cloud?
>
>>
>> It would be a good idea to read one of the Erlang books or the online
>> documentation.
>
> what book is best for me to read?

Modesty prevents me from recommending "Programming Erlang".

> and I cannot find online documentation

http://www.erlang.org/doc.html

> for how to use spawn + receive + messages to scale into the cloud?

What is a cloud?

<< seriously! - Just what is a cloud? -

   I asked myself just this question the other evening -

 Now I did actually go to "a glory glory hallelujah, the cloud is COOL
and WOW - MAN - TWITTER ..." meeting in London - each participant was
given 45 seconds to explain their next greatest "WOW COOL" cloud
application to an assembly of what appeared to be thirteen year old
programmers (the kind that think that PHP is cool) and 25+ ish wannabe
business types
in shiny suits" - I stood at the back for listening for a few minutes
with a group of like-minded souls - after 10 minutes we could take it
no more - we retired to the pub.

   So the the cloud appears to be "a computer at the other end of a
bit of wire, that
somebody else will setup and manage so that you can just pay the bills
and use the thing
and have non of the pain and hassle of actually owning and running a
pile of junk that might on a good day be called a computer."

   This is actually nothing new - we had things called "servers" for a
while now - and
network computing and clusters and virtual machines.

   What seems to be pertinent is management - from my point of view
(and I'm thinking Erlang
here) what I want is a distributed Erlang node up and running on a
remote machine and
I don't care zoot dinkels about how it got there. Since management is
"out of band" I could
happily manage this through a web interface - all need is a form
saying "create N erlang nodes"
I click on "doit" and N erlang nodes are are installed an the
cheapest, lowest-latency, largest
storage, fastest bandwidth kick-arse machine in the universe (has
anybody written this?)

But I digress ...

Tentative answer

     cloud = "a set of abstractions"

     so what are the abstractions?

    The Amazon Web Services says essentially

     Cloud = "CPU + Storage + Queues + management"

     (where CPU = EC2, storage = simple DB etc ...)

     They also have an abstraction "elastic map reduce" (interesting)

     Basing things on messages and queue is of course "the erlang way"
     (ie you send a message to a process mailbox, in order to get it
to do something)
     but for reliable large-scale computing we would need persistent queues with
     reliable delivery etc.

     I guess a good start point for an erlang cloud would be "a set of
erlang nodes"
     (that's the "CPU") - Rabbit MQ (for queues) and a database
back-end of your choice.

     Then you have to manage the system - create all the nodes etc,
make it secure etc.
     This seems to be a mess - how do I create 1000 EC2 instances each
with an Erlang node?
     I just want Erlang, but am forced to install a host OS just for
the purpose. What is the
     minimal OS I need to do this?

     ...


    I'll stop here

>>

>
> that is my main worry: say I have something that uses spawn + receive +
> messages.  how do I make it scale into the cloud?  where can I find online
> documentation on this?  i want my erlang program to run on as many systems
> as I need it to for however many users.  how do I do that?
>
> - Kid Erlang
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>