[erlang-questions] Erlang on supercomputers

[Thomas Lindgren]

--- Olivier BOUDEVILLE <olivier.boudeville@REDACTED>
wrote:

> Hi,
> 
> first, sorry if the question is irrelevant, I am by
> no means an HPC or 
> Erlang expert. 
> 
> I was wondering whether, for the simulation of a
> large-scale distributed 
> system, one could use Erlang on a supercomputer such
> as an IBM's 
> BlueGene/L (see
> http://en.wikipedia.org/wiki/Blue_Gene for a
> description).
...
> Knowing that each processor is a PowerPC (and that
> Erlang works on Mac) 
> running a very minimal (very light-weight,
> mono-process) GNU/Linux OS with 
> limited POSIX support,  what I imagine is that it
> would be feasible to run 
> on each of the, say, 4000 nodes, a stripped-down
> Erlang interpreter that 
> would schedule itself (without much support of the
> OS, no light-weight 
> threads) numerous interacting Erlang processes that
> could communicate 
> preferably with other processes on the same node or,
> otherwise, with 
> processes running on close nodes. 
> 
> Is having Erlang running on these architectures
> technically possible with 
> a limited porting effort ? If yes, am I correct to
> think that, provided we 
> manage to place cleverly the concurrent processes
> (the more two of them 
> are interacting, the "closest" they are in the
> communication topology, a 
> three-dimensional torus), we could in some way take
> advantage of the 
> message-passing nature of Erlang to use (even
> less-than-optimally) the 
> processing power of these machines ?

As long as you're running some Unix-flavour, porting
should be fairly straightforward. The regular Erlang
VM is not excessively large either, so you could start
out by just getting Erlang/OTP running and then pare
things down.

The snags I can see are:

1. Distributed Erlang could have scalability problems,
since it by default uses a mesh of nodes the last time
I looked. You may have to read up on "hidden nodes" to
get this right. (I haven't studied this issue
recently, however.)

2. BlueGene is basically a number cruncher, isn't it?
While it's not hopeless to do that in Erlang, you
probably want to implement the serious numerics in
another language and link them up. Several methods are
available.

MPI: I'm not aware of any existing implementations.
You can probably implement the MPI protocol in Erlang
though. The bit syntax is handy for that sort of
thing. Or perhaps link with an existing
implementation. I think there is a section on how to
implement the Erlang distribution protocol in the
docs.

Best,
Thomas



 
____________________________________________________________________________________
The fish are biting. 
Get more visitors on your site using Yahoo! Search Marketing.
http://searchmarketing.yahoo.com/arp/sponsoredsearch_v2.php