[erlang-questions] bspawner: spawn distribution among distributed nodes

Rick Pettit rpettit@REDACTED
Tue May 27 18:34:40 CEST 2008


On Mon, May 26, 2008 2:04 am, Bhasker Kode wrote:
> Hi everyone,
>
> I just wanted to announce of an the project i've just opened up called
> bspawner.
>
> This project was an attempt to load-balance the task of spawning tasks
> across multiple nodes . The steps involved can be isolated into a
> couple of distinct problems.
>   1. deciding which node needs to spawn a task
>   2. communicating across these nodes
>   3. maintaining a record of nodes, added /removed nodes ,etc

I wonder how this compares to pool(3) ?

-Rick

P.S. I've never used pool(3) personally but thought about playing with it
from time to time.

> This project in its essence, deals with the first part and the
> implementation of the message passing begins with the "messenger.erl"
> program located at
> http://www.erlang.org/doc/getting_started/conc_prog.html and will be
> ported to different message-passing, load-balancing and
> node-information maintenance based on feedback,suggestions ,comments,
> and further changes inspired by the growing involvement of the erlang
> community & encouragement from #erlang in particular.
>
> Example
> ------------
> Assuming that there are three nodes: nodeA,nodeB,nodeC
>
> %%an un-balanced version
>
> 1>messenger:message(node1,{factorial,5}).
> 2>messenger:message(node1,{saveToDisk,"a.dat"}).
> 3>messenger:message(node1,{makeHttpRequest,Url}).
> %%all requests originate from current node,and get spawned at target node1
>
> %%un-balanced version
>
> 1>bspawner:bmessage(node1,{factorial,5}).
> 2>bspawner:bmessage(node1,{saveToDisk,"a.txt"}).
> 3>bspawner:bmessage(node1,{makeHttpRequest,Url}).
>
> %%OR you can directly give in {M,F,A} format
>
> 1>bspawner:bspawn({bspawner_request,{NumberCrunchingMod,factorial,[5]} }).
> 2>bspawner:bspawn({bspawner_request,{io,CustomSaveToDisk,["a.txt"]} }).
> 3>bspawner:bspawn({bspawner_request,{ProxyPortHandler,makeHttpRequest,[Url]}
> }).
>
> %% bmessage
> %% replicates messenger:message(node1,"hi") but internally calls bspawn
>
> 1>bspawner:bmessage(node1,"hi").
> %which is syntactic sugar for
> 2>bspawner:bspawn({ bspawner_request, {bspawner,message,["hi"]} })
>
>
> This was however before i got to know about the pool module which is
> exactly what i wanted in the first place. cheers to that, but it was a
> fun learning experience anyway - and i'm looking to add more clarity
> in terms of which node makes a request Vs which nose do processing,
> possibly add features like to make sets  of nodes to do X,another set
> to do Y, and more TODO's that i'd like to add along the way.
>
> Links
> --------
> http://bspawner.googlecode.com
> http://bosky101.blogspot.com/2008/05/erlang-distribution-bspawner.html
>
>
> Keep Clicking,
> Bhasker V Kode
> bosky101 on #erlang
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>





More information about the erlang-questions mailing list