[erlang-questions] [ANN] Map/Reduce in Erlang and Python - Disco 0.1
Thu Sep 11 17:10:28 CEST 2008
It's not clear from the FAQ, if it's possible to write map and reduce
functions in other languages, except Python. Can they be implemented in
native Erlang and/or C/C++ compiled executable?
Ville H Tuulos wrote:
> Hi all,
> I am happy to announce the availability of Disco (as already featured in
> Reddit, Hacker News etc.), an open-source implementation of the
> Map/Reduce framework for distributed computing. Its
> core is written in Erlang but users typically write jobs in Python.
> Find the project site at
> or see the source code right away at
> We at Nokia Research in Palo Alto have been using it successfully for
> data mining, building probabilistic models, and full-text indexing of
> hundreds of gigabytes of real-world data on hundreds of CPUs in
> parallel. If you don't have a spare cluster available, we provide a
> script that sets up a working cluster automatically on the Amazon's EC2
> It has been a pleasure to use Erlang to implement the job scheduler
> and other core components of the system. It uses SCGI to provide a web
> interface through an external web server, the slave module to start
> Erlang VMs on slave nodes, and normal port commands to launch Python
> workers on the nodes.
> Disco is released under the BSD license. The system is still young,
> there are known bugs, and there is still work to be done on scalability
> issues as well. You're very welcome to try out the system, give
> feedback, and develop the system with us.
> I'll be at the ICFP / Erlang Workshop in Victoria, so if you're
> attending I'd be happy to show a demo and have a chat with you about
> Ville Tuulos
> Member of Research Staff
> Nokia Research Center
> Palo Alto
> erlang-questions mailing list
View this message in context: http://www.nabble.com/-ANN--Map-Reduce-in-Erlang-and-Python---Disco-0.1-tp19431063p19436494.html
Sent from the Erlang Questions mailing list archive at Nabble.com.
More information about the erlang-questions