[erlang-questions] Amazon S3 and SQS implemenation in Erlang

Thu Aug 14 13:52:42 CEST 2008

Hi guys,

I'd like to share with you some softwares I selected from my actual job.

My problem was to use a robust file system for data storage,  
replication and distribution (like Amazon S3),
a messaging system (like Amazon SQS), and computing cloud (like Amazon  
EC2). A little Amazon if you want.
All these, with an open source license if possible.

And here is what I choosed :

* GlusterFS (equiv. to S3) : http://www.gluster.org/

A cluster file-system capable of scaling to several peta-bytes. It  
aggregates various storage bricks over
Infiniband RDMA or TCP/IP interconnect into one large parallel network  
file system. GlusterFS is based on
a stackable user space design without compromising performance.

* Eucalyptus (equiv. to EC2) : http://eucalyptus.cs.ucsb.edu/

Elastic Utility Computing Architecture for Linking Your Programs To  
Useful Systems - is an open-source software infrastructure for  
implementing "cloud computing" on clusters. The current interface to  
EUCALYPTUS is compatible with Amazon's EC2 interface, but the  
infrastructure is designed to support multiple client-side interfaces.  
EUCALYPTUS is implemented using commonly available Linux tools and  
basic Web-service technologies making it easy to install and maintain.

* RabbitMQ (equiv. to SQS) : http://www.rabbitmq.com/
The only exception is here. I need to hide the actual APMQ interface  
behind an SQS one.
Not a big deal as RabbitMQ is an awesome piece of code to hack.

Hope this help you build rock solid, open source, free of charge  
Erlang applications.


Le 12 août 08 à 15:07, Tim Fletcher a écrit :

>> What I'm looking after is a free, simple, and reliable (with
>> replication suport) library to store large
>> number (thousands to million) of very big files (>1gb per file) on
>> secondary storage.
> Hadoop (http://hadoop.apache.org/core/) would be another candidate, if
> it doesn't have to be Erlang.
> _______________________________________________
> erlang-questions mailing list
> http://www.erlang.org/mailman/listinfo/erlang-questions

More information about the erlang-questions mailing list