[erlang-questions] Amazon S3 and SQS implemenation in Erlang
Thu Aug 14 13:52:42 CEST 2008
I'd like to share with you some softwares I selected from my actual job.
My problem was to use a robust file system for data storage,
replication and distribution (like Amazon S3),
a messaging system (like Amazon SQS), and computing cloud (like Amazon
EC2). A little Amazon if you want.
All these, with an open source license if possible.
And here is what I choosed :
* GlusterFS (equiv. to S3) : http://www.gluster.org/
A cluster file-system capable of scaling to several peta-bytes. It
aggregates various storage bricks over
Infiniband RDMA or TCP/IP interconnect into one large parallel network
file system. GlusterFS is based on
a stackable user space design without compromising performance.
* Eucalyptus (equiv. to EC2) : http://eucalyptus.cs.ucsb.edu/
Elastic Utility Computing Architecture for Linking Your Programs To
Useful Systems - is an open-source software infrastructure for
implementing "cloud computing" on clusters. The current interface to
EUCALYPTUS is compatible with Amazon's EC2 interface, but the
infrastructure is designed to support multiple client-side interfaces.
EUCALYPTUS is implemented using commonly available Linux tools and
basic Web-service technologies making it easy to install and maintain.
* RabbitMQ (equiv. to SQS) : http://www.rabbitmq.com/
The only exception is here. I need to hide the actual APMQ interface
behind an SQS one.
Not a big deal as RabbitMQ is an awesome piece of code to hack.
Hope this help you build rock solid, open source, free of charge
Le 12 août 08 à 15:07, Tim Fletcher a écrit :
>> What I'm looking after is a free, simple, and reliable (with
>> replication suport) library to store large
>> number (thousands to million) of very big files (>1gb per file) on
>> secondary storage.
> Hadoop (http://hadoop.apache.org/core/) would be another candidate, if
> it doesn't have to be Erlang.
> erlang-questions mailing list
More information about the erlang-questions