[erlang-questions] scalaris code posted

[Kevin Scaldeferri]

On Aug 7, 2008, at 7:58 AM, john s wolter wrote:

> Oh, that sounds bad.  I'll look into hypertable, maybe I can see  
> someone on a project using it.  Most customers always go with what  
> they know, right or Wrong.
>

My anecdotal 2c:

I went to a couple talks on these projects at OSCON a couple weeks  
ago.  As for Hadoop, the NYT had some great success using Hadoop and  
EC2/S3 to do a bunch of image processing, and they plan to continue  
using it.  Also Hadoop just pulled ahead on the TB sort challenge, so  
at least for some tasks it's pretty fast. (BTW, having Hadoop jobs  
that run continuously for days sounds odd to me.  The whole point is  
to parallelize so you can do in hours or minutes what would have taken  
days.)

Hypertable looks interesting but immature.  Nearly every question from  
the audience was answered, "well, we're still working on that".

Hadoop and Hypertable are not competing projects.  They provide  
distinct services.

>> Anyway, Hadoop basics are simple and one can come up with an Erlang  
>> implementation easily.
>>

Sure, about a million bloggers have written their own "MapReduce".   
Just remember, the "map" and "reduce" bits are the trivial,  
uninteresting part.  The hard bits are reliability, fault-tolerance,  
cluster management, performance (i.e. having the right data in the  
right place at the right time), etc



-kevin