[erlang-questions] large objects in dets

[vijay chakravarthy]

Why is this not a URL? We do exactly the same thing, except for flash movies
generated from powerpoint. The URL we use is actually an S3 url (Amazon S3).

The only issue which you might want to consider is things like replication
and garbage collection for these large objects  - in some cases you would
need to build your own. In our case, its simple, these objects are not
shared, and S3 provides the redundancy.
Of course, instead of something like S3 you could use a decent file system
(ZFS in solaris is nice). Other alternatives are mogileFS, nutch file system
-- if you want a replicated file system with simple semantics. Of course I
would argue that mogileFS is better off rewritten in erlang -- but thats a
project for another day.

Vijay


On 5/10/07, Joe Armstrong <erlang@REDACTED> wrote:
>
> I'd like to store large objects in a persistent disk store.
>
> Dets is "almost" ok.
>
> Dets is a key-value store - but there will be problems if the value is
> large (think a 4 Gbyte movie)
>
> What I need is an interface that allows random access of large values
>
>    dets:reserve_space_for_key(LargeKey, 2000000000)    reserve 2 GB
>    dets:store(LargeKey, Start, <<bin>>)
>    dets:lookup(largeKey, Start, Len) -> <<bin>>
>
> Now I know that dets does not do this now.
>
> Now the questions
>
>    - to those of you who have hacked dets - is this an easy extension to
> dets?
>    - should I hack dets?
>    - should I "roll-my-own"
>    - has anybody already made a large object disk storage thingy
>
> /Joe
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20070510/6d028522/attachment.htm>