[erlang-questions] Amazon S3 backend for Mnesia

[Joel Reymont]

On Jun 4, 2007, at 1:04 PM, Ulf Wiger (TN/EAB) wrote:

> But you basically only need to do this if it's
> important for you to amortize the updates. It might
> be important in other cases _not to_, e.g. if you
> have other readers of the custom backend. In that case,
> data wouldn't be visible after a commit, but only after
> the next log dump, which might result in inconsistencies
> at the application level.

I can now reveal that I'm working on a Amazon S3 backend for Mnesia.  
The goal is to add transactions and locking, as well as make the data  
just written to S3 available right away.

S3 provides neither transactions nor locking. Data written to S3 may  
become available for reading at some point in the future. S3 is  
conceptually a bunch of ets tables, at least I think it can be viewed  
this way.

I'm looking to have a bunch of Mnesia nodes up and a bunch of  
s3_copies tables on each.

Whenever data is written to one of the Mnesia nodes, I'm looking to  
have _that_ node commit the data to S3 and replicate it to other  
nodes as Mnesia does normally. I would also keep a cache of most  
recently written data in a regular Mnesia table.

I understand that Synapse added a bdb_copies type of table and found  
this hard to maintain. Would rdbms be easier to maintain in my scenario?

	Thanks, Joel

--
http://topdog.cc      - EasyLanguage to C# translator
http://wagerlabs.com  - Blog