[erlang-questions] Combining ets and mnesia operations for extremeperformance

Paulo Sérgio Almeida psa@REDACTED
Mon Sep 15 18:05:43 CEST 2008


Paul Fisher wrote:
> Paulo Sérgio Almeida wrote:
>> There is not much information about this operation. Does it provide
>> atomicity? What happens if there is a crash in the middle of it? I have
>> not made it clear, but I need to persist the updates on several tables
>> as an atomic operation.
> 
> As a general strategy for this type of thing I will do the dump (usually 
> ets tables with ets:tab2file/1) and one the operation is complete i will 
> rename the file from <file>.tmp to <file> in order to "commit" the 
> operation for later recovery.

Yes. This works for a single table. My needs include atomicity involving 
several tables. Also I want to be able to postpone the dumping of the 
full table, which leads to an incremental log-based dumping of 
operations, like what mnesia does.

This is tempting me to write a module tailored to what I need: 
persisting ets tables while ensuring some transactional properties, 
namely atomic updates involving several tables, from the durability 
point of view. It would have no distribution or concurrency control. As 
a bonus it would support more general ets tables, not being restricted 
to the fixed-length record based ones from mnesia. Sometimes it feels 
like a waste having a field for the record name when I just want a 
key-value mapping, or having to have at least 2 fields when I want to 
simulate a set and the key is enough. But the important thing is it will 
give me orders of magnitude more efficiency than what I am allowed to do 
"officially" with mnesia.

It promises to be fun.

Cheers,
Paulo



More information about the erlang-questions mailing list