[erlang-questions] Efficiently backing up a DETS files

[Alexei Sholik]

I shied away from the added complexity and learning curve of Mnesia. My
simple use case involves permanently storing key-value data that is not
going to be modified or searched for. I'm not using RAM storage either
because I have modest amounts of memory. Data is read not that often, so
dets works just fine for me.

I wasn't aware of Mnesia's builtin backup facilities though. Thanks for
mentioning that.


On Fri, Jun 27, 2014 at 3:58 PM, T Ty <tty.erlang@REDACTED> wrote:

> Any reason why you don't want to run Mnesia instead and use it's backup
> and restore functions ?
>
>
> On Fri, Jun 27, 2014 at 1:39 PM, Alexei Sholik <alcosholik@REDACTED>
> wrote:
>
>> Hi all,
>>
>> I've been thinking about and googling for ways to back up a running dets
>> database (i.e. with no interruption to the main service using it) and
>> here's what I have come up with so far.
>>
>> 1. On the same node that currently has the dets file open, create an
>> empty ETS table and another empty dets file. Call dets:to_ets and then
>> ets:to_dets to write to the new dets file. After closing the latter file it
>> will be safe to move it to the backup location.
>>
>> 2. Open a new dets file and use dets:bchunk on the production file in
>> combination with dets:init_table. I wasn't sure if init_table would iterate
>> over the original dets file automatically (iow, handling all the
>> continuations before receiving '$end_of_table').
>>
>> 3. Use one of the traversal functions to copy elements from one dets file
>> to another one.
>>
>> I first thought the 1st approach to be the most efficient one (provided
>> there is enough free RAM for the temporary ETS table). However, if bchunk()
>> works like this with init_table():
>>
>>      TableSize = dets:info(my_table, size),
>>      dets:open_file(new_table, [{min_no_slots, TableSize}]),
>>      InitFun = dets:bchunk(my_table, start),
>>      dets:init_table(new_table, InitFun, [{min_no_slots, TableSize},
>> {format, bchunk}]),
>>      dets:close(new_table).
>>
>> then this 2nd approach looks like the best candidate. Here I could also
>> use 'ram_file' option for new_table to speed up the subsequent init_table()
>> call provided there is enough free RAM. Is this correct?
>>
>> I will appreciate anyone pointing out mistakes in my reasoning above.
>>
>> Thanks
>>
>> --
>> Best regards
>> Alexei Sholik
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>


-- 
Best regards
Alexei Sholik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140627/2bc02cd6/attachment.htm>