DETS table auto_save behaviour

[Frank Muller]

How about Mnesia and persistence to disk?


> It's always tricky with open files during some abrupt crashes. OS-level
> file system caching means that not all written data may have been
> physically written to disk.
>
> To detect this, dets has a flag indicating whether the file was properly
> closed. As I understand it, the 'auto-save' does the same thing as when the
> file is closed, except the file stays open.
>
> BR,
> Ulf W
>
> Den ons 26 maj 2021 23:10Mikael Pettersson <mikpelinux@REDACTED> skrev:
>
>> On Tue, May 25, 2021 at 8:43 AM Nicolas Martyanoff <khaelin@REDACTED>
>> wrote:
>> > I was hoping to use DETS as a local persistent buffer in case data
>> > cannot be written to a remote database, but it seems impossible to
>> > guarantee that every entry is being sync-ed to disk.
>>
>> I'm not too familiar with the internals of DETS, but basically data
>> goes straight to/from disk while meta-data about allocated and free
>> areas of the file are cached in memory. I don't know if writes are
>> sync or not. In our experience, DETS files are somewhat fragile, plus
>> they have a hard 2GB size limitation which made them extremely awkward
>> for our use case (large mnesia tables). That's part of the reason we
>> migrated most of our mnesia tables to eleveldb.
>>
>> If I had to have a standalone (not mnesia) local persistent store I'd
>> probably go with eleveldb (or one of its spinoffs) if I needed lookups
>> by key, or a disk_log if I just needed a FIFO buffer. disk_log allows
>> you to choose how sync or async your writes are. _I_ wouldn't use
>> DETS.
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20210527/be4f8fb5/attachment.htm>