[erlang-questions] mnesia silent failure and table corruption

[Ahmed Omar]

Hi Janos,

Indeed, in that case transactions could be considered succeeded while
actual storage operation failed...

The thing is, If mnesia needs to write data to a store (in this case dets),
it doesn't check for results for this operation. Actually, it does even
catch exceptions. (mnesia_tm:do_update/4)

As dets have a maximum of 2GB for file size, I believe the table gets
corrupted after that. (somebody correct me please)
So any operation on that dets table will return the bad_object error.

So the commit is considered succeeded, mnesia logs it, transaction returns
{atomic, ok}

sync_transaction won't help, because it just waits for the commit to be
synced/logged, but also doesn't check results of the store operation.

so you could end up in situation where dirty write operation returns an
error, and in a transaction returns {atomic, ok}

> dets:insert(table1, SomeKey, SomeVal).
{error,{{bad_object,read_buckets},
        "/tmp/table1.DAT"}}
> mnesia:dirty_write(SomeRecord).
{error,{{bad_object,read_buckets},
        "/tmp/table1.DAT"}}
> mnesia:transaction(fun()-> mnesia:write(SomeRecord) end).
{atomic,ok}

You could try to detect that in your transaction by adding an operation
that will fail (like read, first, last, ..etc) but that won't protect you
against corrupting the file.


The question is, should mnesia detect this kind of errors and if yes, what
to do about it?
Also, can dets protect against the corruption somehow? (reject the
operation that could lead to size> 2GB?)


Best Regards,
Ahmed



On Tue, Aug 27, 2013 at 5:06 PM, Janos Hary <janos.n.hary@REDACTED> wrote:

> Thanks, but no luck. It behaves the same with sync_transaction.
>
> Janos
>
> -----Original Message-----
> From: hawk.mattsson@REDACTED [mailto:hawk.mattsson@REDACTED] On Behalf
> Of
> Håkan Mattsson
> Sent: Tuesday, August 27, 2013 4:02 PM
> To: Janos Hary
> Cc: erlang-questions
> Subject: Re: [erlang-questions] mnesia silent failure and table corruption
>
> On Tue, Aug 27, 2013 at 3:05 PM, Janos Hary <janos.n.hary@REDACTED>
> wrote:
> > All,
> >
> > I started to write records into a table from a simple test function.
> > When the table size reaches 2GB write transactions still report
> > success, but they are silently ignored. The table became unusable,
> > read operations return error, and mnesia cannot recover the table. How
> > shall I avoid such situation?
>
> Try using mnesia:sync_transaction() instead of mnesia:transaction().
> Then Mnesia will use syncronized calls to disk_log (and syncronized commits
> when several nodes are involved).
>
> /Håkan
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130904/964b920b/attachment.htm>