[erlang-questions] Mnesia deleting log file

[Arun]

Even after setting the "dump_log_update_in_place" configuration to 
false, the data is not getting retained
after a power outage. Any more configurations that can be tried.

Regards,
Arun P


On Thursday 23 November 2017 02:46 AM, Michael Schmidt wrote:
>
> I agree—setting dump_log_in_place to false makes extremely unlikely 
> for mnesia to fail in this way.  Both JFFS2 and UBIFS ensure that 
> rename is atomic, even when power is pulled.
>
> For desktop applications, the culprit is likely the VM being killed, 
> not the whole system crashing.  Thus the rename was either executed or 
> not;  either way we have a complete DAT file.
>
> There is always a chance of things like this happening: 
> http://www.linux-mtd.infradead.org/doc/ubifs.html#L_unstable_bits, but 
> this shouldn’t happen on most hardware.
>
> Note that the default for dump_log_time_threshold is 3 minutes. Speed 
> vs Safety is always a trade-off, but many people will be surprised to 
> lose data in a crash 2 minutes after saving.
>
> *From:*Jesper Louis Andersen [mailto:jesper.louis.andersen@REDACTED]
> *Sent:* Wednesday, November 22, 2017 7:54 AM
> *To:* Michael Schmidt <Michael.K.Schmidt@REDACTED>
> *Cc:* Arun <arunp@REDACTED>; erlang-questions@REDACTED
> *Subject:* Re: [erlang-questions] Mnesia deleting log file
>
> This thread makes we wonder about the following section in the Mnesia 
> documentation:
>
> "If a power failure occurs during the dump, this can cause the 
> randomly accessed DAT files to become corrupt. If the parameter is set 
> to false, Mnesia copies the DAT files and target the dump to the new 
> temporary files. If the dump is successful, the temporary files are 
> renamed to their normal DAT suffixes. The possibility for 
> unrecoverable inconsistencies in the data files becomes much smaller 
> with this strategy. However, the actual dumping of the transaction log 
> becomes considerably slower. The system designer must decide whether 
> speed or safety is the higher priority."
>
> In UNIX(Posix) a rename(2) call is atomic. The worst case, which is an 
> OS crash might leave you with a temporary file and the target file, 
> hardlinked as if ln() was called, but the move itself is atomic, even 
> under OS crashes. Of course, this assumes you correctly wrote your 
> temporary file, fsync()'ed the file and then performed the rename() as 
> you should.
>
> With that information in mind, what is the scenario which can make 
> mnesia fail if the dump_log_update_in_place is set to false? There may 
> be a window, but the operating system cannot be the provider of this 
> window at all, unless it is incorrectly implemented or configured[1]. 
> Hence my question.
>
> [1] The primary concern are write caches on disks which can have a 
> write hole unless they have proper (working!) battery backup of their 
> cache or are able to write their memory to a small solid state part of 
> the chip. Or if you are using ZFS where copy-on-write semantics 
> completely eliminates the write hole.
>
> On Wed, Nov 22, 2017 at 2:29 PM Michael Schmidt 
> <Michael.K.Schmidt@REDACTED 
> <mailto:Michael.K.Schmidt@REDACTED>> wrote:
>
>     Hi Arun,
>
>     If you happen to reboot when Mnesia is writing out its logfile, it
>     can become corrupt (and lose data).  We fight this issue on
>     embedded devices a lot.
>
>     Look “Configuration Parameters” section here:
>
>     http://erlang.org/doc/man/mnesia.html#id69584
>
>     You will want to set dump_log_update_in_placeto false.  This will
>     make it write out the table to a new file, and rename it the very
>     end.
>
>     One other option to look at is dump_log_time_thresholdto change
>     how often items are flushed to disk.  For configuration data, you
>     can likely set this to a few seconds.
>
>     These options can be set via the sys.config file as well
>
>     Mike
>
>     *From:*erlang-questions-bounces@REDACTED
>     <mailto:erlang-questions-bounces@REDACTED>[mailto:erlang-questions-bounces@REDACTED
>     <mailto:erlang-questions-bounces@REDACTED>] *On Behalf Of *Arun
>     *Sent:* Tuesday, November 21, 2017 10:16 PM
>     *To:* erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
>     *Subject:* [erlang-questions] Mnesia deleting log file
>
>     Dear all,
>
>     I've a program written in erlang which uses Mnesia application as
>     the database application. I've a table created by name
>     "configuration_table" which stores
>
>     certain configurations that need to be persistent. Occasionally,
>     whenever I restart my program the following error is thrown by
>     mnesia and I end up losing
>
>     all the persistent configurations.
>
>     I've searched about this problem in the erlang documentation and
>     all it tells is "Node not running". What could be the probable
>     cause for this problem
>
>     and how do I fix it?
>
>     Mnesia('test@REDACTED <mailto:ommp82_plus@REDACTED>'): Data may
>     be missing, Corrupt logfile deleted:
>     "/home/utl/mnesia_database/configuration_table.DCL",
>     {node_not_running,
>     'test@REDACTED <mailto:ommp82_plus@REDACTED>'}
>
>
>     ______________________________________________________________________
>     This email has been scanned by the Symantec Email Security.cloud
>     service.
>     ______________________________________________________________________
>
>     _______________________________________________
>     erlang-questions mailing list
>     erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
>     http://erlang.org/mailman/listinfo/erlang-questions
>
>
> ______________________________________________________________________
> This email has been scanned by the Symantec Email Security.cloud service.
> ______________________________________________________________________
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20171123/38e53719/attachment.htm>