<div dir="ltr">I'm curious: are the data written as part of a mnesia transaction?<br><br></div><br><div class="gmail_quote"><div dir="ltr">On Thu, Nov 23, 2017 at 12:22 PM Arun <<a href="mailto:arunp@utl.in">arunp@utl.in</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<p>Even after setting the "dump_log_update_in_place" configuration
to false, the data is not getting retained
<br>
after a power outage. Any more configurations that can be tried.</p>
Regards,<br>
Arun P</div><div bgcolor="#FFFFFF" text="#000000"><br>
<br>
<br>
<div class="m_-4093598724064724423moz-cite-prefix">On Thursday 23 November 2017 02:46 AM,
Michael Schmidt wrote:<br>
</div>
</div><div bgcolor="#FFFFFF" text="#000000"><blockquote type="cite">
<div class="m_-4093598724064724423WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">I
agree—setting dump_log_in_place to false makes extremely
unlikely for mnesia to fail in this way. Both JFFS2 and
UBIFS ensure that rename is atomic, even when power is
pulled.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">For
desktop applications, the culprit is likely the VM being
killed, not the whole system crashing. Thus the rename was
either executed or not; either way we have a complete DAT
file. <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">There
is always a chance of things like this happening:
<a href="http://www.linux-mtd.infradead.org/doc/ubifs.html#L_unstable_bits" target="_blank">http://www.linux-mtd.infradead.org/doc/ubifs.html#L_unstable_bits</a>,
but this shouldn’t happen on most hardware.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Note
that the default for dump_log_time_threshold is 3 minutes.
Speed vs Safety is always a trade-off, but many people will
be surprised to lose data in a crash 2 minutes after saving.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">
Jesper Louis Andersen
[<a class="m_-4093598724064724423moz-txt-link-freetext" href="mailto:jesper.louis.andersen@gmail.com" target="_blank">mailto:jesper.louis.andersen@gmail.com</a>]
<br>
<b>Sent:</b> Wednesday, November 22, 2017 7:54 AM<br>
<b>To:</b> Michael Schmidt
<a class="m_-4093598724064724423moz-txt-link-rfc2396E" href="mailto:Michael.K.Schmidt@schneider-electric.com" target="_blank"><Michael.K.Schmidt@schneider-electric.com></a><br>
<b>Cc:</b> Arun <a class="m_-4093598724064724423moz-txt-link-rfc2396E" href="mailto:arunp@utl.in" target="_blank"><arunp@utl.in></a>;
<a class="m_-4093598724064724423moz-txt-link-abbreviated" href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
<b>Subject:</b> Re: [erlang-questions] Mnesia deleting log
file<u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">This
thread makes we wonder about the following section in
the Mnesia documentation:<br>
<br>
"If a power failure occurs during the dump, this can
cause the randomly accessed <span class="m_-4093598724064724423inbox-inbox-code">
DAT</span> files to become corrupt. If the parameter
is set to <span class="m_-4093598724064724423inbox-inbox-code">
false</span>, <span class="m_-4093598724064724423inbox-inbox-code">Mnesia</span>
copies the <span class="m_-4093598724064724423inbox-inbox-code">
DAT</span> files and target the dump to the new
temporary files. If the dump is successful, the
temporary files are renamed to their normal
<span class="m_-4093598724064724423inbox-inbox-code">DAT</span> suffixes.
The possibility for unrecoverable inconsistencies in
the data files becomes much smaller with this
strategy. However, the actual dumping of the
transaction log becomes considerably slower. The
system designer must decide whether speed or safety is
the higher priority."<u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">In
UNIX(Posix) a rename(2) call is atomic. The worst case,
which is an OS crash might leave you with a temporary
file and the target file, hardlinked as if ln() was
called, but the move itself is atomic, even under OS
crashes. Of course, this assumes you correctly wrote
your temporary file, fsync()'ed the file and then
performed the rename() as you should.<u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">With that
information in mind, what is the scenario which can make
mnesia fail if the dump_log_update_in_place is set to
false? There may be a window, but the operating system
cannot be the provider of this window at all, unless it is
incorrectly implemented or configured[1]. Hence my
question.<u></u><u></u></p>
</div>
<p class="MsoNormal">[1] The primary concern are write caches
on disks which can have a write hole unless they have proper
(working!) battery backup of their cache or are able to
write their memory to a small solid state part of the chip.
Or if you are using ZFS where copy-on-write semantics
completely eliminates the write hole.<u></u><u></u></p>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">On Wed, Nov 22, 2017 at 2:29 PM Michael
Schmidt <<a href="mailto:Michael.K.Schmidt@schneider-electric.com" target="_blank">Michael.K.Schmidt@schneider-electric.com</a>>
wrote:<u></u><u></u></p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Hi
Arun,</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">If
you happen to reboot when Mnesia is writing out its
logfile, it can become corrupt (and lose data). We
fight this issue on embedded devices a lot.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Look
“Configuration Parameters” section here:</span><u></u><u></u></p>
<p class="MsoNormal"><a href="http://erlang.org/doc/man/mnesia.html#id69584" target="_blank"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">http://erlang.org/doc/man/mnesia.html#id69584</span></a><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">You
will want to set
</span><span style="font-family:Courier;color:#1a1a1a;background:#f3f3f3">dump_log_update_in_place</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> to
false. This will make it write out the table to a
new file, and rename it the very end. </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">One
other option to look at is
</span><span style="font-family:Courier;color:#1a1a1a;background:#f3f3f3">dump_log_time_threshold</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> to
change how often items are flushed to disk. For
configuration data, you can likely set this to a few
seconds.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">These
options can be set via the sys.config file as well</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Mike</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> </span><u></u><u></u></p>
<div>
<div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">
</span><a href="mailto:erlang-questions-bounces@erlang.org" target="_blank"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">erlang-questions-bounces@erlang.org</span></a><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">
[mailto:</span><a href="mailto:erlang-questions-bounces@erlang.org" target="_blank"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">erlang-questions-bounces@erlang.org</span></a><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">]
<b>On Behalf Of </b>Arun<br>
<b>Sent:</b> Tuesday, November 21, 2017 10:16 PM<br>
<b>To:</b> </span><a href="mailto:erlang-questions@erlang.org" target="_blank"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">erlang-questions@erlang.org</span></a><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><br>
<b>Subject:</b> [erlang-questions] Mnesia
deleting log file</span><u></u><u></u></p>
</div>
</div>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<p>Dear all,<u></u><u></u></p>
<p>I've a program written in erlang which uses Mnesia
application as the database application. I've a table
created by name "configuration_table" which stores<u></u><u></u></p>
<p>certain configurations that need to be persistent.
Occasionally, whenever I restart my program the
following error is thrown by mnesia and I end up
losing<u></u><u></u></p>
<p>all the persistent configurations. <u></u><u></u></p>
<p>I've searched about this problem in the erlang
documentation and all it tells is "Node not running".
What could be the probable cause for this problem<u></u><u></u></p>
<p>and how do I fix it?<u></u><u></u></p>
<p>Mnesia('<a href="mailto:ommp82_plus@10.1.4.16" target="_blank">test@10.1.4.16</a>'):
Data may be missing, Corrupt logfile deleted:
"/home/utl/mnesia_database/configuration_table.DCL",
{node_not_running,
<br>
'test<a href="mailto:ommp82_plus@10.1.4.16" target="_blank">@10.1.4.16</a>'}
<u></u>
<u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><br>
______________________________________________________________________<br>
This email has been scanned by the Symantec Email
Security.cloud service.<br>
______________________________________________________________________<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal">_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal"><br>
______________________________________________________________________<br>
This email has been scanned by the Symantec Email
Security.cloud service.<br>
______________________________________________________________________<u></u><u></u></p>
</div>
</blockquote>
<br>
</div></blockquote></div>