[erlang-questions] Mnesia deleting log file
Thu Nov 23 12:22:01 CET 2017
Even after setting the "dump_log_update_in_place" configuration to
false, the data is not getting retained
after a power outage. Any more configurations that can be tried.
On Thursday 23 November 2017 02:46 AM, Michael Schmidt wrote:
> I agree—setting dump_log_in_place to false makes extremely unlikely
> for mnesia to fail in this way. Both JFFS2 and UBIFS ensure that
> rename is atomic, even when power is pulled.
> For desktop applications, the culprit is likely the VM being killed,
> not the whole system crashing. Thus the rename was either executed or
> not; either way we have a complete DAT file.
> There is always a chance of things like this happening:
> http://www.linux-mtd.infradead.org/doc/ubifs.html#L_unstable_bits, but
> this shouldn’t happen on most hardware.
> Note that the default for dump_log_time_threshold is 3 minutes. Speed
> vs Safety is always a trade-off, but many people will be surprised to
> lose data in a crash 2 minutes after saving.
> *From:*Jesper Louis Andersen [mailto:jesper.louis.andersen@REDACTED]
> *Sent:* Wednesday, November 22, 2017 7:54 AM
> *To:* Michael Schmidt <Michael.K.Schmidt@REDACTED>
> *Cc:* Arun <arunp@REDACTED>; erlang-questions@REDACTED
> *Subject:* Re: [erlang-questions] Mnesia deleting log file
> This thread makes we wonder about the following section in the Mnesia
> "If a power failure occurs during the dump, this can cause the
> randomly accessed DAT files to become corrupt. If the parameter is set
> to false, Mnesia copies the DAT files and target the dump to the new
> temporary files. If the dump is successful, the temporary files are
> renamed to their normal DAT suffixes. The possibility for
> unrecoverable inconsistencies in the data files becomes much smaller
> with this strategy. However, the actual dumping of the transaction log
> becomes considerably slower. The system designer must decide whether
> speed or safety is the higher priority."
> In UNIX(Posix) a rename(2) call is atomic. The worst case, which is an
> OS crash might leave you with a temporary file and the target file,
> hardlinked as if ln() was called, but the move itself is atomic, even
> under OS crashes. Of course, this assumes you correctly wrote your
> temporary file, fsync()'ed the file and then performed the rename() as
> you should.
> With that information in mind, what is the scenario which can make
> mnesia fail if the dump_log_update_in_place is set to false? There may
> be a window, but the operating system cannot be the provider of this
> window at all, unless it is incorrectly implemented or configured.
> Hence my question.
>  The primary concern are write caches on disks which can have a
> write hole unless they have proper (working!) battery backup of their
> cache or are able to write their memory to a small solid state part of
> the chip. Or if you are using ZFS where copy-on-write semantics
> completely eliminates the write hole.
> On Wed, Nov 22, 2017 at 2:29 PM Michael Schmidt
> <mailto:Michael.K.Schmidt@REDACTED>> wrote:
> Hi Arun,
> If you happen to reboot when Mnesia is writing out its logfile, it
> can become corrupt (and lose data). We fight this issue on
> embedded devices a lot.
> Look “Configuration Parameters” section here:
> You will want to set dump_log_update_in_placeto false. This will
> make it write out the table to a new file, and rename it the very
> One other option to look at is dump_log_time_thresholdto change
> how often items are flushed to disk. For configuration data, you
> can likely set this to a few seconds.
> These options can be set via the sys.config file as well
> <mailto:erlang-questions-bounces@REDACTED>] *On Behalf Of *Arun
> *Sent:* Tuesday, November 21, 2017 10:16 PM
> *To:* erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
> *Subject:* [erlang-questions] Mnesia deleting log file
> Dear all,
> I've a program written in erlang which uses Mnesia application as
> the database application. I've a table created by name
> "configuration_table" which stores
> certain configurations that need to be persistent. Occasionally,
> whenever I restart my program the following error is thrown by
> mnesia and I end up losing
> all the persistent configurations.
> I've searched about this problem in the erlang documentation and
> all it tells is "Node not running". What could be the probable
> cause for this problem
> and how do I fix it?
> Mnesia('test@REDACTED <mailto:ommp82_plus@REDACTED>'): Data may
> be missing, Corrupt logfile deleted:
> 'test@REDACTED <mailto:ommp82_plus@REDACTED>'}
> This email has been scanned by the Symantec Email Security.cloud
> erlang-questions mailing list
> erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
> This email has been scanned by the Symantec Email Security.cloud service.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the erlang-questions