[erlang-questions] mnesia backup oddity

[Hal Snyder]

FWIW, I did some informal tests of mnesia backups a few months back.  
This was on a server running i386 Solaris 10. Database had 25 or so  
tables including two tables with 500,000 records each. Baseline  
memory used was 381MB for the node with the tables and latency for  
dirty reads was less than 1 msec.

I monitored memory size with top and ran queries against mnesia while  
the backup was running. Entire backup completed in 12 seconds. During  
that time, memory footprint did not budge. Dirty read latency  
initially spiked to 10 and 30 msec but after 2 seconds went right  
back to baseline for the rest of the backup.

Not bad.

On Feb 27, 2008, at 3:55 AM, Lukas Larsson wrote:

> aha, the data was being modified while I was doing the backup. That
> explains it. Thanks!
>
> Lukas
> Erlang Trainer & Consulting Ltd
>
> On Wed, Feb 27, 2008 at 10:37 AM, Hakan Mattsson  
> <hakan@REDACTED> wrote:
>>
>>  The Mnesia backup is performed in several phases:
>>
>>   - First a checkpoint is activated in order to keep
>>     track of the database state as it was at that
>>     timepoint.
>>
>>   - Then Mnesia performs a first iteratation over the
>>     tables involved in the checkpoint and puts their
>>     contents into the backup.
>>
>>   - After that Mnesia performs a second iteration over
>>     the tables involved in the checkpoint in order to
>>     compensate for updates performed during the backup.
>>     During this phase more backup items (records and
>>     delete marks) are added to the backup.
>>
>>   - At last the checkpoint is deactivated.
>>
>>  When iterating over a backup file you should interpret
>>  the backup items in the file as operations (write and
>>  delete), not as plain data. This means that when you
>>  find the first occurrence of a record you should expect
>>  a delete mark and/or yet another version of that the
>>  record later in the backup file.
>>
>>  /Håkan