mnesia no disk write after restart

[Michael McDaniel]

Now that I realize that it is my observer program gumming
up mnesia's gears, I have looked closer at the command I
was using with my other (working) module.  Aha! different.

Here is the observation command I need to use to not break
mnesia:

erl -sname fu -mnesia extra_db_nodes [module_nodename@REDACTED] \
    -s file script checkblock -s init stop


where "[module_nodename@REDACTED]" is the module's -sname
on this host (i.e. the module which created and 'owns' the db).

I am grateful for any insight on the particular point of why
'-config sys.config' breaks mnesia, though.  Even though it
specifies the mnesia directory, I am only reading the db.


~Michael



On Mon, Apr 17, 2006 at 02:11:17PM -0700, Michael McDaniel wrote:
> 
>  I changed the record name from ip to ipm with same behaviour.
> 
>  What I *did* find, though is that the following sequence causes
>  the problem:
> 
>  1) start module as described below (in original post)
> 
>  2) enter records as described below (in original post)
> 
>  3) from command line run:
>     erl -sname fu -config sys.config -s file script checkblock \
>         -s erlang halt
>  
>     where checkblock contains:
> 
>     mnesia:start().
>     timer:sleep(200).
>     mnesia:wait_for_tables([ip,schema], 7000) ,
>     X = block:dirty_sorted_list_table_localtime().
>     Z = global:registered_names().
>     io:format('~p~n~p entries~nregistered names: ~p~n', [X, length(X), Z]).
>     mnesia:stop().
> 
>     and sys.config contents are described below (in original post)
> 
>  4) stop/restart module and no records get written to disk.  Actually,
>     if I do step '3' above before the first records get written to disk
>     then they never will (get written).
> 
>  The contents of block:dirty_sorted_list_table_localtime() is:
> 
>   dirty_sorted_list_table_localtime() ->
>   io:format('*** LOCAL time ***~n',[]) ,
>   lists:keysort(5, ?MODULE:dlt_localtime()) .
> 
>  and dlt_localtime() is:
> 
>   dlt_localtime() ->
>   [ {element(1, X), element(2,X), element(3,X), element(4,X),
>      calendar:universal_time_to_local_time(element(5,X)), element(6,X)}
>      || X <- mnesia:dirty_match_object(ipm, {ipm, '_', '_', '_', '_', '_'}) 
>   ] .
>   
>  
> 
>  So, it appears to be the heisenberg principal and the more I observerved,
>  the less it worked.
> 
>  It appears that running step '3)' above causes mnesia to get confused
>  about whether it should write to disk or ram; how can this be?
> 
>  Also, in original post I mentioned sending messages to module of what
>  records to write.  That is one method of entering records.  During these
>  tests, the module itself is reading the tail of a file and looking for
>  patterns which triggers it to create a record and write to mnesia.  So,
>  I am only entering text into a file to trigger record writing.  I
>  mention this to eliminate conjecture that my sending process (there
>  isn't one) is interfering with the module writing mnesia.  If anything,
>  it appears that my observation (step '3)') causes a permanent problem
>  with mnesia and/or my module function.
> 
>  Now, the question is "why is this happening?".  I need to be able to
>  look at the db from an external program.
> 
>  Is there some other 'proper' way I should be checking the db contents?
> 
> thanks,
> 
> ~Michael
> 
> 
> 
> 
> On Mon, Apr 17, 2006 at 01:42:51AM -0700, Michael McDaniel wrote:
> > BRIEF: module using mnesia works fine the first time I start it
> >        (when the schema and table get created), and the db is
> >        persistent for the next restart.  Henceforth, no records
> >        write to disk, they only exist in memory, and further
> >        restarts lose all but the records written on the initial
> >        start (when the table was created).
> > 
> > 
> > LONG:
> > 
> > My goal is a *persistent* database when restarting my module.
> > 
> > I am running R10B-10 on Linux.
> > 
> > I have abbreviated the following code.  I think all the pertinent
> > bits are there.
> > 
> > I start my module with the following fun:
> > 
> > 
> > start() -> 
> > 
> >  erlang:set_cookie( node(), some_cookie )
> > 
> >  mnesia:create_schema([node()]) ,
> >  mnesia:start() ,
> > 
> > 
> >  case catch mnesia:table_info(?DB, attributes) of
> > 
> >     L when is_list(L) -> 
> >                         ok ;
> > 
> >     _                 ->
> >        mnesia:create_table(?DB ,
> >           [ {disc_copies, [node()]} ,
> >             {attributes, record_info(fields, ?DB)} ,
> >             {type, set}
> >           ])
> > 
> >  end,
> >     
> >  mnesia:wait_for_tables([?DB,schema], 21000)
> > .
> > 
> > 
> > the write function is:
> > 
> > write(Rec) ->
> >     Trans = fun() -> mnesia:write(Rec) end ,
> >     mnesia:transaction(Trans) ,
> >     os:cmd("/bin/sync")
> > .
> > 
> > 
> > I run a loop that waits for messages of records to write.
> > 
> > I start the module from a script containing:
> > 
> > 
> > #!/bin/sh
> > 
> > HOST=`/bin/hostname`
> > 
> > /bin/su mmcdanie -c "cd /home/erl ; /usr/local/bin/erl              \
> >                  -sname  ${HOST}_block -config /home/erl/sys.config \
> >                  -setcookie xus -env TERM linux -env HOME /home/erl \
> >                  -home /home/erl -heart -detached                   \
> >                  -s block start /var/log/auth"
> > #% end
> > 
> > 
> > and the contents of sys.config:
> > 
> > [{mnesia, [{dir, "/home/erl/Mnesia.block"}  ,
> > %%           {debug, trace}                 ,
> >            {schema_location, disc}]}].
> > 
> > 
> > I start the application the first time with no Mnesia.block
> > directory created.  Module starts fine, directory and db files
> > are created, and I can send messages which get written to
> > the database.  I see the .LOG file change and the record.DCL
> > file get created.
> > 
> > I stop the module and restart it and I can see the records
> > in the database.
> > 
> > I send more messages and the records get written though only
> > to memory.  A new record.DCL file never gets created.  The
> > .LOG file(s) never change.  'Never' is waiting for over 10
> > minutes (and {dump_log_time_threshold,180000}).
> > 
> > I run mnesia:system_info from another node:
> > 
> > (fu@REDACTED)2> mnesia:system_info().
> > ===> System info in version "4.2.5", debug level = none <===
> > disc. Directory "/home/erl/Mnesia.block" is used.
> > use fallback at restart = false
> > running db nodes   = [www_block@REDACTED,fu@REDACTED]
> > stopped db nodes   = []
> > master node tables = []
> > remote             = [ip]
> > ram_copies         = [schema]
> > disc_copies        = []
> > disc_only_copies   = []
> > [{fu@REDACTED,ram_copies},{www_block@REDACTED,disc_copies}] = [schema]
> > [{www_block@REDACTED,disc_copies}] = [ip]
> > 3 transactions committed, 0 aborted, 0 restarted, 0 logged to disc
> > 0 held locks, 0 in queue; 0 local transactions, 0 remote
> > 0 transactions waits for other nodes: []
> > yes
> > 
> > 
> > The '0 logged to disc' is after having written a new record that
> > does show in the database.
> > 
> > 
> > QUESTIONS:
> > 
> > 1) Does anyone see something obvious I am overlooking here?
> > 
> > 2) Any suggestions on what to do differently?
> > 
> > 3) What is the next bit of information anyone needs to help?
> > 
> > 
> > thanks,
> > 
> > ~Michael
> > P.S. I am doing essentially the same thing in another module
> > that I start from yaws and it works fine (i.e. persistent db
> > between restarts).  I have not sleuthed out the cause of the
> > different behaviours!  In the other module (that works) I do
> > not even have the case statement protecting the create_table/2
> > fun.
> 
> -- 
> Michael McDaniel
> Portland, Oregon, USA
> http://autosys.us
> +1 503 283 5284

-- 
Michael McDaniel
Portland, Oregon, USA
http://autosys.us
+1 503 283 5284