[erlang-questions] RecordA serialization ... time ... deserialization to RecordB?

Max Bourinov <>
Sat Nov 12 10:22:39 CET 2011


Hello Erlangers!

So I made a dict as field of my state record and I do serialize it via
term_to_binary and it seems to work like a charm.

One thing which makes me thinking about implementing my own serializer or
dict like library - why dict data structure has many empty lists in it? Is
it for the case of fast growing?

Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:4:4] [rq:4]
[async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.8.5  (abort with ^G)
1> D = dict:new().
{dict,0,16,16,8,80,48,
      {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
      {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}}

Actually those empty lists doesn't hurt me much, but why? If nobody knows I
will check sources :-)

Best regards,
Max




On Sat, Nov 12, 2011 at 1:09 AM, Bob Gustafson <> wrote:

> How you implement your 'database' depends on whether you allow the
> record structure to change over time without rewriting the whole
> database.
>
> A changeable structure database could be implemented by having a
> serializable 'lump' for each record. The structure can be changed by
> adding a new tag/value pair to the existing lump.
>
> Each lump has a unique id number. External indexes can be constructed
> for any/all tags with the id number as the index value.
>
> An update of a record/lump would be done by consulting the index(es) for
> each tag value of interest to determine the lump id value(s). The
> resulting lump(s) would then be accessed, deserialized, (data values
> changed, new tags/values added or deleted), lump(s) re-serialized, and
> index(es) updated.
>
> It can be a very simple database or a very complex database. The
> advantage is that lumps written in the distant past can be accessed and
> their data values utilized in the same way as newer lumps with more (or
> less) tags/values.
>
> I'm sure this scheme has been invented many times in the past.
>
> On Fri, 2011-11-11 at 21:46 +0100, Joe Armstrong wrote:
> > Umm - followup question.
> >
> >
> > Is there a way to consult files containing records. If I do
> > file:consult on a file containing records what happens?
> >
> >
> > I tried this without success the other day
> >
> >
> > /Joe
> >
> > On Fri, Nov 11, 2011 at 11:15 AM, Ulf Wiger
> > <> wrote:
> >         This is not exactly what was asked for, but…
> >
> >
> >         In JOBS, I have created a jobs_info module that includes
> >         jobs.hrl and uses exprecs's -export_records() feature.
> >
> >
> >         https://github.com/esl/jobs/blob/master/src/jobs_info.erl
> >
> >
> >         I use this module to present metadata about the various
> >         objects in JOBS. The only thing needed in order to 'serialize'
> >         a new type of record is to define it in jobs.hrl and add it to
> >         the -export_records() directive.
> >
> >
> >         BR,
> >         Ulf W
> >
> >
> >         -module(jobs_info).
> >
> >
> >
> >         -export([pp/1]).
> >
> >
> >
> >         -include("jobs.hrl").
> >
> >         -include_lib("parse_trans/include/exprecs.hrl").
> >
> >
> >
> >         -export_records([rr, cr, grp, rate, queue, sampler]).
> >
> >
> >
> >
> >
> >         pp(L) when is_list(L) ->
> >
> >             [pp(X) || X <- L];
> >
> >         pp(X) ->
> >
> >             case '#is_record-'(X) of
> >
> >               true ->
> >
> >                   RecName = element(1,X),
> >
> >                   {RecName, lists:zip(
> >
> >                               '#info-'(RecName,fields),
> >
> >                               pp(tl(tuple_to_list(X))))};
> >
> >               false ->
> >
> >                   if is_tuple(X) ->
> >
> >                           list_to_tuple(pp(tuple_to_list(X)));
> >
> >                      true ->
> >
> >                           X
> >
> >                   end
> >
> >             end.
> >
> >
> >
> >
> >         On 11 Nov 2011, at 10:57, Joel Reymont wrote:
> >
> >         > You need to serialize the record fields with their tags.
> >         > Then you can use something like exprecs to set individual
> >         > fields in the new record definition.
> >         >
> >         >
> >         > This assumes you are just adding fields and names of old
> >         > fields stay the same.
> >         >
> >         > ---
> >         > Sent from my iPhone
> >         >
> >         > On Nov 11, 2011, at 10:44 AM, Maxim Treskin
> >         > <> wrote:
> >         >
> >         >
> >         >
> >         > > Max, what are you need indeed? Your words sounds very
> >         > > strange.
> >         > >
> >         > > On 11 November 2011 15:31, Max Bourinov
> >         > > <> wrote:
> >         > >         Hello Erlangers,
> >         > >
> >         > >
> >         > >         What is a best way to serialize record, modify
> >         > >         record's code and then deserialize it back?
> >         > >
> >         > >
> >         > >         I seen https://github.com/esl/parse_trans. Is this
> >         > >         what I need or there are another options?
> >         > >
> >         > >
> >         > >         Maybe I better use another data structure?
> >         > >
> >         > >
> >         > >         p.s. Of course performance is always important.
> >         > >
> >         > >
> >         > >         Best regards,
> >         > >         Max
> >         > >
> >         > >         _______________________________________________
> >         > >         erlang-questions mailing list
> >         > >         
> >         > >         http://erlang.org/mailman/listinfo/erlang-questions
> >         > >
> >         > >
> >         > >
> >         > >
> >         > >
> >         > > --
> >         > > Maxim Treskin
> >         > >
> >         > > _______________________________________________
> >         > > erlang-questions mailing list
> >         > > 
> >         > > http://erlang.org/mailman/listinfo/erlang-questions
> >         > >
> >         > _______________________________________________
> >         > erlang-questions mailing list
> >         > 
> >         > http://erlang.org/mailman/listinfo/erlang-questions
> >
> >
> >         Ulf Wiger, CTO, Erlang Solutions, Ltd.
> >         http://erlang-solutions.com
> >
> >
> >
> >
> >
> >
> >
> >         _______________________________________________
> >         erlang-questions mailing list
> >         
> >         http://erlang.org/mailman/listinfo/erlang-questions
> >
> >
> >
> > _______________________________________________
> > erlang-questions mailing list
> > 
> > http://erlang.org/mailman/listinfo/erlang-questions
>
>
> _______________________________________________
> erlang-questions mailing list
> 
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20111112/d1f2fd09/attachment.html>


More information about the erlang-questions mailing list