[erlang-questions] Misbehaving ets table in a gen_fsm

[Dan Gudmundsson]

By using ets tables you avoid garbage collection, but instead you copy the
data to/from heap
whenever you read or write it.

So for large datasets you may want to use ets tables to avoid long garbage
collection times.


On Fri, Oct 11, 2013 at 2:52 PM, Dmitry Kolesnikov
<dmkolesnikov@REDACTED>wrote:

> Hello,
>
> I think you problems is that ets "owner" process dies and table is
> released… This is most probably if tables is created within gen_fsm process.
>
> 1. ETS one of the performant way to share data among multiple processes…
>
> 2. Most probably if the table was destroyed.
>
> 3. I think terminate is appropriate approach to "checkpoint" state at
> "normal" termination. However, if would not guarantee that internal process
> state is serialised. I would use "checkpoint" by timer or write-through
> approach. ETS would not preserve data if node crashes therefore
> "persistency" might require use of amnesia or other storage. From my
> perspective, ETS is not good approach to "preserve" data in-between of
> process crash.
>
> - Dmitry
>
> On Oct 11, 2013, at 3:36 PM, Ivan Uemlianin <ivan@REDACTED> wrote:
>
> > Dear All
> >
> > I am maintaining some code the original author of which has long gone. A
> gen_fsm has a state variable which is a key/value structure with integer
> keys.  I might use a dict for this, or an array; the author uses an ets
> table.
> >
> > From some error reports, it looks like this ets table occasionally
> "mislays" its keys.  Perhaps empties or renews itself silently.
> >
> > Finally, in the event of a crash/restart, the data in this ets variable
> should be preserved and passed on the new gen_fsm process.  However, the
> gen_fsm module has nothing special to handle this in its terminate/3, and
> init just creates a new table.
> >
> > This is all a bit mysterious so I'm going to write a gen_fsm that has
> nothing but the above, try and replicate the "mislaying" and generally get
> it behaving properly.
> >
> > I have three questions:
> >
> > 1. I can't think of any good reason to use an ets table over a dict or
> an array, other than sharing the data structure with other processes, which
> isn't done.  There are several inconclusive debates on the web about which
> is fastest.  Is there a reason I'm missing for why an ets table might be
> most appropriate?
> >
> > 2. Under what circumstances if any can an ets table mislay keys?  At
> some point the table stops recognising keys it should know about.  There
> are no other errors in the system.
> >
> > 3. Persisting data across crash & restart I might store the data in the
> fsm's terminate/3 (if it was a non-normal termination), and check for the
> stored data in the fsm's init/1.  Is that a common approach?  has any
> approach co-allesced into a "pattern" or are types of storage, kinds of
> data, etc too variable?  Might keeping the data in an ets table the whole
> time remove the need for this step?
> >
> > With thanks and best wishes
> >
> > Ivan
> >
> >
> > --
> > ============================================================
> > Ivan A. Uemlianin PhD
> > Llaisdy
> > Speech Technology Research and Development
> >
> >                    ivan@REDACTED
> >                     www.llaisdy.com
> >                         llaisdy.wordpress.com
> >              github.com/llaisdy
> >                     www.linkedin.com/in/ivanuemlianin
> >
> >                        festina lente
> > ============================================================
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://erlang.org/mailman/listinfo/erlang-questions
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20131011/0a8e0500/attachment.htm>