[erlang-questions] how: prefix matching on external term format

[Paul Mineiro]

Hey, I have an initial release on google code:

http://code.google.com/p/tcerl/

I'm just now starting on integration with mnesia so there will be
additional changes as problems are exposed via that experience.  In the
interim I'm very interested in feedback from mnesia / dets experts.

Thanks,

-- p

On Wed, 4 Jun 2008, Paul Mineiro wrote:

> Oh hey since there's interest, I got this working by forking
> erl_compare_ext and adding special tags for smallest and largest erlang
> term.  My original fear of forking was unfounded since 1) the canonical
> erl_compare_ext has several bugs and 2) the canonical erl_compare_ext does
> not support ERL_NEW_FLOAT_EXT.  That last one was pretty funny since my
> main fear of forking was missing out on new type tags.  Well, the
> directory *does* say "legacy" ...
>
> I'm in the end game now, within a week I should have something on google
> code that sports the same interface as dets but supports ordered_set and
> ordered_bag (and 8EB [9.22e18 bytes] byte file size limit).
>
> -- p
>
> On Wed, 4 Jun 2008, Scott Lystig Fritchie wrote:
>
> > Paul Mineiro <paul-trapexit@REDACTED> wrote:
> >
> > pm> so i've envisioned the following choices:
> >
> > Hrm, I wonder if it's a viable option to avoid the official Erlang
> > external format?  If I recall correctly(*), any tuple stored in ETS is
> > not serialized using the external term format.  Instead, the tuple is
> > copied into a separate heap.  That heap is big enough to store only the
> > tuple.  But the data inside that heap uses the same tagged pointer
> > scheme as any Erlang term inside a process's heap, so all the term
> > comparison macros & functions work, regardless of type (process heap or
> > ETS mini-heap).
> >
> > (*) If I'm wrong, someone please yell loudly at me and the list.  I
> > wouldn't want the falsehood to live unrefuted forever.
> >
> > If my memory is correct, then storing an ETS mini-heap onto disk will
> > suffer from pointer offset problems: the VM's tagged pointers for use in
> > RAM won't point to the necessary byte offset within a file on disk.  The
> > simplest (?) kludge would be to write an ETS mini-heap with all pointers
> > rewritten to be relative to the start of the mini-heap, and then re-set
> > those pointers to RAM-valid values after reading from disk but before
> > any comparisons.  (Simplest = avoid reimplementing term comparison and
> > match spec functions.)
> >
> > -Scott
> >
>
> In an artificial world, only extremists live naturally.
>
>         -- Paul Graham
>

In an artificial world, only extremists live naturally.

        -- Paul Graham