[erlang-questions] how: prefix matching on external term format
Mon Jun 9 06:44:01 CEST 2008
Hey, I have an initial release on google code:
I'm just now starting on integration with mnesia so there will be
additional changes as problems are exposed via that experience. In the
interim I'm very interested in feedback from mnesia / dets experts.
On Wed, 4 Jun 2008, Paul Mineiro wrote:
> Oh hey since there's interest, I got this working by forking
> erl_compare_ext and adding special tags for smallest and largest erlang
> term. My original fear of forking was unfounded since 1) the canonical
> erl_compare_ext has several bugs and 2) the canonical erl_compare_ext does
> not support ERL_NEW_FLOAT_EXT. That last one was pretty funny since my
> main fear of forking was missing out on new type tags. Well, the
> directory *does* say "legacy" ...
> I'm in the end game now, within a week I should have something on google
> code that sports the same interface as dets but supports ordered_set and
> ordered_bag (and 8EB [9.22e18 bytes] byte file size limit).
> -- p
> On Wed, 4 Jun 2008, Scott Lystig Fritchie wrote:
> > Paul Mineiro <> wrote:
> > pm> so i've envisioned the following choices:
> > Hrm, I wonder if it's a viable option to avoid the official Erlang
> > external format? If I recall correctly(*), any tuple stored in ETS is
> > not serialized using the external term format. Instead, the tuple is
> > copied into a separate heap. That heap is big enough to store only the
> > tuple. But the data inside that heap uses the same tagged pointer
> > scheme as any Erlang term inside a process's heap, so all the term
> > comparison macros & functions work, regardless of type (process heap or
> > ETS mini-heap).
> > (*) If I'm wrong, someone please yell loudly at me and the list. I
> > wouldn't want the falsehood to live unrefuted forever.
> > If my memory is correct, then storing an ETS mini-heap onto disk will
> > suffer from pointer offset problems: the VM's tagged pointers for use in
> > RAM won't point to the necessary byte offset within a file on disk. The
> > simplest (?) kludge would be to write an ETS mini-heap with all pointers
> > rewritten to be relative to the start of the mini-heap, and then re-set
> > those pointers to RAM-valid values after reading from disk but before
> > any comparisons. (Simplest = avoid reimplementing term comparison and
> > match spec functions.)
> > -Scott
> In an artificial world, only extremists live naturally.
> -- Paul Graham
In an artificial world, only extremists live naturally.
-- Paul Graham
More information about the erlang-questions