[erlang-questions] how: prefix matching on external term format

[Scott Lystig Fritchie]

Paul Mineiro <paul-trapexit@REDACTED> wrote:

pm> so i've envisioned the following choices:

Hrm, I wonder if it's a viable option to avoid the official Erlang
external format?  If I recall correctly(*), any tuple stored in ETS is
not serialized using the external term format.  Instead, the tuple is
copied into a separate heap.  That heap is big enough to store only the
tuple.  But the data inside that heap uses the same tagged pointer
scheme as any Erlang term inside a process's heap, so all the term
comparison macros & functions work, regardless of type (process heap or
ETS mini-heap).

(*) If I'm wrong, someone please yell loudly at me and the list.  I
wouldn't want the falsehood to live unrefuted forever.

If my memory is correct, then storing an ETS mini-heap onto disk will
suffer from pointer offset problems: the VM's tagged pointers for use in
RAM won't point to the necessary byte offset within a file on disk.  The
simplest (?) kludge would be to write an ETS mini-heap with all pointers
rewritten to be relative to the start of the mini-heap, and then re-set
those pointers to RAM-valid values after reading from disk but before
any comparisons.  (Simplest = avoid reimplementing term comparison and
match spec functions.)

-Scott