sortable serialization format
Sun Oct 25 17:28:22 CET 2009
A while ago I started hacking on a serialization format that
would have the same sorting properties as Erlang terms.
I didn't quite get it to work (negative floats was the most
difficult part), but when I returned to it today, I realized
that it was only a very small problem. Once fixed, all my
QuickCheck suites passed.
In many cases this format is slightly more compact than
the External Term Format. The most notable exception is
negative bignums, but large binaries also take up a little
bit more space (asymptotically approaching 12.5% overhead
for binaries larger than 36 bytes).
(Strings are much bigger, since I don't try to detect that
they are special cases of lists. This should perhaps be fixed.)
You are welcome to give it a spin and report to me if you
find good uses for it. The original idea was to make it
easier to make use of external storage solutions with
ordered set semantics, without foregoing the genericity and
convenience of generic term serialization.
PS Float comparisons are handled in the QuickCheck suite by
coercing to IEEE 764 binary64 format. This is supposedly what
Erlang uses internally, but I encountered lots of weird behaviour
before doing this coercion.
CTO, Erlang Training & Consulting Ltd
More information about the erlang-questions