[erlang-questions] map elements in defined order

[zxq9]

On 2017年10月27日 金曜日 15:42:30 Ulf Wiger wrote:
> We have a scenario where we need to serialize objects partly for network
> comms, partly for storage and partly for signing and cryptographic hasing.
> We also prefer the serialization to work easily across programming
> languages.
> 
> The trouble is of course the signing/crypto-hashing. For this to be stable,
> the order must be fixed. Obviously, adding a function that makes Erlang
> maps appear ordered will not suffice, since other language environments
> would either not expect maps to be ordered at all, or are likely to have a
> different take on how keys are ordered*.

Interesting. I have been dealing with this exact case myself for quite
some time (before the advent of maps, actually). The tradeoff we had to
make was exactly as you describe: Universal ordered representation of
pretty much any K/V sort of data type (especially across languages) only
works as an ordered list, so the external, canonical representation of
the data must be defined as an ordered list (sorted some specific way).
Occasionally this also means internal lists must be themselves sorted.

That is one critical part of the definition of any serialization
procedure for any data that needs to be verifiable via signature.

Either that or build an ASN.1 DER representation for everything; which
isn't actually so bad, but would be way better if there were tutorials
on just the DDL part of ASN.1 for the cool kids to brush up on...

Having maps inside an Erlang program (and dicts in Python and blats in
Frozz and so on) is quite nice, but it can never be relied on for things
like canonical serialization. Imagine if every SQL query return had to
suddenly be ordered!

Those internal representations can only ever be immediate conveniences,
never canonical data representations. I think this is easy to forget
because most of the time we never even have to worry about what a
"canonical representation" even might be for 99% of the data most of us
ever deal with.

-Craig