<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">On 04/25/2017 06:17 AM, Jesper Louis
Andersen wrote:<br>
</div>
<blockquote
cite="mid:CAGrdgiWHZkoryE2ySJ1fhv7xm-_7LT4U2fqyPmL_BkBhyF=zpw@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_quote">
<div dir="ltr">On Tue, Apr 25, 2017 at 1:17 PM Attila Rajmund
Nohl <<a moz-do-not-send="true"
href="mailto:attila.r.nohl@gmail.com">attila.r.nohl@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">It would
affect everybody who saves the opaque data using an earlier<br>
OTP version, then reads it in newer OTP version (e.g. after
upgrade).<br>
Or those who run two nodes on different OTP versions (e.g.
during an<br>
upgrade).<br>
</blockquote>
<div><br>
</div>
<div>I like to think of dict and sets to be "opaque" data
structures. If you are relying on their internal
representation, you will run into trouble. So changing the
representation going forward should definitely be possible.<br>
<br>
</div>
<div>This leaves backwards compatibility. If the
maps-optimized module can read and dynamically change old
sets implementations into the new format on the fly, it may
be possible to gradually replace the old representations
with the new.<br>
<br>
</div>
<div>Some things to look out for in that process is
data-at-rest, stored years ago in a database. At some point
you would have to reprocess such data, or supply a
conversion module which can handle these old formats. <br>
<br>
</div>
<div>I also think that any system must provide some measure of
pushing things forward. That is, each major release of
Erlang could contain a limited set of things you now have to
do differently, with a clear upgrade path from earlier
versions. As long as the set is limited, we can probably
handle the rewrites needed. If you value backwards
compatibility for forever, you run the risk of getting
stale, never upgrading anything.<br>
<br>
</div>
<div>As for the increased memory copy pressure: I think this
should be fixed in the context of "maps" and not be part of
the argument as to why one would keep the old sets
representation.<br>
</div>
</div>
</div>
</blockquote>
The memory allocation related to maps is likely not something to
solve, and should be related to its data structure (<span class="st">HAMT
(Hash-Array Mapped Trie)). Keeping things stable is important and
there should never be changes that sacrifice stability for a
passing whim. It is easy to add new modules for maps used for
sets and it is important to avoid adversely affecting legacy
source code. That doesn't block future improvements, it only
makes sure they are focused on being improvements.<br>
<br>
</span>Best Regards,<br>
Michael<br>
</body>
</html>