[erlang-questions] Noddy question on hashing

[Bob Ippolito]

Larger binaries (refc binaries) in the Erlang VM are sent by reference and
not copied when sent from one process to another (the threshold is 64
bytes, and they use a reference counting strategy). If you have a process
that can de-dupe these binaries then you don't use hashes at all, you just
store the binaries and know that they're shared. I'm not sure if an ets
table (instead of a process) will suffice to do the de-duping, but you
could do some experiments to figure that out.

If the attributes also have names (and there aren't too many possible
names), then atoms are generally the right type for that.

See also: http://www.erlang.org/doc/efficiency_guide/binaryhandling.html


On Thu, Jul 25, 2013 at 2:52 AM, Rick Payne <rickp@REDACTED> wrote:

> Hi,
>
> I've been using erlang for some simple projects, and really enjoying it.
> However, when thinking about some of the things I'd like to do - I'm
> struggling a bit with hashing and how to handle that in a concurrent way in
> erlang. This is probably 101 stuff, sorry...
>
> Imagine we have a set of data streams giving us objects and attributes.
> Attributes can be a few kB but typically are less than 1kB. The objects are
> unique but some set of them share the attributes. Typically in C, I'm used
> to hashing the attributes and pointing at them from the objects. We hash to
> save memory and refcount so we know when we can delete from the hash. The
> good news is that the attributes are never updated in place, apart from the
> refcount.
>
> However, I'm struggling to get my head around how I'd do this in erlang
> efficiently. Any suggestions?
>
> Cheers,
> Rick
>
> ______________________________**_________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/**listinfo/erlang-questions<http://erlang.org/mailman/listinfo/erlang-questions>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130725/0322b929/attachment.htm>