[erlang-questions] Ref counted binary in ETS

[Frank Muller]

Michael,

Thank you for sharing "blookup". I'll look at it today.

Any other ideas to minimise the overhead when sharing large Bins? I'm ok
with changing the overall program design if necessary.

Frank.

Le Friday, October 28, 2016, Michael Truog <mjtruog@REDACTED> a écrit :

> You could use something like https://github.com/okeuday/blookup/ to make
> larger binaries, but ideally you don't have to (since it makes the lookups
> slow).  This would be the concept of grouping keys and values together,
> putting them into various buckets, to decrease the ETS usage but increase
> the binary size.
>
> On 10/27/2016 03:25 PM, Tony Rogvall wrote:
>
> Simple, you may your 512 byte memory chunks bigger :-)
>
> /Tony
>
>
> On 27 okt 2016, at 23:24, Frank Muller <frank.muller.erl@REDACTED> <javascript:_e(%7B%7D,'cvml','frank.muller.erl@REDACTED');> wrote:
>
> Tony,
>
> Wow, 48-bytes is a huge overhead as N ~= 1000, and Every reader reads a binary 10x times or so per second.
>
> Can anyone please suggest a better strategy to efficiently share them with a lesser overhead?
>
> Frank.
>
> Le Thursday, October 27, 2016, Tony Rogvall <tony@REDACTED> <javascript:_e(%7B%7D,'cvml','tony@REDACTED');> a écrit :
> 1. A reference.
>
> It will however build a 6-word structure called ProcBin on the heap for every binary reference that you read.
> On a 64 bit machine that will lead to a 48 byte overhead. Nearly 10%.
>
> /Tony
>
>
> On 27 okt 2016, at 20:47, Frank Muller <frank.muller.erl@REDACTED> <javascript:_e(%7B%7D,'cvml','frank.muller.erl@REDACTED');> wrote:
>
> Hi guys
>
> Suppose an ETS table with only 1 writer and N readers
> This table contains only large binaries (>64B) with the same size 512B exactly.
>
> The single writer update these Bins once a while.
> The N readers regularly read one of these Bins, but never change/update them.
>
> Questions: what a reader gets when reading a Bin from this ETS table:
>
> 1. a reference to that Bin
> 2. or a full copy of that Bin
>
> Hope someone can clarify this (with an example).
>
> Thank you
> Frank
>
> _______________________________________________
> erlang-questions mailing listerlang-questions@REDACTED <javascript:_e(%7B%7D,'cvml','erlang-questions@REDACTED');>http://erlang.org/mailman/listinfo/erlang-questions
>
>
>
> _______________________________________________
> erlang-questions mailing listerlang-questions@REDACTED <javascript:_e(%7B%7D,'cvml','erlang-questions@REDACTED');>http://erlang.org/mailman/listinfo/erlang-questions
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20161027/85cfe72c/attachment.htm>