[erlang-questions] Add a new data structure to erlang VM.

[Constantin Kulikov]

> ETS tables are mutable.  Which is why they are not *Erlang* data
structures.
Just to clarify, are objects copied when we put/get them from an ets table?

Looks like I mixed and confused some of my statements.
Looks like I want my data structure to reside in the Erlang process'
memory, because I don't want to always copy data when I put/get something
in/from that data structure.
However it doesn't necessarily have to be as transparent to the programmer
as the Erlang's lists.

With NIFs I always must copy data between environments. And it looks like
enif_make_copy is doing a full copy of an object:
https://github.com/erlang/otp/blob/3b7a6ffddc819bf305353a593904cea9e932e7dc/erts/emulator/beam/erl_nif.c#L795
https://github.com/erlang/otp/blob/3b7a6ffddc819bf305353a593904cea9e932e7dc/erts/emulator/beam/copy.c#L73
https://github.com/erlang/otp/blob/3b7a6ffddc819bf305353a593904cea9e932e7dc/erts/emulator/beam/global.h#L1127
https://github.com/erlang/otp/blob/3b7a6ffddc819bf305353a593904cea9e932e7dc/erts/emulator/beam/copy.c#L602

And I can only add a term to the environment, but can not free a term other
than freeing the environment entirely with all it's terms.

I would do enif_make_env for each single object that I want to store, but
it looks like a bad idea:
http://erlang.2086793.n4.nabble.com/erl-nif-questions-regarding-resource-handling-td4673810.html#a4673832
> The current implementation of environments from enif_make_env() has a quite
large constant memory overhead.
> A lot of environments with small/few terms in them will not be memory
efficient.

The next step is to make a "custom GC" based on managing these environments
like here: https://github.com/fauxsoup/neural
(hmm, maybe Erlang's GC works like this)


Well, the more I dive into that, the more I see that adding a data
structure to the Erlang VM level is not so easy and doesn't worth it.)
Thanks for your help.


On 12 July 2016 at 01:44, Richard A. O'Keefe <ok@REDACTED> wrote:

>
>
> On 11/07/16 9:07 PM, Constantin Kulikov wrote:
> >> ETS tables are not data structures in an Erlang process.
>
> >Anyway, they are mutable(right?)
> Yes, ETS tables are mutable.  Which is why they are not *Erlang* data
> structures.
>
>> and I didn't ask that a new data structure must necessarily reside in an
>> Erlang process.
>>
> If we are talking about something that Erlang code has a *name* for
> (represented as say a reference) but cannot work with directly, then
> adding a data structure to Erlang is pretty much what NIFs are for.
>
> Objects that are large, long-lived, few, and have explicitly managed
> lifetimes are a good fit for that.  Data base connections, special
> access to devices, that sort of thing.
>
> Objects that are small, multitudinous, and evanescent are not.
>
> The thing is that we are all floundering in the dark because you have
> yet to tell us what it is you actually want to add and what problem
> you expect to solve by adding it, and for that matter why it needs to
> be in the Erlang VM at all instead of being a C node.
>
> Background: I worked on Xerox Quintus Prolog, which put Quintus Prolog
> and Interlisp-D in the same box.  The design we ended up with was the
> least painful and most efficient we could devise, and basically Prolog was
> given a chunk of the address space and Interlisp-D was given the rest
> and cross-language procedure calls involved copying.  We worked hard
> to make sure each language's GC had to know as little as possible about
> the other.  Around the same time, the Poplog system -- which is open
> source these days -- was in flower.  That put Pop-11, Prolog, Common
> Lisp, SML, and I think Scheme in the one address space.  It was and is
> an impressive piece of work but meant that Prolog and SML performance
> suffered.  In contrast, the Prolog component of XQP did not suffer, as
> we didn't have to make compromises.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20160713/e77c1055/attachment.htm>