[erlang-questions] mmap file to binary

[Tony Rogvall]

Last time I checked the orig_bytes was not separated pointer.
It's currently (r14) a dynamic field.

typedef struct binary {
    ERTS_INTERNAL_BINARY_FIELDS
    long orig_size;
    char orig_bytes[1]; /* to be continued */
} Binary;

So this must be changed to be able to mmap data.
You are on the right track ;-)

Check out the ErlNifResource stuff, this will handle the garage collection for data like this.
(yes, it is aso using the magic stuff)
My suggestions for drivers is one ErlDrvObject where you can store the actual object data
inline and pass it back and forth from driver to runtime. It will appear as a binary.
One other suggestion is to make a simple wrapper for C(++) objects ErlDrvHandle to simplify
the allocation when you only want to pass a pointer back and forth. To make it safe the 
resource scheme used by erl_nif.c should be adopted. Just think what will happen if the
driver is unloaded and the destructors are leaving with it ;-)


Still the orig_bytes must be change. There are not to many places in the VM that uses the fact
that orig_bytes is not a pointer, so the change should not be hard (if it's not done already ;-)

/Tony


On 9 apr 2010, at 12.32, Max Lapshin wrote:

> On Fri, Apr 9, 2010 at 2:23 PM, Angel Alvarez <clist@REDACTED> wrote:
>> 
>> Maybe a new alloc for mmaped files will allow manage diferences among OSes in a easy way...
>> 
>> http://www.erlang.org/doc/man/erts_alloc.html
>> 
>> It seems that the mmaped case is already covered using the  mseg_alloc:
>> 
>> mseg_alloc
>> 
> 
> Maybe it will be good to dig inside this. However, there seems to be easy way:
> 
> erl_binary.h has erts_create_magic_binary. It should be changed to:
> 
> ERTS_GLB_INLINE Binary *
> erts_create_mmap_binary(size_t size, int fd)
> {
>    Uint bsize = ERTS_MAGIC_BIN_SIZE(0);
>    Binary* bptr = erts_alloc_fnf(ERTS_ALC_T_BINARY, 0);
>    if (!bptr)
> 	erts_alloc_n_enomem(ERTS_ALC_T2N(ERTS_ALC_T_BINARY), bsize);
>    ERTS_CHK_BIN_ALIGNMENT(bptr);
>    bptr->flags = BIN_FLAG_MAGIC;
>    bptr->orig_size = ERTS_MAGIC_BIN_ORIG_SIZE(size);
>    bptr->orig_bytes = mmap(0, size, PROT_READ, MAP_FILE, fd, 0);
>    erts_refc_init(&bptr->refc, 0);
>    ERTS_MAGIC_BIN_DESTRUCTOR(bptr) = binary_mmap_destructor;
>    return bptr;
> }
> 
> static void binary_mmap_destructor(Binary *bptr)
> {
>  munmap(bptr->orig_bytes, bptr->orig_size);
> }
> 
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>