[erlang-patches] Allow limited pattern matching on magic binaries

[Evax Software]

Thanks for your answer and enlightening explanations.
I'm fine with your proposal, and willing to implement it.

Here's what I have in mind:

* erts/emulator/beam/global.h:
add cmp and eq entries to ErtsMagicBinary and the corresponding
ERTS_MAGIC_BIN_{CMP,EQ} macros

* erts/emulator/beam/erl_binary.h:
update erts_create_magic_binary to handle those two extra callbacks

* erts/emulator/beam/utils.c:
update cmp and eq to use the corresponding callback if we have two magic
binaries and the callbacks match

* Add cmd and eq callbacks to the enif_resource_type_t struct, implement
nif_resource_cmp and nif_resource_eq and modify enif_open_resource_type
to take the extra two parameters (erts/emulator/beam/erl_nif.c)

So the changes will only introduce a small memory overhead and two
functions' signature updates.

As erts_create_magic_binary is only used internally, it's easy enough to
update all users.
On the other hand enif_open_resource_type is widely used in existing
NIFs. I believe these NIFs should made to compile unmodified against the
new code. So, if you think the approach is right, how do you want me to
handle the API change for this function ?

Evax

Le lundi 11 avril 2011 à 12:22 +0200, Sverker Eriksson a écrit :
> Thank you for your explanation, but your patch was not rejected because 
> of lack of understanding. (Hopefully the opposite)
> 
> I understand the need to match NIF resources. But the simple solution of 
> just comparing the (magic) binary data is just not enough as it may 
> cause very strange behavior that violates the functional properties of 
> the language.
> 
> For example, the binary content of NIF resources are allowed to be 
> mutated. Your binary comparison will expose such a mutation and violate 
> the single assignment property. A variable referring a NIF resource may 
> suddenly compare different to what it did earlier.
> 
> Another (too) simple solution could be to compare the pointers *to* the 
> magic binaries instead of their content. That would on the other hand 
> introduce a comparison that is based on identity (rather than equality), 
> which we otherwise don't have in the language.
> 
> I think the right way to go, is to introduce a new callback for resource 
> types, that allows the NIF resource designer to implement how 
> comparisons should be done.
> 
> /Sverker, Erlang/OTP Ericsson
> 
> 
> 
> 
> 
> Evax Software wrote:
> > Ok, this patch has just been rejected.
> >
> > Maybe I was unclear as to the goal of the patch, I'll try to explain it
> > a bit better.
> >
> > NIF resources are implemented as magic binaries.
> > Magic binaries are special binaries holding private data but presenting
> > an apparent length of zero.
> >
> > I understand that this is a good thing as the private data is not
> > exposed directly to the Erlang side, but this has a special side effect
> > for NIF resource users: any resource will match any other resource (even
> > from a different resource type, as well of course as the empty binary.
> >
> > This is somehow dangerous as it prevents us from catching some errors
> > from the Erlang side, where for example assigning a resource to a
> > variable already bound to a resource will NOT cause an error.
> >
> > Moreover not being able to distinguish NIF resources from Erlang causes
> > some extra work. Imagine a NIF project implementing some kind of
> > socket-like mechanism (erlzmq2 or libev-erlang for example), and imagine
> > a process handling several of these sockets in a receive loop (as in
> > active mode), then one must wrap resources in a tuple with a ref to be
> > able to match on them.
> >
> > The patch implements special cases for magic binaries in the eq and cmp
> > operations, causing the private data to be compared, and magic binaries
> > with identical private data to match.
> >
> > So, and maybe the patch name was badly chosen, there is no real
> > sub-binary pattern matching happening on private data here, it's just a
> > simple and straightforward way to tell resources apart, still without
> > exposing the private data to the Erlang side.
> >
> > Evax
> >
> > Le jeudi 17 mars 2011 à 11:59 +0100, Henrik Nord a écrit :
> >   
> >> On 03/16/2011 09:41 PM, Evax Software wrote:
> >>     
> >>> Hi,
> >>>
> >>> The proposed patch allows limited pattern matching on magic binaries.
> >>> It tries to be the as non-intrusive as possible.
> >>> The goal of the patch is to allow matching on NIF resources, but it
> >>> shouldn't cause problems to other magic binary beam users.
> >>>
> >>> git fetch git@REDACTED:evax/otp.git pattern_match_on_magic_binaries
> >>>
> >>> Evax
> >>>
> >>>
> >>>
> >>> ________________________________________________________________
> >>> erlang-patches (at) erlang.org mailing list.
> >>> See http://www.erlang.org/faq.html
> >>> To unsubscribe; mailto:erlang-patches-unsubscribe@REDACTED
> >>>    
> >>>       
> >> Hello
> >>
> >> I have included this in 'pu' and will let it cook for a while.
> >>
> >> /Henrik Nord, Erlang/OTP
> >>
> >> ________________________________________________________________
> >> erlang-patches (at) erlang.org mailing list.
> >> See http://www.erlang.org/faq.html
> >> To unsubscribe; mailto:erlang-patches-unsubscribe@REDACTED
> >>
> >>     
> >
> >
> > _______________________________________________
> > erlang-patches mailing list
> > erlang-patches@REDACTED
> > http://erlang.org/mailman/listinfo/erlang-patches
> >
> >   
> 
>