[erlang-questions] Thread-safety of driver_alloc/realloc/free_binary

[Rickard Green]

They are all thread safe in the emulator with smp support, but they will 
probably never be thread safe in the emulator without smp support.

The thread unsafe part is the reference counting of binaries. Making 
them thread safe would cause overhead for all reference counted 
binaries, not just driver binaries. We don't want to slow down all 
binary management as it is now, but we might make them thread safe in 
the future if threaded drivers should become very common (but don't 
count on it).

BR,
Rickard Green, Erlang/OTP

Chris Newcombe wrote:
> (To the maintainers of erts/emulator/beam/io.c ... )
> 
> Hi,
> 
> I'm working on an update to Scott Lystig Fritchie's Erlang Driver
> Toolkit, including a significantly enhanced driver for Berkeley DB.  I
> hope to be able to release this soon as open-source.
> 
> The update includes features such as multiple private resizable
> threadpools per port instance (supported in both 'linked-in' and
> 'spawned-pipe' modes).
> 
> One problem is that, when getting data from Berkeley DB, the driver
> needs to allocate driver binaries.  Ideally, it would like to to do so
> from within the driver's private threadpools.  But of course,
> driver_alloc/realloc/free_binary are not documented as thread-safe.
> 
> So currently the Berkeley DB driver has to allocate each piece of data
> twice (once with malloc and later with driver_alloc_binary), and copy
> it once.  And the copying is done in the Erlang scheduler thread,
> which blocks the VM unecessarily.
> 
> This appears to be changing.  I found the following comments in
> otp_src_R11B-1/erts/emulator/beam/io.c
> 
>  * - driver_alloc_binary() is thread safe (efile driver depend on it).
>  * - driver_realloc_binary(), and driver_free_binary() are *not* thread safe.
> 
> Unfortunately, to avoid the copying the Berkeley DB driver needs to
> call all 3 of these functions from private threads.
> 
> So, please could the realloc & free functions also be made
> thread-safe, and documented as such?   This would be a major
> performance benefit for many multi-threaded drivers (not just the
> efile driver mentioned in the comment),
> 
> Many thanks for considering this,
> 
> Chris
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>