[erlang-questions] Thread-safety of driver_alloc/realloc/free_binary

[Chris Newcombe]

(To the maintainers of erts/emulator/beam/io.c ... )

Hi,

I'm working on an update to Scott Lystig Fritchie's Erlang Driver
Toolkit, including a significantly enhanced driver for Berkeley DB.  I
hope to be able to release this soon as open-source.

The update includes features such as multiple private resizable
threadpools per port instance (supported in both 'linked-in' and
'spawned-pipe' modes).

One problem is that, when getting data from Berkeley DB, the driver
needs to allocate driver binaries.  Ideally, it would like to to do so
from within the driver's private threadpools.  But of course,
driver_alloc/realloc/free_binary are not documented as thread-safe.

So currently the Berkeley DB driver has to allocate each piece of data
twice (once with malloc and later with driver_alloc_binary), and copy
it once.  And the copying is done in the Erlang scheduler thread,
which blocks the VM unecessarily.

This appears to be changing.  I found the following comments in
otp_src_R11B-1/erts/emulator/beam/io.c

 * - driver_alloc_binary() is thread safe (efile driver depend on it).
 * - driver_realloc_binary(), and driver_free_binary() are *not* thread safe.

Unfortunately, to avoid the copying the Berkeley DB driver needs to
call all 3 of these functions from private threads.

So, please could the realloc & free functions also be made
thread-safe, and documented as such?   This would be a major
performance benefit for many multi-threaded drivers (not just the
efile driver mentioned in the comment),

Many thanks for considering this,

Chris