[erlang-questions] Thread-safety of driver_alloc/realloc/free_binary

[Rickard Green]

It is currently not possible to test whether it is an emulator with smp 
support or not from the driver, but this will be possible some time in 
the future.

 From an Erlang program you can use erlang:system_info(smp_support) 
(since R11B-0). Compile two versions of the driver and decide which one 
to load based on the result returned from erlang:system_info(smp_support).

BR,
Rickard Green, Erlang/OTP

Chris Newcombe wrote:
> This is great news -- thanks.
> 
> Is there a way to test at runtime if the emulator is running in smp mode?
> 
> many thanks,
> 
> Chris
> 
> On 10/20/06, Rickard Green <rickard.s.green@REDACTED> wrote:
>> They are all thread safe in the emulator with smp support, but they will
>> probably never be thread safe in the emulator without smp support.
>>
>> The thread unsafe part is the reference counting of binaries. Making
>> them thread safe would cause overhead for all reference counted
>> binaries, not just driver binaries. We don't want to slow down all
>> binary management as it is now, but we might make them thread safe in
>> the future if threaded drivers should become very common (but don't
>> count on it).
>>
>> BR,
>> Rickard Green, Erlang/OTP
>>
>> Chris Newcombe wrote:
>> > (To the maintainers of erts/emulator/beam/io.c ... )
>> >
>> > Hi,
>> >
>> > I'm working on an update to Scott Lystig Fritchie's Erlang Driver
>> > Toolkit, including a significantly enhanced driver for Berkeley DB.  I
>> > hope to be able to release this soon as open-source.
>> >
>> > The update includes features such as multiple private resizable
>> > threadpools per port instance (supported in both 'linked-in' and
>> > 'spawned-pipe' modes).
>> >
>> > One problem is that, when getting data from Berkeley DB, the driver
>> > needs to allocate driver binaries.  Ideally, it would like to to do so
>> > from within the driver's private threadpools.  But of course,
>> > driver_alloc/realloc/free_binary are not documented as thread-safe.
>> >
>> > So currently the Berkeley DB driver has to allocate each piece of data
>> > twice (once with malloc and later with driver_alloc_binary), and copy
>> > it once.  And the copying is done in the Erlang scheduler thread,
>> > which blocks the VM unecessarily.
>> >
>> > This appears to be changing.  I found the following comments in
>> > otp_src_R11B-1/erts/emulator/beam/io.c
>> >
>> >  * - driver_alloc_binary() is thread safe (efile driver depend on it).
>> >  * - driver_realloc_binary(), and driver_free_binary() are *not* 
>> thread safe.
>> >
>> > Unfortunately, to avoid the copying the Berkeley DB driver needs to
>> > call all 3 of these functions from private threads.
>> >
>> > So, please could the realloc & free functions also be made
>> > thread-safe, and documented as such?   This would be a major
>> > performance benefit for many multi-threaded drivers (not just the
>> > efile driver mentioned in the comment),
>> >
>> > Many thanks for considering this,
>> >
>> > Chris
>> > _______________________________________________
>> > erlang-questions mailing list
>> > erlang-questions@REDACTED
>> > http://www.erlang.org/mailman/listinfo/erlang-questions
>> >
>>
>