[erlang-questions] Thread-safety of driver_alloc/realloc/free_binary

[Chris Newcombe]

This is great news -- thanks.

Is there a way to test at runtime if the emulator is running in smp mode?

many thanks,

Chris

On 10/20/06, Rickard Green <rickard.s.green@REDACTED> wrote:
> They are all thread safe in the emulator with smp support, but they will
> probably never be thread safe in the emulator without smp support.
>
> The thread unsafe part is the reference counting of binaries. Making
> them thread safe would cause overhead for all reference counted
> binaries, not just driver binaries. We don't want to slow down all
> binary management as it is now, but we might make them thread safe in
> the future if threaded drivers should become very common (but don't
> count on it).
>
> BR,
> Rickard Green, Erlang/OTP
>
> Chris Newcombe wrote:
> > (To the maintainers of erts/emulator/beam/io.c ... )
> >
> > Hi,
> >
> > I'm working on an update to Scott Lystig Fritchie's Erlang Driver
> > Toolkit, including a significantly enhanced driver for Berkeley DB.  I
> > hope to be able to release this soon as open-source.
> >
> > The update includes features such as multiple private resizable
> > threadpools per port instance (supported in both 'linked-in' and
> > 'spawned-pipe' modes).
> >
> > One problem is that, when getting data from Berkeley DB, the driver
> > needs to allocate driver binaries.  Ideally, it would like to to do so
> > from within the driver's private threadpools.  But of course,
> > driver_alloc/realloc/free_binary are not documented as thread-safe.
> >
> > So currently the Berkeley DB driver has to allocate each piece of data
> > twice (once with malloc and later with driver_alloc_binary), and copy
> > it once.  And the copying is done in the Erlang scheduler thread,
> > which blocks the VM unecessarily.
> >
> > This appears to be changing.  I found the following comments in
> > otp_src_R11B-1/erts/emulator/beam/io.c
> >
> >  * - driver_alloc_binary() is thread safe (efile driver depend on it).
> >  * - driver_realloc_binary(), and driver_free_binary() are *not* thread safe.
> >
> > Unfortunately, to avoid the copying the Berkeley DB driver needs to
> > call all 3 of these functions from private threads.
> >
> > So, please could the realloc & free functions also be made
> > thread-safe, and documented as such?   This would be a major
> > performance benefit for many multi-threaded drivers (not just the
> > efile driver mentioned in the comment),
> >
> > Many thanks for considering this,
> >
> > Chris
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
>