[erlang-questions] Keeping massive concurrency when interfacing with C

[Peer Stritzinger]

On Mon, Oct 3, 2011 at 10:05 PM, John Smith <> wrote:
>* We are still in the R&D phase (you could say), so I'm not quite sure*>* yet which specific category the number crunching will fall into (I*>* wouldn't be surprised if there are matrices, however). I think what*>* I'll do is write the most intensive parts in both Erlang and C, and*>* compare the two. I'd prefer to stick purely with Erlang though!*

*From time to time the "Erlang is poor at number crunching" can be heard.*

*Mainly this revolves around Erlang being bad at the kind of number
crunching needed for linear algebra/image processing etc.*

*Having a similar requirement as John for a current project I thinking
a lot recently how to use Erlang maybe together with another language
or system together for this (other requirements are quite in favor of
Erlang for my project).*

*When pondering this I noticed that if Erlang would have a flexible
n-dim array type with good performing matrix/vector manipulation
functions I would not need to integrate some external system with all
the complexity required to make the concurrency impedances match.*

*There are several systems where efficient matrix manipulation is
added to languages that would not be considered for numerical
calculations without them.  Examples are NumPy and pdl.perl.org.  *

*Usually these revolve around a basic matrix "buffer" which is
reference counted.  These buffers are referred to by structures of
metadata containing dimensionality, matrix slice info (they support
the concept of having slices of matrices which share the buffers
without copying).*

*The buffers are quite similar to Erlangs binaries, and the no-copy
slicing looks quite functional to me.*

*If we would build a matrix library on top of binaries with only a few
basic operations as NIF's quite powerful things could be built on top
of this using pure Erlang.*

*I would invest some of my time in a library like this, probably not
enough to make it general purpose.  But if others want to participate,
maybe we won't have to repeat the Erlangs to for (all) number
crunching sentiment too often.*

*Cheers*

*-- Peer Stritzinger*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20111004/a74ca8b7/attachment.htm>