[erlang-questions] Machine Learning

[lloyd@REDACTED]

Hello,

I can't claim anything but the shallowest machine learning chops beyond cursory exploration of the literature and what my data scientist son has taught me. But it seems like the first and foremost consideration in determining whether or not to proceed with a machine learning project in Erlang is careful consideration of the problem you're trying to solve.

My sense is that, as Alex points out, Erlang may work quite well for problems in the Machine Learning 101  domain; e.g. limited number of perceptron elements. How do we define limited? Exactly why it may be worthwhile to experiment. Certainly Gene Shor has done interesting work. Does anyone know of worthy follow-up?

That said, big-data machine learning isn't the only game in town. This piece throws down a significant challenge:

Neural modelling: Abstractions of the mind
http://www.nature.com/nature/journal/v531/n7592_supp/full/531S16a.html

All the best,

LRP

-----Original Message-----
From: "Alex Alvarez" <eajam@REDACTED>
Sent: Saturday, March 5, 2016 4:58pm
To: erlang-questions@REDACTED
Subject: Re: [erlang-questions] Machine Learning

_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://erlang.org/mailman/listinfo/erlang-questions
Sorry for my late response to this topic, but I do believe Erlang is 
actually a great language for the ML and statistics space.  Take a basic 
feed-forward NN with back-propagation, for example.  What you'd normally 
have in terms of mathematical computation is mainly addition and 
multiplication.  You only need to put together a perceptron, which are 
only inputs (including a bias) multiplied by respective weights, you add 
them up and pass this value through a function like sigmoid or 
hyperbolic tangent and that's that. Back-propagation, as a way to adjust 
the weights during the training phase, doesn't require math-wise 
anything more complicated.  You combine the perceptrons for the hidden 
and output layers and you got yourself a NN.  In this configuration, 
deep learning will simply be two or more hidden layers, instead of one.  
The key thing to maximize the use of Erlang is certainly to distribute 
the load through processes, so each perception could be one individual 
process, for example.  Definitely, not rocket science.  Now, I concur 
that in some situations it might be advantageous to write a module, say 
for a perception, in C and make use of it within Erlang, but there is no 
reason why you couldn't start with Erlang and gradually move to that 
direction, if need be.

Cheers,
Alex


On 02/10/2016 06:31 AM, Jesper Louis Andersen wrote:
>
> On Wed, Feb 10, 2016 at 10:34 AM, Samuel <samuelrivas@REDACTED 
> <mailto:samuelrivas@REDACTED>> wrote:
>
>     I am not aware of existing ML or linear algrebra libraries in erlang
>     that can be used to quick start an ML project, but just piping into
>     tensorflow (or any other existing library/framework) isn't really
>     doing ML with erlang, is it? You can as well just use tensorflow
>     directly.
>
>
> The question is if this is a practical problem which needs solving or 
> it is for research. If you are researching how to construct, say, SVM 
> or NNs, then surely Erlang is a good vehicle. But in practice, there 
> is a number of things which makes Erlang unsuitable:
>
> * ML is often CPU bound. You don't want a bytecode interpreter to be a 
> limiting factor here. Even if the interpreter in Erlang is 
> state-of-the-art and highly optimized, it is not far fetched that a 
> FP-intensive program will be roughly a factor of 30 faster if compiled 
> in a lower level language.
>
> * GPUs are popular in ML models for a reason: they speed up the FP 
> computations by a factor of 3000 or more. This in itself should hint 
> you that you need something else than Erlang.
>
> * Erlangs word overhead per process and terms means a lower-level 
> model can pack many more entities in memory. This affects caching 
> behavior.
>
> Training of the model is often off-line and using the model is online 
> in the system. How you train your model is less important. This is why 
> I'd just outsource this problem to the libraries built and tuned for 
> it. It is like solving LinAlg problems but forgetting everything about 
> existing LAPACK and ATLAS routines in Fortran.
>
> A model, in Erlang, which could be viable is to use Erlang to produce 
> programs for lower level consumption by compilation. But these are 
> problems for which languages such as Haskell and OCaml dominates for a 
> reason: their type systems makes it far easier to pull off.
>
>
>
> -- 
> J.
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions