[erlang-questions] Machine Learning

[Theepan]

Thanks Jesper and other for the pointers.. My work is more on texts
corpuses than on images. Will revert if there is something that I can share
with the team.. :)

Regards,
Theepan


On Wed, Feb 10, 2016 at 5:01 PM, Jesper Louis Andersen <
jesper.louis.andersen@REDACTED> wrote:

>
> On Wed, Feb 10, 2016 at 10:34 AM, Samuel <samuelrivas@REDACTED> wrote:
>
>> I am not aware of existing ML or linear algrebra libraries in erlang
>> that can be used to quick start an ML project, but just piping into
>> tensorflow (or any other existing library/framework) isn't really
>> doing ML with erlang, is it? You can as well just use tensorflow
>> directly.
>>
>
> The question is if this is a practical problem which needs solving or it
> is for research. If you are researching how to construct, say, SVM or NNs,
> then surely Erlang is a good vehicle. But in practice, there is a number of
> things which makes Erlang unsuitable:
>
> * ML is often CPU bound. You don't want a bytecode interpreter to be a
> limiting factor here. Even if the interpreter in Erlang is state-of-the-art
> and highly optimized, it is not far fetched that a FP-intensive program
> will be roughly a factor of 30 faster if compiled in a lower level language.
>
> * GPUs are popular in ML models for a reason: they speed up the FP
> computations by a factor of 3000 or more. This in itself should hint you
> that you need something else than Erlang.
>
> * Erlangs word overhead per process and terms means a lower-level model
> can pack many more entities in memory. This affects caching behavior.
>
> Training of the model is often off-line and using the model is online in
> the system. How you train your model is less important. This is why I'd
> just outsource this problem to the libraries built and tuned for it. It is
> like solving LinAlg problems but forgetting everything about existing
> LAPACK and ATLAS routines in Fortran.
>
> A model, in Erlang, which could be viable is to use Erlang to produce
> programs for lower level consumption by compilation. But these are problems
> for which languages such as Haskell and OCaml dominates for a reason: their
> type systems makes it far easier to pull off.
>
>
>
> --
> J.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20160211/45a4a625/attachment.htm>