[erlang-questions] The quest for the perfect programming language for massive concurrency.

Sean Cribbs sean@REDACTED
Thu Jan 30 17:46:46 CET 2014


*In a relevant language, create an array of 1000 numbers. Initialize all of
the values in the array to zero. Create two threads that run concurrently
and which increment each element of the array one time.*

Sometimes the framing of the problem is the problem. In Erlang, which has
no shared memory (except maybe ETS? But that's an entirely different
issue), and no mutability, you would make two processes with lists of 1000
0's, which would map them into 1's. Maybe if they all need to be 2's, you
might have a pipeline/chain of processes, each receiving messages and
incrementing by 1. The issue is really the framing of the problem doesn't
make sense in Erlang.


   1. The tools are, well frankly, garbage. Sorry, in 2014 to be pushed
   back to coding with VIM and makefiles is primitive. Rebar is crytptic and
   just the pet project of a guy on GIT. Compared to Gradle, Maven and even
   (though I don't care for it much) SBT, rebar is ... lacking. I want to
   spend time working on my business logic, not fighting tools. There are
   plugins for eclipse and intellij but they have minimal functionality and i
   keep reverting back to vim.

This is really just a matter of familiarity. Yes, rebar could be better, I
know because I use it every day. However, you might be surprised at how
mature the OTP ecosystem is (it has 10 years on Java, by the way). IDEs are
not the way to go with Erlang generally, but Emacs has support for Wrangler
which does automated refactoring and Distel which lets you fire up and
interact with Erlang nodes directly from the editor (including debugging!),
and much more. Dialyzer lets you introduce gradual typing into your program
so you can find many bugs, a thing I've grown attached to lately.

It seems you are just comfortable in your IDE. When you started doing Java
or Scala were you immediately familiar with all the tools in its ecosystem?
I doubt it. Any new language ecosystem will have its own ramp-up time.

   1. Much harder to staff than Scala because it is not Java based.

Actually we have found this to be an advantage. Yes, it's harder to find
*any* programmer, but *good* programmers will try out new things because
they have an interest in learning and are adaptable. Head-count isn't
always the answer. One of our best Erlang devs at Basho came from a shop
where he did mostly Clojure. I think our CTO Justin Sheehy said it more
eloquently here: http://basho.com/erlang-at-basho-five-years-later/

   1. Fewer general purpose libraries and no major central repositories.
   I don't want to write code to create JSON, that isnt part of my business
   scope. I will pick that one of the shelf If i can.

Erlang/OTP contains most of what you need, but there are also plenty of
libraries for mundane things like JSON codecs, etc, and often the top
Erlang projects use them already! Many times a quick search on Github or
Google will help you find what you need.

   1. Records as the only structured data type is ... annoying.

Records aren't "structured data types", they're a compile-time syntax sugar
over tagged tuples. I suspect this criticism comes from your background as
an OO programmer.

I hope you won't dismiss Erlang for the wrong reasons. Yes, it is a totally
different world from JVM projects, but it's one I have enjoyed working in
for a long time.


On Thu, Jan 30, 2014 at 10:26 AM, Aaron France <aaron.l.france@REDACTED>wrote:

> Then what about Clojure?
>
> You can stay on the jvm, use Java  code and it has a focus on concurrency.
>
> Whilst it fails hard on fault tolerance it could be an escape hatch for
> those stuck with the jvm
>
> Aaron
> On 30 Jan 2014 17:22, "kraythe ." <kraythe@REDACTED> wrote:
>
>> Ok right up front, I'm a Java Guru, not a Scala or Erlang one. What that
>> means is that I know more than most debs about the core java language, but
>> enough to know where the problems are. And certainly java has many issues
>> but it also has massive momentum. I think one of the issues with Java can
>> be expressed in one little programming puzzle I came across recently:
>>
>> *In a relevant language, create an array of 1000 numbers. Initialize all
>> of the values in the array to zero. Create two threads that run
>> concurrently and which increment each element of the array one time.*
>>
>>
>> Interesting? The solution in Scala that I came up with is the following.
>>
>> def mtmap[A](data : List[A], threads: Int, f: (A) => A) = {
>>   import scala.concurrent.{ExecutionContext, Future, Await}
>>   implicit val executor =
>> java.util.concurrent.Executors.newFixedThreadPool(threads)
>>   implicit val context =  ExecutionContext.fromExecutorService(executor)
>>   val futures : Future[List[A]]= Future.traverse(data)(x => Future{f(x)})
>>   import scala.concurrent.duration._
>>   import scala.language.postfixOps
>>   val results : List[A] = Await.result(futures, 1 minute)
>>   results
>> }
>>
>>
>> Just thinking of doing this in Java will bring up some of the big
>> problems with Java; I will leave it as a mental exercise for the reader.
>> The problem is that Scala inherits some of them from the JVM and that has
>> made me look into Erlang. The goal being to select a language for the
>> development of a concurrent TCP/IP based application with thousands of
>> users interacting in a small space.
>>
>> So far I think the contenders I have are Scala with Akka or Erlang. And
>> yes, I know there are evangelists to both and I will post this to an Scala
>> list to get their feedback as well (or something similar). Now, right up
>> front I am not peeing on either language. They both rock or they wouldn't
>> be on the list. The question is which should win the prize. There is no
>> going back once development is 1000 hours down the road.
>>
>> *Scala: *
>> *Pros:*
>>
>>    1. Based on Java Ecosystem so easier to staff because good Java devs
>>    can convert to Scala.
>>    2. Decent tools (which are getting better). Many libraries.
>>    3. Static typing with type inference.
>>
>> *Cons:*
>>
>>    1. Based on Java Ecosystem and inherit the problems of that ecosystem
>>    (i.e. immutable is a function of the design of a class, not of the language
>>    so it can't be guaranteed.), Also library code under the hood is not as
>>    rigorous as scala code in enforcing immutability so at some point you are
>>    rolling dice here.
>>    2. Scala is also more heavyweight than Erlang when it comes to
>>    spawning thousands of processes. Erlang was built from the ground up to do
>>    concurrency. For Scala its an Akka bolt on can carries the Java threading
>>    nightmare (shared memory, etc).
>>    3. Scala is not as fast. My server will be doing billions of vector
>>    math calculations per day and they have to be in the terms of milliseconds
>>    of latency. It has to be that if I have a user R in the server at position
>>    V where V is a vector, I need to calculate all other actors within 50 units
>>    and get that answer in milliseconds so that only the network latency is the
>>    bottleneck. Some of this can be helped with algorithms like spatial grids
>>    and so on but still we are looking at a couple of hundred vector math calls
>>    per second.
>>    4. Scala is harder to hook up to dozens of nodes and move actors from
>>    node x to node y than Erlang, mainly because that was one of the design
>>    goals of Erlang.
>>
>>
>> *Erlang:*
>> *Pros:*
>>
>>    1. Built for concurrency. Can handle dozens of hardware nodes, build
>>    massive applications of the kind I am trying to deploy. Think of 100k users
>>    connecting to the cluster with a TCP connection and interacting over that
>>    connection which interacts with any or all of the other 100k actors.
>>    2. Built from the ground up with immutability in mind. Immutability
>>    is a language feature, not compromisable. Its not JVM based and so is not
>>    under the same issues as Java is.
>>    3. There is merit in the thought that static typing is sometimes a
>>    hinderance to a language.
>>
>> *Cons:*
>>
>>    1. The tools are, well frankly, garbage. Sorry, in 2014 to be pushed
>>    back to coding with VIM and makefiles is primitive. Rebar is crytptic and
>>    just the pet project of a guy on GIT. Compared to Gradle, Maven and even
>>    (though I don't care for it much) SBT, rebar is ... lacking. I want to
>>    spend time working on my business logic, not fighting tools. There are
>>    plugins for eclipse and intellij but they have minimal functionality and i
>>    keep reverting back to vim.
>>    2. Much harder to staff than Scala because it is not Java based.
>>    3. Fewer general purpose libraries and no major central repositories.
>>    I don't want to write code to create JSON, that isnt part of my business
>>    scope. I will pick that one of the shelf If i can.
>>    4. Records as the only structured data type is ... annoying.
>>
>>
>> The problem I have is I can't find the perfect solution. Erlang is
>> compelling but also is Scala.
>>
>> Opinions?
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>


-- 
Sean Cribbs <sean@REDACTED>
Software Engineer
Basho Technologies, Inc.
http://basho.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140130/90624cb0/attachment.htm>


More information about the erlang-questions mailing list