[erlang-questions] The quest for the perfect programming language for massive concurrency.

[kraythe .]

Ok right up front, I'm a Java Guru, not a Scala or Erlang one. What that
means is that I know more than most debs about the core java language, but
enough to know where the problems are. And certainly java has many issues
but it also has massive momentum. I think one of the issues with Java can
be expressed in one little programming puzzle I came across recently:

*In a relevant language, create an array of 1000 numbers. Initialize all of
the values in the array to zero. Create two threads that run concurrently
and which increment each element of the array one time.*


Interesting? The solution in Scala that I came up with is the following.

def mtmap[A](data : List[A], threads: Int, f: (A) => A) = {
  import scala.concurrent.{ExecutionContext, Future, Await}
  implicit val executor =
java.util.concurrent.Executors.newFixedThreadPool(threads)
  implicit val context =  ExecutionContext.fromExecutorService(executor)
  val futures : Future[List[A]]= Future.traverse(data)(x => Future{f(x)})
  import scala.concurrent.duration._
  import scala.language.postfixOps
  val results : List[A] = Await.result(futures, 1 minute)
  results
}


Just thinking of doing this in Java will bring up some of the big problems
with Java; I will leave it as a mental exercise for the reader. The problem
is that Scala inherits some of them from the JVM and that has made me look
into Erlang. The goal being to select a language for the development of a
concurrent TCP/IP based application with thousands of users interacting in
a small space.

So far I think the contenders I have are Scala with Akka or Erlang. And
yes, I know there are evangelists to both and I will post this to an Scala
list to get their feedback as well (or something similar). Now, right up
front I am not peeing on either language. They both rock or they wouldn't
be on the list. The question is which should win the prize. There is no
going back once development is 1000 hours down the road.

*Scala: *
*Pros:*

   1. Based on Java Ecosystem so easier to staff because good Java devs can
   convert to Scala.
   2. Decent tools (which are getting better). Many libraries.
   3. Static typing with type inference.

*Cons:*

   1. Based on Java Ecosystem and inherit the problems of that ecosystem
   (i.e. immutable is a function of the design of a class, not of the language
   so it can't be guaranteed.), Also library code under the hood is not as
   rigorous as scala code in enforcing immutability so at some point you are
   rolling dice here.
   2. Scala is also more heavyweight than Erlang when it comes to spawning
   thousands of processes. Erlang was built from the ground up to do
   concurrency. For Scala its an Akka bolt on can carries the Java threading
   nightmare (shared memory, etc).
   3. Scala is not as fast. My server will be doing billions of vector math
   calculations per day and they have to be in the terms of milliseconds of
   latency. It has to be that if I have a user R in the server at position V
   where V is a vector, I need to calculate all other actors within 50 units
   and get that answer in milliseconds so that only the network latency is the
   bottleneck. Some of this can be helped with algorithms like spatial grids
   and so on but still we are looking at a couple of hundred vector math calls
   per second.
   4. Scala is harder to hook up to dozens of nodes and move actors from
   node x to node y than Erlang, mainly because that was one of the design
   goals of Erlang.


*Erlang:*
*Pros:*

   1. Built for concurrency. Can handle dozens of hardware nodes, build
   massive applications of the kind I am trying to deploy. Think of 100k users
   connecting to the cluster with a TCP connection and interacting over that
   connection which interacts with any or all of the other 100k actors.
   2. Built from the ground up with immutability in mind. Immutability is a
   language feature, not compromisable. Its not JVM based and so is not under
   the same issues as Java is.
   3. There is merit in the thought that static typing is sometimes a
   hinderance to a language.

*Cons:*

   1. The tools are, well frankly, garbage. Sorry, in 2014 to be pushed
   back to coding with VIM and makefiles is primitive. Rebar is crytptic and
   just the pet project of a guy on GIT. Compared to Gradle, Maven and even
   (though I don't care for it much) SBT, rebar is ... lacking. I want to
   spend time working on my business logic, not fighting tools. There are
   plugins for eclipse and intellij but they have minimal functionality and i
   keep reverting back to vim.
   2. Much harder to staff than Scala because it is not Java based.
   3. Fewer general purpose libraries and no major central repositories.
   I don't want to write code to create JSON, that isnt part of my business
   scope. I will pick that one of the shelf If i can.
   4. Records as the only structured data type is ... annoying.


The problem I have is I can't find the perfect solution. Erlang is
compelling but also is Scala.

Opinions?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140130/fe4609fc/attachment.htm>