<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Jan 30, 2014 at 5:19 PM, kraythe . <span dir="ltr"><<a href="mailto:kraythe@gmail.com" target="_blank">kraythe@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><span style="font-family:Arial,Helvetica,sans-serif;font-size:13px">Ok right up front, I'm a Java Guru, not a Scala or Erlang one. What that means is that I know more than most debs about the core java language, but enough to know where the problems are. And certainly java has many issues but it also has massive momentum. I think one of the issues with Java can be expressed in one little programming puzzle I came across recently: </span><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px">
<br></div><blockquote style="font-family:Arial,Helvetica,sans-serif;font-size:13px;margin:0px 0px 0px 40px;border:none;padding:0px"><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><i>In a relevant language, create an array of 1000 numbers. Initialize all of the values in the array to zero. Create two threads that run concurrently and which increment each element of the array one time.</i></div>
</blockquote><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px"><br></div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px">
Interesting? The solution in Scala that I came up with is the following. </div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px"><br></div><blockquote style="font-family:Arial,Helvetica,sans-serif;font-size:13px;margin:0px 0px 0px 40px;border:none;padding:0px">
<div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><font face="courier new, monospace" style="margin:0px;padding:0px;border:0px;vertical-align:baseline">def mtmap[A](data : List[A], threads: Int, f: (A) => A) = {</font></div>
</div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><font face="courier new, monospace" style="margin:0px;padding:0px;border:0px;vertical-align:baseline"> import scala.concurrent.{ExecutionContext, Future, Await}</font></div>
</div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><font face="courier new, monospace" style="margin:0px;padding:0px;border:0px;vertical-align:baseline"> implicit val executor = java.util.concurrent.Executors.newFixedThreadPool(threads)</font></div>
</div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><font face="courier new, monospace" style="margin:0px;padding:0px;border:0px;vertical-align:baseline"> implicit val context = ExecutionContext.fromExecutorService(executor)</font></div>
</div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><font face="courier new, monospace" style="margin:0px;padding:0px;border:0px;vertical-align:baseline"> val futures : Future[List[A]]= Future.traverse(data)(x => Future{f(x)})</font></div>
</div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><font face="courier new, monospace" style="margin:0px;padding:0px;border:0px;vertical-align:baseline"> import scala.concurrent.duration._</font></div>
</div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><font face="courier new, monospace" style="margin:0px;padding:0px;border:0px;vertical-align:baseline"> import scala.language.postfixOps</font></div>
</div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><font face="courier new, monospace" style="margin:0px;padding:0px;border:0px;vertical-align:baseline"> val results : List[A] = Await.result(futures, 1 minute)</font></div>
</div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><font face="courier new, monospace" style="margin:0px;padding:0px;border:0px;vertical-align:baseline"> results</font></div>
</div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline"><font face="courier new, monospace" style="margin:0px;padding:0px;border:0px;vertical-align:baseline">}</font></div>
</div></blockquote><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px"><br></div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px">
Just thinking of doing this in Java will bring up some of the big problems with Java; I will leave it as a mental exercise for the reader. The problem is that Scala inherits some of them from the JVM and that has made me look into Erlang. The goal being to select a language for the development of a concurrent TCP/IP based application with thousands of users interacting in a small space. </div>
</div></blockquote><div><br></div><div>A benchmark that tells you how concurrent array updates works won't tell you anything about</div><div>well sockets work.</div><div><br></div><div>Erlang was explicitly designed for managing thousands of simultaneous sessions (ie telecomms applications) - (that's why things like WhatsApp is programmed in Erlang)</div>
<div>try Googling "whatsapp+erlang".</div><div><br></div><div>Java was designed for set-top boxes - both languages are good for what they were designed to be good at. Consequently Erlang is good at handling large numbers of parallel connections and java is really good for set-top boxes. Both languages suck when used outside their design domains. So Erlang is crap</div>
<div>at programming small memory footprint embedded devices and java is not stunning good at handling</div><div>concurrency.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px"><br></div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px">
So far I think the contenders I have are Scala with Akka or Erlang. And yes, I know there are evangelists to both and I will post this to an Scala list to get their feedback as well (or something similar). Now, right up front I am not peeing on either language. They both rock or they wouldn't be on the list. The question is which should win the prize. There is no going back once development is 1000 hours down the road. </div>
<div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px"><br></div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px">
<u><b>Scala:</b> </u></div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px"><b>Pros:</b> </div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px">
<ol><li style="line-height:17px">Based on Java Ecosystem so easier to staff because good Java devs can convert to Scala. <br></li><li style="line-height:17px">Decent tools (which are getting better). Many libraries. </li>
<li style="line-height:17px">Static typing with type inference. </li></ol></div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px"><b>Cons:</b> </div>
<div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px"><ol><li style="line-height:17px">Based on Java Ecosystem and inherit the problems of that ecosystem (i.e. immutable is a function of the design of a class, not of the language so it can't be guaranteed.), Also library code under the hood is not as rigorous as scala code in enforcing immutability so at some point you are rolling dice here. </li>
<li style="line-height:17px">Scala is also more heavyweight than Erlang when it comes to spawning thousands of processes. Erlang was built from the ground up to do concurrency. For Scala its an Akka bolt on can carries the Java threading nightmare (shared memory, etc). <br>
</li></ol></div></div></blockquote><div><br></div><div>How about millions of processes - "thousands" of processes is in the "tiny" league table</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px"><ol><li style="line-height:17px">
</li><li style="line-height:17px">Scala is not as fast. My server will be doing billions of vector math calculations per day and they have to be in the terms of milliseconds of latency. It has to be that if I have a user R in the server at position V where V is a vector, I need to calculate all other actors within 50 units and get that answer in milliseconds so that only the network latency is the bottleneck. Some of this can be helped with algorithms like spatial grids and so on but still we are looking at a couple of hundred vector math calls per second. </li>
</ol></div></div></blockquote><div>I don't get a feeling for what "billions of vector math calculations per day" means. How long does</div><div>one "vector math calculation" take - or is this what you were trying to measure?</div>
<div><br></div><div>If you're doing really heavy number crunching Julia seems the way to go. Julia is outperforming Fortran on many benchmarks - and Fortran always set the gold standard for number crunching.</div><div>
<br></div><div>Something like Erlang for orchestration, Julia for number crunching and 0MQ for messaging would be what I'd look at.</div><div> </div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px"><ol>
<li style="line-height:17px">Scala is harder to hook up to dozens of nodes and move actors from node x to node y than Erlang, mainly because that was one of the design goals of Erlang.</li></ol></div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px">
<br></div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px"><u><b>Erlang:</b></u></div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px">
<b>Pros:</b> </div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px"><ol><li style="line-height:17px"><span style="margin:0px;padding:0px;border:0px;vertical-align:baseline;line-height:normal">Built for concurrency. Can handle dozens of hardware nodes, build massive applications of the kind I am trying to deploy. Think of 100k users connecting to the cluster with a TCP connection and interacting over that connection which interacts with any or all of the other 100k actors. </span></li>
<li style="line-height:17px"><span style="margin:0px;padding:0px;border:0px;vertical-align:baseline;line-height:normal">Built from the ground up with immutability in mind. Immutability is a language feature, not compromisable. Its not JVM based and so is not under the same issues as Java is. </span></li>
</ol></div></div></blockquote><div>What's immutability got to do with anything? If you want to do fast vector operations things like</div><div>Fortran is king - and that's mutable as hell.</div><div><br></div><div>
The kind of immutability that Erlang has is for reasons of fault tolerance - we don't want programs to crash and run wild - fast math needs mutable structures</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px"><ol>
<li style="line-height:17px"><span style="margin:0px;padding:0px;border:0px;vertical-align:baseline;line-height:normal">There is merit in the thought that static typing is sometimes a hinderance to a language. </span></li>
</ol></div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px"><b>Cons:</b> </div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px">
<ol><li style="line-height:17px"><span style="margin:0px;padding:0px;border:0px;vertical-align:baseline;line-height:normal">The tools are, well frankly, garbage. Sorry, in 2014 to be pushed back to coding with VIM and makefiles is primitive. Rebar is crytptic and just the pet project of a guy on GIT. Compared to Gradle, Maven and even (though I don't care for it much) SBT, rebar is ... lacking. I want to spend time working on my business logic, not fighting tools. There are plugins for eclipse and intellij but they have minimal functionality and i keep reverting back to vim. </span></li>
</ol></div></div></blockquote><div>Tools are no substitute for thought. None of the tools you mention help me think - the thing that takes</div><div>time in programming is deciding exactly what problem to solve. Once you know you can usually knock out the program in a few hours.</div>
<div><br></div><div>The best tool we have is (or course) maths - ie logical reasoning and proofs.</div><div> </div><div>The average programmer produces about 10 lines of code per day, but they might write hundreds or</div>
<div>even thousands of lines of code per day that never makes it to the final product. If they knew what to</div><div>do they could just write these ten lines in a few minutes, but they don't know so they have to write</div>
<div>thousands of lines to help try and figure out what the problem is. It also creates the illusion of work</div><div>and they can get paid for it.</div><div><br></div><div>The big problem in programming is understanding the problem. "programming is understanding"</div>
<div>The tools we have today are no better than they were 40 years ago. My brain is still far far better at</div><div>debugging that any fancy tool. I have no idea my brain works - all I know is that I often work on</div>
<div>a problem and then give up. I go to sleep and wake up with the solution. Now that is far far better</div><div>than *any* IDE on the planet.</div><div><br></div><div>I like to spend my time figuring out *exactly* how things related to my applications, there would be</div>
<div>no point in figuring out (say) eclipse works - since editing text is not a problem for me - I'm happy</div><div>with emacs - and even happier with a pen and paper (which boots far quicker, is far more portable,</div>
<div>and far more flexible)</div><div><br></div><div>I spend 65% of my programming time fixing stuff that I need that is broken. Every time a program</div><div>does not install or work exactly as advertised I have to pull it apart and fix it. None of these tools </div>
<div>are of any use when they break. Rebar/Maven/Git etc are great when they work.</div><div><br></div><div>Even emacs is crashing randomly on my mac - the problem is not the tools but that fact that</div><div>broken tools take a long time to fix. (this is why I try to use as little as possible - emacs, make and bash, do it for me)</div>
<div><br></div><div>In the good old days programmers could talk to each other - everybody who could program knew how to write makefile and bash scripts. Now a guy who speaks Maven is floored when sent a rakefile.</div><div>
<br></div><div>And as for eclipse etc. these are unbelievably horrible.</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px">
<ol>
<li style="line-height:17px"><span style="margin:0px;padding:0px;border:0px;vertical-align:baseline;line-height:normal">Much harder to staff than Scala because it is not Java based. </span></li><li style="line-height:17px">
<span style="margin:0px;padding:0px;border:0px;vertical-align:baseline;line-height:normal">Fewer general purpose libraries and no major central repositories. I don't want to write code to create JSON, that isnt part of my business scope. I will pick that one of the shelf If i can. </span></li>
</ol></div></div></blockquote><div>It's often far quicker to write code than a) find a library b) install it c) fix the bugs in the library.</div><div>Creating JSON is trivial:</div><div><br></div><div> io:format("{'firstname':'joe', 'lastname':'armstring'}").</div>
<div><br></div><div>Parsing it is a wee bit messy. </div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px">
<ol>
<li style="line-height:17px"><span style="margin:0px;padding:0px;border:0px;vertical-align:baseline;line-height:normal">Records as the only structured data type is ... annoying. </span></li></ol><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline">
<br></div></div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px">The problem I have is I can't find the perfect solution. Erlang is compelling but also is Scala. </div>
<div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px"><br></div><div style="margin:0px;padding:0px;border:0px;vertical-align:baseline;font-family:Arial,Helvetica,sans-serif;font-size:13px">
Opinions?</div></div></blockquote><div><br></div><div>There is no easy path.</div><div><br></div><div> - choose your languages</div><div> - write the benchmarks in all the languages</div><div> - measure</div><div>
- don't believe what other people say - base the results on your personal experience</div><div> - understand your problem - write a detailed English description of the problem and</div><div> give to to your friends - get them to read it and explain the problem back to you</div>
<div> </div><div>There are also tradeoff to consider.</div><div><br></div><div> You want super fast math (this implies mutable data) </div><div> You want safety (this implies non-mutable data)</div><div> You mention JSON (implies inefficient data transfer)</div>
<div> You want massive communication</div><div>etc.</div><div><br></div><div> These pull in different directions.</div><div><br></div><div>Cheers</div><div><br></div><div>/Joe</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br></blockquote></div><br></div></div>