[erlang-questions] Freedom Box and Erlang

[Tim Watson]

On 21 June 2011 03:36, Michael Truog <mjtruog@REDACTED> wrote:
> On 06/20/2011 02:42 PM, Tim Watson wrote:
>> On 20 June 2011 20:04, Michael Truog <mjtruog@REDACTED> wrote:
>>> True.  CloudFoundry exists and is open source.  However, it is Ruby based so it is unable to scale like an Erlang system (like CloudI).  CloudFoundry is control by VMware, which is an organization, and they are using the Apache License 2.0.  CloudI is independent and uses a BSD license, so it can be reused anywhere.
>> First up, I'm definitely not knocking CloudI here - I think it looks
>> awesome. CloudFoundry is owned and developed by an organisation, but
>> so are plenty of other things (riak, rabbitmq, etc) so that doesn't
>> automatically mean lock-in. I mean Erlang/OTP is controlled by
>> Ericsson right, and by the same measure CloudI is controlled by
>> Michael Truog. The Apache License 2.0 is more restrictive than the BSD
>> type license you've chosen for CloudI - and good on you for making it
>> as open as possible - but I can still use and adapt stuff based on the
>> Apache license for commercial or similarly licensed open source
>> projects.
> I agree.  Often times organizations make strategic decisions to include and exclude support for various types of software, which either benefits their partners/divisions or harms their competition.  Having an open source project that is independent means that it is able to pursue a path to the best combination of technology based on what the community thinks (though limited by time/money).  That way there should be no major influence based on "competitive advantage" decisions that are commonly made by any commercial organization.
>
> I am not a lawyer and I am not giving legal advise.  However, I think it is more likely that BSD licensed software is reused and then sold in a commercial product, when compared with Apache licensed software.  The main difference between the licenses is over patent ownership (BSD is assumed, Apache is explicit), so the differences might be very negligible when you consider the content in each license.  Part of the bias for support of the BSD license may be because of its simplicity.

I'd agree with that - I think the simplicity is for many of us, a
compelling reason for adopting it.

>> Secondly, I agree that Erlang/OTP is a gift from the Old Gods of the
>> North when it comes to fault tolerance and so on, but just because
>> app-1 is written in Erlang and app-2 is written in Ruby, doesn't
>> automatically mean that one will scale better than another. Both these
>> cloud frameworks are based (at least in part) on services running as
>> native OS processes - admittedly you support native Erlang
>> services/jobs - and the other comparative technologies are really
>> around routing and distribution. Here both frameworks are relying on
>> the same kind of underlying TCP stack that takes advantage of
>> non-blocking socket I/O - distributed Erlang using the inet
>> infrastructure running on a beam emulator compiled with --enable-poll
>> or whatever, versus CloudFoundry which runs on event-machine, a
>> framework a la libevent. Distributed Erlang is known to have
>> limitations of its own, that wouldn't necessarily apply to a very
>> large grid built on some other distribution protocol - riak for
>> example uses a custom protocol, as does Scalaris I think and probably
>> quite a few others. The fault tolerance that is built in for you
>> (because you're running on top of OTP) though, is admittedly a big
>> selling point.
> I am judging based on the VMs involved and their known scalability limitations.  Having libevent in a project does not make it scalable.  There are many projects that use libevent that do not scale judging based on loadtest results.  Also, libevent usage, when done incorrectly, makes the system fail prematurely, so it should not be seen as a panacea for scalability.  The approach to scalability is achievable in many ways, but the complexity of the result is a concern since that controls how testable the system is.  Looking at the Erlang VM scale (with epoll/kqueue, the "+K true" (kernel-poll) erl command line argument) using the inet infrastructure and light-weight processes (one Erlang process per socket), 10K+ scalability is easily achievable with simple code that is usable as beginner examples.  When you compare that to other solutions where loadtests rarely hit 10K (usually HTTP servers and different configurations), with code that is actively developed for any length of
> time, I think Erlang is clearly the winner.

Yes I agree, based on my own experience migrating SIP/H.323
applications from Java/J2EE/JAIN-SIP to Erlang/OTP - even supposedly
very stable platforms (e.g., the JVM) don't necessarily compete with
Erlang and when using OTP the resulting system is actually far simpler
and easier to understand and test. My thoughts here were that it is
the (soft) routing that is really under consideration when we compare
the two. Looking at both code bases, I don't see the CloudFoundry code
actually doing that much work with requests - it mainly just forwards
them to the right HTTP endpoint. I wanted to point out that whilst
Ruby VMs are likely inferior to BEAM when it comes to building HTTP
servers, multiplexing socket I/O *can* shave away that advantage if it
is done right. I've seen python network servers out perform Java
applications, which surprised the hell out of a lot of people.

>
> When you consider scalability across separate machines, that is a separate concern.  The distributed Erlang fully connected network topology is an advantage for creating Erlang systems.  However, when the node count needs to be increased beyond 100 (roughly) it is necessary to have a tree network topology to keep things fault-tolerant.  ZeroMQ integration configuration could be used within CloudI to avoid the distributed Erlang node limitation.
>

Yes, and this is definitely an area where Erlang/OTP clearly shines. I
think not depending on distributed Erlang - using some other
integration such as 0MQ or whatever - would be very sensible for
CloudI. Just as I suspect that having a routing engine built in Erlang
would be much better for CloudFoundry IMO. :)

>> Don't get me wrong - I'm not asserting any amazing properties about
>> ruby, nor am I trying to criticise your work. It's just that you're
>> clearly very smart (because you wrote a complex beast like CloudI) so
>> your peers (and even underlings like me) need to hold you to account
>> for intellectual honesty. Saying "it is Ruby based so it is unable to
>> scale..." is FUD unless you qualify it in more detail. Most people on
>> this list would be inclined to think you're on to something, but this
>> is not a black and white area. I'm pretty sure that services such as
>> Github and Heroku are very scalable, and they use both Ruby and Erlang
>> (and probably only Odin knows what else) for various tasks.
> Sorry, not trying to start a language flame-war.  I like Ruby and enjoy how succinct it is.  I just think that it is important to use the right tool for the right job.  Github and Heroku have both talked publicly about how Erlang has helped them scale their back-end systems.  So, I think most companies are becoming aware that a Ruby front-end can scale with an Erlang back-end.  However, I don't see anything within CloudFoundry that can provide scalability like Erlang.

That's better - I was mainly concerned with getting you to state your
reasoning. ;)

What you say makes good sense and I agree that Erlang's benefits are
exemplified by the Github and Heroku architectures (any many others)
which rely on it. I would point out that some backend processes (on
both platform IIRC) that are not front-end are actually implemented in
Ruby (or other tech), but Erlang is right there in the core
nonetheless. I've found that "the right tool for the right job" is
absolutely the best way to go about building software, so I
wholeheartedly support that sentiment.

Thanks for your constructive responses Michael, and very best of luck
with CloudI!

Cheers,

Tim