[erlang-questions] Erlang and the learning curve

[Michael Turner]

"I'm sure I'm not alone on this mailing list in my opinion that idiomatic
Erlang code generally performs very well and needs no specific tuning or
measurement."

And those few who would beg to differ ought to go look at the sobering Great
Shootout benchmarks and compare Erlang with PHP.  PHP isn't just slow.  It's
*gob-smackingly* slow.  Some of the PHP benchmarks fail because they time
out after an *hour* (where the fastest implementations, in languages like
C++ and Ada, finish in seconds, and Erlang versions are hardly more than an
order of magnitude slower than the fastest.)  A great deal of the world's
server-room fan-noise must be generated by the need to carefully scale
PHP/MySQL apps.  I wouldn't be surprised if to learn that there are many
niches where an Erlang/NoSQL solution would stay uniprocessor and reach
market saturation for its services, where the equivalent PHP/MySQL
implementation would require considerable babysitting and diaper changes to
scale up sensibly.

-michael turner

On Wed, Jan 5, 2011 at 1:03 PM, Steve Vinoski <vinoski@REDACTED> wrote:

> On Tue, Jan 4, 2011 at 4:07 PM, Morten Krogh <mk@REDACTED> wrote:
> > On 1/4/11 8:59 PM, Steve Vinoski wrote:
> >>
> >> On Tue, Jan 4, 2011 at 1:35 PM, Morten Krogh<mk@REDACTED>  wrote:
> >>>
> >>> On 1/4/11 4:37 PM, Steve Vinoski wrote:
> >>>>
> >>>> On Tue, Jan 4, 2011 at 6:31 AM, Morten Krogh<mk@REDACTED>
>  wrote:
> >>>>>
> >>>>> Why are one fourth, or whatever, of the posts on this list about nif
> or
> >>>>> c
> >>>>> nodes or ports?
> >>>>
> >>>> Even if we assume your percentage is accurate (which I don't), simply
> >>>> assuming these features are used only for performance reasons is a
> >>>> case of premature optimization. ;-)
> >>>>
> >>>> While some uses of these features involve performance concerns, C
> >>>> nodes, ports, drivers, and NIFs all serve as points for integrating
> >>>> Erlang with other environments. NIFs have already proven themselves
> >>>> particularly useful in this regard. Because of the continued growth in
> >>>> the popularity and application of Erlang, the number of developers
> >>>> interested in using these features to integrate with non-Erlang code
> >>>> is also growing.
> >>>>
> >>>> These are areas of Erlang involving languages other than Erlang, such
> >>>> as C, so it could be they're harder to understand than other parts of
> >>>> the language. It also could be that even with the aforementioned
> >>>> growing popularity of the language, the number of Erlang developers
> >>>> who've ever written C nodes and drivers used in production remains
> >>>> relatively small, and so finding good sources of detailed information
> >>>> about how best to write them is challenging.
> >>>>
> >>>> But still, I'd argue there really aren't that many questions posted
> >>>> here about these features.
> >>>>
> >>>> --steve
> >>>
> >>> My percentage is probably wrong. I didn't count. I just wanted to
> stress
> >>> that I see quite a lot of performance optimization in Erlang.
> >>
> >> As do I, but not by jumping out of Erlang. Using Erlang's profilers
> >> and other tools is very straightforward and can significantly help
> >> improve performance.
> >>
> > Yes, and that is also my point. Because people have think more about
> > performance in Erlang than in C or C++ say, profilers and other tools are
> > needed
> > earlier and should be included in a discussion of the difficulty of
> various
> > languages.  Erlang with profilers and tools is a harder "language" than
> > Erlang without.
>
> So what you're claiming is the fact that Erlang provides profiling
> tools proves that it performs poorly? No offense, Morten, but that
> doesn't make sense. Name me a system that doesn't offer profiling
> tools and I'll name you a system that isn't used for real-world
> development.
>
> Erlang provides these tools so that programmers can analyze their
> programs at the level of abstraction of the language itself. I've
> never tried the following, but maybe you could instead use a tool like
> callgrind at the Erlang VM level to analyze your Erlang programs, but
> I think it would probably be pretty challenging to match your findings
> back to your Erlang source code.
>
> As far as I've seen, I use profiling tools (regardless of language) a
> lot more than most programmers, yet the amount of Erlang code I've
> worked on that I've felt the need to actually profile and measure is
> likely in the 2-3% range. I'm sure I'm not alone on this mailing list
> in my opinion that idiomatic Erlang code generally performs very well
> and needs no specific tuning or measurement.
>
> Also, consider this: some of the first hard Erlang performance issues
> I ever faced was when I tried tackling Tim Bray's original WideFinder
> problem several years ago. I was still relatively new to Erlang then
> and compared to what I know now, I barely knew anything, and yet the
> Erlang solution that I submitted was one of the fastest solutions
> among all languages for awhile. I'm sure if I looked at it now I'd
> find that the code I submitted was actually fairly terrible, and I
> knew nothing about Erlang's profilers then (I was just using the timer
> module to measure how long various functions took). Eventually my
> solution fell by the wayside in the contest because it had a bug
> somewhere that I was too inexperienced too resolve, but IIRC correctly
> the winner ended up being an OCaml program (and yes, it was faster
> than any C or C++ that was submitted).
>
> The point of all this, of course, is that all of this information runs
> contrary to your claims that Erlang requires deep, continuous focus on
> performance. In my experience, it doesn't.
>
> >>> Why were NIFs introduced at all? Why new BIFs all the time? All of that
> >>> could be done in Erlang.
> >>
> >> Only the OTP team can say for sure why they introduced NIFs, but my
> >> understanding is that it allows for easier integration with and access
> >> to C than drivers. I personally don't find drivers difficult, but I
> >> know they can be overkill for simple integration needs.
> >>
> > Maybe, but why C to start with. Performance must be a big issue here, but
> > anyway.
>
> The Erlang runtime system is written in C, so NIFs and drivers are C.
> What else would you write it in? C is a universal portable language,
> so it's a logical choice, especially given the timeframe in which
> Erlang development started. I don't know if Kresten's Erjang,
> implemented in Java, supports NIFs yet, but they'd be written in Java
> just as Erjang's drivers are written in Java.
>
> >> Are new BIFs really being introduced "all the time" strictly for
> >> performance reasons? If so can you cite some examples?
> >
> > I was thinking about the Binary module for example. Isn't that whole
> module
> > just for performance reasons more or less.
>
> I thought it provided a richer way of working with binaries. Are
> binaries present in the language only for performance reasons? Klacke
> can tell us for sure, but I don't think so; they're there to provide a
> simple yet extremely powerful abstraction for manipulating bytes and
> bits. When you're dealing with network protocols, like you do in the
> telecoms world for which Erlang was originally created, the need to do
> that easily and in a way that makes for readable and maintainable code
> is critically important. The fact that binaries and bitstrings also
> turn out to be useful in other app domains should come as no surprise.
>
> > I don't have a list of all new BIFs. It was just my understanding that
> every
> > new release would have some new BIFS, that could be
> > implemented in pure Erlang. What about min and max? Aren't they BIFs?
>
> I don't know off the top of my head, but I actually don't care because
> to me BIFs look and act like regular Erlang.
>
> > Anyway, my point in this thread is just that performance matters, and
> when
> > you compare languages you need to include the whole package including
> > that you some language needs more work on performance than others. I was
> > citing NIFs and BIFs etc to claim that other people than me cares about
> > performance
> > in Erlang.
>
> Sure, performance matters, but only in areas that are proven to be
> deficient via actual measurements of actual running programs. Still,
> performance is only one of a number of engineering trade-offs that
> have to be properly balanced in real-world systems. I invite you to
> read an article I wrote about this topic a few years ago, specifically
> about the misguided focus on middleware performance that seemed to be
> rampant in my world back then:
>
> http://steve.vinoski.net/pdf/IEEE-The_Performance_Presumption.pdf
>
> >>> Steve, I have never seen a NIF that was not performance motivated.
> >>> But you have seen many more NIFs than me.
> >>> Do you have any examples?
> >>
> >> I have two NIFs in the system I work on, for example, neither of which
> >> is performance-oriented:
> >>
> >> * One NIF allows simple access the Linux UUID library so that my
> >> Erlang code can generate UUIDs the same as the C++ components with
> >> which it interoperates over the internal chassis network. This has
> >> nothing to do with performance and everything to do with consistent
> >> UUID generation across all the components in the chassis.
> >>
> >> * The other NIF enables control plane calls to some specialized
> >> offload sockets on our video delivery hardware. The system components
> >> providing these sockets are written in C, so the NIF imports their
> >> header files to pick up all their #defines, constants, and function
> >> prototypes, and then presents that functionality in Erlang in a way
> >> that's natural and simple for the Erlang programmer to use. This NIF
> >> has nothing to do with performance and everything to do with code
> >> coherency, integration with the C code, and minimizing duplication
> >> between Erlang and C.
> >>
> > Fine, so performance is not your reason to use more than pure Erlang. I
> will
> > still claim that it is for many others.
> > But it leads to the same conclusion, namely that functional languages are
> > not so much easier.
> > With such interfaces in your code, you are not supporting the notion of
> > functional languages being easy, quite on the contrary.
>
> I have to call BS on you there, since you can't logically reach your
> conclusions from what's been discussed here. I wrote a relatively
> small amount of C in NIF form to integrate Erlang with the rest of our
> system, which means my fellow Erlang colleagues are able to use these
> facilities from their regular Erlang code without having to think
> about C. They're able to stay in the functional programming world and
> still get our specialized hardware to do what they need it to do --
> it's the best of both worlds.
>
> I don't know how much actual real-world experience you have with
> functional programming, but everything I've ever done with FP tells me
> that yes, things actually *are* easier with FP -- the code is easier
> to write, easier to read, smaller, more maintainable, easier to debug,
> easier to extend, and it performs and scales reasonably well, and all
> of this can be done in less time than with most other languages. I've
> watched half a dozen of my colleagues learn Erlang from the start and
> they were all, without exception, able to become very proficient in a
> very short time, shorter than with any other language I've ever seen.
> If you have different experiences, then fine -- all that means is that
> you have different experiences with FP than many of us on this list.
> It certainly doesn't mean the IMO faulty broad conclusions you've been
> drawing trump our individual experiences.
>
> I'm bowing out of this discussion now, too much work to do.
>
> --steve
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>
>