[erlang-questions] What problem are we trying to solve here? [was Erland users group [was re: languages in use? [was: Time for OTP to be Renamed?]]]

[Garrett Smith]

On Sat, Feb 15, 2014 at 11:08 AM, Fred Hebert <mononcqc@REDACTED> wrote:
> One very important thing that jumped to me while writing LYSE is that it
> takes *forever* before getting from "I'm writing stuff in the shell and
> a module" to "I have a program I can call from the command-line".

I completely agree - and this is my number one personal pain point. I
want to do this:

$ make release

and:

$ make exe

And have it just work!

Maybe this is all there, but last I checked, the tools for making
releases were not easy. I'm now inspired to spend some cycles on this.

> Since the community has apparently settled on, rebar, relx, releases,
> etc. There's a crapload of learning to do before tying your hands to
> production prototype. See how Go's toolchain works there.

Over time I've become ever increasingly antagonistic to rebar. Not
trying to be a meanie poo poo head here -- but independent of its
merits/demerits I don't see the harm in supporting alternatives.

I've been erlang.mk and have really liked it. I don't know if this is
the final landing point in my search for a build tool that works and
across the various common use cases, but for what I've used it for
(simple builds with dependencies) it works very well.

I haven't spent any time with relx. I will.

> I know this likely gets close to your comments from an earlier e-mail --
> we could show OTP sooner to get started faster -- but my objections, I
> think, still stand.

Indeed, I don't think this is either-or. In fact, learning from the
top down requires learning language constructs along the way -- and
learning from bottom up should not preclude a vision about end goals.
I do think that getting a runnable artifact you can put into
production is best done sooner than later.

> It could be interesting to see if a tool-based approach could help.
> Rebar templates to set things up, relx to build the release. Bundle-in
> tools (eper, recon, eflame, etc?), possibly.

I think definitely so.

There's a general angst reflected in the Therapy Session Threads about
scattered libraries and tools. But until we can get *something better*
in the core distribution, I think that's just the way it goes.
Regardless, the first step is to get something that's decent. rebar
has emerged, somehow, as the defacto standard and if that's what the
market wants, maybe it, or pieces or it, could be integrated into an
officially supported tool chain. I don't know. I just want my damn
builds to work and to get releases out without a day's effort.

> Hell, make a beginner project template that uses raw processes and a
> supervisor_bridge above them to get people going *with OTP* without
> first needing to understand it all. This later leaves place to replacing
> various parts of the project without needing a total rearchitecture of
> all connex services running.

I don't see this coming from today's OTP code. Usability was (clearly)
never a priority for these modules. e2 tries to wrap this complexity
and I think does a pretty reasonable job. It could be made easier, but
I'm inclined to think at that point we're talking about another
language -- or some serious parse transforms.

e2 was originally meant to use parse transforms to really dig out from
under the gen_server/supervisor-spec baggage. But the problem I ran
into quickly is that there's only so far you can go in simplifying the
"generic server" API. At a certain point, the programmer loses touch
with the client/server connection, and you've created an abstraction
that leaks way too often to be any good. So e2 is now just an API
wrapper.

But this is clearly not a priority, even remotely, for most Erlang
programmers. I honestly don't understand it. If these APIs were easier
to use, people would use them more often. They'd have canonical Erlang
programs without a sense they're using some heavy weight framework
(comparisons to J2EE? really?).

> Package management and discoverability would likely help there too.
>
> Another one, and I think this is where we lose a lot of people, is
> algorithms. Anybody who uses Erlang to shuttle bytes around the place
> for the network will tend to have a far better time than someone who
> comes around and decides "I'll write a game and rely on math-heavy
> stuff".
>
> Or even more generally, "I'll write a thing that requires an algorithm
> that uses arrays and assumes O(1) updates". Then, you're directly
> screwed. Your A* that uses a loop and a table, your vector
> multiplication or whatever, they all go slower than expected.
>
> Maybe maps will help, maybe not. But to me there's no doubt that
> functional/immutable algorithms are definitely trickier to write given
> the literature. Outside of Okasaki's book (which is for MLs and Haskells
> first, so you need to learn them to translate in Erlang!), there is
> nearly no source of algorithms that are not in papers. Do we need an
> 'algorithms in Erlang' tutorial?

Yeah - I think this is part of the journey into the realm of
functional programming. It has its costs, but it has its benefits.
There are many times when I wish I didn't have to think so damn hard
about how to "do something". But nearly all of the time this is a
benefit rather than a cost.

For the performance sensitive code, I like the jump into C/C++ land.
Heck, even using something like erlport and punting over to Python
(which has some great native libraries like NumPy) is so easy and
elegant in Erlang -- I see this as a real advantage.

> Don't get me wrong -- I think once you're set up, the maintainability of
> Erlang programs is amazingly good. I've written long blog posts on this.
> I also think the structure of programs (like service-oriented, but
> within a node) is brilliant. But before you get there, there's a
> shitload of hurdles to get through, and few apps or exercises targeted
> at beginners to help build their chops or get a good general idea.

Sigh. Yes, it's discouraging that these points -- in particular the
micro service architecture -- aren't talked about *way more often*.

I actually don't think it's all that hard. I know, I've been pimping
this e2 thing -- but seriously -- when you can write OTP compliant
behaviors in with zero boilerplate, it's a *lot* easier, and even fun!

I don't know what your thoughts are on the three hour tutorial that I
run (did this in Toronto) -- but in that amount of time, we (I) build
a full fledged, service oriented, OTP compliant app. I realize this
three hour investment doesn't quality audience members to jump in and
solve business critical problems in Erlang, but heck, they at least
saw that you can build real applications using canonical Erlang
patterns with a straight forward problem solving methodology.

I *think* they saw this. I may be delusional.

> More or less, the learning curve of Erlang is Vim-like or Emacs-like.
> Sadly, if you want to learn Erlang, you often get told to also learn
> Emacs on top of it.
>
> That makes for quite a steep curve, doesn't it?

If it doesn't cost sacrifice important, flattening the curve will only
help everyone. There's lots of friction points that we can smooth out
-- but boy it'd be nice if others saw this as a problem too. But I'm
not judging.

Garrett