[erlang-questions] Erlang as a tool for human communication -- was GUI development

[Fred Hebert]

On 12/22, Lloyd R. Prentice wrote:
>
>At this point, my understanding is that Erlang is a terrific language for distributed back-end systems and, to some extent, scalable web applications. But my sense is that the Erlang community is smaller than the merits of the language deserves, has an aging community and, rather than organic growth, seems to be fragmenting into a cluster of BEAM languages that don't, from what I've seen, bring much vitality back to Erlang OTP.
>

Yes and no. Frankly, I think the community's been stable for a while.  
There's a few newcomers coming from the Elixir world into Erlang's, and 
some very interesting VM improvements that have come from that side of 
the fence.

We borrowed a bunch from them (just hex is a good example) and they 
borrowed a lot from us. I think that with better tool compatibility 
(being able to compile Elixir with an Erlang toolchain, or maybe 
adopting mix as your erlang build tool if you want to do it today), we 
could get the influx of folks that support the web in the Elixir world 
and benefit from their work within Erlang.

The community is small for the people who talk in public, but there's a 
surprising amount of people and corporations that use Erlang behind 
closed doors or will not participate to that many conversations about 
it. It's an iceberg of a programming language with a small public-facing 
group and a massive hidden userbase.

The fact that the language is used to make infrastructure component also 
tends to mean that 2-3 developers may do the work that hundreds of 
corporations will use. This is far less often the case when you have 
user-facing libraries and products, which are, for lack of better terms, 
less fundamental to multiple classes of systems, and will instead tend 
to reside within a few classes at most.

The language, I think, has passed its bubble phase -- that one bit of 
time where all the language hipsters and tech early adopters keep 
jumping wagons and you can grab a fraction of them to do your work.  
Erlang has had killer apps. Many of them. I think we should expect 
stability in the years to come; minor growth, and hopefully minor churn 
in community members.

>The question I've asked myself over recent months is, "Why doesn't Erlang have more libraries supporting human communication with the same effectiveness with which it supports machine-to-machine communication?"
>
>The recent thread GUI development thread is Exhibit Number One.
>
>The various stalemated Erlang-questions threads on Erlang documentation is Exhibit Number Two.
>

I think there are two factors here.

One is the very strong infrastructure focus in Erlang products. You will 
have routers and proxies for all types of protocols, translators for 
fancy data types, queues and building blocks for complex systems, 
databases and foundational components to high-load applications. Those 
are very often machine-to-machine mechanisms, doing a specific thing for 
specific high-end environments. Erlang was often used for these because 
nothing else would be reasonable for it, not necessarily because people 
felt that Erlang was amazing; that part usually comes later.

That means you built a community of infrastructure folks, who speak 
infrastructure to each other. Who cares that unicode is not there? Few 
people did, because few people even handled text in the first place; 
those were byte streams you just needed to shuttle from here to there!  
If we just got into the unicode game properly, I'm not *that* surprised 
that user-facing communication is lagging behind.

Reason number two, I think, are community dynamics related to that. I've 
given a keynote on this topic in Stockholm last year -- 
https://www.youtube.com/watch?v=Z28SDd9bXcE and my position is still 
roughly the same. We've made progress but the behaviour of our community 
has been self-reinforcing for years. Very experienced devs who know a 
crapload of stuff, and are usually fine fixing their own problems. This 
makes newcomers usually look somewhere else over time for an easier 
experience.

I welcomed Elixir's challenge in tooling and usability and while their 
tooling is still often seen as superior, I feel that over time the gap 
has been narrowed (we have caught up a bit); the comments I see online 
have less of a focus on that than there was before. That's good.

We're not done yet though.

>Loïc Hoguin has done some nice work on documentation tools. I do wish they were more widely used and, themselves, better documented.
>
>This makes me wonder why we don't eat our own dog food, that is, develop and adopt standardized documentation tools written in Erlang and fluent across all media?
>

The work that *you* do right now is probably the exact kind of stuff we 
need to keep fighting the inertia we have. The questions you ask and 
these comments help.

Here's an interesting one; we have edoc, there is hexdocs to host it, 
but a lot of us don't even get there (myself included). Hell, I'm still 
late on package publishing unless someone prompts me that way.

I think there's a gap currently between "learning Erlang from a book" 
and "using Erlang within the community" that has not been addressed.  
Maybe there's a decent opportunity there.

In the ruby community, I've been told nobody would consider a library 
until it had tests. In the Erlang community, the first question you get 
asked about a system is "has it been in production?" -- there's a very 
strong focus on field experience and proof for a lot of the work. Maybe 
we'd gain by starting to collectively ask "does it have documentation?" 

Is a component truly ready for production if its only documentation is a 
slideset from a lightning talk given to a random usergroup 3 years ago?

>
>Many of the things that I want to do in Writersglen could be accomplished in other languages and with tools and applications written in those languages. But I want to work in Erlang. I don't see any technical obstacles to building better human communication libraries and tools in Erlang. Selfishly, I don't have enough years ahead to learn other languages. I want to build my web community. I want to build it in Erlang. And I wish I'd been able to launch it two years ago.
>
>But beyond my own interests, my hypothesis is this: With more well-documented and polished open source libraries and applications focused on human communication, Erlang would attract a wider, more diverse, population of programmers and developers. This, in turn, would result in a more vibrant community and wider adoption of Erlang.
>
>Am I wrong?

No you're right (at least in my opinion). One of the things I do in the 
workplace and that I recommend people do is take all their interns and 
new hires, and make them take notes on everything they found painful and 
difficult when they started. Tell them early on to do it.

Take that list of opinions from someone who has a fresh view and a 
different background, and try to address these problems. Commit to 
fixing them. If you're a newcomer, this means taking the notes.

Maybe we'd need a place to gather these comments. Some of them won't be 
useful, some of them might be.

This way of proceeding is part of why Tristan and I had started work on 
Rebar3. We wanted other teams within Heroku to try Erlang but they had a 
bad time with other tools.

I'm doing the same thing at Genetec right now, and I've been adding 
better windows support to the tool (and some features that should drop 
later today) for that reason.

>
>If not, what can we do to make it happen?
>
>Perhaps the problem is that we don't have an appropriate venue or forum for sharing ideas.
>

We make the venues with what we have. The harder part is ownership; 
someone committed to tracking and fixing issues.

What can we do to make it happen? Good question.

I think for you personally, it would be neat to see what the stumbling 
blocks you've had were. Can we categorize them? Find a solution that 
would have fixed things?

We could look in whether this was a documentation problem, a 
discoverability problem, issues with tooling, issues with finding 
resources, or anything else, really. Once we have some concrete problems 
to fix, and that we start putting some work that way, we'll get in a 
better place.

The lucky thing with these types of frustrations is that there's some 
big nasty ones that are difficult to fix, but a lot of them are small 
one-off issues that cause deaths from a thousand papercuts. Those are 
usually easy to fix and as you go, progress becomes easier and easier 
and the barrier of entry lower and lower.

Frankly the hard part is finding people with the experience and the time 
to do all of that, but it starts with any individual deciding to get 
their hands dirty.

Regards,
Fred.