[erlang-questions] Where is the Golden Path?
Tue Jul 16 15:55:21 CEST 2019
A former manager used to say "You hero, sir" when someone did something
Imagine me saying it.
On Mon, 15 Jul 2019 at 19:02, Todd <> wrote:
> From http://zwrob.com/posts/erlang-01/
> I'm slowly learning [Erlang]. It's taken me more than 10 years to get to
> where I am now...which, frankly, isn't very far. I thought I'd share some
> thoughts from a veteran software developer with over 25 years of
> experience in
> the industry.
> # History
> From 2009 to 2019...
> * read books [(Joe, Fred, etc.)]
> * write little code snippets
> * post little [how-to-tutorials] showing how little I understand about
> * wander away as I lose focus and time
> * repeat.
> This year, after Joe died, I thought...I'm not getting any younger either.
> decided to take the summer off and really study Erlang. So far the process
> has been:
> * read the [same books], but only as references
> * have a real problem to solve (I'm basing a business on it)
> * write little code snippets (escripts) to POC business components
> * document what I am learning on my [personal website]
> * design the full scale app
> * implement ... oops hitting the road blocks...
> ## Road blocks
> ### Erlang installation and tools
> I wish I'd known about [asdf] much earlier. That's really simplified my
> dev box
> configuration. I wish that [rebar3] and [kerl] and other docs just
> pointed this
> out. In fact, I wish there was a `golden` path published that just says,
> here's the common accepted tool chain you are going to want to use. Stay on
> this path, and life will be easy. [rebar3] is a big part of this story,
> not the whole thing.
> ### Migrating from escripts to `real` apps
> $ rebar3 new escript myfooscript
> I started with escripts because I didn't want the hassle of understanding
> or anything else as I was writing POC snippets to understand the basic
> libraries. So I'd write an escript to shell out and curl some resource and
> gradually build up an application that ingests data and writes into a
> blah blah. But along the way I could learn about `xmerl` and other useful
> libraries. Oh, and I could run the app and see STDOUT. Cool! Just like any
> other language I'm familiar with.
> So I roll up my sleeves and decide to create an app...
> ### Naming things is hard
> $ rebar3 new release|umbrella myfooapp
> Um, why does 'umbrella' exist as an option here? Is it the same thing as a
> 'release'? Is it necessary? Why is 'release' called 'release'? This is an
> overloaded term, as there are 'releases' which is a thing, and there's a
> 'release' command, and there's a 'release' target in the rebar3 command.
> there's an 'umbrella' in there, too. OK, release == umbrella, so I pick
> $ rebar3 new release myfooapp
> ### Where is STDOUT?
> So, let me say, `rebar3` is great. I barely remember what it was like to
> `sinan` or `faxien`. (And while Loic's Makefiles look amazing, I'm nowhere
> smart enough to use them.) But here's the thing, I had to watch a number
> of videos
> before I found out what the `shell` command was. This is important to me
> the next block I had was... how do I start my application _and_ see the
> from it? I want to watch STDOUT, right? I was watching some conference
> when someone mentioned that 'shell' was the coolest part of rebar3 and I'm
> like, 'what is that'? And voila! It's what I needed.
> This should be on the golden path.
> $ cd myfooapp
> $ rebar3 shell myfooapp
> ### Where is the golden path?
> It's weird that there is no golden path. [In this talk], at 14:50, Fred
> about how it's only far into various Erlang books that authors start
> about running code outside the shell, in contrast to K&R C where it's
> on page 6. In fact, the entire (excellent) presentation is on the terrible
> of tooling in the Erlang ecosystem.
> *begin rant*
> #### Here's what's weird about Erlang : Books are out of date
> * The books are already out of date. Even [LYSEFGG] still talks about
> [systools] and [reltool] even though [rebar3] and [relx]
> supplanted them? What's weird about this is to be reading the rebar3
> documentation, then cross-referencing with relx, and reading the various
> that describe the old ways of building releases and relups, etc.
> #### Docs are hard to follow
> * [relx] documentation is pretty hard to follow
> * Community support seems fragmented
> * [Github shows relx] has 41 open issues and 6 open pull requests,
> 89 contributors
> * [Github shows rebar3] has 107 open issues and 11 open pull
> with 239 contributors
> #### Who's maintaining these tools?
> If I were Tristan or Fred, boy I'd be tired of supporting this stuff.
> #### Where are the videos?
> Here's another thing that's weird...
> I cannot find videos on how to use the Erlang shell. So on top of that
> fact, I'm finding out about erlang by watching Elixir videos. I was
> watching an
> Elixir video on 'wobserver' and realized, 'Hey! I bet that's just a web
> interface on some Erlang thingy called "observer"'. So then I find out
> 'observer' and how useful that is. Great! But things like, how to I do
> stuff like navigate from one process to another, what do I do when
> seems hung and broken, what do real live seasoned Erlang devs do when
> hit's the fan'? How do they go about examining a running system? The web
> to have nothing on this. But that's just weird, b/c Erlang has been around
> over 25 years, and any other language / framework/ runtime whatever has
> of youtube videos.
> *end rant*
> ## Culture of Quick Fixes
> OK, so enough ranting. Let me get into [Fred's mention of the 'Culture of
> Fixes'] (at about 21:00) by introducing my own steaming pile of a quick
> So here's the deal. I'm trying to build a [company]. I have limited
> $$$, I
> can't screw around learning stuff forever. At some point I have to build a
> live app and try to generate some buzz. I'll roughly describe my first
> app/service and run through my decision points...
> ### App
> The app basically pulls data from publicly available but disparate data
> ingests them, normalizes them into a common data store, and then surfaces
> data store for analysis, reporting, AI/ML, etc, etc. In a nutshell, take
> add magic, produce pretty pictures.
> ### Process
> * prototype in bash (or python, or whatever...)
> Seriously, with just `bash` and I can query for my data, extract in, play
> around with it, etc. If I wanted I could hook it up to `cron` and probably
> move on
> to actually using the data rather than learning Erlang and how to build
> ### Prototype in escript
> Here's where I take small tasks and learn the Erlang libraries to
> what I'm trying to do. It takes about as much time to do this as the
> prototyping step. I find this to be a great way to both decompose a
> problem and
> learn a new language.
> ### Start building industrial grade services
> So now, rather than focusing on my business, finding customers, analyzing
> data I've collected, or in any other way providing value that could bring
> revenue and allow me to purchase more Erlang books... I'm veering off into
> weeds and learning how to:
> * build erlang apps for real
> * build erlang releases
> * wire all this into docker containers
> * wire in monitoring and all the 'industrial grade goodies'
> Along the way I tend to post about what I learn [here].
> * Is any of this worthwhile?
> * Am I just engaged in endless pencil sharpening?
> * I dunno. I'm too close to the fire.
> I'm having fun though. And it's incredibly useful to my brain in some way
> I cannot describe.
> ### The Problem
> So back to "The Culture of Quick Fixes". As I'm building my first set of
> components and learning about rebar.config and config/sys.config and more
> stuff, I run into the issue of how to deploy Erlang to docker containers.
> answer is, run the release with the `foreground` command. But this wasn't
> obvious to me at first and took a lot of digging around the 'net, [posting
> question on the rebar3 discussion list], and just general noodling
> But, if I'm building industrial grade applications, then I need to solve
> issues such as:
> * How do I emit metrics from my app?
> * How do I monitor my metrics (e.g. Prometheus)?
> * How do I hook up a visualization tool (e.g. Grafana)? Alarms?
> * And how do I wire all this up into a containerized solution so that it's
> re-usable and shareable?
> * Don't forget my "awesome" Makefiles! I like to have a common control
> plane for
> all my services, and common Makefiles are great for this.
> Now, if you watched the end of Fred's video, he was talking about many of
> same things.
> *And I'll suggest that what I've built is an example of exactly what
> shouldn't be done:*
> * I ignored everything on hex. Why? Um, I don't really understand hex that
> yet, I'd have to go read a bunch of code to understand what folks have
> and I'm still learning Erlang by building things myself. None of these
> is a
> good reason, BTW.
> * I only published my stuff to github, and not to hex. This may be a good
> as I'd just be duplicating other, probably better, efforts.
> * What I built should probably be a rebar3 plugin, and not a Makefile with
> dependencies on rebar3 etc.
> However, I'll show you what this is because I think it satisfies a need and
> perhaps we can all come up with a better implementation strategy. Or learn
> ## Erlang-Seed
> So, here we have 'yet-another-quick-fix' called [erlang-seed]. Let me
> honest, this is a 'quick-fix' to satisfy my own personal needs...
> ### The Good
> What this does is create an erlang application, inject it into a Docker
> container, and then spin up a local Docker cluster that monitors the
> Erlang app
> and surfaces the metrics via Prometheus and Grafana. So with one command I
> generate an OTP application, all the infrastructure necessary to monitor
> it with
> common tooling, _and_ spin it up and run it on my dev box. I find this to
> really useful. YMMV.
> ### The Bad
> This is really crufty. It calls into rebar3 to generate the app. It should
> probably be a rebar3 plugin so that it can better evolve with the tooling.
> could then more intelligently inject the necessary dependencies,
> instrument the
> code with metrics, etc. Right now it's gross, as it just overwrites files
> necessary using mustache templates.
> ### The Ugly
> * Do I have time to maintain this? No.
> * Do I have time to make this a rebar3 plugin? Not right now.
> * Will this probably break over time due to either rebar3, docker, or some
> other dependency changing? Certainly, eventually, yes.
> ## Summary
> The reason I'm sharing this is to just widen the discussion on what the
> education and tooling could look like. I think that generating full stacks
> allow developers to focus more on solving business domain issues rather
> forcing them to understand all the arcane details of whatever is under the
> covers is the 'way to go (TM)'. There's got to be a way to allow
> developers to
> maintain velocity while increasing their depth and scope of knowledge as
> necessary. I've been studying Erlang intensely for at least a month now,
> the books _countless_ times, and read the online docs and I still don't
> understand how this all fits together. I mean I get the big picture. But
> a _lot_ of details.
> Contrast this with Go. I was writing production code in 2 days. The
> quality was
> production quality after about a week. Back in the day it took me about a
> to become proficient in Java, though it required reading Bloch's book "Java
> Concurrency In Practice" to be 'good'. Python was similar, it took about a
> to ramp up, about a month to become comfortable. In all of these languages
> was a relatively short and easy ramp up and then smooth sailing for years
> With Erlang it's an incredibly steep ramp that goes on and on. I guess if I
> remember that I'm learning the equivalent of a language, an orchestration
> framework (like kubernetes), a database system, etc., then it starts to
> Anyway, for my part, I'll keep posting things as I figure them out.
> erlang-questions mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the erlang-questions