[erlang-questions] [ANN} MAD: Erlang Build and Deploy Tool

[Maxim Sokhatsky]

Sounds like one more rebar replacement :-)

History

We came to conclusion that no matter how perfect your libraries are, the comfort and ease come mostly from developing tools. Everything got started when Vladimir Kirillov[1] decided to replace Rusty’s sync beam reloader. As you know sync uses filesystem polling which is neither energy-efficient nor elegant. Also sync is only able to recompile separate modules while common use-case in N2O is to recompile DTL templates and LESS/SCSS stylesheets. That is why we need to recompile the whole project. That’s the story how active emerged. Under the hood active is a client subscriber of fs library, native filesystem listener for Linux, Windows and Mac.

De-facto standard in Erlang world is rebar. We love rebar interface despite its implementation. First we plugged rebar into active and then decided to drop its support, it was slow, especially in cold recompilation. It was designed to be a stand-alone tool, so it has some glitches while using as embedded library. Later we switched to Makefile-based build tool otp.mk.

The idea to build rebar replacement was up in the air for a long time. The best minimal approach was picked up by Sina Samavati[2], who implemented the first prototype called mad[3]. Initially mad was able to compile DTL templated, YECC files, escript (like bundled in gproc), also it had support for caching with side-effects. In a month I forked mad and took over the development under the same name.

Listing 1: Example of building N2O sample

                                   Cold       Hot
    rebar get-deps compile         53.156s    4.714s
    mad deps compile               54.097s    0.899s

Listing 2: Example of building Cowboy

                                   Hot
    make (erlang.mk)               2.588s
    mad compile                    2.521s

Introduction

We were trying to make something minimalistic that fits out Web Stack. Besides we wanted to use our knowledge of other build tools like lein, sbt etc. Also for sure we tried sinan, ebt, Makefile-based scripts.

Synrc mad has a simple interface as follows:

  BNF:
      invoke := mad params
      params := [] | run params
         run := command [ options ]
     command := app | lib | deps | compile | bundle
                start | stop | repl

Single-File Bundling

The key feature of mad is ability to create single-file bundled web sites. Thus making dream to boot simpler than node.js come true. This target escript is ready to run on Windows, Linux and Mac with Erlang installed.

To make this possible we implemented a zip filesytem inside escript. mad packages priv directories along with ebin and configs. You can redefine each file in zip fs inside target escript by creation the copy with same path locally near escript. After launch all files are copied to ETS. N2O also comes with custom cowboy static handler that is able to read static files from this cached ETS filesystem. Also bundle are compatible with active online realoading and recompilation.

Templates

mad also comes with N2O templates. So you can bootstrap a N2O-based site just having a single copy of mad binary.

    # mad app sample
    # cd sample
    # mad deps compile plan bundle web_app

After that you can just run escript web_app under Windows, Linux and Mac and open http://localhost:8000.

  C:\> escript web_app
    Applications: [kernel,stdlib,crypto,cowlib,ranch,cowboy,compiler,syntax_tools,
                   erlydtl,gproc,xmerl,n2o,n2o_sample,fs,active,mad,sh]
    Configuration: [{n2o,[{port,8000},{route,routes}]},
                    {kvs,[{dba,store_mnesia},
                          {schema,[kvs_user,kvs_acl,kvs_feed,kvs_subscription]}]}]
    Erlang/OTP 17 [erts-6.0] [64-bit] [smp:4:4] [async-threads:10] [kernel-poll:false]

    Eshell V6.0  (abort with ^G)
    1>

Also you can create OTP releases with reltool, rebar configure or relx. Internally mad uses reltool to order your dependencies and generation of reltool.config.

    # mad plan
    Ordered: [kernel,stdlib,mnesia,kvs,crypto,cowlib,ranch,
              cowboy,compiler,syntax_tools,erlydtl,gproc,
              xmerl,n2o,n2o_sample,fs,active,mad,rest,sh]

And the good part:

                      Sources        Binary
    mad               567 LOC        39 KB
    rebar             7717 LOC       181 KB

Cheers!

[1] https://twitter.com/darkproger
[2] https://twitter.com/sinasamavati
[3] https://github.com/synrc/mad
[4] https://github.com/synrc/active
[5] https://github.com/synrc/fs
[6] https://github.com/synrc/active

--  
Maxim Sokhatsky  
https://synrc.com