[erlang-questions] erlc speed (or lack thereof), Make and emake

[Loïc Hoguin]

On 01/25/2013 03:28 AM, Loïc Hoguin wrote:
> Anthony Ramine gave me a tip on how to build only modified erl files
> using make's functionality directly, I'll try it out tomorrow and report
> back.

I have finally played with this. It is very nice and works perfectly.

The idea is to recompile only source files newer than the .app file. 
This can be done by making source files depend on the .app file, and 
using $? to obtain the list of changed files.

This can be seen here:

 
https://github.com/extend/cowboy/blob/747bfc29ee951841f38d6e8d471041d6a3008d60/Makefile

If no files have changed:

essen@REDACTED (0) % time make
make[1]: Entering directory `/home/essen/extend/ranch'
make[1]: Leaving directory `/home/essen/extend/ranch'
make  0.04s user 0.01s system 77% cpu 0.060 total

If one file has changed:

essen@REDACTED (0) % time make
erlc -v -Werror +debug_info +warn_export_all  -o ebin/ -pa ebin/ \
	src/cowboy_middleware.erl src/cowboy.erl
make[1]: Entering directory `/home/essen/extend/ranch'
make[1]: Leaving directory `/home/essen/extend/ranch'
make  0.20s user 0.03s system 91% cpu 0.249 total

As you can see I always compile cowboy_middleware, because I need it 
compiled before anything else, as it's a behavior.

If you want to rebuild everything:

essen@REDACTED (0) % time make clean app
make[1]: Entering directory `/home/essen/extend/ranch'
rm -rf ebin/
rm -f test/*.beam
rm -f erl_crash.dump
make[1]: Leaving directory `/home/essen/extend/ranch'
rm -rf ebin/
rm -f test/*.beam
rm -f erl_crash.dump
erlc -v -Werror +debug_info +warn_export_all  -o ebin/ -pa ebin/ \
	src/cowboy_middleware.erl src/cowboy_multipart.erl 
src/cowboy_http_handler.erl src/cowboy_handler.erl src/cowboy_router.erl 
src/cowboy_http.erl src/cowboy_loop_handler.erl src/cowboy_websocket.erl 
src/cowboy_sup.erl src/cowboy_rest.erl src/cowboy_app.erl 
src/cowboy_bstr.erl src/cowboy_protocol.erl src/cowboy_req.erl 
src/cowboy_middleware.erl src/cowboy.erl src/cowboy_client.erl 
src/cowboy_websocket_handler.erl src/cowboy_clock.erl src/cowboy_static.erl
make[1]: Entering directory `/home/essen/extend/ranch'
erlc -v -Werror +debug_info +warn_export_all  -o ebin/ -pa ebin/ \
	src/ranch_transport.erl src/ranch_transport.erl src/ranch_acceptor.erl 
src/ranch_sup.erl src/ranch_listener.erl src/ranch_server.erl 
src/ranch_acceptors_sup.erl src/ranch_conns_sup.erl 
src/ranch_listener_sup.erl src/ranch_ssl.erl src/ranch_app.erl 
src/ranch_protocol.erl src/ranch.erl src/ranch_tcp.erl
make[1]: Leaving directory `/home/essen/extend/ranch'
make clean app  1.39s user 0.10s system 98% cpu 1.512 total

That's pretty fast in my book.

In comparison, rebar:

essen@REDACTED (0) % time rebar clean compile
==> ranch (clean)
==> cowboy (clean)
==> ranch (compile)
Compiled src/ranch_protocol.erl
Compiled src/ranch_transport.erl
Compiled src/ranch_acceptor.erl
Compiled src/ranch_listener.erl
Compiled src/ranch.erl
Compiled src/ranch_tcp.erl
Compiled src/ranch_listener_sup.erl
Compiled src/ranch_conns_sup.erl
Compiled src/ranch_ssl.erl
Compiled src/ranch_acceptors_sup.erl
Compiled src/ranch_server.erl
Compiled src/ranch_sup.erl
Compiled src/ranch_app.erl
==> cowboy (compile)
Compiled src/cowboy_http_handler.erl
Compiled src/cowboy_loop_handler.erl
Compiled src/cowboy_websocket_handler.erl
Compiled src/cowboy_middleware.erl
Compiled src/cowboy.erl
Compiled src/cowboy_bstr.erl
Compiled src/cowboy_router.erl
Compiled src/cowboy_static.erl
Compiled src/cowboy_handler.erl
Compiled src/cowboy_rest.erl
Compiled src/cowboy_protocol.erl
Compiled src/cowboy_sup.erl
Compiled src/cowboy_multipart.erl
Compiled src/cowboy_req.erl
Compiled src/cowboy_clock.erl
Compiled src/cowboy_websocket.erl
Compiled src/cowboy_app.erl
Compiled src/cowboy_client.erl
Compiled src/cowboy_http.erl
rebar clean compile  2.38s user 0.17s system 179% cpu 1.419 total

All this on a state of the art 13" laptop with a good SSD, i7 and 
everything, if you wonder.

I probably won't bother trying to make -j to work. Feel free to send a 
patch if you want to have fun with it, though.

-- 
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu