[erlang-questions] erlc speed (or lack thereof), Make and emake

[Serge Aleynikov]

Loïc,

There's a little trick you can use in a make file to avoid spawning
external shell to comma delimit files:

empty :=
space := $(empty) $(empty)
comma := $(empty), $(empty)

FILES   := $(wildcard src/*.erl)
MODULES := $(subst $(space),$(comma),$(sort $(FILES:src/%.erl=%)))

example:
    @echo "UsingShell = $(shell ls src/*.erl | sed
's/src\///;s/\.erl/,/' | sed '$$s/.$$//')"
    @echo "UsingMake  = $(MODULES)"

And also for the app target you can just do:

ebin/$(PROJECT).app: src/$(PROJECT).app.src
	sed 's/{modules, \[\]}/{modules, \[$(MODULES)\]}/' $< > $@

app: deps/ranch ebin/$(PROJECT).app
	@$(MAKE) -C $(DEPS_DIR)/ranch


My $2c.

Regards,

Serge


On 1/28/2013 8:44 PM, Loïc Hoguin wrote:
> On 01/25/2013 03:28 AM, Loïc Hoguin wrote:
>> Anthony Ramine gave me a tip on how to build only modified erl files
>> using make's functionality directly, I'll try it out tomorrow and report
>> back.
> 
> I have finally played with this. It is very nice and works perfectly.
> 
> The idea is to recompile only source files newer than the .app file.
> This can be done by making source files depend on the .app file, and
> using $? to obtain the list of changed files.
> 
> This can be seen here:
> 
> 
> https://github.com/extend/cowboy/blob/747bfc29ee951841f38d6e8d471041d6a3008d60/Makefile
> 
> 
> If no files have changed:
> 
> essen@REDACTED (0) % time make
> make[1]: Entering directory `/home/essen/extend/ranch'
> make[1]: Leaving directory `/home/essen/extend/ranch'
> make  0.04s user 0.01s system 77% cpu 0.060 total
> 
> If one file has changed:
> 
> essen@REDACTED (0) % time make
> erlc -v -Werror +debug_info +warn_export_all  -o ebin/ -pa ebin/ \
>     src/cowboy_middleware.erl src/cowboy.erl
> make[1]: Entering directory `/home/essen/extend/ranch'
> make[1]: Leaving directory `/home/essen/extend/ranch'
> make  0.20s user 0.03s system 91% cpu 0.249 total
> 
> As you can see I always compile cowboy_middleware, because I need it
> compiled before anything else, as it's a behavior.
> 
> If you want to rebuild everything:
> 
> essen@REDACTED (0) % time make clean app
> make[1]: Entering directory `/home/essen/extend/ranch'
> rm -rf ebin/
> rm -f test/*.beam
> rm -f erl_crash.dump
> make[1]: Leaving directory `/home/essen/extend/ranch'
> rm -rf ebin/
> rm -f test/*.beam
> rm -f erl_crash.dump
> erlc -v -Werror +debug_info +warn_export_all  -o ebin/ -pa ebin/ \
>     src/cowboy_middleware.erl src/cowboy_multipart.erl
> src/cowboy_http_handler.erl src/cowboy_handler.erl src/cowboy_router.erl
> src/cowboy_http.erl src/cowboy_loop_handler.erl src/cowboy_websocket.erl
> src/cowboy_sup.erl src/cowboy_rest.erl src/cowboy_app.erl
> src/cowboy_bstr.erl src/cowboy_protocol.erl src/cowboy_req.erl
> src/cowboy_middleware.erl src/cowboy.erl src/cowboy_client.erl
> src/cowboy_websocket_handler.erl src/cowboy_clock.erl src/cowboy_static.erl
> make[1]: Entering directory `/home/essen/extend/ranch'
> erlc -v -Werror +debug_info +warn_export_all  -o ebin/ -pa ebin/ \
>     src/ranch_transport.erl src/ranch_transport.erl
> src/ranch_acceptor.erl src/ranch_sup.erl src/ranch_listener.erl
> src/ranch_server.erl src/ranch_acceptors_sup.erl src/ranch_conns_sup.erl
> src/ranch_listener_sup.erl src/ranch_ssl.erl src/ranch_app.erl
> src/ranch_protocol.erl src/ranch.erl src/ranch_tcp.erl
> make[1]: Leaving directory `/home/essen/extend/ranch'
> make clean app  1.39s user 0.10s system 98% cpu 1.512 total
> 
> That's pretty fast in my book.
> 
> In comparison, rebar:
> 
> essen@REDACTED (0) % time rebar clean compile
> ==> ranch (clean)
> ==> cowboy (clean)
> ==> ranch (compile)
> Compiled src/ranch_protocol.erl
> Compiled src/ranch_transport.erl
> Compiled src/ranch_acceptor.erl
> Compiled src/ranch_listener.erl
> Compiled src/ranch.erl
> Compiled src/ranch_tcp.erl
> Compiled src/ranch_listener_sup.erl
> Compiled src/ranch_conns_sup.erl
> Compiled src/ranch_ssl.erl
> Compiled src/ranch_acceptors_sup.erl
> Compiled src/ranch_server.erl
> Compiled src/ranch_sup.erl
> Compiled src/ranch_app.erl
> ==> cowboy (compile)
> Compiled src/cowboy_http_handler.erl
> Compiled src/cowboy_loop_handler.erl
> Compiled src/cowboy_websocket_handler.erl
> Compiled src/cowboy_middleware.erl
> Compiled src/cowboy.erl
> Compiled src/cowboy_bstr.erl
> Compiled src/cowboy_router.erl
> Compiled src/cowboy_static.erl
> Compiled src/cowboy_handler.erl
> Compiled src/cowboy_rest.erl
> Compiled src/cowboy_protocol.erl
> Compiled src/cowboy_sup.erl
> Compiled src/cowboy_multipart.erl
> Compiled src/cowboy_req.erl
> Compiled src/cowboy_clock.erl
> Compiled src/cowboy_websocket.erl
> Compiled src/cowboy_app.erl
> Compiled src/cowboy_client.erl
> Compiled src/cowboy_http.erl
> rebar clean compile  2.38s user 0.17s system 179% cpu 1.419 total
> 
> All this on a state of the art 13" laptop with a good SSD, i7 and
> everything, if you wonder.
> 
> I probably won't bother trying to make -j to work. Feel free to send a
> patch if you want to have fun with it, though.
>