[erlang-questions] more flexible code generation

[Ulf Wiger]

On 17 Dec 2012, at 10:38, Tim Watson wrote:

> Ulf - some feedback: this is totally AWESOME!
> 
> Damn I need to find time to go and refactor at least three projects to take advantage of this now.... :D

Thanks for the encouraging feedback. :)

I've just pushed some additions for improving error reporting, which I would also like feedback on.

Handling errors in parse transforms has been a bit tricky. My latest approach is to use a new function, parse_trans:return(Forms, Context), which checks for the presence of {error, Info} or {warning, Info} tuples in the abstract tree. It will then extract them and produce a return that the compiler is comfortable with.

I have also added a function, parse_trans:format_exception(Class, Error), which can be used to produce a less user-hostile exception trace.

To illustrate: I've started working on a module for code generation from YANG specs:

https://github.com/tonyrog/yang/blob/master/src/yang_codegen.erl

I got some horrible exception dumps, and was motivated to address the problem. The parse_transform/2 function in parse_trans_codegen.erl now looks like this:

parse_transform(Forms, Options) ->
    Context = parse_trans:initial_context(Forms, Options),
    {NewForms, _} =
	parse_trans:do_depth_first(
	  fun xform_fun/4, _Acc = Forms, Forms, Context),
    parse_trans:return(parse_trans:revert(NewForms), Context).

where the parse_trans:return(Forms, Context) is new.

A crash in the code generation now looks like:

==> yang (compile)
src/yang_codegen.erl:71: exception error: no case clause matching match_expr
  in function  parse_trans_codegen:gen_function_/4 (src/parse_trans_codegen.erl, line 189)
  in call from parse_trans_codegen:gen_function/5 (src/parse_trans_codegen.erl, line 181)
  in call from parse_trans_codegen:xform_fun/4 (src/parse_trans_codegen.erl, line 156)
ERROR: compile failed while processing /Users/uwiger/FL/git/yang: rebar_abort
make: *** [compile] Error 1

…which, by parse_transform standards, is pretty decent. Note that we get the line info in the original source code.

The error-trapping code in parse_trans_codegen.erl looks like this:

https://github.com/uwiger/parse_trans/blob/master/src/parse_trans_codegen.erl#L180

gen_function(NameF, FunF, L0, L, Acc) ->
    try gen_function_(NameF, FunF, L, Acc)
    catch
	error:E ->
	    ErrStr = parse_trans:format_exception(error, E),
	    {error, {L0, ?MODULE, ErrStr}}
    end.

Now, I *could* rework the existing functions in parse_trans to produce error info in this way. It won't happen immediately, since I have other, more urgent, things on my plate, but - would anyone have objections to this?

It's not entirely trivial. If I have the existing functions add {error, I} and {warning, I} tuples in the form tree, and the parse_transform() functions don't end with a call to parse_trans:return/2, the linter will crash, which is neither pretty, nor informative. If the old functions call parse_trans:return/2 automatically, this will mess up code that makes successive calls to them.

BW compatibility sucks, as Robert Virding once put it. I am open to suggestions.

BR,
Ulf W

> 
> On 16 Dec 2012, at 11:39, Ulf Wiger wrote:
> 
>> 
>> I made a little addition to my parse_trans library, in parse_trans_codegen:
>> 
>> The 'original' method for generating code for a function was e.g.:
>> 
>> g(Name, V) ->
>>   codegen:gen_function(
>>     Name,
>>     fun(L) ->
>>             member({'$var',V}, L)
>>     end).
>> 
>> Where the {'$var', V} notation is a way to insert a reference to a variable, rather than using the variable itself. 
>> 
>> To illustrate, let's first create a little pretty-printer fun:
>> 
>> 1> PP = fun(F) -> io:fwrite("~s~n", [erl_pp:form(F)]) end.
>> #Fun<erl_eval.6.82930912>
>> 
>> 
>> 2> PP(ex_codegen:g(foo,17)).
>> foo(L) ->
>>   member(17, L).
>> 
>> There are a few other, similar, functions, but in some cases, this is still too restrictive. A few days ago, I added support for specifying a more dynamic pattern:
>> 
>> k(L) ->
>>   codegen:gen_function(
>>     lcf,
>>     [fun({'$var',X}) ->
>>              {'$var',Y}
>>      end || {X, Y}  <- L]).
>> 
>> The list comprehension results in a list of clauses based on data resolved at run-time. For example:
>> 
>> 3> PP(ex_codegen:k([ {"a", {1,2,3}}, {"b", {4,5,6}} ])).
>> lcf("a") ->
>>   {1,2,3};
>> lcf("b") ->
>>   {4,5,6}.
>> 
>> https://github.com/uwiger/parse_trans/blob/master/doc/parse_trans_codegen.md
>> 
>> Feedback is welcome.
>> 
>> BR,
>> Ulf W
>> 
>> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.
>> http://feuerlabs.com
>> 
>> 
>> 
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
> 

Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.
http://feuerlabs.com