[erlang-questions] erl_scan, erl_parse, macros and records

[Robert Virding]

Sorry, I have not really been able to understand exactly what you want to do and what you are having problems with. A short review of what is done where: 

erl_scan - generates tokens from characters 
epp (the preprocessor) - handles macros, both definition and expansion, and include files. That's all it does not, no more. It works on tokens and generates tokens. 
erl_parse - parses everything except macros which have been done in epp. So it parses records, both the definition and use of. 
compile (the compiler) - takes the abstract syntax from erl_parse and generates code. All record processing is done here (in a very early pass). 

Erl_scan, erl_parse and the compiler are easy to run directly but unfortunately epp, the preprocessor, only works on files. There have been numerous suggestions to modify it to work directly on tokens but this has never been done. 


So if you want to generate code from within a program without writing to temporary files you can basically do everything, including records, except for macros. Include files can easily handled explicitly by parsing them and including the abstract forms directly in with your own forms. If you are generating your own code it should not be too difficult to get around the lack of macros. 


An alternative to using the standard libraries to generate code is to use the syntax_tools application with the erl_syntax module to build an erlang abstract syntax . These tools can handle macros. It does not internally use the same abstract syntax as the standard tools but it has functions for generating the standard abstract syntax from its representation, which can then be used by the compiler. 

An example of generating a record definition from a string would be: 

{ok,Tokens,Line} = erl_scan:string("-record(foo, {a,b=42,c}). "), 
{ok,FooDef} = erl_parse:parse_form(Tokens) 

where FooDef is the abstract syntax form of the record definition. The abstract syntax of a module is just a list of the abstract syntax of al the forms (attributes and functions) in the module. This is why include files are basically simple to handle. 

At what level do you want to start generating input? From strings, or what? 

Robert 


----- "Dan Kelley" <djk121@REDACTED> wrote: 
> 
> 
Hi, 

> 
I'm working on a router of sorts. I'd like to have a config file which specifies how to classify messages, what channels are available, and routing rules based on classifications and channels. 

> 
When my application starts, I'd like to transform the configuration into a module (let's call it 'router') so I can do someting like 'router:route(Message)'. Message will be a record that encapsulates a bunch of things about the message, and the returned value will be the channel to which the message should be delivered. The Message record is reasonably complicated - the top level thing contains several layers of different records. I also have a large number of macros that I use to extract different fields/subrecords of a top-level Message. 

> 
When I started looking at how to do this, I quickly found erl_scan, erl_parse, and friends. I can easily create a module on the fly with these, but everything falls apart when I need to get at the Message record and macro definitions. Despite the access provided to compiler-ish things by erl_scan, erl_parse and friends, it looks like the only way to get at the preprocessor is to use epp on files. This is a problem because the code I want to dynamically create needs to use the record definitions and macros from the Message header file. 

> 
Trying to figure a way around this led me to aleppo, which neatly solved the problem of including the header file, but aleppo doesn't process records. For a brief and exciting moment, it looked like erl_expand_records might save the day, but it expects forms, and I can't get anything with records through erl_parse:parse_form. 

> 
Am I missing an easier way to dynamically create code that references macros and records? 

> 
Thanks, 

> 
Dan 

> 

> 
> _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110621/47c79ff9/attachment.htm>