[erlang-questions] ANN: toker - erlang parser fun

Ulf Wiger <>
Sun Nov 30 09:23:55 CET 2014


Pluggable parsers for the Erlang compiler

This project started as an experiment with runtime code injection using the parse_trans library.

(Note, 'Toker' is the Swedish name for Dopey the dwarf, but also obviously a word play refering to token transformation.)

This library installs itself into the Erlang compiler, allowing modules to switch parser modules as well as install token transformers.

Example: the toker_test module uses a custom syntax:


-export([double/1, i2l/1]).


double(L) ->
    lists:map(`(X) -> X*2`, L).

i2l(L) ->
    lists:map(`integer_to_list/1, L).

This module will not compile with the standard Erlang parser, but toker's own build chain bootstraps itself and installs a hook in the erl_parse module, which then detects the instruction -toker_parser(toker_erl_parse).


Eshell V5.10.3  (abort with ^G)

1> toker_test:double([1,2,3]).

2> toker_test:i2l([1,2,3]).

Contributions are welcome, although the library is mostly for fun (it’s been gathering dust for a while, and I thought I’d at least publish it).

Ulf W

Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 496 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20141130/62d88daa/attachment.bin>

More information about the erlang-questions mailing list