erl_scan

MODULE

erl_scan

MODULE SUMMARY

The Erlang Token Scanner

DESCRIPTION

This module contains functions for tokenizing characters into Erlang tokens.

EXPORTS

string(CharList,StartLine]) -> {ok, Tokens, EndLine} | Error
string(CharList) -> {ok, Tokens, EndLine} | Error

Types:
CharList = string()
StartLine = EndLine = Line = integer()
Tokens = [{atom(),Line}|{atom(),Line,term()}]
Error = {error, ErrorInfo, EndLine}

Takes the list of characters CharList and tries to scan (tokenize) them. Returns {ok, Tokens, EndLine}, where Tokens are the Erlang tokens from CharList. EndLine is the last line where a token was found.

StartLine indicates the initial line when scanning starts. string/1 is equivalent to string(CharList,1).

{error, ErrorInfo, EndLine} is returned if an error occurs. EndLine indicates where the error occurred.

tokens(Continuation, CharList, StartLine) ->Return

Types:
Return = {done, Result, LeftOverChars} | {more, Continuation}
Continuation = [] | string()
CharList = string()
StartLine = EndLine = integer()
Result = {ok, Tokens, EndLine} | {eof, EndLine}
Tokens = [{atom(),Line}|{atom(),Line,term()}]

This is the re-entrant scanner which scans characters until a dot ('.' whitespace) has been reached. It returns:

{done, Result, LeftOverChars}

This return indicates that there is sufficient input data to get an input. Result is:



{ok, Tokens, EndLine}

The scanning was successful. Tokens is the list of tokens including dot.


{eof, EndLine}

End of file was encountered before any more tokens.


{error, ErrorInfo, EndLine}

An error occurred.


{more, Continuation}

More data is required for building a term. Continuation must be passed in a new call to tokens/3 when more data is available.

reserved_word(Atom) -> bool()

Returns true if Atom is an Erlang reserved word, otherwise false.

format_error(ErrorDescriptor) -> string()

Types:
ErrorDescriptor = errordesc()

Takes an ErrorDescriptor and returns a string which describes the error or warning. This function is usually called implicitly when processing an ErrorInfo structure (see below).

Error Information

The ErrorInfo mentioned above is the standard ErrorInfo structure which is returned from all IO modules. It has the following format:

    {ErrorLine, Module, ErrorDescriptor}

A string which describes the error is obtained with the following call:

apply(Module, format_error, ErrorDescriptor)

Notes

The continuation of the first call to the re-entrant input functions must be []. Refer to Armstrong, Virding and Williams, 'Concurrent Programming in Erlang', Chapter 13, for a complete description of how the re-entrant input scheme works.

See Also

io erl_parse

AUTHORS

Robert Virding - support@erlang.ericsson.se